Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REF: "bare_pytest_raises" to use the ast module #32932

Merged

Conversation

ShaharNaveh
Copy link
Member

@ShaharNaveh ShaharNaveh commented Mar 23, 2020

  • closes #xxxx
  • tests added / passed
  • passes black pandas
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry

There is also a performance boost:

In [1]: import os                                                                                             

In [2]: from scripts.validate_unwanted_patterns import bare_pytest_raises, main                               

In [3]: SOURCE_PATH = "pandas/tests/"                                                                         

In [4]: %timeit main(function=bare_pytest_raises, source_path=SOURCE_PATH, output_format="{source_path}:{line_number}:{msg}.") 

12.6 s ± 76.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) # Master
4.33 s ± 27 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) # PR

Now it uses the ast module instead of the tokenize module
@jbrockmendel
Copy link
Member

@MomIsBestFriend tangential to this bc I was thinking about an ast-based check: one anti-pattern we have is using private functions across modules (e.g. core.indexing._maybe_numeric_slice used in io.formats.style). How hard would it be to track those down?

@ShaharNaveh
Copy link
Member Author

ShaharNaveh commented Mar 23, 2020

@MomIsBestFriend tangential to this bc I was thinking about an ast-based check: one anti-pattern we have is using private functions across modules (e.g. core.indexing._maybe_numeric_slice used in io.formats.style). How hard would it be to track those down?

@jbrockmendel If you meant tracking every use of private function that is not defined is the exact same file, I spouse is doable. One problem I think is going to be with .pyx files, but I can start small and add that feature later on.

@jbrockmendel
Copy link
Member

Dont go out of your way on my account; focus on what you're most interested in. We appreciate the effort your putting in.

@ShaharNaveh
Copy link
Member Author

ShaharNaveh commented Mar 23, 2020

Dont go out of your way on my account; focus on what you're most interested in. We appreciate the effort your putting in.

I want to make progress, I am not really familiar with the code base, I really like when you give me issues you'd like to see me work on, because this is how I make progress.
I will never make progress if I don't do things I am not 100% sure on them.


@MomIsBestFriend tangential to this bc I was thinking about an ast-based check: one anti-pattern we have is using private functions across modules (e.g. core.indexing._maybe_numeric_slice used in io.formats.style). How hard would it be to track those down?

@jbrockmendel If you meant tracking every use of private function that is not defined is the exact same file, I spouse is doable. One problem I think is going to be with .pyx files, but I can start small and add that feature later on.

@jbrockmendel Can you confirm I got it correctly?

Copy link
Member

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - I think can simplify a bit but otherwise looks good

continue
for next_token in tokens[counter:]:
if next_token.type == token.NAME and next_token.string == "match":
def get_fqdn(node: ast.AST) -> Optional[str]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this I think you can just call node.func.value.id to get pytest and node.func.attr for the raises portion

@WillAyd WillAyd added the CI Continuous Integration label Mar 27, 2020
@jreback jreback added this to the 1.1 milestone Mar 29, 2020
@jreback jreback merged commit 0f044f8 into pandas-dev:master Mar 29, 2020
@jreback
Copy link
Contributor

jreback commented Mar 29, 2020

thanks @MomIsBestFriend

@ShaharNaveh ShaharNaveh deleted the CI-unwanted-patterns-refactor-to-use-ast branch March 29, 2020 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants