STYLE pandas-dev flake8 plugin #40826
Comments
MarcoGorelli
added
Code Style
|
I think this would be great to try out
…Sent from my iPhone
On Apr 7, 2021, at 10:49 AM, Marco Gorelli ***@***.***> wrote:
Currently, there are a lot of style checks which are specific to this repo, the result of which is a ton of CI checks and occasionally quite a bit of extra code.
The main repo is already huge, so I'd like to suggest making a pandas-dev/pandas-style-guide repo, which would be a flake8 plugin, with all the custom linting rules we currently have.
Some advantages:
the current gigantic repo would become a bit smaller, as many style checks could be moved over to the pandas-dev/pandas-style-guide one
the plugin could easily be tested on multiple OSs and versions of Python, which is currently difficult with all the stylistic checks living in the main repo. The ast module often changes a bit between versions, so it would be good to test the ast-based checks on different Python versions
linting failures would be reported with the familiar flake8 syntax we're all accustomed to by now
One disadvantage I see is that there might be less visibility if this is in a separate repo.
@pandas-dev/pandas-core @pandas-dev/pandas-triage anyone have any thoughts/objections?
—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Can you be more specific on what are the code checks that you'd like to move? For the grep commands in For |
Mainly the ones in Also, future ast-based scripts. pandas is using more and more typing, and I expect that this will require futher checks that regex won't scale to but which won't be generic enough to be used outside of pandas
The big advantage I see is the second bullet-point I listed above about testing on multiple OSs and Python versions. Decluttering the main repo would also be nice.
Thanks - I'll put something together then and we can take the discussion from there, if people don't like it then that's fine, I can bin it 😄 |
|
~2 years ago there was a discussion about trying to upstream some of our
code checks to existing tools that had their own maintainers. I'm not sure
if anything ever came of that, but if it was a successful model it may be
worth trying again.
…On Wed, Apr 7, 2021 at 11:23 AM Marco Gorelli ***@***.***> wrote:
Can you be more specific on what are the code checks that you'd like to
move?
Mainly the ones in scripts - some of the grep ones may also be more
robust as part of an AST traversal than with a regex
I don't personally see the advantage is they can't be reused
The big advantage I see is the second bullet-point I listed above about
testing on multiple OSs and Python versions. Decluttering the main repo
would also be nice.
------------------------------
I think this would be great to try out
Thanks - I'll put something together then and we can take the discussion
from there, if people don't like it then that's fine, I can bin it 😄
—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
<#40826 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AB5UM6G6K5DZ4WKQRCD6ZA3THSPIRANCNFSM42RHWOBA>
.
|
|
Being able to test the style checking against different Python versions / OSs sounds like a good reason to use a separate repo. Practical question: how do you see the syncing between both repo's (as right now I suppose often a change in the scripts and the related change in the pandas code itself happen in the same PR)? The separate repo has a version, and that version gets bumped from time to time in pandas' CI? One naming related comment: I wouldn't call it like |
|
Nice idea @MarcoGorelli. I'm not really sure how flake8 plugins work but would it still be possible to upstream the lint checks while keeping in the primary pandas-dev repo? In a vacuum keeping things centralized into more of a "monorepo" feels better to me as very often it would seem necessary to cross reference between the style checks and main code base. Also if there is a bug in the linter causing issues in pandas-dev might be more awkward to fix. |
|
Also, for testing the scripts for different Python / OS versions: I suppose those tests don't take very long? In that case it might be fine to actually include them in this repo as well. We could have a github actions workflow that eg only get triggered if something changes in the scripts directory? |
Exactly, it'd get bumped by the (existing) autoupdate job
Less than a minute, probably. For now I'll make the plugin in my own repo, and if people like it we can decide on where to keep it / what to name it. Thanks for the comments, will ping next week when I have something ready |
|
I really like the pandas tools and it would be great if they had some ability to be easily used with other projects. |
We might want to look into making the |
This'd be an argument for keeping it separate - if there's any check in the that's not relevant outside of pandas development, then a great thing about flake8 is that it's easy to configure, so people can disable any rules they don't want I've opened #40906 to show how this would work, and the repo can be viewed here: https://github.com/MarcoGorelli/pandas-dev-flaker Thanks for the comments so far! UPDATE |
Currently, there are a lot of style checks which are specific to this repo, the result of which is a ton of CI checks and occasionally quite a bit of extra code.
The main repo is already huge, so I'd like to suggest making a
pandas-dev/pandas-style-guiderepo, which would be aflake8plugin, with all the custom linting rules we currently have.Some advantages:
pandas-dev/pandas-style-guideoneastmodule often changes a bit between versions, so it would be good to test the ast-based checks on different Python versionsOne disadvantage I see is that there might be less visibility if this is in a separate repo.
@pandas-dev/pandas-core @pandas-dev/pandas-triage anyone have any thoughts/objections?
The text was updated successfully, but these errors were encountered: