STYLE use pandas-dev-flaker

[MarcoGorelli]

• closes STYLE pandas-dev flake8 plugin #40826
• closes STYLE "wrongplaced whitespace" rule doesn't error on its example of a bad string #40873
• tests added / passed
• Ensure all linting tests pass, see here for how to run them
• whatsnew entry

---

This means that, just by running flake8, you'll be able to get all these custom linting errors alongside the regular flake8 ones. E.g. if we make pandas/tests/t.py as follows:

import pytest

import pandas as pd
from pandas import Categorical


def test_foo():
    cat_0 = pd.Categorical([1])
    with pytest.warns(FutureWarning):
        cat_1 = pd.Categorical(1)

then we get:

$ flake8 pandas/tests/t.py 
pandas/tests/t.py:4:1: F401 'pandas.Categorical' imported but unused
pandas/tests/t.py:8:5: F841 local variable 'cat_0' is assigned to but never used
pandas/tests/t.py:8:13: PDF019 found both 'pd.Categorical' and 'Categorical' in the same file
pandas/tests/t.py:9:10: PDF011 found 'pytest.warns' (use 'pandas._testing.assert_produces_warning')
pandas/tests/t.py:10:9: F841 local variable 'cat_1' is assigned to but never used
pandas/tests/t.py:10:17: PDF019 found both 'pd.Categorical' and 'Categorical' in the same file

These can then easily be configured using flake8's configuration options, like # noqa and per-file-ignores in setup.cfg.

This also brings down the number of pre-commit checks.

---

I this is something the team wants, and it's decided to keep it as a separate repo, then some outstanding things to do would be:

• transferring it to pandas-dev
• sorting out the license (it's currently in my name, and I presume that should change)

[jbrockmendel]

i dont know the code for these checks very well; is there any need for concern about the Marco bus factor?

[MarcoGorelli]

i dont know the code for these checks very well; is there any need for concern about the Marco bus factor?

That's definitely a valid concern. What I'd like to first understand is whether you'd find this change useful - if so, then I can do my best to address concerns.

Just to clarify the bus factor question - is the concern more the repo being in my name, or about how most maintainers might not have ast-parsing at the top of their interest list and so this'd be hard to maintain going forward?
If the former, then we could either include it directly inside the pandas repo (as preferred by @dsaxton ) or I could share all access rights with the team. If the latter, then perhaps I could write a guide on how it works, how to debug it, and how to add new checks?

[jreback]

generally looks good. I think including it as a pandas-dev repo is totally fine

[MarcoGorelli]

Thanks @jreback - I've transferred it to pandas-dev/pandas-dev-flaker

For authors / license, am I OK to put "PyData Development Team" (as in the pandas license)?

[jreback]

Thanks @jreback - I've transferred it to pandas-dev/pandas-dev-flaker

For authors / license, am I OK to put "PyData Development Team" (as in the pandas license)?

sure that sounds good. I would add https://github.com/pandas-dev/pandas/tree/master/LICENSES as well

[MarcoGorelli]

I've fixed up the author, and the pandas license is there (https://github.com/pandas-dev/pandas-dev-flaker/blob/main/LICENSES/PANDAS_LICENSE)

Any further thoughts? If this is still seen as potentially controversial, maybe it can be discussed at the dev meeting tomorrow?

[jbrockmendel]

Just to clarify the bus factor question - is the concern more the repo being in my name, or about how most maintainers might not have ast-parsing at the top of their interest list and so this'd be hard to maintain going forward?

The second one. I'm not losing sleep over it.

[jreback]

lgtm.
cc @WillAyd @datapythonista if any concerns.

[WillAyd]

Love this - great work @MarcoGorelli

[datapythonista]

Really nice clean up, thanks @MarcoGorelli

Just a minor personal opinion on the pandas-dev-flaker, this PR looks good.

Instead of using from pandas_dev_flaker.__main__ import run, I'd personally define run in __init__.py, and import it from __main__.py, and from the tests or anywhere else needed with from pandas_dev_flaker import run.

Personally I think __main__.py should be exclusively for when calling python -m module, and nothing should be imported from it. I think that's the original purpose, and keep things better organized.

In any case, just a small suggestion, changes look great.

[MarcoGorelli]

Thanks for your feedback, really appreciate it!

Thanks @datapythonista for the tip - I'll move definitions to __init__ when the next release comes around

[jreback]

very nice @MarcoGorelli