Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PERF: EWMGroupby #39664

Merged
merged 7 commits into from
Feb 8, 2021
Merged

PERF: EWMGroupby #39664

merged 7 commits into from
Feb 8, 2021

Conversation

mroeschke
Copy link
Member

  • tests added / passed
  • Ensure all linting tests pass, see here for how to run them
  • whatsnew entry
(pandas-dev) asv_bench % asv continuous -f 1.1 upstream/master HEAD -b rolling.GroupbyEWM
       before           after         ratio
     [508b9f98]       [c4340436]
     <master>         <ref/ewm_groupby>
-        22.1±4ms       2.57±0.3ms     0.12  rolling.GroupbyEWM.time_groupby_mean('cython')

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.

@mroeschke mroeschke added this to the 1.3 milestone Feb 8, 2021
@mroeschke mroeschke added Performance Memory or execution speed performance Window rolling, ewma, expanding labels Feb 8, 2021
@jreback
Copy link
Contributor

jreback commented Feb 8, 2021

love it. we have sufficient asv's for these cases? (e.g. ewmcov, not that its used much). ?

@mroeschke
Copy link
Member Author

Here are the new benchmarks I added for other methods

       before           after         ratio
     [e889b941]       [1f1d2ef9]
     <ref/ewm_groupby~1^2>       <ref/ewm_groupby>
-      22.6±0.7ms       2.29±0.2ms     0.10  rolling.GroupbyEWM.time_groupby_method('var')
-        22.2±2ms       2.21±0.3ms     0.10  rolling.GroupbyEWMEngine.time_groupby_mean('cython')
-      30.9±0.1ms       2.42±0.3ms     0.08  rolling.GroupbyEWM.time_groupby_method('std')
-       234±0.9ms       4.42±0.2ms     0.02  rolling.GroupbyEWM.time_groupby_method('cov')
-        251±20ms       4.54±0.7ms     0.02  rolling.GroupbyEWM.time_groupby_method('corr')

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.

Looks like the CI / Checks is failing for an unrelated reason

Run ci/code_checks.sh docstrings
Validate docstrings (GL03, GL04, GL05, GL06, GL07, GL09, GL10, SS01, SS02, SS04, SS05, PR03, PR04, PR05, PR10, EX04, RT01, RT04, RT05, SA02, SA03)
Traceback (most recent call last):
  File "ci/../scripts/validate_docstrings.py", line 50, in <module>
    from numpydoc.validate import validate, Docstring  # isort:skip
ImportError: cannot import name 'Docstring' from 'numpydoc.validate' (/usr/share/miniconda/envs/pandas-dev/lib/python3.8/site-packages/numpydoc/validate.py)

@jreback jreback merged commit 306ccdb into pandas-dev:master Feb 8, 2021
@jreback
Copy link
Contributor

jreback commented Feb 8, 2021

very nice @mroeschke

yeah the doc-strings are failing unrelated

@mroeschke mroeschke deleted the ref/ewm_groupby branch February 8, 2021 21:02
@mroeschke mroeschke changed the title PERF: ExpandingGroupby PERF: EWMGroupby Feb 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Performance Memory or execution speed performance Window rolling, ewma, expanding
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants