Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Series.drop() with MultiIndex: inconsistent behaviour #8594

Closed
urraca opened this issue Oct 21, 2014 · 2 comments · Fixed by #30377
Closed

Series.drop() with MultiIndex: inconsistent behaviour #8594

urraca opened this issue Oct 21, 2014 · 2 comments · Fixed by #30377
Labels
API Design Bug MultiIndex Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Milestone

Comments

@urraca
Copy link

urraca commented Oct 21, 2014

The behaviour below occurs in versions '0.15.0rc1-21-g32c5016' and '0.14.1'.

When the label passed to the drop method of a Series is not in the axis:
(a) if the index is not a MultiIndex then an exception is raised (in most cases; see issue #8530)
(b) if the index is a MultiIndex then an exception is not raised; the original series is returned

Examples of current behaviour:
(a)

>>> pd.Series([10, 20, 30], [1, 2, 3]).drop(7)

ValueError: labels [7] not contained in axis

(b)

>>> pd.Series(
    [10, 20, 30],
    pd.MultiIndex(
        levels=[[1, 2, 3], [4, 5, 6]],
        labels=[[0, 1, 2], [0, 1, 2]],
        names=['a', 'b'])
).drop(7, level='a')

a  b
1  4    10
2  5    20
3  6    30
dtype: int64

I propose that in (b) an exception be raised. Furthermore, an exception should be raised in (b) if (for instance) the label passed to drop had been 4, because although 4 is in the axis, it is not in the specified level.

The exception message in (b) would have to refer to the level, e.g.:

ValueError: labels [7] not contained in level
@jreback
Copy link
Contributor

jreback commented Oct 21, 2014

xref to #6599

@jreback jreback added API Design MultiIndex Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Oct 21, 2014
@jreback jreback added this to the 0.16.0 milestone Oct 21, 2014
@jreback jreback modified the milestones: 0.16.0, Next Major Release Mar 6, 2015
@wesm wesm added the Bug label Jul 6, 2018
@wesm
Copy link
Member

wesm commented Jul 6, 2018

This bug is still present in 0.23.2

@jreback jreback modified the milestones: Contributions Welcome, 1.0 Dec 26, 2019
andrewsanchez added a commit to andrewsanchez/q2-diversity that referenced this issue Feb 12, 2020
This prevents attempting to drop columns that don't exist in merged.columns
after setting the index, while still dropping columns that are present in
merged.columns.  Attempting to do see raises an exception in pandas >= 1.

Please see pandas-dev/pandas#8594
for details.
andrewsanchez added a commit to andrewsanchez/q2-diversity that referenced this issue Feb 12, 2020
This prevents attempting to drop columns that don't exist in merged.columns
after setting the index, while still dropping columns that are present in
merged.columns.  Attempting to do see raises an exception in pandas >= 1.

Please see pandas-dev/pandas#8594
for details.
andrewsanchez added a commit to andrewsanchez/q2-diversity that referenced this issue Feb 13, 2020
This prevents attempting to drop columns that don't exist in merged.columns
after setting the index, while still dropping columns that are present in
merged.columns.  Attempting to do so raises an exception in pandas >= 1.

Please see pandas-dev/pandas#8594
for details.
nbokulich pushed a commit to qiime2/q2-diversity that referenced this issue Feb 14, 2020
* Use MultiIndex.codes instead of labels (no longer support)

Please see these links for the rational if interested:

pandas-dev/pandas#13443

pandas-dev/pandas#23752

* Unpack and name values returned by _reindex_with_metadata

This makes it easier to see what's going on below with the values
returned by this function.

* Initial patch to handle new pandas error

This prevents attempting to drop columns that don't exist in merged.columns
after setting the index, while still dropping columns that are present in
merged.columns.  Attempting to do so raises an exception in pandas >= 1.

Please see pandas-dev/pandas#8594
for details.

* Avoid mutating `merged` in place by assigning to new variable

This avoids attempting to drop columns that had already been dropped in previous
calls to _reindex_with_metadata in the for loop in `alpha_rarefaction`.

Co-authored-by: Matthew Dillon <matthewrdillon@gmail.com>

Co-authored-by: Matthew Dillon <matthewrdillon@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Design Bug MultiIndex Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants