Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

REGR: assignment of pd.NA with enlargement gives object dtype with IntegerArray #47284

Closed
2 of 3 tasks
simonjayhawkins opened this issue Jun 8, 2022 · 4 comments · Fixed by #47372
Closed
2 of 3 tasks

REGR: assignment of pd.NA with enlargement gives object dtype with IntegerArray #47284

simonjayhawkins opened this issue Jun 8, 2022 · 4 comments · Fixed by #47372
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate NA - MaskedArrays Related to pd.NA and nullable extension arrays Regression Functionality that used to work in a prior pandas version
Milestone
1.4.3

Comments

@simonjayhawkins
Copy link
Member

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

# gets upcast to object
df = pd.DataFrame({"a": [1, 2, 3]}, dtype="Int64")
df.loc[4] = pd.NA

Issue Description

xref #32346 (comment), #47214

Expected Behavior

In [1]: df = pd.DataFrame({"a": [1, 2, 3]}, dtype="Int64")
...: df.loc[4] = pd.NA

In [2]: df
Out[2]:
a
0 1
1 2
2 3
4

In [3]: df.dtypes
Out[3]:
a Int64
dtype: object

In [4]: pd.version
Out[4]: '1.3.5'

Installed Versions

.
@simonjayhawkins simonjayhawkins added Bug Needs Triage Issue that has not been reviewed by a pandas team member Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Regression Functionality that used to work in a prior pandas version NA - MaskedArrays Related to pd.NA and nullable extension arrays and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Jun 8, 2022
@simonjayhawkins simonjayhawkins added this to the 1.4.3 milestone Jun 8, 2022
@simonjayhawkins simonjayhawkins added the Indexing Related to indexing on series/frames, not to indexes themselves label Jun 8, 2022
simonjayhawkins added a commit to simonjayhawkins/pandas that referenced this issue Jun 8, 2022
@simonjayhawkins
code sample for pandas-dev#47284
9c6d7d7
@simonjayhawkins
Copy link
Member Author

first bad commit: [084c543] BUG/API: concat with empty DataFrames or all-NA columns (#43507)

cc @jbrockmendel

@simonjayhawkins simonjayhawkins mentioned this issue Jun 8, 2022
Open
@jbrockmendel
Copy link
Member

Looks like we get to _setitem_with_indexer_missing with value=pd.NA and end up doing value = Series(value, index=self.obj.columns, name=indexer), which comes back as object-dtype. the concat that follows preserves object dtype.

We may need to special-case pd.NA here, or otherwise avoid getting object-dtype back.

@jorisvandenbossche jorisvandenbossche mentioned this issue Jun 17, 2022
Merged
@jbrockmendel
Copy link
Member

Does #47372 include a test for this?

@simonjayhawkins
Copy link
Member Author

test_setitem_with_expansion_row

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate NA - MaskedArrays Related to pd.NA and nullable extension arrays Regression Functionality that used to work in a prior pandas version
Projects
None yet
Milestone
1.4.3
Development

Successfully merging a pull request may close this issue.

REGR: revert behaviour change for concat with empty/all-NaN data jorisvandenbossche/pandas
2 participants
@jbrockmendel @simonjayhawkins