Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Fix concat DataFrame and Series with ignore_index=True #60983

Merged
merged 20 commits into from
Mar 7, 2025
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
3d1cb46
modified the files according to bug#60237
Anurag-Varma Feb 18, 2025
a153646
Merge branch 'bug#60237' of https://github.com/Anurag-Varma/pandas in…
Anurag-Varma Feb 18, 2025
3d40bca
Update doc/source/whatsnew/v3.0.0.rst
Anurag-Varma Feb 18, 2025
43e97c5
moved test case to frame and serier folders
Anurag-Varma Feb 18, 2025
0b47d24
fix pyarrow import error
Anurag-Varma Feb 18, 2025
f4922c2
inconsistent issue fix
Anurag-Varma Feb 22, 2025
252e5b2
added test cases and fixed old pr test cases
Anurag-Varma Feb 22, 2025
f98a814
added rst and small changes in tests file
Anurag-Varma Feb 22, 2025
c73a931
Merge branch 'main' of https://github.com/Anurag-Varma/pandas into bu…
Anurag-Varma Feb 22, 2025
d9de374
fixed column name issue for column wise concat
Anurag-Varma Feb 23, 2025
e19e820
fixed text case for concat
Anurag-Varma Feb 23, 2025
42c51ec
fix test cases issue
Anurag-Varma Feb 23, 2025
a6ef45a
Trigger redeployment
Anurag-Varma Feb 23, 2025
1750d6a
Merge branch 'main' into bug#60723
Anurag-Varma Feb 24, 2025
952e292
Merge branch 'main' into bug#60723
Anurag-Varma Feb 25, 2025
dba9778
fixed reviewed changes and added extra test cases
Anurag-Varma Mar 4, 2025
5787d95
Merge branch 'main' of https://github.com/pandas-dev/pandas into bug#…
Anurag-Varma Mar 4, 2025
48d2e06
Merge branch 'bug#60723' of https://github.com/Anurag-Varma/pandas in…
Anurag-Varma Mar 4, 2025
d53dc0a
removed duplicate test case
Anurag-Varma Mar 4, 2025
bf40bb5
Merge branch 'main' into bug#60723
Anurag-Varma Mar 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v3.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -779,6 +779,7 @@ Reshaping
- Bug in :meth:`DataFrame.pivot_table` incorrectly subaggregating results when called without an ``index`` argument (:issue:`58722`)
- Bug in :meth:`DataFrame.stack` with the new implementation where ``ValueError`` is raised when ``level=[]`` (:issue:`60740`)
- Bug in :meth:`DataFrame.unstack` producing incorrect results when manipulating empty :class:`DataFrame` with an :class:`ExtentionDtype` (:issue:`59123`)
- Bug in :meth:`concat` where concatenating dataframe and series with ignore_index = True drops the series name (:issue:`60723`, :issue:`56257`)

Sparse
^^^^^^
Expand Down
7 changes: 5 additions & 2 deletions pandas/core/reshape/concat.py
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,7 @@ def _sanitize_mixed_ndim(

else:
name = getattr(obj, "name", None)
name_none_flag = False
if ignore_index or name is None:
if axis == 1:
# doing a row-wise concatenation so need everything
Expand All @@ -485,10 +486,12 @@ def _sanitize_mixed_ndim(
else:
# doing a column-wise concatenation so need series
# to have unique names
name = current_column
if name is None:
name_none_flag = True
name = current_column
current_column += 1
obj = sample._constructor(obj, copy=False)
if isinstance(obj, ABCDataFrame):
if isinstance(obj, ABCDataFrame) and name_none_flag:
obj.columns = range(name, name + 1, 1)
else:
obj = sample._constructor({name: obj}, copy=False)
Expand Down
21 changes: 19 additions & 2 deletions pandas/tests/reshape/concat/test_concat.py
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,8 @@ def test_concat_mixed_objs_index(self):
def test_concat_mixed_objs_index_names(self):
# Test row-wise concat for mixed series/frames with distinct names
# GH2385, GH15047
# GH #60723 & GH #56257 (Updated the test case,
# as the above GH PR ones were incorrect)

index = date_range("01-Jan-2013", periods=10, freq="h")
arr = np.arange(10, dtype="int64")
Expand All @@ -341,8 +343,11 @@ def test_concat_mixed_objs_index_names(self):
result = concat([s1, df, s2])
tm.assert_frame_equal(result, expected)

# Rename all series to 0 when ignore_index=True
expected = DataFrame(np.tile(arr, 3).reshape(-1, 1), columns=[0])
expected = DataFrame(
np.kron(np.where(np.identity(3) == 1, 1, np.nan), arr).T,
index=np.arange(30, dtype=np.int64),
columns=["foo", 0, "bar"],
)
result = concat([s1, df, s2], ignore_index=True)
tm.assert_frame_equal(result, expected)

Expand Down Expand Up @@ -943,3 +948,15 @@ def test_concat_with_moot_ignore_index_and_keys():
msg = f"Cannot set {ignore_index=} and specify keys. Either should be used."
with pytest.raises(ValueError, match=msg):
concat([df1, df2], keys=keys, ignore_index=ignore_index)


def test_concat_of_series_and_frame_with_names_for_ignore_index():
# GH #60723 and #56257
ser = Series([4, 5], name="c")
df = DataFrame({"a": [0, 1], "b": [2, 3]})

result = concat([df, ser], ignore_index=True)
expected = DataFrame(
{"a": [0, 1, None, None], "b": [2, 3, None, None], "c": [None, None, 4, 5]}
)
tm.assert_frame_equal(result, expected)
Loading