Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: scatter and line matplotlib plots not compatible for sharex=True datetime plots #61005

Closed
3 tasks done
jonbanks87 opened this issue Feb 25, 2025 · 6 comments · Fixed by #61244
Closed
3 tasks done
Assignees
Labels

Comments

@jonbanks87
Copy link

jonbanks87 commented Feb 25, 2025

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

datetime_list = [datetime.datetime(year = 2025, month = 1, day = 1, hour = n) for n in range(23)]
y = [n for n in range(23)]

df = pd.DataFrame(columns = ['datetime','y'])
for i, n in enumerate(datetime_list):
    df.loc[len(df)] = [n,y[i]]

#Plotting with pandas - first subplot shows up as blank
fig, ax = plt.subplots(2, sharex=True)
df.plot.scatter(x = 'datetime', y = 'y', ax = ax[0])
df.plot(x = 'datetime', y = 'y', ax = ax[1])

#Plotting with matplotlib - this works
fig, ax = plt.subplots(2, sharex=True)
ax[0].scatter(df['datetime'],df['y'])
ax[1].plot(df['datetime'],df['y'])

Issue Description

When I am trying to plot a line plot and scatter plot using df.plot and df.plot.scatter, the scatter plot shows up as blank when I select sharex=True when making the figure and axes.

#Plotting with pandas - first subplot shows up as blank
fig, ax = plt.subplots(2, sharex=True)
df.plot.scatter(x = 'datetime', y = 'y', ax = ax[0])
df.plot(x = 'datetime', y = 'y', ax = ax[1])

Image

However, if I plot using the standard matplotlib way of plotting (ax.scatter and ax.plot), both subplots show up correctly when I set sharex=True for the figure.

#Plotting with matplotlib - this works
fig, ax = plt.subplots(2, sharex=True)
ax[0].scatter(df['datetime'],df['y'])
ax[1].plot(df['datetime'],df['y'])

Image

Expected Behavior

I would expect for the both subplots to show up, not blank. The behavior should be more similar to the traditional way of plotting with matplotlib.

Installed Versions

INSTALLED VERSIONS

commit : 0691c5c
python : 3.13.1
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19045
machine : AMD64
processor : Intel64 Family 6 Model 158 Stepping 9, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United States.1252

pandas : 2.2.3
numpy : 2.2.2
pytz : 2025.1
dateutil : 2.9.0.post0
pip : 24.3.1
Cython : None
sphinx : None
IPython : 8.32.0
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : 4.13.3
blosc : None
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : 3.1.5
lxml.etree : None
matplotlib : 3.10.0
numba : None
numexpr : None
odfpy : None
openpyxl : 3.1.5
pandas_gbq : None
psycopg2 : None
pymysql : None
pyarrow : None
pyreadstat : None
pytest : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : 1.15.1
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
tzdata : 2025.1
qtpy : None
pyqt5 : None

@jonbanks87 jonbanks87 added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Feb 25, 2025
@snitish
Copy link
Member

snitish commented Feb 26, 2025

I was able to reproduce the bug on main. This seems to be due to the two plots (scatter and line) having different xaxis types. The scatter plot keeps the 'datetime' column as is, but the line plot converts it to a PeriodIndex due to

data = maybe_convert_index(self._get_ax(0), self.data)

I was able to fix this by setting x_compat = True in the line plot:

df.plot(x = 'datetime', y = 'y', ax=ax[1], x_compat=True)

For a more robust fix, I'm guessing we should probably support automatic tick resolution in scatter plots similar to line plots?

@rhshadrach
Copy link
Member

Agreed @snitish - a fix for scatter plot would be welcome!

@rhshadrach rhshadrach added Visualization plotting and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Mar 2, 2025
@Nivya-21
Copy link

Nivya-21 commented Mar 2, 2025

take

@MartinBraquet
Copy link
Contributor

MartinBraquet commented Apr 7, 2025

@snitish and others, why are all the functionalities related to timeseries (i.e., x axis) in LinePlot? It seems like they should apply to all types of plot (i.e., in MPLPlot).

self.x_compat = plot_params["x_compat"]
if "x_compat" in self.kwds:
self.x_compat = bool(self.kwds.pop("x_compat"))
@final
def _is_ts_plot(self) -> bool:
# this is slightly deceptive
return not self.x_compat and self.use_index and self._use_dynamic_x()
@final
def _use_dynamic_x(self) -> bool:
return use_dynamic_x(self._get_ax(0), self.data)

Am I missing something, or is a big refactor required?

@MartinBraquet
Copy link
Contributor

take

@MartinBraquet
Copy link
Contributor

The x ticks of a scatter plots must be converted from Period to float via @register_pandas_matplotlib_converters.

MartinBraquet added a commit to MartinBraquet/pandas that referenced this issue Apr 7, 2025
This resolves an issue where line and scatter plots were not aligned when using Series.plot. The fix ensures proper alignment and improves plot consistency. Refer to issue pandas-dev#61005 for further details.
mroeschke pushed a commit that referenced this issue Apr 9, 2025
* Refactor time series plotting logic for improved clarity

Extract and streamline time series preparation steps into `prepare_ts_data`, replacing redundant logic across methods. Simplifies axis frequency handling and improves code readability while maintaining functionality.

* Add test to validate xtick alignment for scatter and line plots

This test ensures that the x-axis ticks are consistent between scatter and line plots when sharing the same axis. It addresses a potential issue related to GH#61005, verifying proper rendering of datetime x-axis labels.

* Fix bug in Series.plot misalignment for line and scatter plots

This resolves an issue where line and scatter plots were not aligned when using Series.plot. The fix ensures proper alignment and improves plot consistency. Refer to issue #61005 for further details.

* Update scatter plot test to support datetime.time data

Datetime.time is now supported in scatter plots due to added converter implementation in ScatterPlot. Removed the test expecting a TypeError and updated it to validate the new functionality.

* Refactor handling of x_data in matplotlib plotting.

Simplify and streamline the code by directly assigning x_data from the data variable and replacing the intermediate Series object with a clearer `s` variable. This improves readability and maintains the existing functionality.

* Move test_scatter_line_xticks from Series to DataFrame tests

Relocated the `test_scatter_line_xticks` test from `test_series.py` to `test_frame.py` for better alignment with DataFrame-specific functionality. This refactor ensures the test resides in the appropriate context based on its usage and focus.

* Refactor `prepare_ts_data` to improve type annotations.

Added precise type annotations to the function signature for better clarity and type checking. Replaced `data` with `series` and `kwds` with `kwargs` to enhance readability and consistency.

* Refactor test_scatter_line_xticks to simplify DataFrame creation

The DataFrame creation in the test has been streamlined for clarity and conciseness by replacing the loop with a list comprehension. This improves code readability and maintains the same functionality.

* Refactor Series import to optimize scope and maintain consistency

Moved the `Series` import inside relevant function scopes to minimize unnecessary top-level imports and align with existing import patterns. This helps improve code readability and ensures imports are only loaded where needed.

* `Reorder import statement in _make_plot method`

Moved the import of `Series` within the `_make_plot` method to comply with styling or runtime considerations. This ensures consistency and avoids potential import-related issues.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants