Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when max with a datetime.date Series #24109

Open
YanxuanLI opened this issue Dec 5, 2018 · 2 comments
Open

Error when max with a datetime.date Series #24109

YanxuanLI opened this issue Dec 5, 2018 · 2 comments
Labels
datetime.date stdlib datetime.date support Enhancement Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Reduction Operations sum, mean, min, max, etc.

Comments

@YanxuanLI
Copy link

YanxuanLI commented Dec 5, 2018

import datetime as dt
s = pd.Series([dt.date(2018,1,1), None, dt.date(2018,1,1)])
print(s.max()) # TypeError: '>=' not supported between instances of 'datetime.date' and 'float'

s1 = pd.to_datetime(s).dt.date
print(s1.max()) # TypeError: '>=' not supported between instances of 'datetime.date' and 'float'
print(max(s1)) # OK

s2 = pd.Series([dt.datetime(2018,1,1), None, dt.datetime(2018,1,1)])
print(s2.max()) # OK

Problem description

It's a little tricky for this bug.
I received an error when I tried to maximize a series type of date with None value, but it failed throwing an error.
Then using to_datetime().dt.date, it converts to date. Still not work for s1.max(), but this time, works fine for max(s1), Too noisy.
While for the datetime, it works fine for everything.

Expected Output

Output of pd.show_versions()

[INSTALLED VERSIONS

commit: None
python: 3.6.5.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 142 Stepping 9, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.23.4
pytest: None
pip: 18.1
setuptools: 39.0.1
Cython: None
numpy: 1.15.1
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.2.3
openpyxl: 2.5.9
xlrd: 1.1.0
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
None]

@WillAyd WillAyd added Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Datetime Datetime data dtype labels Dec 5, 2018
@mroeschke
Copy link
Member

datetime.date is not a first class data type in Pandas; many operations will probably not be supported when manipulating datetime.dates. It's preferable to convert them to datetime.datetime (centered at midnight) and export as datetime.dates

@mroeschke mroeschke changed the title Error when max with a date Series Error when max with a datetime.date Series Apr 1, 2020
@jbrockmendel jbrockmendel added the Reduction Operations sum, mean, min, max, etc. label Sep 21, 2020
@jbrockmendel
Copy link
Member

So in nanops.nanmax we default to masking NA values with -inf (actually in _get_values). For the first case with a None that might be reasonable, but in the second case with a NaT it is for sure fishy. Sufficient to get the desired behavior (not sure if we should do this yet) is to add inside _get_values:

    # get our fill value (in case we need to provide an alternative
    # dtype for it)
    fill_value = _get_fill_value(
        dtype, fill_value=fill_value, fill_value_typ=fill_value_typ
    )

+    if skipna and dtype == object and lib.infer_dtype(values) == "date":
+        # GH#24109
+        fill_value = NaT

@mroeschke mroeschke added datetime.date stdlib datetime.date support and removed Datetime Datetime data dtype labels Jun 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datetime.date stdlib datetime.date support Enhancement Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Reduction Operations sum, mean, min, max, etc.
Projects
None yet
Development

No branches or pull requests

4 participants