ENH: Consistent NA handling in unique(), and nunique()

[olek-osikowicz]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

Currently Series.nunique has a default parameter dropna=True.
However Series.unique does not accept the dropna the parameter.

This can cause the unexpected behaviour when: s.nunique() is not nessesarly equal to len(s.unique()).
See example below:

>>> import pandas as pd
>>> s = pd.Series([pd.NA, 1, pd.NA])
>>> s.unique()
array([<NA>, 1], dtype=object)
>>> len(s.unique())
2
>>> s.nunique()
1

I believe it should be addressed to avoid implicit behaviour.

Feature Description

Simplest way to addess it would be to change the default parameter of Series.nunique to dropna=False.
Analogously the same default parameter for DataFrame.nunique.

This would be consistent with current summary of the method:

Count number of distinct elements in specified axis.
Return Series with number of distinct elements. Can ignore NaN values.

"Can ignore NaN values.", hints that should be optional parameter not enabled by default.

Alternative Solutions

Another approach to force consistent NaN handling by default would be to addapt Series.unique to accept dropna and set it to True by default.

Although possible, this is more laborious and more impactful change on Pandas API.

Additional Context

No response

EDIT: Typos

[HoqueUM]

take

[snitish]

I think it should be dropna=True by default, so your alternative solution, i.e. add dropna to Series.unique (with default set to True) makes more sense to me. cc: @rhshadrach

[rhshadrach]

Related: #53094

While I would prefer pandas not dropping NA values by default, that isn't the case today. However if we are going to eventually change the default of dropna to False, then I would be hesitant of changing the default behavior of unique just to then change it back.

In this particular case I think we should wait for dropna to default to False, and then decide if we really want a dropna argument in this method. The main blocker for this is work on pivot_table behaviors, which I plan to take up after 3.0 is released.