BUG: take nans correctly into consideration in complex and tuple

[realead]

• closes _libs.hashtable.ismember incorrect with tuples containing nan #41836
• tests added / passed
• Ensure all linting tests pass, see here for how to run them
• whatsnew entry

NaNs as Float are already handled correctly (all nans are the same equavalency class), this PR adds similar functionality for complex and tuple.

It doesn't handle frozenset (see also discussion in #41836) because it would require to duplicate a lot of code/internal details from setobject.

[realead]

For the upcoming issues with Py3.10 and hash-functions (see this comment #41836 (comment) and #41953) there will be a separate PR soonish.

[realead]

It looks as if this exampe fails:

def test_isin_nan_not_pypy():
        idx = MultiIndex.from_arrays([["foo", "bar"], [1.0, np.nan]])
>       tm.assert_numpy_array_equal(idx.isin([("bar", np.nan)]), np.array([False, False]))

pandas\tests\indexes\multi\test_isin.py:37: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

left = array([False,  True]), right = array([False, False]), err_msg = None

To me it seems, that the new behavior is correct. At least it would be result if np.nan would be replaced through something like 2.

[jreback]

does this change outward behavior for <= 3.9?

asking if we should put this in 1.3 (inclination is yes)

[realead]

@jreback The only behavioral change I'm aware of is fixing the bug #41836 (which also lead to this change in the test suitehttps://github.com//pull/41952/commits/0b02e27eb14f7f059740081eb71639550832d05f).

The performance didn't really suffer much under this change, there is a typical run (somehow on my machine the variance is quite high between different runs):

+     2.42±0.02ms      2.58±0.06ms     1.07  algos.isin.IsInFloat64.time_isin('Float64', 'many_different_values')
-         531±3ms          503±3ms     0.95  algos.isin.IsinWithRandomFloat.time_isin(<class 'numpy.object_'>, 900000, 'inside')
-         283±2μs          261±4μs     0.92  algos.isin.IsinWithRandomFloat.time_isin(<class 'numpy.object_'>, 1300, 'outside')
-     1.28±0.02ms      1.16±0.01ms     0.91  algos.isin.IsinWithRandomFloat.time_isin(<class 'numpy.object_'>, 7000, 'outside')
-         423±2ms          382±7ms     0.90  algos.isin.IsinWithRandomFloat.time_isin(<class 'numpy.object_'>, 900000, 'outside')
-     1.34±0.04ms      1.19±0.01ms     0.89  algos.isin.IsinWithRandomFloat.time_isin(<class 'numpy.object_'>, 7000, 'inside')
-         369±2ms          321±2ms     0.87  algos.isin.IsinWithRandomFloat.time_isin(<class 'numpy.object_'>, 750000, 'inside')
-      16.0±0.7ms       12.7±0.3ms     0.80  algos.isin.IsinWithRandomFloat.time_isin(<class 'numpy.object_'>, 80000, 'outside')
-         374±4ms          253±7ms     0.68  algos.isin.IsinWithRandomFloat.time_isin(<class 'numpy.object_'>, 750000, 'outside')
-        710±10μs          471±8μs     0.66  algos.isin.IsInForObjects.time_isin('nans', 'nans')

[jreback]

ok labeling this for 1.3

[realead]

@jreback Do you think a what's new entry is needed?

[jreback]

@jreback Do you think a what's new entry is needed?

yeah why don't you add one just in case. merge master and ping on green.

[realead]

@jreback it is green(ish) - the failures seem to be unrelated to this PR.

[simonjayhawkins]

@jreback it is green(ish) - the failures seem to be unrelated to this PR.

agreed. restarted ci to be sure

[jreback]

this ref is an internal one. does this for example affect Series.isin in a user visible way?

[jbrockmendel]

yes, the motivating case was MultiIndex.isin

[jreback]

can you add some comments before these of what is going on / why doing is & the issue reference

[jreback]

lgtm @realead just a few doc comments, ping when green

[realead]

@jreback green.

[jreback]

thanks @realead

[jreback]

@meeseeksdev backport 1.3.x

[lumberbot-app]

Something went wrong ... Please have a look at my logs.