Skip to content

Subclassed reshape clean #15655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 25 commits into from
Closed

Subclassed reshape clean #15655

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
642bfce
add subclassed stack/unstack/pivot tests
delgadom Mar 8, 2017
0295604
add melt test
delgadom Mar 8, 2017
60a2cfd
use _constructor* properties to create Series and DataFrame objects t…
delgadom Mar 8, 2017
d65cff5
document _Unstacker
delgadom Mar 8, 2017
9d1cf63
fix bug in wide_to_long_test, add GH issue numbers
delgadom Mar 11, 2017
3efb82f
add whatsnew entry
delgadom Mar 11, 2017
200c752
flake8 cleanup
delgadom Mar 11, 2017
5e480c6
fix bug in existing docs ``internals.rst:220`` ``{A, [`` --> ``{A: [``
delgadom Mar 11, 2017
4f3319c
clarify language and add subclassed reshape and math examples to doc/…
delgadom Mar 11, 2017
8af21c1
additional clarification in doc/source/internals.rst
delgadom Mar 11, 2017
027f36a
remove references to Panel from doc/source/internals.rst subclassing …
delgadom Mar 11, 2017
8a61374
change code-block to ipython directives in doc/source/internals.rst
delgadom Mar 12, 2017
6715a25
change from python to ipython code blocks in docs
delgadom May 13, 2017
f751a85
reformat docstrings
delgadom May 13, 2017
ca85796
add subclassed stack/unstack/pivot tests
delgadom Mar 8, 2017
b0bc8f4
add melt test
delgadom Mar 8, 2017
246a464
use _constructor* properties to create Series and DataFrame objects t…
delgadom Mar 8, 2017
1c672a9
document _Unstacker
delgadom Mar 8, 2017
eff151e
fix bug in wide_to_long_test, add GH issue numbers
delgadom Mar 11, 2017
16dae8e
flake8 cleanup
delgadom Mar 11, 2017
66b2e42
fix bug in existing docs ``internals.rst:220`` ``{A, [`` --> ``{A: [``
delgadom Mar 11, 2017
7641812
clarify language and add subclassed reshape and math examples to doc/…
delgadom Mar 11, 2017
be66ce0
additional clarification in doc/source/internals.rst
delgadom Mar 11, 2017
d27034d
remove references to Panel from doc/source/internals.rst subclassing …
delgadom Mar 11, 2017
dc3b07e
merge conflicts
delgadom May 13, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
174 changes: 100 additions & 74 deletions doc/source/internals.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
np.random.seed(123456)
np.set_printoptions(precision=4, suppress=True)
import pandas as pd
from pandas import Series, DataFrame
pd.options.display.max_rows = 15

*********
Expand Down Expand Up @@ -130,82 +131,94 @@ Property Attributes ``Series`` ``DataFrame`` ``Panel

The below example shows how to define ``SubclassedSeries`` and ``SubclassedDataFrame`` classes, overriding the default constructor properties.

.. code-block:: python

class SubclassedSeries(Series):

@property
def _constructor(self):
return SubclassedSeries

@property
def _constructor_expanddim(self):
return SubclassedDataFrame

class SubclassedDataFrame(DataFrame):

@property
def _constructor(self):
return SubclassedDataFrame

@property
def _constructor_sliced(self):
return SubclassedSeries
.. ipython:: python

In [1]: class SubclassedSeries(Series):
...:
...: @property
...: def _constructor(self):
...: return SubclassedSeries
...:
...: @property
...: def _constructor_expanddim(self):
...: return SubclassedDataFrame
...:

In [1]: class SubclassedDataFrame(DataFrame):
...:
...: @property
...: def _constructor(self):
...: return SubclassedDataFrame
...:
...: @property
...: def _constructor_sliced(self):
...: return SubclassedSeries
...:

Overriding constructor properties allows subclass families to be preserved across slice and reshape operations:

.. code-block:: python
.. ipython:: python

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually I am puzzled why these are code-blocks at all, they should all simply be ipython-blocks. much more simple and will show an error if something is wrong. can you change.

>>> ser = SubclassedSeries([1, 2, 3])
>>> ser
In [1]: ser = SubclassedSeries([1, 2, 3])
In [1]: ser
Out[1]:
0 1
1 2
2 3
dtype: int64
>>> type(ser)
In [1]: type(ser)
Out[1]:
<class '__main__.SubclassedSeries'>

>>> to_framed = s.to_frame()
>>> type(to_framed)
In [1]: to_framed = s.to_frame()
In [1]: type(to_framed)
Out[1]:
<class '__main__.SubclassedDataFrame'>

>>> df = SubclassedDataFrame({
... 'A': ['a', 'a', 'b', 'b'],
... 'B': ['x', 'y', 'x', 'y'],
... 'C': [1, 2, 3, 4]})
>>> df
In [1]: df = SubclassedDataFrame({
...: 'A': ['a', 'a', 'b', 'b'],
...: 'B': ['x', 'y', 'x', 'y'],
...: 'C': [1, 2, 3, 4]})
In [1]: df
Out[1]:
A B C
0 a x 0
1 a y 1
2 b x 2
3 b y 3

>>> type(df)
In [1]: type(df)
Out[1]:
<class '__main__.SubclassedDataFrame'>

>>> sliced1 = df[['A', 'B']]
>>> sliced1
In [1]: sliced1 = df[['A', 'B']]
In [1]: sliced1
Out[1]:
A B
0 a x
1 a y
2 b x
3 b y
>>> type(sliced1)
In [1]: type(sliced1)
Out[1]:
<class '__main__.SubclassedDataFrame'>

>>> sliced2 = df['C']
>>> sliced2
In [1]: sliced2 = df['C']
In [1]: sliced2
Out[1]:
0 0
1 1
2 2
3 3
Name: A, dtype: int64
>>> type(sliced2)

In [1]: type(sliced2)
Out[1]:
<class '__main__.SubclassedSeries'>

>>> stacked = df.stack()
>>> stacked
In [1]: stacked = df.stack()
In [1]: stacked
Out[1]:
0 A a
B x
C 1
Expand All @@ -219,40 +232,47 @@ Overriding constructor properties allows subclass families to be preserved acros
B y
C 4
dtype: object
>>> type(stacked)
In [1]: type(stacked)
Out[1]:
<class '__main__.SubclassedSeries'>

>>> pivoted = df.pivot(index='A', columns='B', values='C')
>>> pivoted
In [1]: pivoted = pd.pivot(index='A', columns='B', values='C')
In [1]: pivoted
Out[1]:
B x y
A
a 1 2
b 3 4
>>> type(pivoted)
In [1]: type(pivoted)
Out[1]:
<class '__main__.SubclassedDataFrame'>

Most data operations also preserve the class:

.. code-block:: python
.. ipython:: python

>>> squared = pivoted**2
>>> squared
In [1]: squared = pivoted**2
In [1]: squared
Out[1]:
B x y
A
a 1 4
b 9 16
>>> type(pivoted)
In [1]: type(pivoted)
Out[1]:
<class '__main__.SubclassedDataFrame'>

>>> interped = ser.loc[[0, 0.5, 1, 1.5, 2]].interpolate()
>>> interped
In [1]: interped = ser.loc[[0, 0.5, 1, 1.5, 2]].interpolate()
In [1]: interped
Out[1]:
0.0 1.0
0.5 1.5
1.0 2.0
1.5 2.5
2.0 3.0
dtype: float64
>>> type(interped)
In [1]: type(interped)
Out[1]:
<class '__main__.SubclassedSeries'>


Expand All @@ -266,42 +286,48 @@ To let original data structures have additional properties, you should let ``pan

Below is an example to define 2 original properties, "internal_cache" as a temporary property and "added_property" as a normal property

.. code-block:: python

class SubclassedDataFrame2(DataFrame):

# temporary properties
_internal_names = pd.DataFrame._internal_names + ['internal_cache']
_internal_names_set = set(_internal_names)

# normal properties
_metadata = ['added_property']
.. ipython:: python

@property
def _constructor(self):
return SubclassedDataFrame2
In [1]: class SubclassedDataFrame2(DataFrame):
...:
...: # temporary properties
...: _internal_names = DataFrame._internal_names + ['internal_cache']
...: _internal_names_set = set(_internal_names)
...:
...: # normal properties
...: _metadata = ['added_property']
...:
...: @property
...: def _constructor(self):
...: return SubclassedDataFrame2

.. code-block:: python
.. ipython:: python

>>> df = SubclassedDataFrame2({'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]})
>>> df
In [1]: df = SubclassedDataFrame2({'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]})
In [1]: df
Out[1]:
A B C
0 1 4 7
1 2 5 8
2 3 6 9

>>> df.internal_cache = 'cached'
>>> df.added_property = 'property'
In [1]: df.internal_cache = 'cached'
In [1]: df.added_property = 'property'
Out[1]:

>>> df.internal_cache
In [1]: df.internal_cache
Out[1]:
cached
>>> df.added_property
In [1]: df.added_property
Out[1]:
property

# properties defined in _internal_names is reset after manipulation
>>> df[['A', 'B']].internal_cache
In [1]: df[['A', 'B']].internal_cache
Out[1]:
AttributeError: 'SubclassedDataFrame2' object has no attribute 'internal_cache'

# properties defined in _metadata are retained
>>> df[['A', 'B']].added_property
In [1]: df[['A', 'B']].added_property
Out[1]:
property
Loading