Skip to content

Behavior of generator expressions doesn't match the docs when passed a non-iterable #143493

@markshannon

Description

@markshannon

Bug report

Bug description:

>>> def f(s):
...     return (x for x in s)
...     
>>> f(1)
<generator object f.<locals>.<genexpr> at 0xe8529f655c70>

According to the docs (and in earlier versions) the correct behavior is:

>>> def f(s):
...     return (x for x in s)
...     
>>> f(1)
TypeError: 'int' object is not iterable

The current behavior on main is arguably more logical as it is consistent with generators, but Hyrum's law dictates we need to be backwards compatible.

https://docs.python.org/3/reference/expressions.html#generator-expressions

This issue has a bit of history.
To fix #125038 we moved the GET_ITER instruction into the generator body. This broke stuff, so we added a supposedly redundant GET_ITER to the callsite, but that also caused problems #127682.
Eventually, for 3.14 we added an additional check in FOR_ITER that the iterator is indeed an iterator.
This adds a little bit of unnecessary overhead and is an obstacle to optimization.

On main, we use "virtual iterators" and have kept the GET_ITER in the body of the generator. This is efficient and optimizes nicely, but doesn't conform to the docs.

We can keep both the nice performance and conform to the docs, by keeping the GET_ITER in the generator function, but moving it into generator creation and out of the generator body.
Instead of:

RETURN_GENERATOR
...
LOAD__FAST 0
GET_ITER

we have:

LOAD_FAST 0
GET_ITER
RETURN_GENERATOR
...

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Metadata

Metadata

Assignees

Labels

3.15new features, bugs and security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions