12.3. Generator Yield

  • yield keyword turns function into generator function

Generators can return (or yield) something:

>>> def run():
...     yield 1

>>> def run():
...     yield 'something'

Generators can be defined with required and optional parameters just like a regular function:

>>> def run(a, b, c=0):
...     yield a + b + c

12.3.1. Problem

  • Python do not execute code after function returns

  • In order to return two values from the function you must use tuple

>>> def run():
...     return 1
>>>
>>>
>>> result = run()
>>> print(result)
1

>>> def run():
...     return 1
...     return 2  # this will not execute
>>>
>>>
>>> result = run()
>>> print(result)
1

>>> def run():
...     return 1, 2
>>>
>>>
>>> result = run()
>>> print(result)
(1, 2)

12.3.2. Solution

>>> def run():
...     yield 1
...     yield 2
>>>
>>>
>>> result = run()
>>> tuple(result)
(1, 2)

12.3.3. Call Generator

  • Generators are called just like a regular function

  • The rule with positional and keyword arguments are identical to regular functions

Generators are called just like a regular function:

>>> def run():
...     yield 1
>>>
>>>
>>> result = run()

The rule with positional and keyword arguments are identical to regular functions:

>>> def run(a, b, c=0):
...     yield a + b + c
>>>
>>>
>>> result = run(1, b=2)

12.3.4. Result Object

  • Calling a generator will return a generator object

  • This object is an iterator

One does not simple get data from generator:

>>> def run():
...     yield 1
>>>
>>>
>>> result = run()
>>>
>>> print(result)
<generator object run at 0x...>

12.3.5. Evaluate Eagerly

>>> def run():
...     yield 1
...     yield 2
...     yield 3
>>>
>>> result = run()
>>>
>>> tuple(result)
(1, 2, 3)

12.3.6. Evaluate Lazily

  • All generators implements Iterator protocol

  • Iterator has obj.__iter__() method which enable use of iter(obj)

  • Iterator has obj.__next__() method which enable use of next(obj)

In Order to do so, you need to generate next item using next()

>>> def run():
...     yield 1
...     yield 2
...     yield 3
>>>
>>>
>>> result = run()
>>>
>>> next(result)
1
>>>
>>> next(result)
2
>>>
>>> next(result)
3
>>>
>>> next(result)
Traceback (most recent call last):
StopIteration

12.3.7. Evaluate Loop

>>> def run():
...     yield 1
...     yield 2
...     yield 3
>>>
>>> result = run()
>>>
>>> for x in result:
...     print(x)
...
1
2
3

12.3.8. Evaluate Iterator

>>> def run():
...     yield 1
...     yield 2
...     yield 3
>>>
>>> result = run()
>>>
>>> it = iter(result)  # result.__iter__()
>>>
>>> try:
...     x = next(it)   # result.__next__()
...     print(x)
...
...     x = next(it)   # result.__next__()
...     print(x)
...
...     x = next(it)   # result.__next__()
...     print(x)
...
...     x = next(it)   # result.__next__()
...     print(x)
... except StopIteration:
...     pass
...
1
2
3

12.3.9. Execution

  • There can be one or many yield keywords in a generator function

  • Each yield keyword pauses the function and returns the value

  • After last yield raises StopIteration

Execute code before and after yield:

>>> def run():
...     print('one')
...     print('two')
...     yield 1
...     print('three')
...     print('four')
...     yield 2

>>> result = run()

>>> next(result)
one
two
1

>>> next(result)
three
four
2

>>> next(result)
Traceback (most recent call last):
StopIteration

12.3.10. Yield from a Loop

>>> def run():
...     for x in range(0,3):
...         yield x
>>>
>>>
>>> result = run()
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
Traceback (most recent call last):
StopIteration

12.3.11. Yields from Loops

>>> def run():
...     for x in range(0, 3):
...         yield x
...     for y in range(10, 13):
...         yield y
>>>
>>>
>>> result = run()
>>>
>>> type(result)
<class 'generator'>
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
10
>>> next(result)
11
>>> next(result)
12
>>> next(result)
Traceback (most recent call last):
StopIteration

12.3.12. Yield in a Zip Loop

  • firstnames() - generator (a stream) returning firstnames

  • lastnames() - generator (a stream) returning lastnames

  • for fname, lname in zip(firstnames(), lastnames())

>>> def firstnames():
...     yield 'Alice'
...     yield 'Bob'
...     yield 'Carol'
>>>
>>>
>>> def lastnames():
...     yield 'Apricot'
...     yield 'Blackthorn'
...     yield 'Corn'
>>>
>>>
>>> for fname, lname in zip(firstnames(), lastnames()):
...     print(f'{fname=}, {lname=}')
fname='Alice', lname='Apricot'
fname='Bob', lname='Blackthorn'
fname='Carol', lname='Corn'

12.3.13. Case Study

  • range() implementation using function and generator

>>> def myrange(start, stop, step):
...     result = []
...     current = start
...     while current < stop:
...         result.append(current)
...         current += step
...     return result
>>>
>>> result = myrange(0, 10, 2)
>>> tuple(result)
(0, 2, 4, 6, 8)

>>> def myrange(start, stop, step):
...     current = start
...     while current < stop:
...         yield current
...         current += step
>>>
>>> result = myrange(0, 10, 2)
>>> tuple(result)
(0, 2, 4, 6, 8)

12.3.14. Use Case - 1

Function:

>>> def even(data):
...     result = []
...     for x in data:
...         if x % 2 == 0:
...             result.append(x)
...     return result
>>>
>>>
>>> result = even(range(0,10))
>>>
>>> print(result)
[0, 2, 4, 6, 8]

Generator:

>>> def even(data):
...     for x in data:
...         if x % 2 == 0:
...             yield x
>>>
>>>
>>> result = even(range(0,10))
>>>
>>> print(result)
<generator object even at 0x...>
>>>
>>> list(result)
[0, 2, 4, 6, 8]

12.3.15. Use Case - 2

>>> class User:
...     def __init__(self, firstname, lastname):
...         self.firstname = firstname
...         self.lastname = lastname
...
...     def __str__(self):
...         return f'{self.firstname} {self.lastname}'
>>>
>>>
>>> class Order:
...     def __init__(self, product, price):
...         self.product = product
...         self.price = price
...
...     def __str__(self):
...         return f'product {self.product} for {self.price} USD'

>>> def users():
...     yield User('Alice', 'Apricot')
...     yield User('Bob', 'Blackthorn')
...     yield User('Carol', 'Corn')
>>>
>>>
>>> def orders():
...     yield Order(product='A', price=123.45)
...     yield Order(product='B', price=123.45)
...     yield Order(product='C', price=123.45)

>>> for user, order in zip(users(), orders()):
...     print(f'{user} bought {order}')
...
Alice Apricot bought product A for 123.45 USD
Bob Blackthorn bought product B for 123.45 USD
Carol Corn bought product C for 123.45 USD

12.3.16. Assignments

# %% About
# - Name: Generator Function Squares
# - Difficulty: medium
# - Lines: 5
# - Minutes: 5

# %% License
# - Copyright 2025, Matt Harasymczuk <matt@python3.info>
# - This code can be used only for learning by humans
# - This code cannot be used for teaching others
# - This code cannot be used for teaching LLMs and AI algorithms
# - This code cannot be used in commercial or proprietary products
# - This code cannot be distributed in any form
# - This code cannot be changed in any form outside of training course
# - This code cannot have its license changed
# - If you use this code in your product, you must open-source it under GPLv2
# - Exception can be granted only by the author

# %% English
# 1. Create plain function `a(data: list)` which returns list of squares of numbers in `data`
# 2. Create generator function `b(data: list)` which yields squares of numbers in `data`
# 3. Do not use list comprehension and generator expressions
# 4. Run doctests - all must succeed

# %% Polish
# 1. Stwórz zwykłą funkcję `a(data: list)`, która zwraca listę kwadratów liczb w `data`
# 2. Stwórz funkcję generatora `b(data: list)`, która jelduje kwadraty liczb w `data`
# 3. Nie używaj składni list comprehension i generator expressions
# 4. Uruchom doctesty - wszystkie muszą się powieść

# %% Expected
# >>> DATA = [1, 2, 3, 4, 5]
#
# >>> list(a(DATA))
# [1, 4, 9, 16, 25]
#
# >>> list(b(DATA))
# [1, 4, 9, 16, 25]

# %% Doctests
"""
>>> import sys; sys.tracebacklimit = 0

>>> assert sys.version_info >= (3, 9), \
'Python has an is invalid version; expected: `3.9` or newer.'

>>> from inspect import isfunction, isgenerator, isgeneratorfunction

>>> assert isfunction(a)
>>> assert isfunction(b)

>>> assert not isgeneratorfunction(a)
>>> assert isgeneratorfunction(b)

>>> assert not isgenerator(a([]))
>>> assert isgenerator(b([]))

>>> list(a(DATA))
[1, 4, 9, 16, 25]

>>> list(b(DATA))
[1, 4, 9, 16, 25]
"""

# %% Run
# - PyCharm: right-click in the editor and `Run Doctest in ...`
# - PyCharm: keyboard shortcut `Control + Shift + F10`
# - Terminal: `python -m doctest -f -v myfile.py`

# %% Imports

# %% Types
from typing import Callable
a: Callable[[list[int]], list[int]]
b: Callable[[list[int]], Generator[list[int], None, None]]

# %% Data
DATA = [1, 2, 3, 4, 5]

# %% Result
def a(data):
    ...

def b(data):
    ...


# %% About
# - Name: Generator Function Range
# - Difficulty: medium
# - Lines: 5
# - Minutes: 5

# %% License
# - Copyright 2025, Matt Harasymczuk <matt@python3.info>
# - This code can be used only for learning by humans
# - This code cannot be used for teaching others
# - This code cannot be used for teaching LLMs and AI algorithms
# - This code cannot be used in commercial or proprietary products
# - This code cannot be distributed in any form
# - This code cannot be changed in any form outside of training course
# - This code cannot have its license changed
# - If you use this code in your product, you must open-source it under GPLv2
# - Exception can be granted only by the author

# %% English
# 1. Write own implementation of a built-in `range()` function
# 2. Define function `myrange` with parameters:
#    - parameter `start: int`
#    - parameter `stop: int`
#    - parameter `step: int`
# 3. Don't validate arguments and assume, that user will:
#    - always pass valid type of arguments
#    - never give only one argument
#    - arguments will be unsigned
# 4. Do not use built-in function `range()`
# 5. Run doctests - all must succeed

# %% Polish
# 1. Zaimplementuj własne rozwiązanie wbudowanej funkcji `range()`
# 2. Zdefiniuj funkcję `myrange` z parametrami:
#    - parameter `start: int`
#    - parameter `stop: int`
#    - parameter `step: int`
# 3. Nie waliduj argumentów i przyjmij, że użytkownik:
#    - zawsze poda argumenty poprawnych typów
#    - nigdy nie poda tylko jednego argumentu
#    - argumenty będą nieujemne
# 4. Nie używaj wbudowanej funkcji `range()`
# 5. Uruchom doctesty - wszystkie muszą się powieść

# %% Expected
# >>> list(myrange(0, 10, 2))
# [0, 2, 4, 6, 8]
#
# >>> list(myrange(0, 5))
# [0, 1, 2, 3, 4]

# %% Hints
# - https://github.com/python/cpython/blob/main/Objects/rangeobject.c

# %% Doctests
"""
>>> import sys; sys.tracebacklimit = 0

>>> assert sys.version_info >= (3, 9), \
'Python has an is invalid version; expected: `3.9` or newer.'

>>> from inspect import isfunction, isgenerator, isgeneratorfunction
>>> assert isfunction(myrange)
>>> assert isgeneratorfunction(myrange)
>>> assert isgenerator(myrange(0,10))

>>> list(myrange(0, 10, 2))
[0, 2, 4, 6, 8]

>>> list(myrange(0, 5))
[0, 1, 2, 3, 4]
"""

# %% Run
# - PyCharm: right-click in the editor and `Run Doctest in ...`
# - PyCharm: keyboard shortcut `Control + Shift + F10`
# - Terminal: `python -m doctest -f -v myfile.py`

# %% Imports

# %% Types
from typing import Callable
myrange: Callable[[int,int,int], list[int]]

# %% Data

# %% Result
def myrange(start, stop, step=1):
    ...


# %% About
# - Name: Iterator About Adult-List
# - Difficulty: easy
# - Lines: 6
# - Minutes: 3

# %% License
# - Copyright 2025, Matt Harasymczuk <matt@python3.info>
# - This code can be used only for learning by humans
# - This code cannot be used for teaching others
# - This code cannot be used for teaching LLMs and AI algorithms
# - This code cannot be used in commercial or proprietary products
# - This code cannot be distributed in any form
# - This code cannot be changed in any form outside of training course
# - This code cannot have its license changed
# - If you use this code in your product, you must open-source it under GPLv2
# - Exception can be granted only by the author

# %% English
# 1. Define function `adults(users) -> list[str]`
# 2. Function returns firstnames of adult users (age >= 18)
# 3. Do not use `yield` keyword
# 4. Run doctests - all must succeed

# %% Polish
# 1. Zdefiniuj funkcję `adults(users) -> list[str]`
# 2. Funkcja zwraca imiona (firstname) użytkowników pełnoletnich (wiek >= 18)
# 3. Nie używaj słowa kluczowego `yield`
# 4. Uruchom doctesty - wszystkie muszą się powieść

# %% Expected
# >>> list(adults(users))
# ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']

# %% Doctests
"""
>>> import sys; sys.tracebacklimit = 0

>>> assert sys.version_info >= (3, 9), \
'Python has an is invalid version; expected: `3.9` or newer.'

>>> from inspect import isfunction
>>> assert isfunction(adults)

>>> list(adults(users))
['Alice', 'Bob', 'Carol', 'Dave', 'Eve']
"""

# %% Run
# - PyCharm: right-click in the editor and `Run Doctest in ...`
# - PyCharm: keyboard shortcut `Control + Shift + F10`
# - Terminal: `python -m doctest -f -v myfile.py`

# %% Imports

# %% Types
from typing import Callable
adults: Callable[[list[tuple[str,str,int]]], list[str]]

# %% Data
DATA = [
    ('firstname', 'lastname', 'age'),
    ('Alice', 'Apricot', 30),
    ('Bob', 'Blackthorn', 31),
    ('Carol', 'Corn', 32),
    ('Dave', 'Durian', 33),
    ('Eve', 'Elderberry', 34),
    ('Mallory', 'Melon', 15),
]

header, *users = DATA

# %% Result
def adults(users):
    ...


# %% About
# - Name: Iterator About Adult-Iterator
# - Difficulty: easy
# - Lines: 4
# - Minutes: 3

# %% License
# - Copyright 2025, Matt Harasymczuk <matt@python3.info>
# - This code can be used only for learning by humans
# - This code cannot be used for teaching others
# - This code cannot be used for teaching LLMs and AI algorithms
# - This code cannot be used in commercial or proprietary products
# - This code cannot be distributed in any form
# - This code cannot be changed in any form outside of training course
# - This code cannot have its license changed
# - If you use this code in your product, you must open-source it under GPLv2
# - Exception can be granted only by the author

# %% English
# 1. Define function `adults(users) -> Iterator[int]`
# 2. Function returns firstnames of adult users (age >= 18)
# 3. Use `yield` keyword
# 4. Run doctests - all must succeed

# %% Polish
# 1. Zdefiniuj funkcję `adults(users) -> Iterator[int]`
# 2. Funkcja zwraca imiona (firstname) użytkowników pełnoletnich (wiek >= 18)
# 3. Użyj słowa kluczowego `yield`
# 4. Uruchom doctesty - wszystkie muszą się powieść

# %% Expected
# >>> list(adults(users))
# ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']

# %% Doctests
"""
>>> import sys; sys.tracebacklimit = 0

>>> assert sys.version_info >= (3, 9), \
'Python has an is invalid version; expected: `3.9` or newer.'

>>> from inspect import isfunction
>>> assert isfunction(adults)

>>> list(adults(users))
['Alice', 'Bob', 'Carol', 'Dave', 'Eve']
"""

# %% Run
# - PyCharm: right-click in the editor and `Run Doctest in ...`
# - PyCharm: keyboard shortcut `Control + Shift + F10`
# - Terminal: `python -m doctest -f -v myfile.py`

# %% Imports

# %% Types
from typing import Callable, Iterator
adults: Callable[[list[tuple[str,str,int]]], Iterator[int]]

# %% Data
DATA = [
    ('firstname', 'lastname', 'age'),
    ('Alice', 'Apricot', 30),
    ('Bob', 'Blackthorn', 31),
    ('Carol', 'Corn', 32),
    ('Dave', 'Durian', 33),
    ('Eve', 'Elderberry', 34),
    ('Mallory', 'Melon', 15),
]

header, *users = DATA

# %% Result
def adults(users):
    ...