itertools.groupby in Python: Group Consecutive Records

Quick answer: itertools.groupby groups consecutive records by a key. It is not a global grouping operation: sort or arrange the input by the same key when equal records are separated, and materialize a group before advancing if it must be reused.

Python itertools.groupby() groups consecutive items from an iterable that have the same key. It returns pairs of (key, group_iterator), which makes it useful for summarizing sorted records, grouping rows, and processing streams without building every group first.

The most important detail is that groupby() does not search the whole iterable for matching keys. It starts a new group whenever the key changes. If you want all matching items together, sort the data by the same key before calling groupby().

Basic groupby example

from itertools import groupby

items = ["A", "A", "B", "A"]
result = [(key, list(group)) for key, group in groupby(items)]
print(result)

Output:

[('A', ['A', 'A']), ('B', ['B']), ('A', ['A'])]

The final "A" is a separate group because it is not next to the first two "A" values. This is the behavior that surprises most beginners.

Sort before grouping

When you need one group per key, sort first:

from itertools import groupby
from operator import itemgetter

orders = [
    {"status": "paid", "total": 40},
    {"status": "open", "total": 25},
    {"status": "paid", "total": 15},
    {"status": "open", "total": 10},
]

orders.sort(key=itemgetter("status"))
summary = {
    status: sum(order["total"] for order in group)
    for status, group in groupby(orders, key=itemgetter("status"))
}
print(summary)

Output:

{'open': 35, 'paid': 55}

The sort key and the group key should match. Here, both use itemgetter("status"), so each status appears once in the summary.

Grouping anagrams

A common real-world pattern is to group strings by a derived key. For anagrams, the key can be the sorted letters of each word:

from itertools import groupby

words = ["eat", "tea", "tan", "ate", "nat", "bat"]
words.sort(key=lambda word: "".join(sorted(word)))

anagrams = {
    key: list(group)
    for key, group in groupby(words, key=lambda word: "".join(sorted(word)))
}
print(anagrams)

Output:

{'abt': ['bat'], 'aet': ['eat', 'tea', 'ate'], 'ant': ['tan', 'nat']}

The dictionary key is not the final label you would show to users; it is the grouping key that tells Python which words belong together.

How group iterators work

Each group returned by groupby() is an iterator over the current consecutive group. It shares the underlying iterable with the outer groupby object. If you need the values later, convert the group immediately with list(group) or consume it before the outer loop advances.

for key, group in groupby(items):
    values = list(group)
    print(key, values)

This iterator behavior is memory efficient for streams, logs, and sorted files, but it means you should not store the raw group iterator for later use.

groupby vs Counter and defaultdict

Use groupby() when order matters or when you can sort the input once and then process each group in sequence. It is especially useful for sorted CSV rows, log entries, database exports, and other streams where equal keys are already adjacent.

If you only need counts, collections.Counter is usually clearer. If you need to collect unsorted values into lists, collections.defaultdict(list) avoids the sorting step. The tradeoff is memory: dictionary-based grouping keeps every collected value in memory, while groupby() can process one consecutive group at a time.

from collections import Counter

items = ["A", "A", "B", "A"]
print(Counter(items))

This returns total counts across the whole iterable, not consecutive groups. Choose the tool based on whether you care about sequence order, total aggregation, or streaming behavior.

When to use groupby

• Use groupby() when the input is already sorted or naturally arrives in grouped order.
• Use it when you want to process groups lazily instead of building a large dictionary of lists.
• Use a dictionary, defaultdict(list), or collections.Counter when you need to aggregate unsorted data without sorting first.
• Use the same key function for sorting and grouping to avoid split groups.

Related Python guides

• Python itertools.islice()
• Python itertools.product()
• itertools combinations
• Convert tuple to string in Python
• Python 2D list
• Sort dictionary by key in Python

Official references

• Python itertools.groupby documentation
• Python Sorting HOWTO
• Python operator.itemgetter documentation

Conclusion

itertools.groupby() is best for grouping consecutive items by a key. Sort the iterable first when you need all equal keys together, convert each group to a list if you need to reuse it, and choose a dictionary-based approach when the data is unsorted and sorting is not desirable.

Group Consecutive Values

groupby emits a key and an iterator for each run of equal keys. This is useful for already ordered event streams and avoids building a full index.

from itertools import groupby

values = ["a", "a", "b", "b", "a"]
for key, group in groupby(values):
    print(key, list(group))

Sort For Global Groups

If the same key appears in several separated runs, sort first when order does not carry meaning. Sorting by a different key will produce incorrect group boundaries.

from itertools import groupby

records = [{"team": "red", "score": 2}, {"team": "blue", "score": 4}, {"team": "red", "score": 3}]
ordered = sorted(records, key=lambda record: record["team"])
for team, group in groupby(ordered, key=lambda record: record["team"]):
    print(team, list(group))

Materialize Before Advancing

Each group shares the underlying iterator. Convert it to a list or consume it immediately when later code needs the values after the outer loop advances.

from itertools import groupby

values = [1, 1, 2, 2]
groups = []
for key, group in groupby(values):
    groups.append((key, list(group)))
print(groups)

Use A Key Function

A key function can project a record onto the grouping field. Keep the key function consistent with sorting and document whether case or missing values are normalized.

from itertools import groupby

words = ["Apple", "apricot", "banana"]
ordered = sorted(words, key=str.lower)
for first, group in groupby(ordered, key=lambda word: word[0].lower()):
    print(first, list(group))

Choose The Right Grouping Tool

groupby is a streaming tool, so it is a good fit when records are ordered and you want to process one run at a time. It is not a replacement for a dictionary or database aggregation in every situation. If the input is large and already sorted, groupby can avoid storing every group at once. If the input is unordered, sorting may be expensive and a dictionary keyed by the grouping value may be clearer. Make the choice explicit in the code and document whether original order, memory use, or one-result-per-key behavior matters.

Also decide how empty input, missing keys, and mixed types should behave. A key function that normalizes case or converts a field should be tested independently. Small tests with separated duplicate keys are especially valuable because they reveal the most common misunderstanding of groupby.

Python’s itertools.groupby() documentation explains consecutive grouping and shared iterators. Related references include foreach-style iteration, sorting records, and key transformations.

For related iteration and ordering, compare foreach-style loops, sorting records, and key transformations when grouping data.

Frequently Asked Questions

Does groupby group all equal values automatically?

It groups consecutive values with the same key, so equal records separated by another key produce separate groups.

Should I sort before groupby?

Sort by the same key when you need one group per key and the input order does not already make equal keys consecutive.

Why does a group disappear after iteration?

Each group is an iterator tied to the source iterator; consume or materialize it before advancing to the next group.

Can groupby use a custom key?

Yes. Pass a key function that returns the grouping value, such as operator.itemgetter for records.