A deep dive into Collections, Sequences, and Iterators in Swift – Donny Wals

If you write for merchandise in record the compiler quietly units a number of equipment in movement. Often writing a for loop is a fairly mundane process, it is not that advanced of a syntax to jot down. Nevertheless, it is at all times enjoyable to dig a bit deeper and see what occurs underneath the hood. On this publish I’ll unpack the items that make iteration tick so you’ll be able to cause about loops with the identical confidence you have already got round optionals, enums, or outcome builders.

Right here’s what you’ll decide up:

• What Sequence and Assortment promise—and why iterators are virtually at all times structs.
• How for … in desugars, plus the pitfalls of mutating when you loop.
• How async iteration and customized collections lengthen the identical core concepts.

Understanding Sequence

Sequence is the smallest unit of iteration in Swift and it comes with a really intentional contract: “when someone asks for an iterator, give them one that may hand out components till you’re out”. Meaning a conforming sort must outline two related sorts (Factor and Iterator) and return a recent iterator each time makeIterator() known as.

public protocol Sequence {
    associatedtype Factor
    associatedtype Iterator: IteratorProtocol the place Iterator.Factor == Factor

    func makeIterator() -> Iterator
}

The iterator itself conforms to IteratorProtocol and exposes a mutating subsequent() operate:

public protocol IteratorProtocol {
    associatedtype Factor
    mutating func subsequent() -> Factor?
}

You’ll see most iterators applied as structs. subsequent() is marked mutating, so a value-type iterator can replace its place with none additional ceremony. If you copy the iterator, you get a recent cursor that resumes from the identical level, which retains iteration predictable and prevents shared mutable state from leaking between loops. Courses can undertake IteratorProtocol too, however worth semantics are a pure match for the contract.

There are two necessary implications to bear in mind:

• A sequence solely needs to be single-pass. It’s completely legitimate at hand out a “consumable” iterator that can be utilized as soon as after which returns nil ceaselessly. Lazy I/O streams or generator-style APIs lean on this behaviour.
• makeIterator() ought to produce a recent iterator every time you name it. Some sequences select to retailer and reuse an iterator internally, however the contract encourages the “new iterator per loop” mannequin so for loops can run independently with out odd interactions.

If you happen to’ve ever used stride(from:to:by:) you’ve already labored with a plain Sequence. The usual library exposes it proper subsequent to ranges, and it’s good for strolling an arithmetic development with out allocating an array. For instance:

for angle in stride(from: 0, by: 360, by: 30) {
    print(angle)
}

This prints 0, 30, 60 … 360 after which the iterator is completed. If you happen to ask for one more iterator you’ll get a brand new run, however there’s no requirement that the unique one resets itself or that the sequence shops all of its values. It simply retains the present step and arms out the subsequent quantity till it reaches the top. That’s the core Sequence contract in motion.

So to summarize, a Sequence incorporates n objects (we do not know what number of as a result of there is no idea of depend in a Sequence), and we are able to ask the Sequence for an Iterator to obtain objects till the Sequence runs out. As you noticed with stride, the Sequence would not have to carry all values it’ll ship in reminiscence. It will possibly generate the values each time its Iterator has its subsequent() operate referred to as.

If you happen to want a number of passes, random entry, or counting, Sequence received’t offer you that by itself. The protocol doesn’t forbid throwing the weather away after the primary cross; AsyncStream-style sequences do precisely that. An AsyncStream will vend a brand new worth to an async loop, after which it discards the worth ceaselessly.

In different phrases, the one promise is “I can vend an iterator”. Nothing says the iterator might be rewound or that calling makeIterator() twice produces the identical outcomes. That’s the place Assortment steps in.

Assortment’s Further Ensures

Assortment refines Sequence with the guarantees we lean on day-to-day: you’ll be able to iterate as many instances as you want, the order is secure (so long as the gathering’s personal documentation says so), and also you get indexes, subscripts, and counts. Swift’s Array, Dictionary, and Set all conform to the Assortment protocol for instance.

public protocol Assortment: Sequence {
    associatedtype Index: Comparable

    var startIndex: Index { get }
    var endIndex: Index { get }

    func index(after i: Index) -> Index
    subscript(place: Index) -> Factor { get }
}

These additional necessities unlock optimisations. map can preallocate precisely the correct quantity of storage. depend doesn’t have to stroll the whole knowledge set. If a Assortment additionally implements BidirectionalCollection or RandomAccessCollection the compiler can apply much more optimizations free of charge.

Value noting: Set and Dictionary each conform to Assortment though their order can change after you mutate them. The protocols don’t promise order, so if iteration order issues to you ensure you decide a kind that paperwork the way it behaves.

How for … in Truly Works

Now that a bit extra about collections and iterating them in Swift, right here’s what a easy loop seems like in case you have been to jot down one with out utilizing for x in y:

var iterator = container.makeIterator()
whereas let factor = iterator.subsequent() {
    print(factor)
}

To make this concrete, right here’s a small customized sequence that can depend down from a given beginning quantity:

struct Countdown: Sequence {
    let begin: Int

    func makeIterator() -> Iterator {
        Iterator(present: begin)
    }

    struct Iterator: IteratorProtocol {
        var present: Int

        mutating func subsequent() -> Int? {
            guard present >= 0 else { return nil }
            defer { present -= 1 }
            return present
        }
    }
}

Operating for quantity in Countdown(begin: 3) executes the desugared loop above. Copy the iterator midway by and every copy continues independently due to worth semantics.

One factor to keep away from: mutating the underlying storage when you’re in the course of iterating it. An array iterator assumes the buffer stays secure; in case you take away a component, the buffer shifts and the iterator now not is aware of the place the subsequent factor lives, so the runtime traps with Assortment modified whereas enumerating. When it’s essential cull objects, there are safer approaches: name removeAll(the place:) which handles the iteration for you, seize the indexes first and mutate after the loop, or construct a filtered copy and exchange the unique when you’re completed.

Right here’s what an actual bug seems like. Think about an inventory of duties the place you need to strip the finished ones:

struct TodoItem {
    var title: String
    var isCompleted: Bool
}

var todoItems = [
    TodoItem(title: "Ship blog post", isCompleted: true),
    TodoItem(title: "Record podcast", isCompleted: false),
    TodoItem(title: "Review PR", isCompleted: true),
]

for merchandise in todoItems {
    if merchandise.isCompleted,
       let index = todoItems.firstIndex(the place: { $0.title == merchandise.title }) {
        todoItems.take away(at: index) // ⚠️ Deadly error: Assortment modified whereas enumerating.
    }
}

Operating this code crashes the second the primary accomplished process is eliminated as a result of the iterator nonetheless expects the previous structure. It additionally calls firstIndex on each cross, so every iteration scans the entire array once more—a straightforward technique to flip a fast cleanup into O(n²) work. A safer rewrite delegates the traversal:

todoItems.removeAll(the place: .isCompleted)

As a result of removeAll(the place:) owns the traversal, it walks the array as soon as and removes matches in place.

If you happen to favor to maintain the originals round, construct a filtered copy as a substitute:

let openTodos = todoItems.filter { !$0.isCompleted }

Each approaches hold iteration and mutation separated, which suggests you received’t journey over the iterator mid-loop. Every thing we’ve checked out to date assumes the weather are prepared the second you ask for them. In fashionable apps, it is not unusual to need to iterate over collections (or streams) that generate new values over time. Swift’s concurrency options lengthen the very same iteration patterns into that world.

Async Iteration in Follow

Swift Concurrency introduces AsyncSequence and AsyncIteratorProtocol. These look acquainted, however the iterator’s subsequent() methodology can droop and throw.

public protocol AsyncSequence {
    associatedtype Factor
    associatedtype AsyncIterator: AsyncIteratorProtocol the place AsyncIterator.Factor == Factor

    func makeAsyncIterator() -> AsyncIterator
}

public protocol AsyncIteratorProtocol {
    associatedtype Factor
    mutating func subsequent() async throws -> Factor?
}

You eat async sequences with for await:

for await factor in stream {
    print(factor)
}

Underneath the hood the compiler builds a looping process that repeatedly awaits subsequent(). If subsequent() can throw, change to for strive await. Errors propagate identical to they’d in another async context.

Most callback-style APIs might be bridged with AsyncStream. Right here’s a condensed instance that publishes progress updates:

func makeProgressStream() -> AsyncStream {
    AsyncStream { continuation in
        let token = progressManager.observe { fraction in
            continuation.yield(fraction)
            if fraction == 1 { continuation.end() }
        }

        continuation.onTermination = { _ in
            progressManager.removeObserver(token)
        }
    }
}

for await fraction in makeProgressStream() now suspends between values. Don’t overlook to name end() once you’re completed producing output, in any other case downstream loops by no means exit.

Since async loops run inside duties, they need to play properly with cancellation. The simplest sample is to test for cancellation inside subsequent():

struct PollingIterator: AsyncIteratorProtocol {
    mutating func subsequent() async throws -> Merchandise? {
        strive Job.checkCancellation()
        return await fetchNextItem()
    }
}

If the duty is cancelled you’ll see CancellationError, which ends the loop routinely until you resolve to catch it.

Implementing your personal collections

Most of us by no means should construct a group from scratch—and that’s an excellent factor. Arrays, dictionaries, and units already cowl nearly all of instances with battle-tested semantics. If you do roll your personal, tread rigorously: you’re promising index validity, multi-pass iteration, efficiency traits, and all the opposite traits that callers anticipate from the usual library. A tiny mistake can corrupt indices or put you in undefined territory.

Nonetheless, there are reputable causes to create a specialised assortment. You may want a hoop buffer that overwrites previous entries, or a sliding window that exposes simply sufficient knowledge for a streaming algorithm. Everytime you go down this path, hold the floor space tight, doc the invariants, and write exhaustive assessments to show the gathering acts like a regular one.

Even so, it is value exploring a customized implementation of Assortment for the sake of finding out it. Right here’s a light-weight ring buffer that conforms to Assortment:

struct RingBuffer: Assortment {
    personal var storage: [Element?]
    personal var head = 0
    personal var tail = 0
    personal(set) var depend = 0

    init(capability: Int) {
        storage = Array(repeating: nil, depend: capability)
    }

    mutating func enqueue(_ factor: Factor) {
        storage[tail] = factor
        tail = (tail + 1) % storage.depend
        if depend == storage.depend {
            head = (head + 1) % storage.depend
        } else {
            depend += 1
        }
    }

    // MARK: Assortment
    typealias Index = Int

    var startIndex: Int { 0 }
    var endIndex: Int { depend }

    func index(after i: Int) -> Int {
        precondition(i < endIndex, "Can not advance previous endIndex")
        return i + 1
    }

    subscript(place: Int) -> Factor {
        precondition((0..

A couple of particulars in that snippet are value highlighting:

• storage shops optionals so the buffer can hold a set capability whereas monitoring empty slots. head and tail advance as you enqueue, however the array by no means reallocates.
• depend is maintained individually. A hoop buffer is likely to be partially crammed, so counting on storage.depend would lie about what number of components are literally out there.
• index(after:) and the subscript settle for logical indexes (0 by depend) and translate them to the suitable slot in storage by offsetting from head and wrapping with the modulo operator. That bookkeeping retains iteration secure even after the buffer wraps round.
• Every accessor defends the invariants with precondition. Skip these checks and a stray index can pull stale knowledge or stroll off the top with out warning.

Even in an instance as small because the one above, you’ll be able to see how a lot duty you tackle when you undertake Assortment.

In Abstract

Iteration seems easy as a result of Swift hides the boilerplate, however there’s a surprisingly wealthy protocol hierarchy behind each loop. As soon as you know the way Sequence, Assortment, and their async siblings work together, you’ll be able to construct knowledge buildings that really feel pure in Swift, cause about efficiency, and bridge legacy callbacks into clear async code.

If you wish to hold exploring after this, revisit the posts I’ve written on actors and knowledge races to see how iteration interacts with isolation. Or take one other have a look at my items on map and flatMap to dig deeper into lazy sequences and purposeful pipelines. Both approach, the subsequent time you attain for for merchandise in record, you’ll know precisely what’s taking place underneath the hood and the way to decide on the suitable strategy for the job.