Refactoring Streaming API Pipelines with Go 1.24 range-over-func — Comparing Channel, Callback, and Iterator Patterns

Last year during an on-call shift, I noticed the goroutine count quietly creeping past 2,000 on a service monitoring dashboard. Tracing the root cause, I found a leak in a channel-based DB streaming function where the producer goroutine was stuck waiting to write to the channel after the consumer had called break. The function itself looked perfect and had passed code review. All it took was a single missing ctx.Done() check.

The range-over-func iterator, officially stabilized in Go 1.23 and extended across the standard library in Go 1.24, solves this class of problem at the language level. In this article, we compare channel-based pipelines, callback patterns, and iter.Seq iterators side by side in real streaming scenarios to explore when each pattern is the right fit.

Three Patterns: What Makes Them Different

There are three main ways to deliver a sequence of values to a consumer in Go.

Channel pattern: Place a channel between goroutines and send data through it. Natural for parallel produce/consume, but if the consumer stops with break, the producer goroutine gets permanently stuck trying to write to the channel — a leak.

Callback pattern: Pass a function of the form func(item T) bool and call it for each item. No break means no leak risk, and it's fast — but you can't use for range syntax, and nesting adapters hurts readability.

range-over-func: A function that takes a yield func(V) bool, where the compiler automatically transforms the loop body into a yield closure. When break/return/panic occurs, false is automatically passed to yield, allowing the producer to clean up immediately.

This diagram compares the early-exit behavior and parallelism characteristics of all three patterns.

Basic Structure of range-over-func

The two core types in the iter package:

// iter package definitions (Go 1.23+)
type Seq[V any]     func(yield func(V) bool)
type Seq2[K, V any] func(yield func(K, V) bool)

The producer side calls yield to push values in, and the consumer side receives them with for range.

func Integers(start, end int) iter.Seq[int] {
    return func(yield func(int) bool) {
        for i := start; i < end; i++ {
            if !yield(i) { // returns false when consumer calls break
                return     // run cleanup logic here
            }
        }
    }
}
 
for n := range Integers(1, 100) {
    if n > 10 {
        break // safe: false is automatically passed to yield
    }
    fmt.Println(n)
}

★ Insight ─────────────────────────────────────

• range-over-func is a compiler transformation. for n := range Integers(...) is compiled into a function call of the form Integers(...)(func(n int) bool { ... }). No new abstraction is added at runtime — it's pure syntactic sugar.
• Ignoring the return value of yield does not cause a compile error. Writing yield(i) without if !yield(i) compiles just fine. This is the most common footgun. Make sure to add it to your team's code review checklist. ─────────────────────────────────────────────────

push vs pull: Choosing Between iter.Seq and iter.Pull

range-over-func is a push model: the iterator pushes values to the consumer. This is natural for most streaming scenarios.

However, when you need to advance two iterators simultaneously and interleave their values (zip), or connect an iterator to an external API's pull interface, you need a pull conversion.

iter.Pull uses coroutines internally. Coroutines added to the Go 1.23 runtime differ from goroutines: two functions exchange control (yield/resume) on a single OS thread, bypassing the OS scheduler and channel mutexes. While channel-based communication runs at ~400ns/op per context switch, coroutine-based iter.Pull achieves ~20ns/op.¹

next, stop := iter.Pull(someIterator)
defer stop() // required! omitting this leaves the backing coroutine permanently blocked
 
for {
    val, ok := next()
    if !ok {
        break
    }
    // process val
}

¹ Based on benchmarks from bwplotka's "Optimizing in-process gRPC with Go 1.23 Iterators and Coroutines" (Jan 2025). Apple M-series chip, Go 1.23, in-process gRPC scenario.

What Changed in the Standard Library with Go 1.24

Go 1.23 added iterator-returning functions to the slices and maps packages; Go 1.24 extended this to the strings and bytes packages as well.

// Go 1.24 strings package
for line := range strings.Lines(bigText) {
    process(line)
}
 
for field := range strings.FieldsSeq(csvLine) {
    columns = append(columns, field)
}
 
// slices package (Go 1.23+)
for i, v := range slices.All(mySlice) {
    fmt.Println(i, v)
}

You'll encounter iterators naturally when writing new code.

Real-World 1: DB Streaming Query

Loading millions of user records into memory for batch processing causes OOM. Channel-based streaming was the common choice, but it has problems.

Channel approach — early-exit risk

func StreamUsers(ctx context.Context, db *sql.DB) <-chan User {
    ch := make(chan User)
    go func() {
        defer close(ch)
        rows, err := db.QueryContext(ctx, "SELECT * FROM users")
        if err != nil {
            return // no way to propagate the error (brevity — in practice, a separate error channel is needed)
        }
        defer rows.Close()
        for rows.Next() {
            var u User
            rows.Scan(&u.ID, &u.Name, &u.Email)
            select {
            case ch <- u:
            case <-ctx.Done(): // without this, a break causes the goroutine to block permanently
                return
            }
        }
    }()
    return ch
}

Forgetting the ctx.Done() check is an easy mistake that slips through code review. The 2,000-goroutine incident I mentioned at the start originated from exactly this pattern.

Iterator approach — safe, with error propagation

func StreamUsers(ctx context.Context, db *sql.DB) iter.Seq2[User, error] {
    return func(yield func(User, error) bool) {
        rows, err := db.QueryContext(ctx, "SELECT * FROM users")
        if err != nil {
            yield(User{}, err)
            return
        }
        defer rows.Close()
 
        for rows.Next() {
            var u User
            if err := rows.Scan(&u.ID, &u.Name, &u.Email); err != nil {
                // scan error: yield the error for this row and continue to the next.
                // assumes batch processing that tolerates partial row failures.
                // if any single error should abort, change to yield then return.
                if !yield(User{}, err) {
                    return
                }
                continue
            }
            if !yield(u, nil) {
                return // consumer called break, clean up immediately
            }
        }
        if err := rows.Err(); err != nil {
            yield(User{}, err)
        }
    }
}
 
// consumer
for u, err := range StreamUsers(ctx, db) {
    if err != nil {
        log.Printf("error: %v", err)
        continue
    }
    if u.ID > 1000 {
        break // safe: rows.Close() is called via defer. no leak even without ctx.
    }
    process(u)
}

The moment break is hit, false is passed to yield, and defer rows.Close() inside the iterator runs.

Real-World 2: HTTP SSE Streaming Pipeline

Code for processing an SSE (Server-Sent Events) stream from an AI API. We build a pipeline that receives chunks, filters, and transforms them using iterator adapters.

General-purpose Map / Filter adapters

func Map[In, Out any](seq iter.Seq[In], fn func(In) Out) iter.Seq[Out] {
    return func(yield func(Out) bool) {
        for v := range seq {
            if !yield(fn(v)) {
                return
            }
        }
    }
}
 
func Filter[V any](seq iter.Seq[V], pred func(V) bool) iter.Seq[V] {
    return func(yield func(V) bool) {
        for v := range seq {
            if pred(v) {
                if !yield(v) {
                    return
                }
            }
        }
    }
}

Actual AI API streaming pipeline

type Chunk struct {
    Text string
    Done bool
}
 
func StreamAIResponse(ctx context.Context, prompt string) iter.Seq[Chunk] {
    return func(yield func(Chunk) bool) {
        resp, err := callAIAPI(ctx, prompt)
        if err != nil {
            return
        }
        defer resp.Body.Close()
 
        scanner := bufio.NewScanner(resp.Body)
        for scanner.Scan() {
            chunk, ok := parseSSELine(scanner.Text())
            if !ok {
                continue
            }
            if !yield(chunk) {
                return // consumer stopped, close resp.Body immediately
            }
            if chunk.Done {
                return
            }
        }
    }
}
 
func ProcessStream(ctx context.Context, prompt string) iter.Seq[string] {
    raw := StreamAIResponse(ctx, prompt)
    nonEmpty := Filter(raw, func(c Chunk) bool { return c.Text != "" })
    return Map(nonEmpty, func(c Chunk) string { return c.Text })
}
 
func handler(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/event-stream")
 
    for text := range ProcessStream(r.Context(), r.URL.Query().Get("prompt")) {
        fmt.Fprintf(w, "data: %s\n\n", text)
        if f, ok := w.(http.Flusher); ok {
            f.Flush()
        }
    }
}

The Filter → Map chain operates with lazy evaluation. No intermediate slices are created — each chunk passes directly through the pipeline.

★ Insight ─────────────────────────────────────

• The Map and Filter adapters are higher-order functions that return iterators. This structure is fundamentally the same as Haskell's lazy lists or Rust's Iterator trait. Go iterators bring a well-established idea from functional languages into the Go style.
• Calling ProcessStream() does not trigger an HTTP request. The first chunk is pulled only when for range begins. Iterator adapters are pure lazy evaluation — no computation happens until consumed. ─────────────────────────────────────────────────

Real-World 3: Wrapping gRPC Server Streams with iter.Pull2

When wrapping gRPC server streaming responses as iterators, watch out for the pointer reuse trap.

Pointer reuse bug (do not use)

// incorrect: the same msg pointer is repeatedly overwritten by RecvMsg
// the next call overwrites that memory even if the consumer still holds a reference to the previous value
func RecvAllBad[T any](stream grpc.ClientStream, msg *T) iter.Seq2[*T, error] {
    return func(yield func(*T, error) bool) {
        for {
            if err := stream.RecvMsg(msg); err != nil {
                if err != io.EOF {
                    yield(nil, err)
                }
                return
            }
            if !yield(msg, nil) { // danger: the next RecvMsg will overwrite the contents of this pointer
                return
            }
        }
    }
}

Correct implementation — yield by value

func RecvAll[T any](recv func() (*T, error)) iter.Seq2[T, error] {
    return func(yield func(T, error) bool) {
        for {
            msg, err := recv()
            if err == io.EOF {
                return
            }
            if err != nil {
                var zero T
                yield(zero, err)
                return
            }
            if !yield(*msg, nil) { // yield by copy — consumer can safely hold a reference
                return
            }
        }
    }
}
 
// usage
for resp, err := range RecvAll(stream.Recv) {
    if err != nil {
        return fmt.Errorf("stream error: %w", err)
    }
    process(resp)
}

Merging two streams with iter.Pull2

When you need to advance two Seq2 streams simultaneously — such as reading alternately from two gRPC streams — use iter.Pull2. iter.Pull2 takes a Seq2[K, V] and returns a next() (K, V, bool) / stop() pair.

func MergeStreams[T any](a, b iter.Seq2[T, error]) iter.Seq2[T, error] {
    return func(yield func(T, error) bool) {
        nextA, stopA := iter.Pull2(a)
        defer stopA()
        nextB, stopB := iter.Pull2(b)
        defer stopB()
 
        for {
            vA, errA, okA := nextA()
            if okA && !yield(vA, errA) {
                return
            }
            vB, errB, okB := nextB()
            if okB && !yield(vB, errB) {
                return
            }
            if !okA && !okB {
                return
            }
        }
    }
}

Forgetting stop() leaves the backing coroutine permanently blocked. Make it a habit to immediately write defer stop() right after any iter.Pull / iter.Pull2 declaration.

Four Common Mistakes

1. Ignoring the yield return value

// incorrect — no compile error, but buggy
func BadIterator() iter.Seq[int] {
    return func(yield func(int) bool) {
        for i := 0; i < 1_000_000; i++ {
            yield(i) // return value ignored! loop continues even after consumer calls break
        }
    }
}
 
// correct
func GoodIterator() iter.Seq[int] {
    return func(yield func(int) bool) {
        for i := 0; i < 1_000_000; i++ {
            if !yield(i) {
                return
            }
        }
    }
}

Add this to your team's code review checklist: "Does every yield(...) call inside an iterator function check its return value with if !yield(...)?"

2. Missing stop() with iter.Pull

// incorrect — backing coroutine permanently blocked
next, stop := iter.Pull(someSeq)
for {
    v, ok := next()
    if !ok { break }
    process(v)
}
 
// correct
next, stop := iter.Pull(someSeq)
defer stop() // defer immediately upon declaration

3. Panicking inside an iterator

In range-over-func, the iterator function and the consumer loop body run on the same goroutine's call stack. A panic inside the iterator therefore propagates up the call stack into the consumer. Handle risky operations as errors, or use recover() inside the iterator.

func SafeIterator(data []string) iter.Seq2[int, error] {
    return func(yield func(int, error) bool) {
        for _, s := range data {
            n, err := strconv.Atoi(s)
            if err != nil {
                if !yield(0, err) {
                    return
                }
                continue
            }
            if !yield(n, nil) {
                return
            }
        }
    }
}

4. Calling next() concurrently from multiple goroutines

The next() returned by iter.Pull is not goroutine-safe. This is explicitly stated in the official documentation. Calling it concurrently from multiple goroutines causes a data race.

// incorrect — data race
next, stop := iter.Pull(seq)
defer stop()
for range workers {
    go func() {
        v, ok := next() // concurrent calls are forbidden!
        process(v)
    }()
}

If you need parallel processing, it's much clearer to design with a channel-based pipeline from the start. Iterators are inherently sequential; forcing parallelism requires the extra work of serializing next() calls with a mutex.

Pattern Comparison Table

¹ Callbacks and push iterators both operate at the function-call level and have similar performance. May vary depending on compiler inlining.

Channels are better suited for parallel produce/consume patterns. This is a clear advantage of the channel pattern. The sequential nature of iterators fits most streaming scenarios where a single stream is processed in order, but if you need to process multiple sources in parallel, design with channels from the start.

Closing Thoughts

The key contribution of range-over-func is making existing callback patterns usable with for range syntax. On top of that, the compiler guarantees correct handling of break/return/panic, eliminating at the language level the goroutine leak risks that had to be managed by hand in the channel pattern. The parallelism advantage of channel-based patterns remains intact, so the two are not competing approaches — they're tools you choose based on use case.

As of Go 1.24, the slices, maps, strings, and bytes packages already return iterators, so you'll encounter them naturally in new code. There's no need to rewrite existing channel or callback APIs wholesale. The practical approach is to design new APIs with iter.Seq / iter.Seq2 from the start and migrate incrementally.

Three steps you can take right now:

1. Find existing channel iterators where break is dangerous and replace them with iter.Seq
2. Design streams with errors — like DB queries or HTTP streams — using iter.Seq2[V, error]
3. Make defer stop() a reflex whenever you use iter.Pull / iter.Pull2

References

• Range Over Function Types — Go Official Blog (2024.08)
• Go 1.23 Release Notes
• Go 1.24 Release Notes
• iter package official documentation
• Optimizing in-process gRPC with Go 1.23 Iterators and Coroutines — bwplotka (2025.01)
• Streaming Database Queries in Go 1.23 Using Iterators
• First impressions of Go 1.23's range-over-func feature — Boldly Go
• Ranging over functions in Go 1.23 — Eli Bendersky
• Go's range-over-func: 4 Footguns the Compiler Won't Warn You About
• Go Iterators: A Practical Guide to the iter Package
• Channel iteration and goroutine leak — Redowan's Reflections
• Go Concurrency Patterns: Pipelines and cancellation — Go Official Blog