Go Slice Aliasing: The `append` That Silently Overwrites Your Caller's Data

Go · Intermediate · 6 min read · published

What this solves: Slicing a Go slice and appending to the result can overwrite elements the original slice still owns, producing corrupted data that only appears once capacity exceeds length. This shows you how to spot and prevent it.

The Problem

A pagination helper in a Go service:

func page(items []Result, offset, size int) []Result {
    return items[offset : offset+size]
}

Callers get a page, then decorate it:

p := page(all, 0, 20)
p = append(p, sponsoredResult) // pin an ad to the end of the page

In unit tests with a 20-item fixture, perfect. In production with 10,000 results, users on page 1 see the ad — and users on page 2 see the ad as their first result, while the genuine item that belonged there has vanished. The bug rate is exactly 1 item per page, it never panics, and no log line mentions it. Support tickets say "a product disappeared from search."

Nothing was deleted. append overwrote it.

Why the Obvious Fix Falls Short

The reflex is: "slices are references, so I'll copy defensively." Reasonable — but developers usually reach for one of these:

copied := items[offset : offset+size] // still the same array
copied := append([]Result{}, items[offset:offset+size]...) // works, allocates always

The first does nothing at all: re-slicing never copies. The second is correct but heavy — you allocate and memcpy every page even in the 99% of calls that never append.

The deeper reason the reflex fails is that people believe append always returns a new array, or that "it only reuses capacity if there is room" — and then assume a slice of length 20 has no room. It has room: capacity is measured from the slice's start to the end of the backing array, not to the end of the slice. items[0:20] out of a 10,000-element array has len 20, cap 10000. append sees 9,980 spare slots and happily writes into index 20, which is a live element of the parent.

So the real fix isn't "copy more" — it's "stop lying to append about how much of the array you own."

How It Actually Works

A slice is a three-word header: pointer, length, capacity. Re-slicing produces a new header pointing into the same array, inheriting whatever capacity remains behind it.

append follows one rule: if len < cap, write in place and bump the length. Otherwise, allocate a bigger array, copy, and return a header pointing at the new one. That branch is why the bug is intermittent-looking: when cap == len, append allocates and everything is safe; when cap > len, it mutates shared memory.

flowchart TD
    subgraph arr["backing array (10000 Results)"]
      i0["[0..19] page 1"]
      i20["[20] first item of page 2"]
      i21["[21..] rest"]
    end
    P["page 1 header<br/>ptr=&arr[0] len=20 cap=10000"] --> i0
    Q["page 2 header<br/>ptr=&arr[20] len=20 cap=9980"] --> i20
    A["append(page1, ad)"] -->|"len(20) &lt; cap(10000)<br/>⇒ write in place at index 20"| i20
    A -.->|"if cap had been 20<br/>⇒ allocate new array"| N["new array, copy of 20 + ad"]

The three-index slice expression s[low:high:max] sets capacity explicitly: cap == max - low. Writing s[a:b:b] yields a slice whose capacity equals its length, so the very next append is guaranteed to allocate. This is a full slice expression, and it is free — no copy, just a different capacity word in the header.

Before and After

// BEFORE: capacity leaks the entire tail of the array.
// append() writes into items[offset+size], which belongs to the next page.
func page(items []Result, offset, size int) []Result {
    if offset+size > len(items) {
        size = len(items) - offset
    }
    return items[offset : offset+size] // len=size, cap=len(items)-offset
}

// AFTER: clamp cap to the slice's own length with the three-index form.
// Any append by the caller is forced to allocate, so items is never touched.
func page(items []Result, offset, size int) []Result {
    end := min(offset+size, len(items))
    return items[offset:end:end] // len == cap == end-offset
}

// If callers also mutate elements in place (not just append),
// capacity clamping isn't enough — hand them their own array:
func pageCopy(items []Result, offset, size int) []Result {
    end := min(offset+size, len(items))
    out := make([]Result, end-offset)
    copy(out, items[offset:end])
    return out
}

When NOT to Use This

Gotchas

Key takeaway: Any slice you derive with `s[a:b]` shares a backing array and inherited capacity — use the three-index form `s[a:b:b]` or an explicit `copy` before appending if the caller still owns that data.

Real-world challenge

A batch processor splits a 1,000-item `[]Event` into 100-item chunks with `events[i:i+100]` and hands each chunk to a goroutine. Each worker calls `chunk = append(chunk, retryEvent)` when it needs to re-enqueue a failure. Under load, your metrics show duplicate processing and occasionally an event that was never in the input at all. With small batches (under 100 items, one chunk) everything is fine. What is happening and how do you fix it?

Diagnosis

Each chunk events[i:i+100] has len == 100 but cap == 1000 - i — the tail of the original array. When a worker appends, it writes the retry event directly over events[i+100], which is chunk i+1's first element. Two goroutines are now racing over the same memory, which is also a data race (go test -race will flag it).

With a single chunk there is no next chunk to clobber, so the bug hides.

Add a quick probe:

log.Printf("chunk %d len=%d cap=%d", i, len(chunk), cap(chunk)) // cap != len ⇒ danger

Fix

Clamp the capacity at the chunk boundary with the three-index slice expression so append is forced to allocate:

end := min(i+100, len(events))
chunk := events[i:end:end] // cap == len; append copies instead of overwriting

If workers also mutate elements in place, capacity clamping is not enough — give each worker its own array:

chunk := make([]Event, end-i)
copy(chunk, events[i:end])