When avoiding a copy keeps memory alive

A service reads a response into an 8 MiB buffer. A parser needs only a 16-byte header. Keeping a view of that header sounds cheaper than copying it.

It avoids copying 16 bytes. It can also keep all 8 MiB alive.

Those are two different costs: work performed now and memory retained afterwards. An optimization can improve the first while making the second worse.

A view does not own a smaller buffer

Here is a Python example you can inspect without a local setup. The numbers below are byte counts, not a benchmark. A mebibyte (MiB) is 1,048,576 bytes.

Expected output:

Visible bytes: 16
Backing bytes: 8388608
Header: b'0123456789abcdef'

The local name response disappears when the function returns. The underlying object is still needed: header refers to storage exported by that bytearray. The view selects a range; it does not replace the original allocation with a 16-byte allocation.

bytes(header) copies the selected bytes for display. So even this demonstration should not be called an entirely copy-free program.

Copy at the lifetime boundary

Suppose a queue keeps the header for later, while the rest of the response is no longer useful. Make the queue's data independent:

This copies only the selected 16 bytes. Once the function returns, its temporary view is released and no reference in this example retains the response. The queue can keep the header without keeping that response buffer.

That does not promise an immediate 8 MiB drop in the process's resident memory. Object lifetime, allocator reuse, and what the operating system reports are separate questions. The example establishes a reference relationship, not an RSS result.

Copying the whole response would be a different tradeoff. The useful choice here is a small copy at the point where a small piece of data acquires a much longer lifetime.

A view also shares changes

Lifetime is only half the contract:

The output is bcd bXd. The earlier copy has independent storage. The view observes the modified source.

A producer that reuses a buffer before a delayed consumer finishes can therefore change the consumer's input. Passing a view does not create a snapshot. A safe design might copy the needed data, transfer ownership, or keep a pooled buffer reserved until all its consumers have finished. Which contract works depends on the API and the application's concurrency.

What keeps the buffer alive?

Imagine 100 queued headers, each referring to a different 8 MiB response. The useful headers total 1,600 bytes, but the views retain about 800 MiB of backing payload, plus object overhead.

If those 100 views all refer to the same response, they retain one 8 MiB payload instead. Counting views alone is insufficient. You need to know which storage they share.

This is why a benchmark that creates a view and immediately discards it can miss a production problem. The application may retain the view across a queue, cache, callback, or asynchronous operation. Measure that lifetime too.

What to check in a real service

Start with the object that remains reachable. Find the queue, cache, closure, or request state holding it. Follow that reference to the backing allocation and determine when the last consumer lets go.

Then compare a small copy with a view under the same arrival rate and consumer behavior. Observe retained allocations, queue depth, throughput, and latency. A reduction in retained memory is useful evidence; it is not automatically proof of a latency improvement.

For AI serving, the same question applies to request bodies and preprocessing buffers held while requests wait. This Python example says nothing about GPU tensor ownership or a framework's allocator. Those need their own ownership contracts and measurements.

Name the copy you eliminated

"Zero-copy" is useful only when the boundary is clear. A parser might avoid duplicating a payload into a second application buffer while the rest of the system still moves data through other buffers, caches, or devices.

Ask: which bytes were not copied, between which two places, and how long must the original storage remain alive?

Sometimes the right answer is a view. Sometimes it is a deliberate 16-byte copy.

This article grew out of the memory and data movement section in Systems Unpacked. The platform currently has 19 free foundation lessons; the first lesson in each section is public and the others require a free verified account. The AI-serving queue model is public.

Sources: Python 3.12 memoryview documentation (opens in a new tab) and Python memory management (opens in a new tab). The examples demonstrate Python object relationships and run without Linux-specific tools. They are not timing or GPU benchmarks.

Editorial note: prepared with AI assistance; example outputs were checked before publication.