Foundation section · 10 lessons · Available
Memory and data movement
From bytes and object lifetime to virtual memory, caches, data layout, and ownership. Learn why an optimization that avoids copying can retain far more memory than expected.
- 1Bytes have positions. Meaning comes from interpretation.A network message, an image, and a model weight file all become bytes. That does not make their contents interchangeable. To interpret a sequence, you need rules: where a value starts, how many bytes it occupies, and what those bytes…Model · Python snippets~4 min estimated reading
- 2Who keeps this data alive?A helper creates a buffer, returns something derived from it, and finishes. Is the returned value still valid? The answer depends on what was returned and who retains the storage. “The function ended” is not enough to decide.Model · Python snippets~4 min estimated reading
- 3An address is a name, not a RAM coordinateTwo processes can print the same numeric address without sharing the same data. A process can also have a large address range mapped without every byte currently occupying physical RAM. Both become less surprising when you separate an…Model~4 min estimated reading
- 4You allocated memory. How much RAM did you use?A dashboard shows a process with a large virtual size and a much smaller resident size. That can be normal. Another process releases a large object but its resident size barely falls. That can also have an explanation other than “free…Recorded native evidence~3 min estimated reading
- 5Why does using nearby data often help?Two pieces of code read the same values and compute the same answer. One visits neighbors; the other jumps around. To understand why that can matter, we need a model of what gets retained between accesses.Model~4 min estimated reading
- 6The data has to get thereF3 · MEM-02 introduction · Memory access and localityRecorded native evidence15–20 min estimated reading10–15 min exercises / exploration
- 7Store data for the work you actually doA service stores records containing a score, an identifier, and other metadata. One hot operation scans only scores. Another operation reads all fields of one record. Should they use the same layout? The answer starts with access…Model · Python snippets~3 min estimated reading
- 8A copy costs work. A view creates a responsibility.A parser needs four bytes from a larger buffer. Copying them gives the parser independent storage. Keeping a view can avoid that payload copy, but now two parts of the program refer to the same underlying bytes. Which is preferable…Model · Python snippets~4 min estimated reading
- 9Waiting for one byte and moving a million are different problemsA system can move large blocks quickly yet take a noticeable time to begin each transfer. Another can answer small requests promptly but struggle with a large stream. “Fast memory” or “fast network” hides at least two questions: how…Model~3 min estimated reading
- 10Investigation: the parser that kept the whole responseA service fetches a large response, extracts a tiny identifier, and caches that identifier. Someone replaces a small copy with a view to reduce allocations. The code still returns the right identifier. Yet memory usage rises as the…Constructed case~4 min estimated reading
The first lesson needs no account; remaining lessons require a verified free account. Signed-in learners can open lessons in any order. Reading estimates exclude exercises and challenges.
About this section
A program does not run on abstract values floating in space. Those values have a representation, occupy storage for some lifetime, and must be available where work happens. A layout choice can change which bytes are touched. A view can avoid a copy while retaining an entire response. A large virtual allocation may mean something different from the resident memory on a dashboard.
This section connects those facts into one way of reasoning. We begin with bytes so the address arithmetic is concrete. We add lifetime and ownership so the bytes remain meaningful. Then we separate virtual mapping from physical residency, inspect locality, and connect layout and copying to transfer costs.
The destination is a complete investigation of a small parser optimization that causes large memory retention. You will justify a repair with representation and ownership evidence rather than rely on a single “memory usage” number. This is a complete introductory section, not the end of memory engineering; NUMA, coherence, atomics, and device-specific memory systems remain deeper topics.
The complete section
| Lesson | Question you will be able to answer |
|---|---|
| 1. Bytes, addresses, and interpretation | What do these bytes mean, and where does an element begin? |
| 2. Who keeps this data alive? | Which owner keeps storage valid, and which names share it? |
| 3. An address is a name | Why can equal virtual addresses refer to different backing memory? |
| 4. Allocation and residency | Which layer does a memory measurement count? |
| 5. Caches and locality | What does an explicit retention policy predict? |
| 6. The data has to get there | What happens when compiled code traverses the same data in another order? |
| 7. Store data for the work | Which representation groups the useful fields for this operation? |
| 8. Copies, views, and ownership | What responsibility comes with avoiding a payload copy? |
| 9. Latency and bandwidth | Is the operation limited by startup, volume, or dependencies? |
| 10. The parser that kept the whole response | Can I explain retention and choose a correct, measurable repair? |
Enter where it helps
Basic programming is sufficient. The execution section is useful background, not a mandatory prerequisite. If you already know bytes and address arithmetic, jump to the question you need. A learning path is a recommendation, not a lock.
Tables and small models expose exact relationships; the native traversal and Linux mapping exercises show where real observations add uncertainty. No example assumes you have a local compiler or terminal. Optional local reproduction is provided for readers who want it. Measured examples show recorded evidence from a stated environment; models show worked traces and can run in your browser.
For every performance claim, ask: did we preserve the result, what did we count, and which alternative explanation remains? For every memory change, ask: who owns the bytes now, and for how long? These questions carry directly into network buffers, inference state, and serving concurrency.
Completion and next steps
The final investigation asks for an ownership diagram, a payload calculation, correctness checks, and a measurement plan. You can complete it through reading and reasoning before any advanced lab infrastructure exists.
Afterward, try the AI endpoint preview. The full curriculum shows where networking, GPUs, and inference go next. Upcoming items are explicitly unavailable; they are not paid content or mandatory next steps.