Preview · AI inference and serving
The model is fast. Why is the response slow?
A request can wait before model work begins, while another request holds a worker slot, or after the server prepares a reply. Follow a small serving system and see why faster computation does not automatically mean a faster user experience.
Start with one question: eight requests arrive together, and each occupies a single worker for 200 ms. Why does the final request take much longer than 200 ms? Change the number of workers and account for every request.
Environment: conceptual schedule model in the browser. The deeper reference walkthrough uses real loopback TCP requests with simulated model work. Neither is GPU inference or a network benchmark.
Account for every request
All requests arrive together at time zero. Each occupies one worker slot for a fixed duration. This demo uses fixed simulated service times. It explains queueing, not model or network performance.
Make a prediction, then select Reveal to see the schedule as a timeline and a table.
Learn what the CPU executes — the first execution lesson is public, with no account needed.
Full AI inference and AI networking courses are planned
Not available yet — these courses are being prepared. No release dates are promised.
- Planned
Where inference time and memory go
Follow preparation, model execution, retained state, and response delivery; distinguish a useful measurement from a headline number.
Not available yet — being prepared
- Planned
AI networking: follow the bytes
Trace application buffers, kernel and transport boundaries, and data movement between machines before choosing an optimization.
Not available yet — being prepared
- Planned
Build an inference scheduler
Explore how admission, batching, and cancellation affect waiting, completed work, and fairness.
Not available yet — being prepared