01 / THE QUESTION
Where does a usable setup run out of memory?
Loading the model is only the first step. A useful deployment also needs room for the input, generated output, runtime overhead and the number of simultaneous requests it promises to serve.
02 / SETUP
Make the conditions explicit.
Record the exact GPU and VRAM, driver, model revision, weight format, serving runtime and cache format. Keep the host and runtime settings fixed within each comparison.
03 / PROPOSED METHOD
A repeatable path.
- Measure idle memory after loading the model and record the available cache capacity.
- Increase input length in bounded steps using the same prompts and output allowance.
- Measure first-token delay, generation speed and peak memory for each completed request.
- Report any reduced precision, CPU offload or concurrency limit alongside the result. Keep stability testing separate from a short fit check.
04 / WHAT TO MEASURE
More than a single number.
- Successful input and output allowance
- Peak VRAM and host memory
- First-token delay and generation speed
- Concurrency and observed failure modes
No results are available for this proposed study.
05 / LIMITATIONS
What this would not prove.
A successful single request does not establish multi-user capacity or long-term reliability. Different quantizations can change answer quality. Rental pricing and hardware availability also need a dated source.
06 / NEXT STEP
Before collecting evidence.
Choose the exact model, runtime and hardware comparison before taking measurements.
Download draft protocol ↓