Lab: capturing first-run evidence

By the end, you can

• Capture real latency evidence from a running local model using a runtime's own timing output, and real memory evidence using a runtime's own process listing (LA-7).
• Read what those captured numbers mean — including load-vs-warm timing and CPU/GPU memory placement — without overgeneralizing one machine's results (LA-7).
• Write a model-selection rationale that states a job, a hardware constraint, a chosen model and quant, and the measured evidence connecting them (LA-7).

Before you start

This is Lesson 4 of 4, the execution-and-reflection lesson closing Running models locally and this module's lab split. It uses the working Ollama install and the `llama3.2:1b` model Lesson 3's setup lab left you with — if you haven't completed that lesson's install and pull yet, do that first. This lesson carries the module's designated final-assessment contribution: a model-selection rationale with hardware justification and measured latency/memory evidence — the same artifact named in this course's Final Assessment Blueprint, so what you produce here is real groundwork for it, not a throwaway exercise.

Capturing latency evidence: --verbose and what its numbers mean

Ollama's `run` command takes a documented `--verbose` flag — its own source describes it plainly as showing "timings for response." Run it against the model from Lesson 3:

``` ollama run llama3.2:1b --verbose ```

Send it one short prompt, read the reply, then look at the timing block printed after it. Ollama's own code prints these exact labels: `total duration`, `load duration`, `prompt eval count` (with a token count), `prompt eval duration`, `prompt eval rate` (in tokens/s), `eval count` (with a token count), `eval duration`, and `eval rate` (in tokens/s). Ollama's own API documentation defines what each duration actually measures: `total_duration` is "how long the response took to generate," `load_duration` is "how long the model took to load," `prompt_eval_count` and `eval_count` are input and output token counts respectively, and `prompt_eval_duration`/`eval_duration` are the time spent on each — all timing values reported in nanoseconds at the API level, though `--verbose` converts them to readable units for you. The same documentation gives the formula behind the rate figures directly: tokens per second is `eval_count` divided by `eval_duration`, scaled by 10^9 to convert nanoseconds to seconds — worth doing once by hand on your own captured numbers, to see the printed `eval rate` is not a mysterious figure but exactly that arithmetic.

One detail that will otherwise look like a bug: run the same command a second time within a few minutes, and `load duration` will likely drop close to zero. That is expected, not an error — Ollama documents a `keep_alive` setting controlling "how long the model will stay loaded into memory following the request," defaulting to 5 minutes. Your first request in a session pays the cost of loading the model; a second request shortly after does not, because the model is still warm in memory. If you want two genuinely comparable cold-start measurements, wait past that window (or explicitly unload the model) between runs.

Capturing memory evidence: ollama ps and what its columns mean

While a model is still loaded — within that same keep_alive window — run:

``` ollama ps ```

Ollama's own documentation shows the columns this prints, with a real example row: `NAME`, `ID`, `SIZE`, `PROCESSOR`, `CONTEXT`, `UNTIL` — for instance, `gemma4:latest c6eb396dbd59 9.6 GB 100% GPU 131072 2 minutes from now`. Read each column against what this module has already covered: `SIZE` is the model's actual loaded memory footprint — the real, measured number behind Lesson 1's disk-size-as-memory-stand-in heuristic, now checkable directly rather than inferred from a download size. `PROCESSOR` reports where that memory actually landed — `100% GPU`, `100% CPU`, or a split like `48%/52% CPU/GPU` if part of the model was offloaded, exactly the evidence Lesson 1 described in the abstract and this lesson now has you observe for real. `CONTEXT` reports the context length actually allocated for this run — the number Lesson 1 of The model underneath warned can be smaller than a model's own advertised maximum, and Lesson 1 of this module warned adds to memory use as it grows. `UNTIL` is the keep_alive countdown from the previous section, ticking down to when the model will be unloaded if nothing else uses it.

Reading the numbers: what to conclude, and what not to

Your numbers will differ from anyone else's, including any figures quoted earlier in this module — those came from a specific benchmark on a specific machine that this module's sources didn't fully disclose, or from a specific different model. That is not a flaw in the exercise; it is the whole point of capturing your own evidence rather than trusting a rule of thumb. A few honest, general things worth watching for, without treating them as universal thresholds:

• If `eval rate` feels too slow for the job you actually have in mind, the two levers this module has given you are model size and quantisation level (Lesson 2) — a smaller model, or the same model at a more aggressive quant, will generally run faster, at whatever accuracy cost that quant level carries.
• If `PROCESSOR` shows a CPU/GPU split and speed disappoints you, the model didn't fully fit in VRAM; the fix is the same two levers, aimed at getting the whole model onto the faster pool instead of accepting a mixed one.
• If `SIZE` sits comfortably below what your hardware has available, per Lesson 1's sizing, there is real headroom to raise the context length for a task that needs it — and `CONTEXT` and `SIZE` together, watched before and after that change, are exactly how you'd confirm the extra memory the earlier lessons warned about actually showed up.
• One run of one prompt is an anecdote, not a benchmark. A comparison that actually means something holds the prompt reasonably constant across the runs being compared — this course's earlier coverage of sampling and determinism (The model underneath, Lesson 3) is a direct reminder that the same prompt can still produce a different-length, differently-timed response run to run, so don't treat a single number as more precise than it is.

From evidence to a model-selection rationale

The course's Final Assessment Blueprint names a specific artifact this module builds toward: "model-selection rationale with hardware justification and measured latency/memory." A rationale worth that name states, plainly and in order: the job the model needs to do; the hardware it has to run on; the specific model and quantisation level chosen; and the measured evidence — captured the way this lesson just showed you — that the choice actually holds up on that hardware. What you write in this lesson's exercise is real practice for that artifact, on the small model this module's lab used, not a substitute for it; the course's later final assessment will ask for the same shape of rationale against a task of your own choosing.

Accessibility notes

This lesson is text-first, with no images, audio, video or downloadable artifacts. All CLI commands, output columns and formulas are given as plain inline text, readable and copyable by assistive technology and keyboard-only users without relying on a screenshot of terminal output. The practice exercise's model answer sits behind a native disclosure control that is reachable and operable by keyboard and correctly announced by screen readers. The knowledge check uses native radio-button inputs with a visible question and options, and posts its result to a live status region so assistive technology announces the outcome without a page reload. This lab's commands run in the learner's own terminal application, outside this page, the same accessibility boundary Lesson 3 notes.

Sources and limits

This lesson synthesises the sources below into a practical learning model. It is not a security standard, legal advice or a guarantee that any particular agent design is safe.

1. ollama/ollama cmd/cmd.go — Ollama (GitHub). Documents the run command's --verbose flag as "Show timings for response".
2. ollama/ollama api/types.go — Ollama (GitHub). Documents the exact metric labels and format --verbose prints — total duration, load duration, prompt/eval counts, durations and rates.
3. Usage — API response metrics — Ollama. Defines total_duration, load_duration, prompt_eval_count and eval_count/eval_duration, all measured in nanoseconds.
4. ollama/ollama docs/api.md — Ollama (GitHub). Documents the tokens-per-second formula (eval_count / eval_duration * 10^9) and the keep_alive parameter's default 5-minute in-memory retention.
5. Context length — ollama ps output — Ollama. Documents the ollama ps columns (NAME, ID, SIZE, PROCESSOR, CONTEXT, UNTIL) with a real example row.