
Ollama’s 0.40.17-rc2 release candidate is tagged with a change to saved conversation history in its MLX runner. The maintainer’s proposal describes replacing a fixed allowance for saved Metal history with a policy that looks at available memory and the workspace needed to run the model.
What this changes on Apple silicon
When a user returns to a previous conversation, the runner may be able to reuse work already done for its earlier messages. That can reduce time to the first token on a revisit. If memory pressure grows, idle histories can be discarded, and the next revisit may need to process more of the prompt again. The proposal preserves active requests and describes CUDA changes as follow-up work.
Maintainer tests compare selected requests on particular Macs and models. They report faster revisits in some cases, alongside higher memory use. Those results depend on workload and runtime. OpenSourceCoder.com has not run the candidate. If you evaluate it, compare first-token latency and memory pressure during both a cold start and revisits under your usual workload.
See the Ollama repository profile for project information. The earlier rc1 coverage discussed a separate batching change.
EXPLORE THE SOURCE
Projects in this story
GitHub repositories
- ollama/ollama ↗View project details
Primary sources reviewed. No independent hands-on tests.



