A 5.95 GB file can make a 27B-class model downloadable on ordinary hardware. It does not guarantee that the full application will fit, run quickly or preserve the tasks that matter to you.
Bonsai 2 27B is PrismML’s ternary version of Qwen3.8 27B. The official GGUF repository lists a 5.95 GB weights file, an Apache 2.0 license and llama.cpp launch instructions. PrismML positions the model for local reasoning, coding, vision and agent work, but the useful buying question is not simply whether the file downloads. It is whether the complete runtime fits your hardware and passes your workload.
What PrismML actually released
| Item | Published detail | What to verify |
|---|---|---|
| Base | Qwen3.8 27B | Tokenizer, chat template and tool-call behavior match the application. |
| Weight format | Ternary GGUF, 5.95 GB | The chosen llama.cpp build supports the file and intended accelerator. |
| License | Apache 2.0 | Review the model card, bundled notices and any upstream obligations. |
| Vision | Separate multimodal projector | Add the projector’s memory and confirm image tasks independently. |
| Claimed benchmark retention | 98.2 percent of PrismML’s full-precision aggregate | Treat as company-reported until reproduced on a disclosed harness. |
| Claimed peak speed | Up to 143 tokens per second on RTX 5090 | Measure prompt processing and generation separately on your hardware. |
The company says the model scored 83.9 across a 20-benchmark suite compared with 85.4 for its full-precision reference. That is a useful release claim, but an aggregate can hide uneven losses. Coding, tool use, visual reasoning and long-context retrieval can move in different directions even when one combined percentage remains high.
Why 5.95 GB is not the RAM requirement
The downloaded weights are only one part of a running system. The process also needs a KV cache for the active context, temporary compute buffers, tokenizer state, the inference engine, the application and, for images, the multimodal projector. GPU offload can split those components across VRAM and system RAM rather than remove them.
- Short text chat: usually has the smallest cache and is the easiest fit.
- Large documents: grow the KV cache as context expands.
- Parallel users: may require a cache and scheduler capacity for each active sequence.
- Vision prompts: add the projection file, decoded images and preprocessing buffers.
- Agents: accumulate tool output and conversation history unless the application compacts it.
An 8 GB machine might load the weights yet leave too little headroom for a useful context or responsive desktop. A 16 GB machine has more room, but shared-memory devices still divide capacity among the operating system, graphics and inference. Treat the 5.95 GB number as a storage and minimum-loading clue, not a deployment guarantee.
A hardware test matrix that reveals the real fit
| Target | First test | Failure signal |
|---|---|---|
| 8 GB RAM or VRAM | Text-only, short context, one user | Swapping, out-of-memory, or a context too small for the task. |
| 16 GB unified memory | Text, then 8K and 32K context | Sharp time-to-first-token growth or system-wide memory pressure. |
| Consumer NVIDIA GPU | Fixed offload layers and batch size | Fallback to CPU changes the speed comparison. |
| Apple Silicon | Same prompts through the same llama.cpp revision | Thermal or memory pressure makes the sustained rate diverge from the first run. |
| Vision workload | One image, then several resolutions | Projection memory or preprocessing dominates the budget. |
Compare capability at an equal total memory budget
The fairest comparison is not ternary weights against an unconstrained full-precision model. Compare Bonsai 2 with a conventional 4-bit or 5-bit quantization that fits the same total memory after cache and overhead. Use the same llama.cpp commit, prompt format, sampling settings, context length and hardware power mode.
- Select 20 to 50 prompts from the work you will deploy, not only public benchmarks.
- Record cold-load time, time to first token, prompt-processing speed and generation speed.
- Run each prompt several times to expose sampling variance.
- Score exact constraints, tool schemas and code tests rather than judging prose fluency.
- Repeat at the longest context users will actually reach.
- Track peak resident memory, VRAM, system RAM, swap and energy draw.
If the compressed model saves enough memory to enable a larger context, that can improve a real task even when per-token quality falls slightly. The reverse is also possible: a large theoretical context can become unusable if prompt processing is too slow or retrieval accuracy decays.
Vision and agents add separate acceptance tests
The GGUF repository documents a separate vision projection file. Verify image understanding with documents, screenshots, charts and photos that resemble production input. Do not infer visual performance from text benchmarks. For agent use, test malformed tool arguments, repeated calls, refusal to stop, recovery after an error and the growth of conversation history.
Our MiniCPM5-2B local AI guide explains why context size and local fit need separate measurements. The 50 AI GitHub repositories reference includes the runtimes and evaluation tools useful for building a repeatable local test.
The practical verdict
Bonsai 2 27B is notable because a 27B-class multimodal model is available as a roughly 6 GB artifact under a permissive license. That lowers the cost of experimenting locally. It does not settle the harder questions about runtime memory, long-context reliability, tool behavior or sustained speed. The release becomes useful when teams publish the full memory envelope and task-level results, not only the weight-file size.
Primary sources
Checked September 19, 2026. Size, benchmark-retention and speed claims are company-reported. MustHave.ai has not independently reproduced them.
