Decision model · v2.1
Manchego · MLX 4-bit
Smaller, and honest about it.
The smallest Manchego build: 2.4 GB for memory-constrained Apple devices, with its accuracy cost measured and stated.
- Runtime
- MLX · mlx-lm ≥ 0.31.3
- Precision
- 4-bit, group size 64
- Size
- 2.4 GB
- Cost
- −7.5 points on task rows
- License
- Apache-2.0
- Updated
- 23 Sep 2026
Overview
Manchego v2.1 converted with mlx_lm at 4 bits for Apple silicon: a quarter of the bf16 footprint, with the same interface.
The cost is measured, not assumed. Against bf16, four bits lose six of the 80 held-out task rows (7.5 points), as they did for v2, and one of the 200 real-text rows. Choose it when memory decides; otherwise use the 8-bit build.
Fig. 01 · Formats
Size against fidelity
Accuracy on the same held-out rows, one prompt per forward pass. The tick on each track is the untrained base.
View as table
| Format | Size | 80 task rows (CE) | 200 real-text rows (CE) |
|---|---|---|---|
| bf16 · Transformers | 9.3 GB | 0.950 (0.177) | 0.905 (0.280) |
| MLX 8-bit | 4.5 GB | 0.938 (0.180) | 0.910 (0.279) |
| MLX 4-bit | 2.4 GB | 0.875 (0.288) | 0.900 (0.290) |
| Untrained base, bf16 | — | 0.650 | 0.690 |
Usage
from mlx_lm import load
import mlx.core as mx
model, tok = load("oraculumai/Manchego-MLX-4bit")
prompt = "Is this message spam?\n\nState:\nWIN a free cruise! Reply YES now.\n\nOptions:\nA = yes\nB = no\n\nReply with only the letter of the best option."
letters = "AB" # the letters you offered
text = tok.apply_chat_template([{"role": "user", "content": prompt}], tokenize=False,
add_generation_prompt=True, enable_thinking=False)
logits = model(mx.array([tok.encode(text, add_special_tokens=False)]))[0, -1]
cand = [tok.encode(L, add_special_tokens=False)[0] for L in letters]
probs = mx.softmax(logits[mx.array(cand)].astype(mx.float32))
print(dict(zip(letters, probs.tolist())))- Text-only conversion. Load it with
mlx_lm, notmlx-vlm: the vision tower and the multi-token-prediction head are not included. - Score one prompt at a time when exact reproducibility matters. Padded batches move probabilities slightly at low precision.
Read the main card first
Results, training data and the full list of weaknesses are on the main Manchego card. These are small format checks, not a guarantee for a new application: test the decision path you intend to deploy.
Disclosures
- JevBench-informed. The training families of v2 and v2.1 were designed from an earlier version’s per-family JevBench hard-tier scores (the published specification only, no item text). Read its JevBench result with that in mind.
- Phrase audit. An early audit used Jev, a hosted decision service, to flag phrases in two data pools; 86 phrases were dropped. Jev never produced a training label.
About this page
Summarized from the public model card and its published files. Repository updated 23 Sep 2026; reviewed 26 Sep 2026. The Hugging Face card is the source of truth for licenses and current details.
Upstream: Oraculum / Manchego