The Goldilocks Zone: 1.5B tokens for $400/m
Model strategy has a habitable band. Most teams camp at one of the extremes.
All-frontier, all the time
Flagship intelligence on every call — and unit economics that collapse the moment you build loops or scale up delegation.
Multi-family, OSS models
Ensembles of OSS models hit frontier-level quality at loop-friendly prices — with true frontier models held back for surgical moments.
Self-hosting for everything
Hits an intelligence ceiling. The model's blind spots become your blind spots — and a single point of failure.
Chinese Models, American Providers
Open weight models are simple, verifiable math formulas — not software.
Anyone can download the weights and run them anywhere. Nothing about inference has to touch the lab that trained the model.
The provider is the risk surface — not the model.
We don't trust Chinese-hosted endpoints, so we don't use them. Distrusting a host doesn't mean discarding the model.
US providers with zero data retention — no exceptions.
Every token we run goes through American providers with ZDR policies: prompts and outputs aren't stored, logged, or used for training.
Two Providers, Full Model Redundancy
From the creators of PyTorch, NVIDIA-backed, and publicly praised by Jensen Huang. Raised a $1.5B Series D in July 2026 at a $17.5B valuation on $1B+ annualized revenue.
Our default endpoint for every family we run.
Newer, and usually beats Fireworks on price. $107M Series B raised in May 2026, new Toronto data center opened in August 2026. Latency is less consistent and we've hit the occasional small bug.
Crucially: it hosts the same models we already run.
Same weights on both sides means failover is a routing change — not a prompt-engineering project.
IRL: the GLM-5.2 incident
Fireworks wobbles
Fireworks starts having trouble serving GLM-5.2, our most popular endpoint, and a 90 minute outage ensues.
Flip the switch
Traffic reroutes to DeepInfra, which hosts the same GLM-5.2. Same weights, same prompts, same harness.
Nothing else changes
Work carries on without incident — near-100% behavioral consistency, because we never changed the model.
Redundancy across providers. Consistency within the model. That's the whole design.
Keep the frontier — Subscribe, don't meter
The Key
Run Anthropic's flagship models exclusively through flat $25–$125/m subscription tiers on a business team agreement, never API rates. Use Opus & Fable as surgeons' tools; break glass in case of emergency. Extremely useful — deliberately rare.
- Reserve it for absolute-flagship reasoning.
- Leverage it for complex debugging and long-form writing.
- Make it your default for cheap deep research.
- Build loops on frontier models.
- Assume planning must start at the absolute frontier.
- Pay API rates for capacity a subscription already covers.
The ground shifted — We rebuilt around it.
of our day-to-day inference now runs on open-weight models.
cheaper than Anthropic models running on Claude Code.
production model families (e.g. DeepSeek, GLM, Qwen, Kimi, Nvidia, MiniMax).
The frontier labs still own the very top end — the game is deciding where those tokens are actually worth it.
Auxiliary Models: delegate case by case
prompt + screenshot"] Q["Qwen 3.8 Max (Vision)
describes the image in detail"] G["GLM-5.2 (Default)
text-only default"] A["DeepSeek Flash (Worker)
executes the task"] T --> G T --> Q Q --> G G <--> A classDef box fill:#141414,stroke:#2a2a2a,stroke-width:1.4px,color:#ece8e3; classDef cold fill:#141414,stroke:#7fa5c4,stroke-width:1.6px,color:#7fa5c4; classDef gold fill:#141414,stroke:#c676e4,stroke-width:1.6px,color:#c676e4; classDef out fill:transparent,stroke:#c2825c,stroke-width:1.6px,color:#ece8e3; classDef green fill:#141414,stroke:#5fc770,stroke-width:1.6px,color:#5fc770; class T box class Q cold class G gold class A green
Pick the default on text strength alone.
Vision inference only as neeeded.
Swap per gap without touching the harness.
MoA: Don't bet on a single model family
reference model"] G["GLM 5.2
reference model"] K["Kimi K3
reference model"] A["Qwen 3.8 Max
aggregator model"] O["Frontier-level
answer"] P --> D P --> G P --> K D --> A G --> A K --> A A ==> O classDef box fill:#141414,stroke:#2a2a2a,stroke-width:1.4px,color:#ece8e3; classDef cold fill:#141414,stroke:#7fa5c4,stroke-width:1.6px,color:#7fa5c4; classDef gold fill:#141414,stroke:#c676e4,stroke-width:1.6px,color:#c676e4; classDef green fill:#141414,stroke:#5fc770,stroke-width:1.6px,color:#5fc770; class P box class D,G,K cold class O green class A gold
Share the prompt across multiple models.
Aggregate each model response.
Return a final answer with frontier quality.
Planner → Workers → Reviewers
Stop asking one model to do everything. Route each phase to the cheapest model that can own it.
Planner 30%
Use a top OSS model or MoA to turn the task into an explicit plan before any execution starts. This is where quality gets decided.
Workers 60%
The plan is delegated to one or more DeepSeek Flash-class models that grind through execution. This is where loop volume lives.
Reviewers 10%
High-intelligence, but lower context window models grade the output — ideally several independent reviewers across several families.
The Goldilocks Playbook — and what's next
- 01
Open weights on trusted hosts — Fireworks AI primary + DeepInfra fallback.
- 02
Primary + fallback, same models — Fireworks first, DeepInfra right behind it.
- 03
Keep the frontier subscription — use for hard calls and edge cases, not loops.
- 04
Diversify across model families — balance your quality and spend.
Trial Fireworks Nexus as a general harness endpoint
Fireworks new model router offers a new savings strategy to route requests efficiently based on your preferences (balanced, max-savings, and max-intelligence). Fireworks reports 54% overall AI cost savings with their early access customers.
Not too hot, not too cold — frontier results at open-source prices.