Inputs
Example Data Table
| Scenario | Window | Current | Turns | User/Turn | Assistant/Turn | Tool/Turn | Growth | Buffer |
|---|---|---|---|---|---|---|---|---|
| RAG chat with light tools | 128,000 | 8,000 | 18 | 450 | 700 | 60 | 2% | 10% |
| Agent loop with verbose outputs | 32,000 | 6,500 | 22 | 380 | 950 | 140 | 5% | 12% |
| Short planning session | 16,000 | 1,200 | 8 | 220 | 420 | 30 | 0% | 8% |
Formula Used
The calculator models token growth across turns. Each turn has a base token cost:
If the conversation grows by g = 1 + growth% each turn, additional tokens are a geometric series:
additional = per_turn × N (for g = 1)
Total projected tokens:
A safety buffer reduces usable capacity: effective_capacity = context_window × (1 − buffer%). Overflow risk increases as total / effective_capacity approaches or exceeds 1.
How to Use This Calculator
- Enter your model context window and any current tokens already in memory.
- Set planned turns and average token sizes for user and assistant messages.
- Add tool overhead if you call functions or return structured tool data.
- Choose a growth rate if prompts expand due to conversation accumulation.
- Apply a safety buffer to reduce surprises from tokenization variance.
- Submit and review risk, overflow amount, and max safe turns.
- If risk is high, reduce output length, summarize history, or use retrieval.
Context windows and real token budgets
Context limits define how much history travels with each request. A 32,000 window with a 10% buffer yields 28,800 usable tokens, while 128,000 yields 115,200. If the thread already holds 8,000 tokens, only remaining effective capacity can absorb new turns. Using effective capacity avoids truncation when tokenization shifts.
Per turn cost components you can measure
Per turn demand is user input plus assistant output plus tool overhead. Example: 420 user tokens, 780 assistant tokens, and 90 tool tokens equal 1,290 per turn before growth. Response style scales output: concise 0.75x and detailed 1.35x shift budgets by hundreds. Measuring averages from logs improves estimates. Today.
Growth compounding explains sudden failures
Many chats grow because prompts include previous summaries, retrieved passages, and tool results. A 3% growth rate means the 20th turn is about 1.03^19 ≈ 1.75 times larger than the first. The total added tokens follow a geometric series, so small growth can create large late-stage jumps. Modeling compounding is the fastest way to predict overflow risk.
Reading the risk score and overflow amount
Utilization compares projected total tokens to capacity. Low risk typically stays under 70% effective utilization, moderate approaches 90%, and high risk sits near the limit. Critical indicates projected total exceeds effective capacity, and the overflow amount estimates how many tokens must be removed. Max safe turns shows how long the session can run with the same per-turn budget and growth.
Mitigation levers that move the curve
Overflow risk drops quickly when you shrink large outputs and stabilize context. Summarizing the conversation every 8-12 turns caps growth and reduces cumulative tokens. Converting tool outputs to compact JSON fields, trimming duplicate citations, and lowering response style can save 20-40% of per turn tokens. Retrieval with short passages beats pasting full documents.
Production monitoring checklist
In production, monitor median and p95 tokens for both user and assistant messages, plus tool payload sizes. Track retries, since repeated calls multiply context usage. Set alerts when effective utilization crosses 85% so you can shorten answers or summarize early. Keep a reserve budget for system instructions, safety text, and structured outputs to remain consistent.
FAQs
Q1. What does "effective capacity" mean?
Effective capacity is the context window after subtracting your safety buffer. It is the practical limit you plan against, because tokenization, formatting, and tool payloads can fluctuate from run to run.
Q2. How do I estimate current tokens used?
Use your platform's token counter when available, or sample recent messages and scale by conversation length. Include pasted documents, retrieved passages, system instructions, and tool outputs, because they all occupy context.
Q3. Why can overflow happen even below the window limit?
Most systems need room for the model's response and hidden formatting. If you operate at 98-100% of the raw window, small differences can push the next request over the edge and trigger truncation.
Q4. What growth rate should I enter?
Start with 0% for tightly controlled prompts. Use 2-5% when you repeatedly append summaries, citations, or tool results. If you see larger prompts over time in logs, increase the rate until projections match reality.
Q5. How can I reduce per-turn tokens quickly?
Shorten assistant verbosity, cap tool fields, and avoid repeating large excerpts. Replace long pasted content with retrieval links or brief quotes. Summarize earlier turns and keep only the minimum instructions needed.
Q6. Is "max safe turns" exact?
It is a planning estimate based on averages and the growth model. Real conversations vary, so treat it as a conservative guide. If risk is high, reduce turns or budget, then re-check before running.