Model APIs charge by the token
Most model APIs charge by the token, not by the call. A short question and a long essay cost very different amounts.
Estimating the cost before you call is how you avoid a surprise bill. This lesson shows you how.
By the end of this lesson
Note
I can estimate what a call costs before I make it.
Key words
| Word | What it means |
|---|---|
| Token | a small piece of text that is counted and billed |
| Input tokens | the tokens you send in the prompt |
| Output tokens | the tokens the model writes back |
| Price per million | what a million tokens cost |
| Budget | the most you decide to spend |
The cells students fill in are left blank in this preview.
A call bills input and output tokens
You pay for the input tokens you send and the output tokens the model writes. Output usually costs more per token.
A price is quoted per million tokens. The prices in this lesson are invented. Real prices change, so read the provider's page.
Worked example: one call
| Part | Tokens | Price per million, in pence | Cost, in millionths of a penny |
|---|---|---|---|
| Input | 1,000 | 200 | 200,000 |
| Output | 500 | 800 | 400,000 |
| Total | 1,500 | Not applicable | 600,000, which is 0.6 of a penny |
The cells students fill in are left blank in this preview.
Estimate tokens from characters
docker exec -w /home/node/project agent node -e "const text='Explain what an API key is in two sentences.';console.log('Characters: '+text.length);console.log('Estimated tokens: '+Math.ceil(text.length/4))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
You can guess tokens before you call
For English text, one token is about four characters. So a text of 400 characters is about 100 tokens. It is a rough guess, and other languages often need more tokens for the same meaning.
After a call, the response usually reports the real counts.
Call a model and read the tokens
docker exec -w /home/node/project agent node -e "const u=(process.env.AI_URL||'http://ai:4000')+'/v1/chat/completions';const show=(a,b)=>{console.log('Tokens in: '+a+', tokens out: '+b);console.log('Cost at the lesson prices: '+(a*200+b*800)+' millionths of a penny')};(async()=>{try{const r=await fetch(u,{method:'POST',headers:{'content-type':'application/json',authorization:'Bearer lab'},body:JSON.stringify({model:'lab',messages:[{role:'user',content:'Explain what an API key is in one sentence.'}],max_tokens:60})});const j=await r.json();if(r.ok===false)throw new Error(r.status+' '+j.error.message);console.log('Reply: '+j.choices[0].message.content);if(j.usage)show(j.usage.prompt_tokens,j.usage.completion_tokens);else console.log('No token counts reported.')}catch(e){console.log('No answer ('+e.message+'). Recorded sample:');console.log('Reply: An API key is a secret that proves a request is yours.');show(14,16)}})()"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A limit keeps the output small
You control the input by writing short prompts. You control the output with max_tokens, which stops the answer at that many tokens.
In a chat, every new turn sends the earlier messages again. So a long chat costs more with each turn.
Stop when the budget runs out
docker exec -w /home/node/project agent node -e "const calls=[[1000,500],[2000,800],[1500,1200],[3000,2000],[500,100]];let spent=0;const limit=4000000;for(let i=0;i<calls.length;i++){const c=calls[i][0]*200+calls[i][1]*800;if(spent+c>limit){console.log('Call '+(i+1)+' would cost '+c+', so stop. Spent '+spent);break}spent+=c;console.log('Call '+(i+1)+' cost '+c+', spent '+spent)}"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people think every call costs the same. Cost grows with the tokens in and out, and a long chat resends its history. Count tokens, set a limit and decide your budget before a loop runs.