How programs talk over a network
HTTP is the way two programs talk across a network: one sends a request and the other sends back a response. JSON is the text format they usually use for the data.
Every model API works this way. Once you can read a request and a response, you can use any of them.
By the end of this lesson
Note
I can read and write an HTTP request and a JSON body.
Key words
| Word | What it means | Example |
|---|---|---|
| Request | what a program sends to ask for something | |
| Response | what comes back to answer a request | |
| Header | a named detail about a request or response | content-type |
| Status code | a number that says how the request went | 200 |
| JSON | a text format for data made of keys and values |
The cells students fill in are left blank in this preview.
A request has four parts
The method says what to do: GET reads and POST sends data. The URL says where. Headers add details, such as the type of the data. The body is the data itself.
A model API takes a POST with a JSON body that holds your messages.
Worked example: a request, part by part
| Part | What it holds | In our call |
|---|---|---|
| Method | What to do | POST |
| URL | Where to send it | http://ai:4000/v1/chat/completions |
| Header | The type of the body | content-type: application/json |
| Body | The data, as JSON | model, messages, max_tokens |
The cells students fill in are left blank in this preview.
Print the request a script would send
docker exec -w /home/node/project agent node -e "const body={model:'lab',messages:[{role:'user',content:'Say hello'}],max_tokens:20};console.log('POST /v1/chat/completions');console.log('content-type: application/json');console.log(JSON.stringify(body,null,2))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
JSON is text with keys and values
JSON writes data as text. An object is curly brackets around "key": value pairs. A list is square brackets. Keys are always in double quotes.
Your program turns data into JSON text to send it (JSON.stringify) and turns JSON text back into data to read it (JSON.parse). In Python the same two jobs are json.dumps and json.loads.
A response has a status and a body
The first thing in a response is the status code. A number from 200 to 299 means it worked. A number from 400 to 499 means the request was at fault. A 500 or above means the service was.
Read the status before you read the body.
Worked example: status codes you will meet
| Status | Meaning | What you do |
|---|---|---|
| 200 | It worked | Read the body |
| 400 | The request is wrong | Fix the body |
| 401 | No key, or a wrong one | Check the key |
| 403 | Not allowed | Read the message |
| 429 | Too many requests | Wait, then try again |
| 503 | Not ready | Try again later |
The cells students fill in are left blank in this preview.
Call a model and read the status
docker exec -w /home/node/project agent node -e "const u=(process.env.AI_URL||'http://ai:4000')+'/v1/chat/completions';(async()=>{try{const r=await fetch(u,{method:'POST',headers:{'content-type':'application/json',authorization:'Bearer lab'},body:JSON.stringify({model:'lab',messages:[{role:'user',content:'Say hello in three words.'}],max_tokens:20})});const j=await r.json();if(r.ok===false)throw new Error(r.status+' '+j.error.message);console.log('Status: '+r.status);console.log('Content type: '+r.headers.get('content-type'));console.log('Top level keys: '+Object.keys(j).join(', '));console.log('Reply: '+j.choices[0].message.content)}catch(e){console.log('No answer from the model ('+e.message+'). Recorded sample:');console.log('Status: 200');console.log('Content type: application/json');console.log('Top level keys: id, object, model, choices, usage');console.log('Reply: Hello there, friend.')}})()"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people think status 200 means the answer is right. It only means the request worked. A model can return 200 with a wrong or odd answer. Check the status first, then check the content.
Going further: change the request
Press the Run that prints the request again. In the terminal, press the up arrow to bring the line back and edit it. Add temperature: 0 to the body and run it. Then change max_tokens to 5.
Write down which part of the request you changed each time: the method, the URL, a header or the body.