How a chatbot writes its replies: Free Lesson

A chatbot predicts the next word

A chatbot is an AI program you talk to by typing. It writes each reply by predicting the next word, again and again.

This matters because it explains its strangest habit. A chatbot can sound completely sure and still be wrong.

By the end of this lesson

Note

I can explain how a chatbot writes a reply.

Key words

Word	What it means
Chatbot	an AI program you talk to by typing
Prompt	the words you send to a chatbot
Predict	to guess what comes next
Likely	seen often in the examples
Confident	sounding sure

The cells students fill in are left blank in this preview.

A chatbot writes one word at a time

A chatbot has learned from a huge amount of text. When you send a prompt, it predicts the next word and adds it. Then it predicts the word after that.

It keeps going until the reply is finished. It does not plan the whole answer first.

Worked example: what comes next?

The words so far	Next word	Seen out of 10
After college I like to play	football	5
After college I like to play	games	3
After college I like to play	guitar	2

The cells students fill in are left blank in this preview.

Count the next words yourself

The Python below learns from four short sentences. It counts which words came straight after a word you choose.

Press Run. Then change "play" in the last line to "like" and run it again.

Which words came next?

Code: python

# Four sentences the program learns from.
sentences = [
    "after college I like to play football",
    "after college I like to play games",
    "on Saturday I like to play football",
    "after college I like to eat chips",
]

def next_words(word):
    counts = {}
    for sentence in sentences:
        words = sentence.split()
        for i in range(len(words) - 1):
            if words[i] == word:
                after = words[i + 1]
                if after not in counts:
                    counts[after] = 0
                counts[after] = counts[after] + 1
    return counts

print(next_words("play"))

Ask the lab chatbot

Note

Start the lab, then press the next two Runs. Each asks the lab chatbot to finish a sentence. If your college has no AI switched on, the Run prints a recorded sample instead, clearly labelled.

Ask for the next word

Command

docker exec -w /home/node/project agent node -e "const u=(process.env.AI_URL||'http://ai:4000')+'/v1/chat/completions';const post=(m,t,x)=>fetch(u,{method:'POST',headers:{'content-type':'application/json',authorization:'Bearer lab'},body:JSON.stringify({model:'lab',messages:[{role:'user',content:m}],max_tokens:t,temperature:x})}).then(r=>r.json());post('Finish the sentence with one word and nothing else: After college I like to play',12,0.7).then(j=>console.log('Chatbot: '+j.choices[0].message.content)).catch(e=>console.log('Recorded sample (no chatbot answered): football'))"

Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.

Ask the same thing twice

Command

docker exec -w /home/node/project agent node -e "const u=(process.env.AI_URL||'http://ai:4000')+'/v1/chat/completions';const post=(m,t,x)=>fetch(u,{method:'POST',headers:{'content-type':'application/json',authorization:'Bearer lab'},body:JSON.stringify({model:'lab',messages:[{role:'user',content:m}],max_tokens:t,temperature:x})}).then(r=>r.json());const s=['football','games'];(async()=>{for(let i=0;i<2;i++){let t;try{const j=await post('Finish the sentence with one word and nothing else: After college I like to play',12,1);t='Chatbot: '+j.choices[0].message.content}catch(e){t='Recorded sample (no chatbot answered): '+s[i]}console.log('Try '+(i+1)+': '+t)}})()"

Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.

Sounding sure is not being right

A chatbot picks words that are likely. It does not check that they are true. A wrong sentence can still be a likely one.

So a chatbot can write something false in a calm, confident voice. Sounding sure is not the same as being right.

A common mistake

Warning

Many people think a chatbot looks the answer up, like a search. It does not. It writes likely words from patterns, so check anything that matters before you use it.

Going further: how it works inside

A real chatbot uses the same idea as your counts, on a far bigger scale. It looks at many words before the next one, not only the last.

If you are curious, the undergraduate module "How language models work: transformers from scratch" shows how that works.