How a language model picks the next word

A new member of staff asks a chatbot when the shop opens on Sundays. It answers at once, in a polite and certain sentence. The hours it gives are wrong.

Nothing broke. The model did exactly what it was built to do. This lesson shows what that is, with a model small enough to work by hand.

A model does not read letters the way you do. It cuts text into tokens first. A large model's tokens are common words and pieces of rarer words, often with the space attached.

Our tiny model keeps it simple. It makes everything lower case, treats a full stop as a token, and splits on spaces.

sentence = "The shop opens at nine. The shop closes at six."
# Lower case, and make the full stop a token of its own.
tokens = sentence.lower().replace(".", " .").split()
print(tokens)
print(len(tokens), "tokens")
# Add a sentence of your own. How many tokens now?

A bigram model learns one thing: which token tends to follow which. Go through the training text and count every pair of neighbours.

To get the probabilities after a word, divide each count by how often that word was followed by anything. bigram.py does this for the word "the" in a short text about the shop.

docker exec jupyter python3 steps/bigram.py

# Count what follows one word in a short text.
text = "the bus is late . the bus is full . the train is late ."
words = text.split()
word = "is"
counts = {}
total = 0
for n in range(len(words) - 1):
    if words[n] == word:
        after = words[n + 1]
        if after in counts:
            counts[after] = counts[after] + 1
        else:
            counts[after] = 1
        total = total + 1
for after in counts:
    print(word, after, counts[after], "of", total)
# Change word to "the". Which word is most likely next?

To write, the model picks a next token, often at random weighted by the probabilities. Then it does it again from the new token, and again.

Nothing in that loop checks whether the sentence is true. A large model is a neural network that looks at thousands of earlier tokens, not one. It is trained on a huge amount of text, but it is still choosing likely next tokens.

docker exec jupyter python3 steps/generate.py

A model can only echo the patterns in its training text. Our text says "the shop" three times, so the model reaches for "shop" whatever the topic.

A large model does the same with people. If its text mostly pairs one job with one sex, or one country with one crime, it repeats that. This is bias from data, and it is often invisible in a single answer.

Treat an answer as a draft from a fast writer who never checks. Ask where a fact comes from, then open that source yourself. A source that does not exist is a warning.

Work any number out again. Ask the same question a second way and compare. Never paste in private data, such as customers' names, unless your organisation allows that tool for it.

Copy generate.py's text into a notebook cell and make the model look at two earlier words instead of one. Count which word follows each pair, such as "the shop".

Generate four sentences with the same seed. Are they closer to the training text? Explain why a model that looks further back writes better text but can still make things up.