Numbers Are Just Text To It

The invoice that did not add up

You paste four invoice totals into a chat and ask for the sum.

£1,284.50 · £963.00 · £2,417.25 · £188.75

The answer comes back instantly, formatted neatly, with a short note about what the total covers.

It is wrong by about forty pounds.

The strange part is that the same model, in the same conversation, will happily write you a working spreadsheet formula, explain compound interest correctly, and debug someone else's Python. So what happened here?

There is nowhere to put the number

When you type 4817, the model does not receive the quantity four thousand eight hundred and seventeen.

It receives text, chopped into chunks. Perhaps 48 and 17. Perhaps 4, 81, 7. The split depends on the tokeniser, and it is not designed around mathematics.

There is no register inside the model holding a value. No place where 4817 sits one greater than 4816. The digits are symbols, handled with exactly the same machinery as the word "invoice".

So what is it actually doing?

Predicting.

It has seen an enormous amount of text containing arithmetic, so it has learned what sums tend to look like. Small ones it usually gets right, because they appeared often enough to be memorised outright. Two plus two is a phrase it has read a million times.

Larger or less common ones get reconstructed from the pattern of what a plausible answer looks like. Right length. Right magnitude. Often right for the first few digits, because those are the most predictable.

And then wrong somewhere in the middle, where the pattern runs out.

Why this is worse than a normal error

A calculator that breaks gives you nonsense. You notice.

This does not give you nonsense. It gives you 14,434.40 when the answer was 14,334.40.

Right shape. Right size. Right number of decimal places. Confidently formatted, possibly with a short explanation of the method that is itself entirely correct.

The explanation being right is the trap. It describes a calculation that was never performed.

The letters test

If you want to feel the mechanism rather than take it on faith, ask a model how many times the letter r appears in strawberry.

Models have gotten better at this specific question, because it became famous enough to train on directly. But try it with a longer or stranger word and the failure comes straight back.

It is the same cause. The word arrives as chunks, not as a sequence of letters, so counting letters means counting something the model cannot see.

What this predicts

You can now say in advance where arithmetic will break:

• Small, common sums are usually fine, because they were memorised
• Long numbers get less reliable as they get longer
• Chains of operations compound, because each step inherits the last error
• Counting anything inside text is shaky for the same reason
• Percentages of awkward numbers are a classic failure

None of this makes the model useless with numbers. It makes the chat window the wrong place to do the sum.