You Think an LLM Is a Faster Monkey. It Is Not a Monkey at All.

Late at night, you open a chat box and type: “Write a monologue in Shakespeare’s voice about a programmer still fixing bugs at 3 a.m.” You hit Enter. Three seconds later, a passage appears on the screen: rhythmic, theatrical, full of thou and hath, even carrying a little rhyme. It was not copied from a book. This passage did not exist before. The model just made it up.

You stare at it for a few seconds and a thought appears. We all grew up hearing the story: put a monkey in front of a typewriter, let it hammer keys until the end of time, and one day it will type the complete works of Shakespeare. Is this machine just an upgraded version of that monkey? Did it compress that absurdly slow process by billions of times, turning “almost impossible” into “one essay every three seconds”?

That intuition is extremely natural. So natural, in fact, that it is almost everyone’s first reaction.

It is also almost entirely wrong.

The mistake is not easy to explain in one sentence, because it quietly merges two questions that look similar but are actually miles apart. In 1941, Borges imagined the Library of Babel: a library containing every book that can be formed from a certain character set, including every masterpiece and infinitely many volumes of pure gibberish. In that library, the question is not “Does Shakespeare exist?” but “How far must you walk before you stumble into him?” The monkey, π, and the machine in front of you that writes in three seconds are three completely different ways of “finding the book.” Put them side by side and you can see what LLMs changed — and what they did not.

Monkeys and LLMs Are Not in the Same Lineage

The “typing monkey” metaphor is much older than computers. The earliest traceable version comes from the French mathematician Émile Borel, in a 1913 book on probability. He imagined monkeys randomly hitting typewriter keys to explain a counterintuitive point: an event with a probability so tiny it can be ignored is not the same thing, mathematically, as an impossible event. The astrophysicist Eddington later used the same image to discuss time scales in statistical mechanics. Borges traced even older shadows of the idea, from Aristotle and Cicero to Pascal and Swift: humans have repeatedly wondered whether arbitrary combinations of letters might accidentally produce meaning. (Wikipedia: Infinite monkey theorem; MacTutor: Borel)

What Borel and Eddington cared about was the sense of time hidden behind the phrase “almost surely.” The standard statement of the infinite monkey theorem is: if a monkey types independently and randomly for an infinite amount of time, it will almost surely produce any given finite text, including the complete works of Shakespeare. “Almost surely” is a precise probabilistic term. It means probability 1, but probability 1 is not the same as logical necessity. More importantly, everything rests on the word “infinite.” The theorem’s entire force depends on infinity. Once you replace that with the lifetime of a real universe, the conclusion changes shape. Countably infinite trials are enough to push the probability to 1; reality does not offer countably infinite chances.

The person who handed language over to probability was someone else. In 1948, Shannon founded information theory; in 1951, he wrote “Prediction and Entropy of Printed English,” the first paper to treat a language as a probabilistic source that emits characters. (Shannon 1951) His question was simple: after seeing the previous letters, how well can you guess the next one? The better you can guess, the more structure the language has and the lower its entropy. That paper planted a seed. The monkey theorem assumes every character is selected uniformly at random; Shannon showed that real language is not like that at all. Characters strongly constrain one another.

Seventy years later, LLMs are playing Shannon’s guessing game, only at the limit. Back then, humans guessed the next letter from intuition. Today, a network with hundreds of billions of parameters consumes half the internet’s text and estimates an entire probability distribution over the next token. From this perspective, an LLM is not an upgraded monkey. It belongs to a different lineage altogether. The monkey lineage asks, “What can infinite time accidentally produce?” The Shannon lineage asks, “How much predictable structure is there in language?” Our confusion today mostly comes from smashing those two lineages together under the sentence “AI can write Shakespeare.”

You Are Actually Asking Two Questions

You are really asking two questions. They just look similar enough that it is easy to mash them together.

One question is: how hard is it for a machine to write a new sentence that reads like Shakespeare? The other is: how hard is it for that machine to reproduce the full text of Hamlet, character for character? LLMs moved the first problem from “nearly impossible” to “trivial.” They did not help with the second problem at all.

Start with the second question. Suppose a keyboard has 27 keys and a book has n characters. If a monkey types blindly, the probability of typing exactly that book in one attempt is 1/27^n. If the book has one million characters, the probability is 1/27^(1,000,000). How small is that number? In 2024, two mathematicians in Sydney calculated that even if you recruited the roughly 200,000 chimpanzees alive today, had each type one key per second, and kept them typing until the heat death of the universe, they still would not get close to Shakespeare’s 850,000-word collected works. A single chimp typing for a lifetime of about thirty years would have only a 5% chance of producing the word “bananas”; the chance of typing “I chimp, therefore I am” is around 1/10^30. (Woodcock & Falletta, 2024; BBC)

An even harsher version appears in Kittel and Kroemer’s thermodynamics textbook. Turn every proton in the observable universe, roughly 10^80 of them, into a typing monkey from the Big Bang to the end of the universe, and to reach even a success probability of 1/10^500 would require another 360,000 orders of magnitude of time. Their conclusion is cold: the probability of Hamlet is zero for any operational purpose. (Wikipedia)

Here is the often-missed turn. The infinite monkey theorem says the probability is 1, and that is correct. But it is an asymptotic conclusion propped up by the word “infinite.” Pull time back into the length of a real universe and that 1 collapses back into nearly zero. That is why the 2024 paper calls the theorem “correct but misleading.”

Now return to the first question — the one you actually care about. LLMs really do make it easy to write something Shakespeare-like. But they do not do this by making 1/27^n larger. They do it by not wandering around the 27^n space at all. They take another route.

One thing must be made explicit here, otherwise the rest turns into hallucination. Reproducing a specified book word for word has a ceiling no one can pass. Information theory says that if a text contains H bits of information, then any model — monkey or GPT — can assign it probability at most 2^(-H). H is the complexity of that text itself; it does not depend on which model you use. What an LLM can do is make its estimated probability approach that upper bound. It cannot lift the bound itself. So “typing out an entire specified book it has not memorized” remains an unimaginably tiny probability for LLMs too. That was never where their strength lived.

LLMs Change the Distribution, Not the Ceiling

So what did LLMs change? The clearest way to see it is to place one beside a monkey.

Every time the monkey hits a key, all 27 keys have equal odds. In its world there is no “context”; the millionth character depends on luck just as much as the first. The first thing a language model does is break the assumption of equality. After reading the preceding text, it produces a probability distribution over what the next token should be. After “to be, or not to,” the probability of “be” rises sharply, while the probability of “xylophone” falls close to zero. Generating a passage is just repeating that operation: predict a distribution, sample a token, append it, then predict the next one. (Raschka: next-token prediction)

You can even adjust how steep that distribution is with a knob. The model’s final layer emits raw scores called logits, which are turned into probabilities by softmax. Temperature is the knob: lower the temperature and the distribution sharpens, so the model nearly always picks the highest-probability token — stable but dull. Raise the temperature and the distribution flattens, allowing low-probability tokens to appear — lively but more chaotic. The model’s weights do not change at all. By turning this one knob, you can slide it from “recorder” to “nonsense generator.” You are not changing its knowledge. You are changing how tightly probability mass is piled up. (IBM: LLM Temperature)

Why does it know “be” should be more likely than “xylophone”? Because language itself is not that random. In a famous 1951 experiment, Shannon asked people to guess the next letter in English and, using human linguistic intuition, estimated English entropy at roughly 0.6 to 1.3 bits per character. (Shannon 1951) If 27 characters were truly selected uniformly, entropy would be log2(27), about 4.75 bits per character. Those missing three or four bits are the regularities inside language: articles are likely to be followed by nouns; q is almost always followed by u. The monkey uses none of this and brute-forces its way through a 4.75-bit space. The LLM has absorbed these regularities into its parameters; in practice, it is walking through a space just above one bit per step.

There is an even prettier way to say this, from DeepMind’s 2023 paper “Language Modeling Is Compression.” A good predictive model and a good lossless compressor are mathematically the same object; arithmetic coding lets you convert one into the other. The better the model predicts the next symbol, the fewer bits you need to encode it. They found that Chinchilla 70B compressed images and audio better than specialized compressors. (Delétang et al., 2023) So you can understand what LLMs do this way: they do not change the information content of Shakespeare’s book. They find an encoding in which “Shakespeare-like text” gets short codes and gibberish gets long codes. The search space is still enormous, but probability mass has been redistributed.

This mechanism brings two problems it cannot shake off. The first is error accumulation. During training, the model sees real preceding context; during generation, it must continue from text it just wrote itself. This gap is called exposure bias. Once a step goes wrong, the error can roll forward through the sequence. (Exposure bias overview) So the longer the target text is, the harder exact word-for-word hitting becomes. Your intuition is right about that direction. It is worth adding that newer research suggests the decay is not a simple exponential: in long contexts, models can sometimes pull themselves back midstream. But the broad rule remains: the longer the target, the harder exactness becomes.

The second problem is the entanglement of memorization and generalization. LLMs can sometimes reproduce training text exactly. Carlini et al. quantified this in 2023: the probability of extraction rises log-linearly with three factors — model size, how many times the text appeared in training, and how long the prompt context is. (Carlini et al., 2023) Note that this is not monkey-style luck. The model reproduces the text because it memorized it, not because it got lucky wandering through 27^n.

π, Parrots, and a Lawsuit Still in Progress

Following the thread of memorization, three real-world comparisons help pin down where LLMs sit on the spectrum.

First: π. People often say the decimal expansion of pi contains the complete works of Shakespeare, your birthday, and your ID number. For that to be true, π would need to be a normal number: every fixed-length digit string would appear with the uniform frequency it should. If π is normal, every finite string will eventually appear. The problem is that no one has proved π is normal. Even the much weaker claim that π contains infinitely many 7s remains unproved. (Scientific American; Normal number) More importantly, π is a completely deterministic sequence. Every digit is already fixed. There is no randomness in it. When we say “π may contain Shakespeare,” we are using a statistical analogy — “it looks random enough” — not describing a real random process like monkeys typing. Mixing those two up is a common flaw in popular explanations.

Second is the debate over how to characterize LLMs themselves. In 2021, Bender, Gebru, and colleagues introduced the phrase “stochastic parrots,” arguing that LLMs are systems that stitch linguistic forms together according to probabilistic information without touching meaning. (Stochastic Parrots) The label is sharp and has been strongly contested. What is interesting is that both sides share one factual foundation: an LLM’s behavior is driven by a probability distribution. The disagreement is over whether that distribution amounts to “understanding.” That is precisely why the distribution-shaping discussed here is the part both sides accept.

The third comparison is the most concrete: an ongoing lawsuit. In late 2023, The New York Times sued OpenAI. One exhibit, Exhibit J, lists 100 examples where giving GPT-4 the first half of a Times article allegedly caused it to emit large stretches of the original article verbatim. (NYT Complaint, Exhibit J) OpenAI’s defense was that such verbatim reproduction is a “rare bug.” In our framework, there is no contradiction. The texts a model can reproduce are precisely the texts that appeared frequently in training and were solidly memorized — exactly matching the pattern Carlini quantified. It did not stumble into the Times through 27^n. It recited something it had memorized. In other words, the only texts an LLM can truly “hit verbatim” are the small set it already remembers. For any specified long text it has not memorized, the information-theoretic ceiling still presses down.

So What Did It Actually Improve?

Return to the original question. Did LLMs increase the probability of typing Shakespeare?

Now you have a ruler sharp enough to split the question. Whenever someone asks this, ask back: do you mean “write something new that sounds like Shakespeare,” or “reproduce a specified book exactly, word for word”? The answers point in opposite directions. For the first, LLMs raise the probability from astronomically tiny to something you can get in the blink of an eye. For the second, if the model has not memorized that book, the probability remains unimaginably small, because an information-theoretic ceiling cannot be lifted.

The second ruler is to distinguish “reshaping a distribution” from “breaking a lower bound.” Everything LLMs do belongs to the former. They do not make any arbitrary text easier to assemble. They move probability mass away from infinitely many strings of gibberish and pile it onto text that looks plausible. A book’s information content is its own property; no model, however powerful, makes it smaller. Once you understand this, you will not be fooled by claims like “if the context window reaches one million tokens, won’t it reproduce long texts exactly?” The limit is not the context window. The limit is the information itself.

Third, keep π, monkeys, and LLMs separate. Monkeys are a true random process: they rely on infinite time, and in reality they will not get there. π is a fully deterministic sequence: whether it contains Shakespeare depends on an unproved mathematical conjecture. An LLM is neither a random process nor a fixed sequence; it is a probability distribution shaped by massive amounts of data and filled with strong bias. The only thing these three share is that people use them to answer “Can a machine write Shakespeare?” But their versions of “can” and “cannot” live on completely different levels.

So the next time someone says, “AI can write Shakespeare now — I guess the infinite monkey theorem was right,” you may want to laugh. The monkey story never worked in reality. And LLMs can write Shakespeare precisely because they are not doing what monkeys do. They do not rely on luck. They rearrange probability.