It is not true that a pretrained transformer is reducible to a Markov chain in the same way that a computation is reducible to a turing machine. While any computation could be achieved by a turing machine (given arbitary time), not every pretrained transformer could be described by a (sufficiently complicated) Markov chain of arbitrary length.

One reason is the attention mechanism, which allows a transformer to weight some tokens in a sequence as differently than others.

But the biggest difference is that a Markov chain can only consider an entire state at once, and produce the next state sequentially, while transformers can consider many parts of a sequence as individual states. Transformers are also capable of autoregression (they’re basically just RNNs+attention heads), while Markov chains are not - new states transform old states, and do not simply append.

Now, if you take the attention mechanism out of the transformer, you’re basically just left with an RNN, and if you then take the autoregression out of the RNN, you’ve got a Markov chain in the limit, probably. So you could argue a Markov chain is in the limit of an LLM (kind of obvious, since you should expect an LLM trained on a Markov chain to predict it totally), but you could never argue an LLM can be produced in the limit of a Markov chain (you can train any arbitrarily large Markov chain and it will never get anywhere close to an LLM).

Another point of anecdata is that I’ve read that vibe coders say that non-reasoning models lead to better results for coding tasks because they are faster and they tend to hallucinate less because they don’t pollute with automated CoT. I’ve seen people recommend Deepseek V3 03/2025 release (with deep think turned off) over R1 for that reason.

my money is on the higher hallucination rate being a result of the data being polluted with synthetic information. I think its model collapse

But that is effectively what happening with RLMs and refeed. LLMs have statistical weights between model and inputs. For example RAG models will add higher weights to the text retrieved from your source documents. RLM reasoning is a fully automated CoT prompting technique. You don’t provide the chain, you don’t ask the LLM to create the chain, it just does it all the time for everything. Meaning the inputs becomes more polluted with generated text which reinforces the existing biases in the model.

For example if we take the em dash issue, the idea is that LLMs already generate more em dashes than exist in human written text. Let’s say turn 1 you get an output with em dashes. On Turn 2 this is fed back into the machine which reinforces that over indexing on em dashes in your prompt. This means turn 2’s output is going to potentially have more em dashes, because the input on turn 2 contained output from turn 1 that had more em dashes than normal. Your input over time end up accumulating the model’s biases through the history. The shorter your inputs on each turn and the longer the conversation the faster the conversation input converges on being mostly LLM generated text.

When you do this with an RLM, you have even more output being added to the input automatically with a CoT prompt. Meaning that any model biases accumulate in the input even faster.

Another reason I suspect the CoT refeed vs training data pollution is that GPT-4.5 which is the latest (Feb 2025) non-reasoning model seems to have a lower hallucination rate on SimpleQA than o1. If the training data were the issue we’d see rates closer to o3/o4.

The other big difference between o1 and o3 and o4 that may explain the higher rate of hallucinations is that the o1’s reasoning is not user accessible, and it’s purposefully trained to not have safe guards on reasoning. Where o3 and o4 have public reasoning and reasoning safeguards. I think safeguards may be a significant source of hallucination because they change prompt intent, encoding and output. So on a non-o1 model that safeguard process is happening twice per turn once for reasoning and once for output, then being accumulated into the next turn’s input. On an o1 model that’s happening once per turn only for output and then being accumulated.

From my other comment about o1 and o3/o4 potential issues:

The other big difference between o1 and o3 and o4 that may explain the higher rate of hallucinations is that the o1’s reasoning is not user accessible, and it’s purposefully trained to not have safe guards on reasoning. Where o3 and o4 have public reasoning and reasoning safeguards. I think safeguards may be a significant source of hallucination because they change prompt intent, encoding and output. So on a non-o1 model that safeguard process is happening twice per turn once for reasoning and once for output, then being accumulated into the input. On an o1 model that’s happening once per turn only for output and then being accumulated.

It happens faster to images because of the way LLMs work. LLM’s work on “tokens”, a token for text is typically a character, fragment of a word, a word, a fragment of a sentence. With language it’s much easier to encode meaning and be more precise because that’s what language already does. The reason NLP is/was difficult is because language is not algorithmically consistent, it evolves, and rules are constantly broken. For example Kai Cenat is credited with more contributions to the English language than the vast majority of people because children decided to talk like him. Point being is that language does the heavy lifting in terms of encoding a string of characters into something meaningful.

With images, it’s a whole different ball game. Image tokenizers often work in several different ways, there are two types of token hard and soft. Hard tokens for example could be the regions of the image, part could be the colors, the alpha channels.

Hard tokens are also the visual encoders of meaning so a chair, table, or car could be a hard tokens based on their bounding boxes. These tokenization techniques are based in a lot of other types of machine learning.

Note that these tokens often overlap in practice and consume regions of other tokens, however as “hard” tokens they are considered distinct entities, and this is where the trouble starts esp. for image generation (that’s roughly why a lot of AI did and still does things like draw extra fingers).

The next type of tokens are soft tokens, and they’re a bit harder to explain, but basically the idea is that soft tokens are encoded by detecting continual statistical distributions within images. It’s a very abstract way of reading an image. Here’s where the trouble compounds.

So now when we’re writing an image, what do we write the image with? You guessed it. Tokens. The reason that those AI drift time lapses exist is because LLMs are statistical and not “functional”. They don’t have the mathematical concept of “identity”. Otherwise they’d try to recreate the same image by copying the data in the exact tokens (or just copy the image itself) , instead they try to regenerate the image by generating new tokens with the same attributes that it read from the image.

To illustrate this lets say an image contains a blue car and the AI can only tokenize it as blue car. Asking an LLM to run an identity function on that image will result in a different car because the resolution of the token is only like 2 dimensions “blue” and “car” which roughly means it will output the average “blue” “car” from its training data. Now with human made things it’s actually a lot easier. There’s a finite variation of cars. However there’s an infinite variation of things that can happen to a car. So an AI theoretically can run an identity function off of a particular make/model/year of a vehicle but if the paint is scratched or the paint job is unique it will start to introduce drift there’s also other sources of drift like camera angle etc. With natural objects this becomes a whole different ball game because of the level of variation, this complexity compounds with scenes.

So identity functions on text are extremely easy in comparison for example:

This works because the tokens are simpler and there is less of a loss of “resolution” from the text to the tokenize form. E.g. word “Poopy” is token “poopy”. But once you get into interpreting an image, and re-encoding those interpretations onto a new canvas it becomes much more difficult. e.g. image of “Dwayne the Rock Johnson” is most likely a series of tokens like “buff man”, “bad actor”, etc.

This is a rough explanation because there’s a lot of voodoo, and I’m more of a Software Engineer than I am a statistics/data guy so I approach the alchemy a little bit from an alchemical standpoint.