It Was Never Hallucination. You Do The Same Thing Every Day.

@solopribuilds asked on X this week why nobody talks about AI hallucinating anymore. It is a fair question, and I think the honest answer is that we were using the wrong word from the start.

Pri @solopribuilds · Jul 31

Nobody talks about AI hallucinating anymore. Why?

Hallucination means perceiving something that is not there. That is not the failure mode. What these systems do is closer to confabulation, which is a different thing entirely. You have a gap, you fill the gap with something plausible, and then you believe the fill, because from the inside there is nothing that marks it as different from a real memory.

I want you to test that on yourself before you decide it is only a machine problem, because I think your own head is the better demonstration.

Pick a moment from five years ago. Something ordinary, not a wedding or a funeral, just a regular Tuesday you happen to remember. Now describe the room. What was on the walls, what you were wearing, who spoke first, and what they actually said word for word.

You just produced most of that. You did not retrieve it.

Your brain held a rough version of the information and "generated" the pieces to fit your idea, and the generated details arrived carrying exactly as much confidence as the real ones. There was no flag on them. There is never a flag on them.

I believe we do this constantly, and the only reason it does not cause you more trouble is that almost none of your reconstructions ever get checked against anything.

That is the part that has changed, and it changed for your business specifically.

Somebody Built This in 1988

The part which changed how I think about all of this is that reconstruction was never a defect. It was not a bug we stumbled into. It was a design goal, forty years ago, and the man who wrote it down was trying to model us.

In 1988, at NASA Ames, Pentti Kanerva published a concept called Sparse Distributed Memory. Ordinary computer memory needs an exact address: you hand it a number, and it hands you back exactly what you put there. Kanerva built a memory you address by content.

You hand it a pattern, and it finds every piece of data sitting within a certain distance of that pattern. It sums up what all of those neighbors are holding and shapes that sum into an answer.

Read that operation again, because it is the entire point of this article.

The information the system returns is assembled from many neighbors at once. The word which comes out might be a word that was never actually written in the first place. Kanerva was not apologizing for that behavior. He reproduced it. He noticed that human long-term memory does the exact same thing, and he wanted the math for it.

So, the machine that reconstructs a plausible answer from whatever sits nearby is not a 2020s accident. It is a 1988 architecture, built deliberately, as a model of your own head. We have spent the last three years calling the output a malfunction, and the man who formalized the mechanism called it the point.

The Reconstruction Used to Be Private

For most of the time I have been doing this kind of work, a confabulated memory remained insignificant. You misremembered a restaurant and told a friend the wrong thing about it, and the friend either went anyway or did not. Nobody built a business on your recollection.

Now the reconstruction is the answer box, and it is sitting on top of your category.

In this example you can see that the fundamentals I've followed over the last two weeks have successfully routed the agent to what Google currently defines as authoritative sources for who Ryan Lenk is. This was performed in an incognito browser, not logged in, behind a VPN.

Somebody asks an engine about your company. There is a version of an answer available, assembled out of whatever the model can reach, and the gaps in that version get filled the same way yours do. The output arrives clean, confident, formatted, and sitting at the top of the page where the answer is supposed to be. Nothing in it is labeled as untrue. And the person reading it takes the first two or three sentences and stops.

That is the thing I keep trying to get people to understand, and it is not "AI makes stuff up." Everybody knows that and nobody does anything about it. The part that I believe matters is that the information comes from whatever is nearest, and for most businesses, nothing authoritative is anywhere near.

If you have not put a clear answer about your business where a machine can reach it, you have not opted out of the process; you have just left your authority to somebody else.

What It Looks Like When It Goes the Other Way

I do this work for Krafty Bandit, a small handmade shop, and I will give you my own numbers rather than someone else's.

One page on that site was cited 104 times between June 10 and July 23. One page, carrying most of the property's citations for that window. The engine is Microsoft Copilot and its partner surfaces, which is the only one that reports anything back to me, so I genuinely cannot tell you what the others are doing with that page.

However, I've come up with a process which anyone can perform to get a general read. Open an incognito browser, and ask 10 questions to each AI platform about your business. See what sources this platform is pulling from. If you're cited, great. This means you have tangible data to start tracking KPIs for your business. You can begin to track where your largest sources of authority are, and how to leverage that authority in an efficient and profitable manner.

Krafty Bandit has a Domain Rating of 4.9. That page has zero external backlinks that Ahrefs can find. Nothing about this is an authority story. It is not a big site, it did not buy its way in, and there is no link-based explanation available to me at all.

What the page has is a clear answer to a specific question, written plainly, in one place, where a machine could reach it. Add on basic SEO fundamentals like structured data, and you have a perfect storm for AI citation.

Here is what I've noticed people rarely put in their case studies. Thirteen more citations landed between July 25 and July 28, and they sent exactly zero visits. What I'm now searching for is something that is NOT measurable without a real life case study with real human beings.

"What made you click that link?"

The Library Exercise

I want you to think about what that transaction actually is, because I do not think your industry has priced it yet.

You drive out to the library because you need the answer to exactly one thing. You check out the book, you read the prologue, and you put it back on the shelf. The library counts your visit. The author gets nothing.

That is your trade now. You can be the source of the answer and receive none of the traffic for having been it. Being cited is not being visited, and those two things have come apart while most of your industry was still optimizing for the second one.

I would rather you hear that from somebody whose own numbers say it than from somebody selling you a package.

Why Did the Conversation Stop?

Partly, I believe, the systems have become better at the obvious errors, the invented court cases and the fake citations that made such good screenshots.

Mostly, I think, we stopped noticing it. The reconstruction by AI agents has become the interface. When the data is smooth enough and confident enough and it appears exactly where an answer belongs, you stop auditing it, in the same way you stopped auditing your own account of that Tuesday five years ago.

I believe it is not laziness. It is that nothing in the output you received after your query invites an audit.

You are not going to fix the mechanism, and I am not going to sell you a way to. It is not a bug; it is what the answer engine does, and it is what I believe you and I do every day.

You can run a free AI visibility scan for your website to see where your gaps are.

The Solution

I believe we are slowly getting to the solution to all of this.

Open-source, open-weight models are becoming more available each day. I would call upon the big players in the arena, such as @sama, @finkd, @elonmusk, and @JensenHuang to continue to push for open-source, open-weight, and transparent models.

The truth can be jarring; sometimes it slaps us right in the face, and it can be a hard pill to swallow. But we need to continue navigating how to make that truth available and digestible to all walks of life. Humanity itself depends on it.

Afterword

Dear reader,

If you made it this far, I want to say thank you. You are one of the few who have resisted what I like to call the "attention drought." You have resisted the urge to scroll mindlessly.

You have triumphed, but at the same time, fallen into the algorithmic trap that says, "Keep the user on the platform for as long as possible."

So, I invite you to do what some refer to as "touch grass." Go tell someone you care about that you love them. Go ask someone, "What's on your mind?" instead of the superficial, "What are you up to?" Invite introspection with someone in your life, and you may just learn something new yourself.

- Ryan Lenk