No, AI will not eliminate genuine expertise; rather it will make it more valuable

These problems are like distant locations that you would hike to. And in the past, you would have to go on a journey. You can lay down trail markers that other people could follow, and you could make maps.

AI tools are like taking a helicopter to drop you off at the site. You miss all the benefits of the journey itself. You just get right to the destination, which actually was only just a part of the value of solving these problems.—Terence Tao interviewed in The Atlantic, February 24, 2026 [HT: Ryan Muldoon]

It’s just a year ago since I last wrote about modern AI and real patterns it uncovers (recall here). It’s time for another, modest reflection on the state of play.

In the quoted passage, Terence Tao (a Field’s medalist in mathematics) describes a very specific species of ignorance, what I (recall; or here) have been calling ‘Humphreys opacity’ (or, if you prefer, ‘epistemic opacity.’) This involves the inability to surveil the steps of a process from a known input to a known desirable (or truthful, useful, beautiful, etc.) output in a timely manner to the decision-maker or responsible agent. I put it like that to make clear that this ignorance is pragmatic in character and could be modelled in terms of trade-offs between the quality or benefit of the output and the cost of surveillance. (Of course, sometimes the opacity is not pragmatic, but ontological in character.) In addition, I use the ambiguous language of ‘surveillance’ because the process can be computational, social, or natural in character.

What’s neat about this particular instance, is that at the moment Tao’s state of opacity about the process (the ‘journey’) that led to the AI proof mirrors the opacity of the machine that ‘helicoptered’ there. At the moment there is no way of recovering the machine’s journey to its answer. (Presumably with time and effort some kind of reverse engineering might be possible, even if it involves an odd intentional stance.)

Tao’s view is that in mathematics the process of discovery is very valuable, and so reaching truth quickly, or more precisely, without access to the mathematical landscape, involves a trade-off between truth and (let’s call it) informativeness (or, perhaps, understanding). Keep this in mind.

Durin the last ten days, I had opportunity to have long talks with Katie Creel (Northeastern) and Ryan Muldoon (Buffalo) during my visits to their programs. And today’s digressions are informed by them. I think the world of both them, and if I am onto something here they may claim credit. As an aside, the older I get the more I recognize that the side-conversations with faculty and students during a campus visit are really the icing on the academic journeyman’s cake.

I should note that in the interview, as reported, Tao never uses the phrase ‘truth.’ Rather, he phrases his analysis in terms of the ‘answer’ the machines provide. It’s worth conveying how he puts it:

One very basic thing that would help the math community: When an AI gives you an answer to a question, usually it does not give you any good indication of how confident it is in this answer, or it will always say, I’m completely certain that this is true. Humans do this. Whether they are confident in something or whether they are not is very important information, and it’s okay to tentatively propose something which you’re not sure about, but it’s important to flag that you’re uncertain about it. But AI tools do not rate their own confidence accurately. And this lowers their usefulness. We would appreciate more honest AIs.

So, there are two issues here: first, the one highlighted by Tao: the machine does not report its own ‘confidence’ in its own answer accurately. Second, even if it offered such confidence accurately, it could still be wrong about the answer (and, perhaps, misreporting its own confidence.) After all, there is no evidence that AIs have eliminated hallucinations altogether, or that this is even possible (at low enough cost and time).

To be sure, the current generation of commercially available flagship LLMs (GPT 5/Claude OPUs 4.5, etc.) are genuinely impressive. (And presumably the ChatGPT that solved these outstanding math puzzles is ahead of them, etc.) During the last month, they have finally reached the level of interesting research assistance in my own field. But second, don’t let anyone claim they have stopped hallucinating. Crucially, for a lot of purposes this makes LLMs worse tools, because you often can’t just eyeball the errors--you really need to pay attention to their output. Keep this in mind, too.

Now, to be sure, there are super-interesting issues lurking here about what it would mean to have AI’s internally model or represent their own confidence. (Would they be simulating human confidence reports as if Terence Tao or some much lesser mathematician, or would they develop their own approach; would they have debates about Bayes? etc.) But that’s not my present main interest.

So much for set-up.

There is a persistent strain of thought that AI will eliminate all knowledge work. The apparent fate of junior and mid-level computer coders in the moment foreshadows a more general disruptiveness. Let’s stipulate that AI will indeed threaten lots of white- collar work (I call this the ‘next great transformation’).¹ And that even in the sciences it will transform discovery and how disciplines will interact with each other, as Tao suggests. (Go read the interview.) So, philosophy of science will have a busy time ahead.

My main interest is this: Tao’s comments alerts us to the fact that there are a class of problems where answers supplied without surveyable information on the means or steps for finding it are themselves fragile. (OMG, I may begin to see the utility of Gettier!) Somebody very skilled needs to check the ‘answer’ at the research frontier. This is why even in mathematics there is a social component to justification. And, as AI eliminates all the low hanging fruits, the difficulty and costs of checking themselves go up as understanding of the landscape becomes very thin.

Even if we could build machines to check the AI (and so on), there would be need to be diagnostic tools that need to be maintained and repaired and so on. Since these machines will suffer from Humphreys opacity, this challenge becomes endemic.

So, rather, what Tao’s remarks suggest is that genuine expertise will be at a premium as we transform to a world suffused with modern AI. And this is because modern AI systematically introduces Humpreys opacity alongside the cutting-edge answers it provides. To what degree genuine expertise will be able to capture that value in our economy is a different question.

That’s the main point I wanted to make. But there is a second point lurking here. The institutional infrastructure of a universe full of Humphreys opacity is itself quite dense. If we take Rousseau’s (1755) Third Discourse as the moment of general awareness about the governance significance of such epistemic opacity, then we can discern a fairly large growth in institutional structures to manage it over time since. This is, by the way, the enduring lesson of Walter Lippmann’s (197) The Good Society. Accurate information that is appropriately public — perhaps even, to adapt a phrase from Tom Pink, witnessed as truth (recall) — requires an enormous machinery of record and a legal infrastructure that helps adjudicate conflicts over identity of and property rights in that information and the consequences of its use. Even some lawyers may survive the next great transformation.

1. It would be nice to see public commentators apply principles of comparative advantage to their modeling exercises. But that’s for another time.