On The Governance of LLMs, and The University (of Chicago)

I heard, then, that at Naucratis, in Egypt, was one of the ancient gods of that country, the one whose sacred bird is called the ibis, and the name of the god himself was Theuth. He it was who invented numbers and arithmetic and geometry and astronomy, also draughts and dice, and, most important of all, letters. Now the king of all Egypt at that time was the god Thamus, who lived in the great city of the upper region, which the Greeks call the Egyptian Thebes, and they call the god himself Ammon. To him came Theuth to show his inventions, saying that they ought to be imparted to the other Egyptians. But Thamus asked what use there was in each, and as Theuth enumerated their uses, expressed praise or blame, according as he approved or disapproved. The story goes that Thamus said many things to Theuth in praise or blame of the various arts, which it would take too long to repeat; but when they came to the letters, “This invention, O king,” said Theuth, “will make the Egyptians wiser and will improve their memories; for it is an elixir of memory and wisdom that I have discovered.” But Thamus replied, “Most ingenious Theuth, one man has the ability to beget arts, but the ability to judge of their usefulness or harmfulness to their users belongs to another; and now you, who are the father of letters, have been led by your affection to ascribe to them a power the opposite of that which they really possess. For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise.—Phaedrus, 274-275, translated by Harold N. Fowler.

Sometimes people that know and like each other, and that would never employ snark with each other, can still talk entirely past each other online. Carlo Ludovico Cordasco wrote a fruitful and prudent sub-stack post (here) on the ‘longstanding debate on AI and deskilling.’ As he notes it was prompted by my Kvetching about a 2 June announcement by The University of Chicago’s President that it has a contract with Anthropic to give all of its students, and all of its staff and faculty, full access to Claude Enterprise.

Now, I viewed that announcement by Paul Alivisatos (the University President) the way I interpret many of that university’s public announcements during the last two decades: as a cynical, branding ploy aiming to keep the university in the eyes of parents who might pay full tuition for their kids. In my view, in its public communication, the UofC has stopped trying to be the academy for would-be-academics and those closely adjacent to it. The fate of skill was far from my mind. I viewed the UofC as vice-signaling its path away from all that is noble and beautiful about higher education. (I will explain that near the end of this post.)

I was never an undergraduate at UofC. But on my second day of graduate school there, I remember walking in the undergraduate library (yes, there was an undergrad library!) surrounded by kids who clearly had pulled an all-nighter (mostly with Homer, but a smattering of other thick books—some were asleep with the books on the desks others were engaged in intense discussions with each other. The most popular T-Shirt in my time there, was ‘Where Fun Goes to Die’ (and closely followed ‘Hell Freezes Over.’) The place had an ethos centered on its undergraduate core that involved reading, discussion, and writing and that permeated it. (Graduate students often taught in The Core so it wasn’t just for the kidz.)

While I was contemplating a response to Carlo’s essay, I decided to do what countless Substack essayists have done while confronting the significance of LLMs: I went back to a familiar passage (quoted above this post) in the Phaedrus. I read it with fresh eyes.¹

Naucratis was a commercial port on the Nile that in Socrates’ time had, somewhat unusually, been populated, as Herodotus attests, by Greeks for several generations. So, it is fascinating that Socrates makes the God of the Underworld, and patron of scribes, Thoth/Theuth dwell there in that cosmopolitan port. One is made to wonder whether this God didn’t pick up the use of letters from some travelers (say Phoenicians), or invented it to mediate or judge a commercial dispute.

As an aside, I would be amazed if Socrates (or Plato) were not alluding to Herodotus in this narrative. For, according to Herodotus, Cadmus brought the Phoenician alphabet to Greece and founded Boeotian Thebes. And the fact that there are two Thebes means one must presuppose a wider world to keep them apart. That is to be distinguished, of course, from entities with multiple names.

Now, while it is tempting to rush ahead to the part of the story where deskilling is discussed, I want to pause for a second at the division of labor between Theuth and Thamus/Ammon. Theuth invents, but Ammon is the regulator who decides whether an invention can be circulated and to what extent. Wisely, these ancient Egyptians don’t let the inventor himself decide on issues of public safety and public utility. This part of the myth is usually skipped because governance is nowhere near as sexy as debating the possible deskilling effects of a new technology. Carlo’s piece nicely illustrates this, all the hard social and political challenges get solved magically in it.²

What is most notable about Ammon in his dealing with Theuth is that he focuses on two features of an invention before he gives permission to circulate it: first, its practical use. And here he is quite clearly guided by the inventor’s views. And, second, the potential harms (or, let’s be cheeky, inductive risks) involved. And notably, when it comes to foreseeing harms Ammon’s own views are more decisive than any inventors not even a God. And the reason for this is explicitly spelled out: would be inventors tend to misperceive the true nature of their inventions, primarily seeing the benefits only. I return to this below.

Obviously, Socrates is simplifying the division of labor a bit by using Ammon. Most kings may (if they know their Hobbes) wish to overawe all the citizens in a godlike fashion, but they will not be in the same epistemically advantageous decision siruation as (let’s stipulate) Ammon. So, how they will judge the possible harms of new technology will be, in fact, a rather challenging governance question.

My point is not that Socrates presents us with the right governance model. But importantly, he presents with such a model. In fact, lurking here is a more important point: that an art of governance is itself the effect of the invention of the very writing-technology that we are discussing here. For writing makes bureaucracy possible. And this raises new problems. As Socrates alerts us.

Conveniently enough (for my new debate with Carlo), the harms Ammon is alert to in the invention of writing do involve, alongside the acquisition of a new skill (writing), some de-skilling, including the art of memory. Now, one might have thought that Ammon would have made a simple cost-benefit analysis to the individual or to the population of the gaining of one skill (writing) and the loss of another (worse memory). Is writing more efficient than relying on memory?

But what’s interesting about Ammon’s diagnosis is that he diagnoses an externality, actually two externalities, that are especially important to a King-Ruler. And the externality has everything to do, I think, with the fact that writing is the tool of bureaucracy. As a close reader of Plato, Rousseau (recall here), reminds us (in the Third Discourse) bureaucracies are invented when the domain to be managed cannot be surveilled by single person or family. A bureaucracy is a division of labor that is spatially and temporarily extended, and relies on proper record-keeping to manage socially complex affairs.

But in this bureaucratic managing there is also a useful fiction: that the recorded ‘data’ convey the facts, and that the person reading the report on the data understands what they signify. Writing makes a form of impersonal acquaintance possible. But the risk that Ammon (correctly) identifies is that the bureaucrats and those they serve (governors like Ammon himself) will “read many things without instruction and will therefore seem to know many things, when they are for the most part ignorant.” The reader of the report will end up with knowledge of the reported reality, but not (without further instruction) with its connection to the underlying facts. And if they forget this distinction, as the written world takes on its own reality, they will seem wise when they may not be so. How to manage a bureaucracy that systematically generates this kind of vulnerability, is a central problem in the art of governance. When things remain on the rails, this challenge is usually invisible to most. But when it goes off the rails — public infrastructure is not repaired, corruption goes unpunished, etc. — it is often too late for minor fixes (not the least in circumstances of emergency that Carlo acknowledges).

The written word, when used as a technology of governance, inevitably creates legibility (of the population and society) and simultaneously all kinds of new opacities. As I have noted before (recall here), this is a feature of modern LLMs, whose internal architecture and the process by way which they convey content, remains largely opaque to end-users. This is especially important because the error rates of LLMs remains non-trivial. This output externality is only visible if you look, and care.

I could stop here. But I promised a second externality as noted by Ammon. And this is a more important social externality. The users of a new technology will feel more confident. And, let’s stipulate, their own confidence is not misplaced. Like Carlo’s ‘off-loaders’ they get more important things done in less time. Unfortunately, human nature being what it is, it will encourage boasting and self-aggrandizement. And, even if this self-satisfaction were fully merited, as Ammon recognizes this is a dynamic that predictably generates social instability. Not the least because off-loading boasters refuse to be bothered by the real challenges new technology introduces. The ethos of a polity may change for the worse with new technology—and it doesn’t take much imagination to discern that Ammon is worried about new idols and new modes. This may be inevitable, even if we are often willing to pay the price.

Now, Carlo treats the university as a site of skill-transference, and in which Claude is one tool of many. And this is no surprise, because if you return to the press release of Alivisatos’ statement, you will see that the key word in it is ‘tool.’ Tools are used. And that’s part of the way Alivisatos sells his message.

But he doesn’t convey at all that he has evaluated the risks (a word he does not use which is notable for a modern executive) and externalities a new tool predictably generates for the ‘community’ (his word) he has been charged to lead. All he notes are the “astonishing and inspiring ways” of the new tool.

In fairness, to Alivisatos: when you read the full press release, he is setting up a quite elaborate governance structure for new AI. So, it is quite possible that he has assimilated the first lesson I wished to derive from Socrates’ myth. (And he could have been told it if he had asked to relate his policy plans to the myth of Theuth to a desktop AI.) But I don’t think a fair reading of his text suggests he is signaling concern over or apt care about what might be lost in the process.

Even so, I used the language of governance throughout this post, because I was most struck by his message to his students:

To our students: Many of you already are reflecting on how a skeptical, ethical, and ambitious approach to AI applies to your education. Your faculty and instructors are developing practices in the classroom within a philosophical framework to support you in opening and sharpening your mind during your journey here. That framework will prepare you well to meet the opportunities of a changing world. You can understand more about how your faculty are thinking about helping you to learn by reading about a recent series of workshops sponsored by the Center for Teaching and Learning. Policies vary by course, and you are required to follow them closely; but the sum across your full curriculum will give you diverse learning experiences with and without AI assistance. Instructors at all levels are navigating a fast-evolving landscape, and the University has a duty of care to ensure that the education offered to you is responsive to these technological developments by teaching you how to think with machines, how to think without them, and how to think about them.

There are a lot of action words in this paragraph (reflecting, developing, reading, navigating, and teaching not to least how to think) and a few technocratic terms (framework, opportunities, policies, diverse learning experiences, landscape)), including a few commands (required). What’s notably absent in this message are reading, interpretation, and (most notably) discussion. That’s not just a matter of vocabulary. Something has gone off the rails here, and by this I don’t mean the total absence of reference to the world of books and intellectual culture in his message.

Back in the day (and I amplified with some twists), Agnes Callard suggested that “A university is a place devoted to the problem of how to make serious use of free time.” It would be wrong to suggest that the proper use of intellectual tools has no place in this vision of the academy. But as Callard notes such a university presupposes what she calls ‘inquisitive leadership.’ Alivisatos’ rhetoric is very far from it. He leads not by teaching and learning here, but by cheerleading and commanding.³ And this is why I called it vice-signaling.

1. After I drafted this post, I also asked Google’s AI, “"if applied LLMs what would be the main implication of the myth of ammon and theuth in phaedrus?” It answered “If applied to Large Language Models (LLMs), the main implication of the myth of Ammon and Theuth is the warning that outsourcing human thought to AI will replace genuine, internalized understanding with a superficial “illusion of wisdom.” I think this misses something important I wish to convey. (When I subsequently asked about governance lessons, it articulated something much closer to the issues articulated in my post.)

2. Here’s Carlo’s description of the process, which I quote in full. And I ask you to look for who decides the hard political and financial/tax questions whose answers his description in the first person plural presupposes:

The second is the emergency, and its logic is different. Here the trouble is that the need for a skill can arrive faster than it could ever be learned. For a small number of jobs, the day comes when the machine fails and there is no time to catch up, so the skill has to be there already, built in the years when the machine was handling things and the person did not seem to need it. We do not deal with this by asking everyone to keep every skill, just in case, which would be both impossible and pointless. We deal with it the way we always have, by deliberately keeping certain abilities alive in the particular people who will need them. We train a few surgeons to operate when the equipment dies and pilots to fly when the automation quits, and we carry the cost because the stakes are high and the warning is short.

3. It’s possible that Callard supports Alivisatos’ AI policy.