On the limits of Cost Benefit Analysis in Heath's Philosophy of the Executive, pt 3

This is my third post on Joseph Heath’s The Machinery of Government: Public Administration and the Liberal State (OUP, 2020). [The first one is here; the second here.] In both earlier posts I indicated that I consider the lengthy, (fifth) chapter “On Cost-Benefit Analysis as an Expression of Liberal Neutrality” a masterpiece of exposition and analysis. The strength of the chapter is that it patiently articulates and then corrects many misconceptions about Cost-Benefit Analysis (hereafter: CBA), including ones articulated by otherwise authoritative and informed philosophers. Substantially, the chapter shows how CBA is a tool for the growth of the Executive in government and, simultaneously, the expansion of rationalization (in Weber’s sense) in society.

Most striking of all, the chapter shows that CBA “as institutionalized and employed in contemporary welfare states is a straightforward extension of modern liberal principles.” (p. 186, emphasis in original; see also p. 188). Heath argues that CBA instantiates a commitment to resolving collective action problems that are the effect of market failure. On Heath’s account the civil service/the executive treats CBA as a means “to determine what outcome private individuals would have contracted to— which is to say, agreed upon— absent some market failure.” (p. 188)

I offer four preliminary observations about this point. First, as the language of ‘contract’ hints at Heath rejects the idea that CBA exhibits commitment to utilitarianism. (p. 194.) Rather, the use of CBA expresses commitment to Pareto efficiency. Second, that CBA generally uses “money as a “metric of value” is “not a consequence of any underlying commitment to utilitarianism, or “economism”; it is a consequence of an underlying commitment to equality.” (p. 194) I can’t emphasize enough how interesting and persuasive Heath’s argument is on this score.

Third, crucially, what CBA does is to create a relatively transparent process in which a counterfactual (“would have contracted”) is established. In fact, CBA involves discrete steps, including constitutively “a market-simulating exercise.” (p. 198) As Heath explains, CBA “needs to produce the outcome that the market would have produced, had the market failure not occurred.” (p. 198) So, fourth — and this is not intimated in at all by Heath — CBA reflects Oskar Lange’s insight back in the 1930s that from the perspective of social planning simulation might well replace actual market processes.* Crucially, here CBA often takes places, as Heath emphasizes (e.g. p. 201) against the background of existing market processes (which give rise to the market failure, and also help establish some of the input in a CBA process.)

Because in what follows I will be critical of Heath’s account, I want to be explicit that my argument here is distinct from my criticism of Heath’s tendency to downplay the risks of the widespread use of Pareto efficiency. (That I criticized in my (recall) second post on his book.) But if I am right in what follows, my argument does reinforce a concern I expressed there. (More on that below.)

I want to articulate three concerns about the reliance by the executive on CBA. The first two are connected, with the second more important than the first. Only the third is related to my worry about his reliance on Pareto efficiency.

Now, before I articulate my main concern of the use of CBA it is worth noting that despite his masterful survey of philosophical criticisms related to CBA, Heath misses the philosophically most satisfying criticism lurking in the vicinity of his argument. The underlying insight goes back to Smith and Marx, but that Frank Knight concisely expressed as that “wants which impel economic activity and which it is directed toward satisfying are the products of the economic process itself” (1922, 457; Knight credits J.M. Clark). Or, if you like slightly more up-to-date formulation, we can use Elster’s thought that “preferences underlying a choice may be shaped by the [given] constraints” that constitute that choice (Sour Grapes, ix). That is, on this view, market processes don’t just help us discover the relative prices and opportunity costs of what we want, but also the content of what we want. And in so far as our wants, partially constitute who we are, markets help constitute the very agents who do buying and selling.

The first problem for CBA is, thus, not that it simulates to discover a counterfactual, but that we can’t know that the key factor simulated — our desires/preferences/wants — would have stayed the same in the very process simulated. So, if our given wants are the effect of a market failure [as might be demonstrated in a CBA], we can’t be secure in claiming that these would have been ‘our’ wants in a better functioning market. CBA presupposes a social ontology — that our wants remain constant across institutional contexts — that may be based on wishful thinking.

Before I continue, I need to make two qualifications. First, notice that the argument here does not touch, say, an exercise of the sort we do in Rawls’ original position. For, in the original position we are establishing the basic structure of society, not our preferences in the context of particular market failure. So, the present argument is not a criticism of the use of decision procedures that rely on hypothetical preferences as such. (As regular readers know, I think Rawls was a rather sophisticated reader of Knight.)

The second qualification is that I don’t view this criticism as fatal to Heath’s argument in defense of the widespread (albeit cautious, rule-governed) use of CBA as a tool of government. CBA may be limited as a tool, and this just is one of the more conceptually fundamental limitations of the tool. As Heath recognizes the “value of CBA as a procedure depends entirely on the quality of the valuations it uses as inputs, which it must take as given.” (p. 215) My claim is that users of CBA have to be made aware that they are not only trying to establish a counterfactual, but that the inputs are, in part, constituted by counterfactuals all the way down. So, even though CBA “attempts to measure the right thing” (p. 217, emphasis in original) it shouldn’t overlook that what that thing is may itself be a construct of the exercise.

Now, I just quoted from Heath’s section in which he addresses the ‘garbage-in-garbage-out’ objection to CBA (section 5.3.2, pp. 215-219). Both my first criticism just articulated and the second I am about to develop are a variants of it. But while drawing on Peter Diamond and Jerry Hausman, Heath treats the ‘garbage-in-garbage-out’ objection to CBA primarily in terms of the cognitive biases that respondents to “contingent valuation” surveys done in the context of CBA might have. My claim is distinct from the general concern with cognitive biases (except, perhaps, that stability of preferences under counterfactual conditions may itself be a cognitive bias).

My second criticism is inspired by a (1992) paper “On Measuring the Social Opportunity Cost of Labour in the Presence of Tariffs and an Informal Sector,” by my friend, the mathematical economist M.A. Khan (Hopkins), that he published back in the day in The Pakistan Development Review. The paper was once out of the way and hard to find, but thanks to Jstor much easier accessible now. The paper is a philosophically sophisticated response to the influential and celebrated effort by Harberger back in 1971 to develop “accepted set of professional standards for” (785) cost benefit analysis in project management. (That’s central to how economic advice is treated in development.)**

The problem that Khan diagnoses starts from the observation that CBA is intrinsically dependent on models. Again, that may be unavoidable. But the real problem is that the models may not be robust at all. (Oddly, while throughout the book Heath sings the praises of economic modeling, Heath ignores this entirely.) One way to think about lack of robustness is when a model is highly sensitive to the choice of parameters that shape outcomes. As Khan notes, even simple models need not be robust in this sense.

But, crucially, even when one is working with relatively robust models one must, unavoidably, make choices about how model and reality are to be related. Khan captures the significance of this with a passage worth quoting. Such choices, “represents a double movement - from reality to model to reality. Each such movement represents a compromise and it makes little sense to me to search for the compromise.” (Khan 1992: 557)

So, now I can state my second (and main) criticism of CBA. Even in the best circumstances a CBA is incredibly sensitive to modeling decisions and, so, indirectly by those who control access to the process. Even if, as Heath suggests, CBA is a tool for standardization and uniformity, any given CBA is vulnerable to being (tacitly) captured intellectually. The way of putting it like that is to show that the problem I am pointing to is compatible with a CBA that is “procedurally proper and complete.” (Heath, p. 218 This is the main point I wished to convey today.

A CBA has the benefit of creating transparency about modeling decisions. So, each step in a CBA can be recovered and explained. But, and this is another way of stating today’s main point, a CBA cannot guarantee that when somebody else redoes the whole modeling exercise independently from the original decisions they will end up with the same numbers.

But I did promise a third criticism. This builds on my more general unease (recall) with how Heath defends Pareto efficiency as central to output legitimacy of the Executive. Recall that the reason why I am made uneasy by Heath’s argument is that, in practice, if the administrative state pursues Pareto efficiency improvements without requiring of itself actual compensation “(as is the case with many regulations)” (p. 250), then it is predictable that, in practice, there will be structural losers. By structural losers I mean social individuals or groups who lack the political clout or social standing to impose on the state the obligation to pay out actual compensation when the administrative state pursues, in principle Pareto improvements across multiple domains. As I noted, this is likely to occur in conditions of structural inequalities, especially in contexts where the permanent upper civil service is drawn disproportionally from social classes other than those who will be structural losers.

When I expressed this concern last week, I did not provide a mechanism by which a well-functioning state can generate structural losers. But today I have suggested that the widespread adoption of CBA is a vehicle by which even a well meaning state can generate a pattern of costs and benefits that systematically disadvantage (at least relatively) some groups rather than others. This is so when the concerns of these groups are imperfectly internalized in otherwise procedurally correct CBA process (and so survive scrutiny by courts).

To put the observation politically. I don’t think it is completely a coincidence that across the most successful liberal democracies, we encounter a seep-seated unease and even successful mobilization against the administrative state as such.

---

*Lange expresses the claim in terms of a process of trial-and-error that can characterize computations.

**Harberger’s is the best technical articulation of neoliberalism in economic practice, but oddly ignored by the massive industry studying neoliberalism.