Mäki Rules on Rodrik’s Rules

Mäki, Uskali. “Rights and wrongs of economic modelling: refining Rodrik.” Journal of Economic Methodology 25.3 (2018): 218-236.

Introduction: I must confess to having admired Dani Rodrik. His research was iconoclastic, fearlessly going after many sacred cows of economics. So, I was saddened and disappointed by his defense of Economics: Rodrik, Dani (2015) Economics Rules. Why Economics Works, When It Fails, and How to Tell the Difference. Oxford UP. Rodrik uses “rules” in a dual sense; a set of rules to discriminate between good and bad economics, as well as an assertion of the superiority of economics. This post is a fairly longish 2100 word summary of the first half of  Mäki’s trenchant critique the book, linked above. If I find the time and energy, I might do the second half later. However, this should suffice to save the reader the pain of reading Rodrik’s incoherent and conflicted defense of the indefensible. In my post title, I also use rules in two senses — as a verdict, and as a judgment of Mäki’s sophistication relative to Rodrik’s naivete.

Uskali Mäki is truly a gentleman and a scholar – his devastating put-down of Rodrik reads like a loving uncle gently correcting an errant child. There are powerful arguments, but no sound-bites. This post provide a summary of Mäki’s critique.

Rodrik’s First Defense: problems are due to economists, rather than economics.

Mäki’s Counter: It is hard to define economics and economists in such a way as to neatly separate the two. If we use the famous aphorism, economics is what economists do, then this defense collapses. Alternatively, we can use more sophisticated definitions of the discipline of economics. The concept of a scientific discipline is usually considered to contain an idea of regular and regulated behaviour, of disciplinary practice. Academic disciplines don’t exist independently of such practices, nor are they reduced to them, such practices rather partly constitute academic disciplines. So if there is a problem with such practices, there is a problem with the respective discipline. It is not possible for the discipline of economics to be just fine if economists regularly misbehave, especially if this regularity is supported by customary attitudes.

To try to articulate a version of Rodrik’s defense that might work, Mäki considers the possibility that some academic practices are good and some are bad. This does not work well because, as Rodrik later admits, bad practices are dominant in the profession – so saving it along these line does not work.  An alternative line of defense might be to create a boundary within economics, classifying some parts as good and others as bad. To sustain this we must ask how Rodrik defines economics. Here Mäki cites Rodrik (p171) as stating: Economics not only consists of models and modelling methods, but also of variouscustomary “practices and professional biases”. But if economics is defined by certain practices and professional biases, and these practices are often bad (as Rodrik admits later, see below), then Rodrik is admitting that economics is flawed, after all.

Mäki provides an illuminating analogy with the gun lobby slogan: “Guns don’t kill people, people kill people”. Rodrik is arguing it is not the economic models which are bad, but rather the economists who misuse them in various ways. Even if we buy this argument, it leads to two possible policy prescriptions – prevent people from having guns, or to give them more guns. Rodrik argues that economic models are good, and the more there are, the better. But he cannot coherently maintain that this will make economics better, because he concedes that most economists misuse models.

Interlude: Mäki’s concept of a Model

Before going on to a critique of Rodrik’s models, Mäki presents his own conception of models. This is actually very useful, and worth learning and understanding on its own merits. I present the diagram here because I found it very useful in understanding and thinking about a large number of different issues related to the use of models. I also add summarized commentary and explanation of the diagram from Mäki, before we go on to consider Rodrik’s paper.

Nothing is a model in itself. Modelhood requires a larger structure within which an object becomes a model. This larger structure embodies many of the dimensions that are characteristic of scientific disciplines. Some agent A (individual or group) needs to consider and use an object as a model, M. This involves having an idea of the model being a model of some target R, actual or possible. It also involves some purpose P for which the model can be used (such as predicting, explaining, or policy advice). The audience (fellow economists, general public, policy makers, etc) is also relevant for judging suitability of the model.

Spelling out these constitutive components and relations helps us to see why Rodrik’s divisionist assessment strategy (economics vs economists’ attitudes and behaviour) is difficult to maintain.

Back to Rodrik’s Defense

Rodrik’s theoretical models are fantasy simplified mini-worlds created by the modeller, designed to represent a complex maxi-worlds, or sometime even simplifications of a complex theoretical world. A crucial and largely un-noticed role is played by the Model  Commentary C. It is here that we find – and he finds — much of the deficiency in economic modelling. Model commentary contains and conveys ideas about how the other components in the modelling endeavour play out their roles in coordination with one another. What is the point of using radically unrealistic assumptions? When is unrealisticness alright and when not? What roles exactly can a model play in assertions or hypotheses about the world? Is it about an actual or merely a possible target? What’s the proper domain of application of a model? What precise purpose(s) can a given model be used for? What uncertainties are involved in model use? A commentary provides (good or bad) answers to such questions. When Rodrik states that economists often have a limited understanding of some important aspects of modelling, he is implying that economists hold a deficient model commentary.  Superficially, this can be used to save Rodrik’s arguments – economic models are good, but economists commentaries on the models are not good. But there are complications in this view, as Mäki points out.

RODRIK: Models are transparent That’s because they are transparent about their critical assumptions and behavioral mechanisms. They come with explicit user’s guides – teaching notes on how to apply them. (p73)

MÄKI: My contention isthat models are not transparent in this way in themselves; that is why a model commentary is needed. For example, “critical assumptions” don’t identify themselves as such; they need to be identified by the agent using an apposite commentary. A good commentary makes as much of the model and modelling process transparent as is possible and needed – but this is NOT common practice among economists.

RODRIK:  In accordance with the subtitle – Why Economics Works, When It Fails, and How to Tell the Difference. – Rodrik attempts to explain the rights and wrongs of economic models.

MÄKI: To define success, we must first define a target, a goal with respect to which success is to be measures. Second, we must have some measure of proximity – how to evaluate the distance between the goal and the model, in order to judge success or failure. In this context, Mäki offers the following considerations.

A simple version of success is to achieve suitable resemblance to some target reality. Suitability must be judged relative to audience, and to purpose of model. Models which attempt to approximate a target reality are called surrogate models. However, in economics most models are treated as substitutes for reality. Surrogate models can be wrong (or right) in terms of how close they get to target reality. However, substitute models cannot even be wrong about the world (since they are not presented and examined as being about the world). This second type of failure is both common, and invisible, since there is no apparent failure at all. See my paper on “Models & Reality: How did models divorced from reality become epistemologically acceptable?” for a deeper discussion.

RODRIK: Mathematical form guarantees transparency. The critical assumptions and derivations are out there for all to see. This means that models are fine, but can be misused by those who ignore the conditions required for use of these models.

MÄKI: This is not true; use of mathematical models does not create transparency. {{AZ: For a specific example of how un-identified critical assumptions can be buried within a maze of mathematics, see Romer’s Trouble With Macro, }}. In addition, use of mathematics creates a tendency towards use of substitute models, where mathematics substitutes for reality. As Rodrik himself admits:  “too many economists fall in love with math […] Excessive formalization – math for its own sake – is rampant in the discipline.” The effect of this failure to connect mathematics with reality is also admitted by Rodrik:

“Asked point-blank, they can state chapter and verse all the assumptions needed to generate a particular result […] But ask them whether the model is more relevant to Bolivia or to Thailand, or whether it resembles more the market for cable TV or the market for oranges, and they will have a hard time producing an articulate answer.” (p172)”

In response to this admission, Mäki makes his most stringent comments:

In other words, many economists don’t quite know what they are doing, and are doing too little. [ …They don’t know …] how their models are connected to any external targets and what specific purposes they can be supposed to serve. What is remarkable is that this is accepted practice: “accepted practice does not require economists to think through the conditions under which their models are useful” (Rodrik: p172). Thus Rodrik admits that the disciplinary conventions of economics are flawed – instead of merely economists misbehaving in not abiding with the proper conventions. So, economists often fail to understand the models they use, and therefore often misuse them.

RODRIK: “The antidote of a bad model is a good model” rather than “no model” (29).

MÄKI: This may sound like a pro-gun person saying there are good guns and there are bad guns. A more sympathetic reading is that Rodrik is trying to say that models should be evaluated with respect to the purpose for which they are being used. But this relativity of “goodness” creates a host of problems not considered by Rodrik. We can always find a “purpose” to turn any model into a good model. With this framework, Rodrik’s understanding of progress can be parsed as follows:

Rodrik’s idea of how economics makes progress in terms of models: Making progress, as well as creating a capacity of making progress, are of course kinds of success. Rodrik suggests that progress in economics is different from progress in natural sciences (p71) as economic knowledge does not accumulate “vertically” by rejecting earlier bad models and replacing them with new good models, but rather “horizontally” by expanding the pool of models (p67). “The newer generations of models do not render the older generations wrong or less relevant … Older models remain useful; we add to them.” (71) Taken literally, this would seem to imply that all economic models ever built are relevant for some present (or perhaps future) respectable purpose.

{{AZ: This ties in closely to my paper on “Models and Reality: How Did Models Divorced from Reality Became Epistemologically Acceptable?”. Rodrik takes the position that all models are good (for some unspecified purpose). So conflicts with reality, and modifying and improving models are irrelevant. Economics does not progress by replacing bad models by good models, but by expanding the collection of models. Given that these models are “substitutes” for reality, there is never any need to examine them for match with reality. Unless this anti-scientific methodology is abandoned, there is no hope of progress.}}

Rodrik on the Sociology of Knowledge: “The authority of [a piece of research] derives from its internal properties – how well it is put together, how convincing the evidence is – not from the identity, connections, or ideology of the researcher.” (p78)

Mäki: This is a blind spot in Rodrik. Rubinstein writes that the elites exercise tremendous influence: “The job market for junior economists is an illustration of the unfairness associated with the power of the elite.” {{AZ: Romer explains that he can be critical of economics and economists in ways that insiders cannot, because he is immune to the power of the elite. See Quotes Critical of Economics for a Krugman quote showing that one had to wrap real economics within an artificial framework required by the dominant ideology. Similarly, Card and Krueger complain of severe alienation caused by their rejection of the sacred cow of Supply and Demand, which led to their abandonment of research on the topic.