Mäki Rules on Rodrik’s Rules

Mäki, Uskali. “Rights and wrongs of economic modelling: refining Rodrik.” Journal of Economic Methodology 25.3 (2018): 218-236.

Introduction: I must confess to having admired Dani Rodrik. His research was iconoclastic, fearlessly going after many sacred cows of economics. So, I was saddened and disappointed by his defense of Economics: Rodrik, Dani (2015) Economics Rules. Why Economics Works, When It Fails, and How to Tell the Difference. Oxford UP. Rodrik uses “rules” in a dual sense; a set of rules to discriminate between good and bad economics, as well as an assertion of the superiority of economics. This post is a fairly longish 2100 word summary of the first half of  Mäki’s trenchant critique the book, linked above. If I find the time and energy, I might do the second half later. However, this should suffice to save the reader the pain of reading Rodrik’s incoherent and conflicted defense of the indefensible. In my post title, I also use rules in two senses — as a verdict, and as a judgment of Mäki’s sophistication relative to Rodrik’s naivete.

Uskali Mäki is truly a gentleman and a scholar – his devastating put-down of Rodrik reads like a loving uncle gently correcting an errant child. There are powerful arguments, but no sound-bites. This post provide a summary of Mäki’s critique.

Rodrik’s First Defense: problems are due to economists, rather than economics.

Mäki’s Counter: It is hard to define economics and economists in such a way as to neatly separate the two. If we use the famous aphorism, economics is what economists do, then this defense collapses. Alternatively, we can use more sophisticated definitions of the discipline of economics. The concept of a scientific discipline is usually considered to contain an idea of regular and regulated behaviour, of disciplinary practice. Academic disciplines don’t exist independently of such practices, nor are they reduced to them, such practices rather partly constitute academic disciplines. So if there is a problem with such practices, there is a problem with the respective discipline. It is not possible for the discipline of economics to be just fine if economists regularly misbehave, especially if this regularity is supported by customary attitudes.

To try to articulate a version of Rodrik’s defense that might work, Mäki considers the possibility that some academic practices are good and some are bad. This does not work well because, as Rodrik later admits, bad practices are dominant in the profession – so saving it along these line does not work.  An alternative line of defense might be to create a boundary within economics, classifying some parts as good and others as bad. To sustain this we must ask how Rodrik defines economics. Here Mäki cites Rodrik (p171) as stating: Economics not only consists of models and modelling methods, but also of variouscustomary “practices and professional biases”. But if economics is defined by certain practices and professional biases, and these practices are often bad (as Rodrik admits later, see below), then Rodrik is admitting that economics is flawed, after all.

Mäki provides an illuminating analogy with the gun lobby slogan: “Guns don’t kill people, people kill people”. Rodrik is arguing it is not the economic models which are bad, but rather the economists who misuse them in various ways. Even if we buy this argument, it leads to two possible policy prescriptions – prevent people from having guns, or to give them more guns. Rodrik argues that economic models are good, and the more there are, the better. But he cannot coherently maintain that this will make economics better, because he concedes that most economists misuse models.

Interlude: Mäki’s concept of a Model

Before going on to a critique of Rodrik’s models, Mäki presents his own conception of models. This is actually very useful, and worth learning and understanding on its own merits. I present the diagram here because I found it very useful in understanding and thinking about a large number of different issues related to the use of models. I also add summarized commentary and explanation of the diagram from Mäki, before we go on to consider Rodrik’s paper.

MakiModel

Nothing is a model in itself. Modelhood requires a larger structure within which an object becomes a model. This larger structure embodies many of the dimensions that are characteristic of scientific disciplines. Some agent A (individual or group) needs to consider and use an object as a model, M. This involves having an idea of the model being a model of some target R, actual or possible. It also involves some purpose P for which the model can be used (such as predicting, explaining, or policy advice). The audience (fellow economists, general public, policy makers, etc) is also relevant for judging suitability of the model.

Spelling out these constitutive components and relations helps us to see why Rodrik’s divisionist assessment strategy (economics vs economists’ attitudes and behaviour) is difficult to maintain.

Back to Rodrik’s Defense

Rodrik’s theoretical models are fantasy simplified mini-worlds created by the modeller, designed to represent a complex maxi-worlds, or sometime even simplifications of a complex theoretical world. A crucial and largely un-noticed role is played by the Model  Commentary C. It is here that we find – and he finds — much of the deficiency in economic modelling. Model commentary contains and conveys ideas about how the other components in the modelling endeavour play out their roles in coordination with one another. What is the point of using radically unrealistic assumptions? When is unrealisticness alright and when not? What roles exactly can a model play in assertions or hypotheses about the world? Is it about an actual or merely a possible target? What’s the proper domain of application of a model? What precise purpose(s) can a given model be used for? What uncertainties are involved in model use? A commentary provides (good or bad) answers to such questions. When Rodrik states that economists often have a limited understanding of some important aspects of modelling, he is implying that economists hold a deficient model commentary.  Superficially, this can be used to save Rodrik’s arguments – economic models are good, but economists commentaries on the models are not good. But there are complications in this view, as Mäki points out.

RODRIK: Models are transparent That’s because they are transparent about their critical assumptions and behavioral mechanisms. They come with explicit user’s guides – teaching notes on how to apply them. (p73)

MÄKI: My contention isthat models are not transparent in this way in themselves; that is why a model commentary is needed. For example, “critical assumptions” don’t identify themselves as such; they need to be identified by the agent using an apposite commentary. A good commentary makes as much of the model and modelling process transparent as is possible and needed – but this is NOT common practice among economists.

RODRIK:  In accordance with the subtitle – Why Economics Works, When It Fails, and How to Tell the Difference. – Rodrik attempts to explain the rights and wrongs of economic models.

MÄKI: To define success, we must first define a target, a goal with respect to which success is to be measures. Second, we must have some measure of proximity – how to evaluate the distance between the goal and the model, in order to judge success or failure. In this context, Mäki offers the following considerations.

A simple version of success is to achieve suitable resemblance to some target reality. Suitability must be judged relative to audience, and to purpose of model. Models which attempt to approximate a target reality are called surrogate models. However, in economics most models are treated as substitutes for reality. Surrogate models can be wrong (or right) in terms of how close they get to target reality. However, substitute models cannot even be wrong about the world (since they are not presented and examined as being about the world). This second type of failure is both common, and invisible, since there is no apparent failure at all. See my paper on “Models & Reality: How did models divorced from reality become epistemologically acceptable?” for a deeper discussion.

RODRIK: Mathematical form guarantees transparency. The critical assumptions and derivations are out there for all to see. This means that models are fine, but can be misused by those who ignore the conditions required for use of these models.

MÄKI: This is not true; use of mathematical models does not create transparency. {{AZ: For a specific example of how un-identified critical assumptions can be buried within a maze of mathematics, see Romer’s Trouble With Macro, }}. In addition, use of mathematics creates a tendency towards use of substitute models, where mathematics substitutes for reality. As Rodrik himself admits:  “too many economists fall in love with math […] Excessive formalization – math for its own sake – is rampant in the discipline.” The effect of this failure to connect mathematics with reality is also admitted by Rodrik:

“Asked point-blank, they can state chapter and verse all the assumptions needed to generate a particular result […] But ask them whether the model is more relevant to Bolivia or to Thailand, or whether it resembles more the market for cable TV or the market for oranges, and they will have a hard time producing an articulate answer.” (p172)”

In response to this admission, Mäki makes his most stringent comments:

In other words, many economists don’t quite know what they are doing, and are doing too little. [ …They don’t know …] how their models are connected to any external targets and what specific purposes they can be supposed to serve. What is remarkable is that this is accepted practice: “accepted practice does not require economists to think through the conditions under which their models are useful” (Rodrik: p172). Thus Rodrik admits that the disciplinary conventions of economics are flawed – instead of merely economists misbehaving in not abiding with the proper conventions. So, economists often fail to understand the models they use, and therefore often misuse them.

RODRIK: “The antidote of a bad model is a good model” rather than “no model” (29).

MÄKI: This may sound like a pro-gun person saying there are good guns and there are bad guns. A more sympathetic reading is that Rodrik is trying to say that models should be evaluated with respect to the purpose for which they are being used. But this relativity of “goodness” creates a host of problems not considered by Rodrik. We can always find a “purpose” to turn any model into a good model. With this framework, Rodrik’s understanding of progress can be parsed as follows:

Rodrik’s idea of how economics makes progress in terms of models: Making progress, as well as creating a capacity of making progress, are of course kinds of success. Rodrik suggests that progress in economics is different from progress in natural sciences (p71) as economic knowledge does not accumulate “vertically” by rejecting earlier bad models and replacing them with new good models, but rather “horizontally” by expanding the pool of models (p67). “The newer generations of models do not render the older generations wrong or less relevant … Older models remain useful; we add to them.” (71) Taken literally, this would seem to imply that all economic models ever built are relevant for some present (or perhaps future) respectable purpose.

{{AZ: This ties in closely to my paper on “Models and Reality: How Did Models Divorced from Reality Became Epistemologically Acceptable?”. Rodrik takes the position that all models are good (for some unspecified purpose). So conflicts with reality, and modifying and improving models are irrelevant. Economics does not progress by replacing bad models by good models, but by expanding the collection of models. Given that these models are “substitutes” for reality, there is never any need to examine them for match with reality. Unless this anti-scientific methodology is abandoned, there is no hope of progress.}}

Rodrik on the Sociology of Knowledge: “The authority of [a piece of research] derives from its internal properties – how well it is put together, how convincing the evidence is – not from the identity, connections, or ideology of the researcher.” (p78)

Mäki: This is a blind spot in Rodrik. Rubinstein writes that the elites exercise tremendous influence: “The job market for junior economists is an illustration of the unfairness associated with the power of the elite.” {{AZ: Romer explains that he can be critical of economics and economists in ways that insiders cannot, because he is immune to the power of the elite. See Quotes Critical of Economics for a Krugman quote showing that one had to wrap real economics within an artificial framework required by the dominant ideology. Similarly, Card and Krueger complain of severe alienation caused by their rejection of the sacred cow of Supply and Demand, which led to their abandonment of research on the topic.

4 thoughts on “Mäki Rules on Rodrik’s Rules

  1. If models have to have a purpose -p, is this true of blog posts? What is the purpose of this blog? is it to suggest that all economics departments should be closed down? If economics cannot be separated from the sins of economists, should the practice of economics be prohibited? Where are we going with this argument?

    There are at least two possible responses to bad economics. One is to try and do good economics. Another is to moan endlessly about economists in general. The second response appears to be more popular than the first – no doubt because it is much easier.

  2. This is very useful, thank you. It is going to stimulate me to getting round to reading Maki’s paper. I had the same reaction to reading Rodrik’s book: I had always regarded him as one of the better economists around, and found his defence of conventional economic modelling really disappointing. And I totally agree with your comment, “Given that these models are “substitutes” for reality, there is never any need to examine them for match with reality. Unless this anti-scientific methodology is abandoned, there is no hope of progress”.

    On the question of how modelling can be used in a way that is far from anti-scientific, in fact it is an enhancement of science, I think it is salutary to compare the type of modelling that is common in economics with the modelling that has been done to inform the policy response to the Covid-19 pandemic, e.g. Ferguson et al 2020 (at https://www.imperial.ac.uk/media/imperial-college/medicine/sph/ide/gida-fellowships/Imperial-College-COVID19-NPI-modelling-16-03-2020.pdf). Also Brian Castellani’s blogs:
    https://sacswebsite.blogspot.com/2020/03/modelling-coronavirus-why-all-public_19.html
    https://sacswebsite.blogspot.com/2020/03/part-2-modelling-covid-19-so-what-does.html
    https://sacswebsite.blogspot.com/2020/03/part-3-simulation-and-coronavirus.html
    https://sacswebsite.blogspot.com/2020/03/part-4-social-networks-and-coronavirus.html

    The next question is, given that the practice of economics by a large proportion of economists is fatally flawed, what to replace it with. For my ideas on this, see https://evidence-based-economics.org/, especially section A that contains short articles/blogs, and for more depth my paper “Causal theories, models and evidence in economics—some reflections from the natural sciences”, which is B1 on the website, and also available at https://www.tandfonline.com/doi/full/10.1080/23322039.2017.1280983. Other people will have other ideas to contribute – the key thing is never to lose sight of the need to come up with better methodology, and to use it in practice, not just to criticise existing economists/economics.

  3. I have now read Uskali Maki’s paper. I have to say that although it is very impressive, it is also disappointing, because it remains in reactive mode. It does not question the focus on models, but instead goes into great detail about how models should operate. I suppose I should not be surprised, because his work is mainly focused on models in economics. But to start doing good economics, modelling is not a good place to start.

    The natural sciences, like biology that also faces a highly complicated, open-ended reality, start by uncovering the causal processes that bring about the various phenomena of interest. Modelling comes later, embedded in empirically-based causal theories that explain these phenomena. (Note that by “theory” here, I do not mean what economists usually mean, which is indistinguishable from modelling – it is the same usage as in “the germ theory of disease”, i.e. an empirically-based causal theory.)

    For links to more of my writings on this, see my previous post – especially the paper “Causal theories …”.

  4. Rodrik seems a decent enough sort. He is obviously intelligent and concerned about actual events and people. But he is hampered by being an economist. When a member of the general public can learn more about the current Western economic arrangements from the movie “Wall Street” than from any economist, including Rodrik, what exactly is the usefulness of economics and economists? These are some of the insights on the 2008 crisis by Gillian Tett, an anthropologist who writes for the Financial Times.

    The position of Timothy Geithner, the youthful president of the New York Federal Reserve, was more ambivalent still. Unlike the men running the Fed and the Bank of England, Geithner had no background in academic economics. He arrived at his post in October 2003, aged just forty-two, after a career in the Treasury. Free from rigid economic dogmas, he was a deeply pragmatic man who sometimes observed that his aim in life was merely to do “the least bad job possible.” Like almost every other American policy maker and official, Geithner believed that in an ideal world, banking should be based on the principles of free-market competition. In the real world, though, he recognized that governments sometimes needed to jump in. In his eyes, the financial system was often plagued with what he called “collective action problems”—or cases when the banks were so busy pursuing their own interests in a competitive and greedy fashion that they failed to rationally consider long-term outcomes. Competitive forces, in other words, did not always produce efficient or safe results, Geithner believed. (Fool’s Gold, 2009)

    Or, I will add not socially beneficial ones.

    Chapter Three related the tale of UBS to show that silos inside institutions can also make people blind to risks. This chapter provides another twist on this silo issue by telling the story of what happened to economists at places such as the Bank of England and the London School of Economics (or the U.S. Federal Reserve and Harvard University) before the 2008 crisis. Silos do not just arise inside institutions. They can also affect entire social groups. Networks of experts can become captured by silos, in the sense of displaying blinkered thinking and tribal behavior, even if they work in different institutions and countries. This is not a problem that is unique to the economic profession, any more than silos are an issue that is just found in banks. However, the story of the economics tribe is distinctly revealing, not least because it shows how skilled experts can become so confident in their own ideas that they end up missing dangers hidden in plain sight. Like the villagers in Bourdieu’s dancehall, economists were so busy watching the “dancers” (or the pieces of the economic picture that everyone was expected to watch), that they ignored the “nondancers” (or the parts of the picture shrouded in social silence). “Why did the crisis happen? It was partly about epistemology, the knowledge systems that we used,” Paul Tucker, the deputy governor of the Bank of England, later observed. Or as Charles Goodhart observed: “[The credit crisis] isn’t really just a story about the structure of the Bank, or the Federal Reserve or any other organisation, but about the mental map we used—in academia, in policymaking, everywhere.
    “Ideas matter and economists were all using the same ideas.” They were sitting in the same mental silo. (The Silo Effect: The Peril of Expertise and the Promise of Breaking Down Barriers, 2015)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.