2-Simpson’s Paradox

In the previous post (1-Simpson’s Paradox), we constructed an example where both Humanities and Engineering Departments favor women, giving them substantially greater % admissions than men. However, overall admit ratios for the university favor men, who receive greater % admissions than women. This reversal is called the Simpson’s Paradox. Once the causal basis of the paradox is understood, it becomes very simple; however, standard econometric and statistical analysis completely ignores causality, and continues to be puzzled. In a nutshell, the explanation is as follows. There are TWO causal factors which influence admit rates. One is gender – being a women is a plus. The second is department – applying to Engineering is a plus. When women apply more to Humanities and men apply more to Engineering then these two factors work at cross purposes and the adverse effect of Humanities can overwhelm the favorable effect of being woman. However, causal structures can be more complex, and call for different types of analyses, as we now proceed to discuss.

CONTINUING from previous post (1-Simpson’s Paradox) —

Berk’s idea that apparent discrimination against women (displayed in the lower overall amit ratio for women) is due to women choosing to apply to the more difficult department (Humanities) is only a conjecture. Many other possible causal factors could change this conclusion. For example, suppose that admissions are gender-blind, and based solely on grade point averages.  Suppose that in Engineering admit percentages are (A:90%, B:70%, C:50%) while in Humanities the admit percentages are (A:50%,B:30%,C:10%). In this case, if the female applicant pool is divided into 60% A’s, 30% B’s and 10% C’s, while the male applicant pool has significantly lower grade profile of 10% A’s, 30% B’s, and 60% C’s, then exactly the same admissions patterns would be observed, but there would be no discrimination by gender. Then the causal question would become – why do better male candidates not apply to Berkeley, while better female candidates do? The answer could be that there is a better all-male university which is preferred by males by not available to females (for example). These are the “unobserved, real, structural” factors which are not present in the numbers being analyzed.

The main point here is that the numbers, by themselves, do not contain causal information. However, understanding the meaning of the numbers requires understanding of the causal factors. Econometrics methods currently in use are deeply flawed because they provide us with no way of inputting relevant causal information and deriving results which vary according to the causal sequencing. There is an implicit assumption in econometrics that causality does not matter, and the Simpson’s paradox shows that this assumption is wrong. Causality is of essential important in understanding numbers. For the above data set, the causal sequence hypothesized by Berk can be graphed as follows.

Gender affects choice of department, and ALSO affects the admit ratio. The Departments chosen also have an affect on the admit ratio. In this causal diagram, if we want to study the effect of department choice on the admit ratio, then Gender is a confounding variable. It affects both choice of department and the admit ratio. The SOLUTION to the problem of confounding here lies in CONDITIONING on the confounder. That is, hold gender constant, and do calculations separately for females and for males. With this conditioning, we can correctly calculate effects of departments choice on admit rates, for each gender separately. If the goal is to calculate the effect of Gender on Admit_Ratio, then Department is NOT a confounding variable. Rather, Gender acts on Admit Ratio in TWO DIFFERENT ways. One is a direct effect, and the second is an indirect affect through choice of department. Both the direct and indirect effect must be taken into consideration, and we CANNOT condition on department. If John and Jane apply to Berkeley, and we do not know which department they have applied to, then we should use the overall rates 56% and 44% for their chance of admission. This can be seen by the following tree diagrams which traces the probabilities of the various outcomes and paths. The gender affects both choice of department and admit ratios, so the decision tree diagram is drawn separately for males and females. For females, the decision tree looks like this:

 

By tracing the branches, we can calculate the probabilities of all the outcomes. Only 10% of Females choose Engineering, but those who do have an 80% chance of admission. The 90% females who apply to Humanities have a 40% chance of admission. Overall,  Female admission probability is 90% x 40% + 10% x 80% = 36% + 8% = 44%.

Because Gender affects both choice of department AND admit probabilities, we must condition on gender and draw separate diagrams for males and females. The same diagram for males looks like the following:

Males have a 90% chance of applying to Engineering (or 90% of males choose to apply to engineering), and a 60% chance of admission in this department. For the 10% Males who apply to humanities, the chance of admission is only 20%. Note that BOTH departmental admission probabilities are LOWER than those for females. However, overall Male admission probability is 10% x 20% + 90% x 60% = 2% + 54% = 56%, which is higher than the 44% admission probability of Females. This is the Simpsons Paradox, where it seems that all departments within the university favor females, but the university as a whole favors males. The paradox arises because gender affects admissions through two channels – one is the direct channel of the admit rate which is better for females, and the other is the indirect channel of choice of department which again affects the admit rate.

EXERCISE: To understand this better, modify the numbers so that admissions is gender-blind in both departments of Humanities and Engineering. Men and Women are admitted in exactly the same proportions. However, make Humanities tougher, with lower admit ratios, and Engineering easier, which higher admit ratios. Show that if a greater percentage of women apply to Humanities while a greater percentage of men apply to Engineering, we will see apparent discrimination against women in the overall admit ratios — women will have lower overall admit ratio, while men will have higher overall admit ratios.

In the causal diagram of Berk, Gender is a confounding factor which impacts on Department Choice and on Admission Rates. If we want to study Admission Rates by Department, then we must condition on Gender, holding it constant. However, in this same diagram, Department is NOT a confounding variable when it comes to assessing the impact of gender on admissions rates. This is because choice of department is PART of the gender effect. Women CHOOSE the more difficult departments, so the choice cannot be separated from the gender. This point is a bit subtle and complex, and failure to understand it creates massive confusions in discussions of confounding. Deeper understanding can be developed by studying a variety of causal structures which can generate the same patterns of admissions, and noting how they lead to entirely different conclusions about the cause of, and the remedies for, discrimination.  We proceed to analyze different causal structures for this pattern of admissions. The DEEPER point we are pursuing here is the NEED to look beneath the surface of the data. The causal structures are NOT PRESENT in the observable statistics, and yet these unobserved real structures MUST be taken into account for a sound data analysis.

NEXT POST: Alternative Causal Structures for Admissions (3-Simpson’s Paradox)

 

 

1-Simpson’s Paradox

The previous 6 posts have discussed some aspects of the methodology of modern economics, in order to understand why current models are so disastrously wrong, and how they could be fixed. These posts are listed below

• Unrealistic Mental Models 6
• Three Types of Models 5
• Errors of Empiricism 4
• Thinking about Thinking 3
• Models and Realities 2
• Mistaken Methodologies of Science 1

For the next 3/4 posts, we will temporarily switch topics, and discuss the Simpson’s Paradox. The reason for this is that it provides a crystal clear and concrete illustration of some of the abstract and vague concepts about models that we have been discussing. In particular, we will see that econometrics used “observational” or “Baconian” models. The hidden real-world structures which generate the connections between observations are the causal sequences. These unknown causal sequences radically affect how we interpret the observed data. So it is impossible to ignore real-world structures in a meaningful data analysis. Yet, econometrics does data analysis without incorporating causal information – in fact, econometrics does not even contain the language required to express causal linkages. As a consequence of this attempt to do the impossible, econometrics ends up doing massively meaningless data analyses. The clearest and most transparent way to establish this is to study the Simpson’s Paradox in detail, which we proceed to do now

Recent Revolution in Causal Inference

There has been a revolution in terms of understanding causal inference, launched by Judea Pearl and associates, and based on a graph-theoretic approach. As  Pearl, Glymour, and Jewell (2016. Causal Inference: A Primer) state: “More has been learned about causal inference in the last few decades than the sum total of everything that had been learned about it in all prior recorded history. … Yet this excitement remains barely seen among statistics educators, and is essentially absent from textbooks of statistics and econometrics.” Current practice of econometrics is very much like archaic medical treatments, which inflicted more pain and injury to the patient than the disease. Those who make the effort to learn the theory of causality, can be pioneers of an exciting new approach to statistics and econometrics, which will allow us to distinguish between real relationships and spurious ones. For example, use the WDI data set and regress almost any country’s consumption on any other country’s GNP. In more than 90% of the cases, this will give a highly significant relationship. The data do not offer us any clue as to how to tell the genuine relationship – consumption of a country on the GNP for the same country – from the fake one. Similarly, the following regression of Life Expectancy in a country regress on the log of the number of newspapers published provides and excellent and robust fit. How can we tell whether publishing more newspapers will lead to a rise in Life Expectancy, or whether this is a spurious regression?

LE (Life Expectancy) = 45.0 + 5.48 LN (Log Newspapers per Capita)+ Error

Conventional econometrics currently being taught to students around the globe has no answers to these questions. Study of causality offers us answers to these essential questions. The topic is not difficult, but it is very different from what we have learnt in econometrics before, so it requires adjusting our mindset and some  flexibility in thought. This is an introductory article which explains how it is essential to explicitly consider and model causality (contrary to conventional econometric practice), in order to extract meaningful information from any data set. We begin with a discussion of Simpson’s Paradox, which provides a clear illustration of how and why it is necessary to understand causal linkages, in order to do sensible data analysis. The analysis below is based on real examples. However, we have deliberately changed the numbers to make the calculations easy, and to use identical numbers across several examples. This is to show that dramatically different analyses are required when the unobserved and hidden causal structures are different, even though the actual numerical data remains exactly the same. This point is rarely highlighted in texts, which create the contrary impressions that the data by itself provides us with sufficient information to enable analysis. This illusion is especially created and sharpened by “Big Data” and “Machine Learning” technologies, which appear to inform us that data by itself, in sufficient quantities, can provide us with all necessary information.

The Berkeley Admissions Case

Suppose that there are only two departments, Engineering (E) and Humanities (H), at Berkeley. Engineering Department has a higher admit rate while Humanities has a lower admit rate in general. For Female Applicants, 80% are admitted and 20% are rejected in Engineering, while 40% are admitted and 60% are rejected in Humanities.

Question 1: What is the OVERALL admit rate of Female Applicants into Berkeley?

Female Admit Ratios for Engineering and Humanities

Answer 1: The data given does not allow us to determine this. If all females apply to engineering and none to humanities, then the overall admit rate for females would be 80%. If all females applied only to humanities and none to engineering the overall admit rate would be 40%. For other combinations, the overall admit rate would be a weighted average of the two numbers 80% and 40%. Here is a table which illustrates the possibilities for Female Applicants: (see also,  LINK to Table  showing how overall Female Admit Ratio depends on proportions of appllicants to Engineering and Humanities)

Engineering (females)			Humanities (females)			Totals (females)
Applied	Admits	%Admit	Applied	Admits	%Admit	Applied	Admits	%Admit
1800	1440	80%	200	80	40%	2000	1520	76%
1500	1200	80%	500	200	40%	2000	1400	70%
1000	800	80%	1000	400	40%	2000	1200	60%
500	400	80%	1500	600	40%	2000	1000	50%
200	160	80%	1800	720	40%	2000	880	44%

As the table shows, the overall admit ratio is a weighted average of the two admission percentages of 80% and 40%. The overall admit ratio is a weighted average of these two numbers, where the weights depend on the proportion of females which apply to Engineering and Humanities.

Lower Male Admit Ratios in Each Department

Next suppose that Berkeley discriminates systematically against men. In each of the two departments the admit ratios for males are significantly lower than those for females. For example, suppose that only 60% of male applicants get admission into engineering (compared to 80% for females). Also suppose that only 20% of males get admitted to Humanities (as opposed to 40% for females). A table similar to the one above for females can be constructed as follows: (see also, LINK to table)

Engineering (males)			Humanities (males)			Totals (males)
Applied	Admits	%Admit	Applied	Admits	%Admit	Applied	Admits	%Admit
1800	1080	60%	200	40	20%	2000	1120	56%
1500	900	60%	500	100	20%	2000	1000	50%
1000	600	60%	1000	200	20%	2000	800	40%
500	300	60%	1500	300	20%	2000	600	30%
200	120	60%	1800	360	20%	2000	480	24%

As for females, the overall admit ratio for males is a weighted average of 60% and 20%, with weights proportional to applicants in in Engineering and Humanities.

Simpson’s Paradox

Now it is easy to see that if proportionally more males apply to engineering, the overall admit rate for males would be closer to 60%. If proportionally more females apply to humanities, the overall admit rate for females would be closer to 40%. So, it is possible that the overall admit rate for males is GREATER than the overall admit rate for females. For example, as the above tables show, suppose that of 2000 male applicants, 1800 apply to Engineering and 200 apply to Humanities. Then overall admission rate for males will be 56%. On the other hand, suppose that 200 females apply to Engineering and 1800 apply to Humanities. Then overall admit rate for females will be 44%, which is much lower than 56% for males.

This is what leads to the paradox. Suppose someone does not have detailed data at the departmental level, but has the overall figures. He will see that 2000 males applied to Berkeley and 1120 (56%) were admitted. At the same time, 2000 females applied to Berkeley and only 880 (44%) were admitted. Running a statistical test (as Berk et. al. 1975  did) leads to a clear-cut conclusion of discrimination against females, on the basis of the overall data. Yet at the departmental level, the same statistical logic shows that each department strongly favors females over and above males. The paradox is that there are only two departments. Each department favors women. However, the university as a whole favors men. How can that be? Another way to put the question is: should females sue Berkeley for discrimination against women, on the basis of overall admissions ratio being strongly biased against women? Or, should the males sue Berkeley for discrimination against men, on the basis that each department is heavily biased against men when it comes to admission ratios? The analysis by Berk comes to the following conclusion. They argue that the department-wise data is reliable, and Berkeley discriminates in favor of women in each department. However, women choose to apply to the difficult department – Humanities – while men choose to apply to the easy department – Engineering. Because of this preference of women for humanities, they end up with a lower admit ratio than that of men.

TO BE CONTINUED – In the next post (2-Simpson’s Paradox) we will see that Berk’s explanation is only one of many possible underlying causal structures. Each of the different possibilities has radically different implications, so we cannot afford to be ignorant or abstain from thinking about the hidden structures (as counselled by Kant, by Empiricists, and by many other categories of philosophers of science). This is the problem at the heart of modern methodology for economics – it gives up on the attempt to figure out hidden causal structures, making it impossible to make progress in understanding the real world around us.

Postscript: A summary and overview of all five posts, with links to each one, is available from the RWER Blog Post on Simpson’s Paradox.

Unrealistic Mental Models 6

In the previous post (Three Types of Models 5), we discussed three types of models. The first type is based purely on patterns in observations, and does not attempt to go beyond what can be seen. This is an “observational” or Baconian model. The second type attempt to look through the surface and discover the hidden structures of reality which generate the observations we see. The best approach to this type of models has been developed by Roy Bhaskar, so we can call it a critical realist model or a Bhaskarian model. The third type of model creates depth and structures in our minds which create the patterns we see in the observations. The question whether our mental structures match reality is considered irrelevant. These may be called Kantian, or mental models.  The models of modern Economics are largely Kantian, while Econometric models are largely Baconian. The key defect of both of these approaches is that they GIVE UP on the idea of finding the truth. Max Weber’s ideas about methodology played an important role in this abandonment of the search for truth, but it would take us too far away from our current concerns to discuss this in any detail. Briefly, Weber thought that heterogeneity of human motives made “explanation” of social realities via “truth” impossibly complex. Instead, he argued that we should settle for a weaker concept based on “ideal-types” – deliberately over-simplified models of behavior which create a match to observed aggregated patterns of outcomes. We now discuss the disastrous consequences of this abandonment of truth in greater detail.

There is a famous article of Milton Friedman on methodology in economic theory, which recommends the abandonment of truth: “Truly important and significant hypotheses will be found to have “assumptions” that are wildly inaccurate descriptive representations of reality, and, in general, the more significant the theory, the more unrealistic the assumptions.” For more details, see “Friedman’s Methodology: A Stake Through the Heart of Reason.”  What Friedman expresses, in inaccurate language, is the idea that an assumed structure of reality which is a mental model designed to match observations, need not match the true hidden structures of reality. All that matters is that observable implications of the model match the our observed data. This idea is called “saving the appearances”. For example, if we imagine that there is a heavenly sphere surrounding the earth and the moon is pasted on that sphere. Motions of the moon occur because of the rotations of the sphere. According the idea of “saving appearances”, as long as the observed motion of the moon matches the predictions of our model, we need not be concerned with whether or not the heavenly sphere actually exists.

This is fundamental methodological mistake at the heart of economics: the idea that we can make up any crazy model we like. As long as our models produce a match to the observations, it does not matter if we make wildly inaccurate assumptions. This has led to DSGE models, currently the dominant macroeconomic models, which have been held responsible for the fact that the profession of economists as a whole was blindsided by the Global Financial Crisis. Economists make completely unrealistic assumptions without any discomfort, because of Friedman’s idea that “wildly inaccurate” assumptions will lead to truly important and significant hypothesis. In a previous portion of this article, we documented the fact that economists are not bothered by conflicts between their models and reality. Below we provide quotes which document the crazy models that now dominate economics because of adherence to Friedman’s Folly: the crazier the assumptions, the better the model.

Keynes: The classical theorists resemble Euclidean geometers in a non-Euclidean world who, discovering that in experience straight lines apparently parallel often meet, rebuke the lines for not keeping straight as the only remedy for the unfortunate collisions which are occurring. Yet, in truth, there is no remedy except to throw over the axiom of parallels and to work out a non-Euclidean geometry. Something similar is required today in economics. (GT)

Solow: Suppose someone sits down where you are sitting right now and announces to me that he is Napoleon Bonaparte. The last thing I want to do with him is to get involved in a technical discussion of cavalry tactics at the battle of Austerlitz. If I do that, I’m getting tacitly drawn into the game that he is Napoleon. Now, Bob Lucas and Tom Sargent like nothing better than to get drawn into technical discussions, because then you have tacitly gone along with their fundamental assumptions; your attention is attracted away from the basic weakness of the whole story. Since I find that fundamental framework ludicrous, I respond by treating it as ludicrous — that is, by laughing at it — so as not to fall into the trap of taking it seriously and passing on to matters of technique.

Narayana Kocherlakota: Minneapolis Federal Reserve President (2010-2015), “Toy Models”, July 14 2016  “The starting premise for serious models is that there is a well-established body of macroeconomic theory… My own view is that, after the highly surprising nature of the data flow over the past ten years, this basic premise of “serious” modeling is wrong: we simply do not have a settled successful theory of the macroeconomy.”

Olivier Blanchard IMF Chief Economist (2010-2015), “Do DSGE Models Have a Future?”, August 2016  “DSGE models have come to play a dominant role in macroeconomic research. Some see them as the sign that macroeconomics has become a mature science, organized around a microfounded common core. Others see them as a dangerous dead end…”  and “There are many reasons to dislike current DSGE models. First: They are based on unappealing assumptions. Not just simplifying assumptions, as any model must, but assumptions profoundly at odds with what we know about consumers and firms.”

All of these authors are expressing the same complaint, in different forms. Structures of our Mental Models have no match to the true Structures of External Reality. The only job mental models have to do is to produce a match to the observed data. Whether or not mental models are realistic has no bearing on whether or not they are good models. There is complete lack of concern about whether our mental models make assumptions which are realistic. The mystery of how models based on false assumptions can help us “understand” and “explain” the real world has been the subject of a long and complex methodological debate. For example, a leading methodologist, Mary Morgan, writes that “Despite the ubiquity of modelling in modern economics, it is not easy to say how this way of doing science works. Scientific models are not self-evident things, and it is not obvious how such research objects are made, nor how a scientist reasons with them, nor to what purpose.” In the “Explanation Paradox”, Julian Reiss writes that it is widely accepted that: (1) economic models are false; (2) economic models are nevertheless explanatory; and (3) only true accounts explain. A whole subsequent issue of the Journal of Economic Methodology is devoted to the attempt to EXPLAIN how all THREE of Reiss’ premises can be true. Alexandrova and Northcott – philosopher-outsiders – point out the obvious: economic models do not explain. However, this simple explanation falls on deaf ears; economists are too much addicted to meaningless mathematical models to realize that these models are mental structures which are “castles hanging in the air, having no contact with reality”. For full references and discussion on these issues, see  Tony Lawson’s Beyond Deductivism 

NEXT: We plan to discuss how econometric models are mostly observational models, while economic models are mental models. However, differentiating econometric models from real structural models requires a discussion of the hidden and unobservable causal structures which generate the observable data. This is done in a sequence of five posts on Simpson’s Paradox. These posts provide a leisurely pedagogical introduction to the topic, and also relate the paradox to causal structures, something not available in the conventional statistical literature on the subject. After covering Simpson’s Paradox, and some related materials on causality and econometric regression models, we will come back to the present topic – the nature of economic and econometric models, and why those models create results conflicting with easily observable reality.

Three Types of Models 5

It is important to understand that there are three type of models, corresponding the following diagram. The simplest type of model is a pattern in the data that we observe. A second type of model is a “mental model”. This is a structure we create in our own minds, in order to understand the patterns that we see in the observations. The third type of model is a structure of the hidden real world, which generates the patterns that we see. Some examples will be helpful in clarifying these ideas about the typology of models.

 

Empirical Models: The simplest kind of model consists of a pattern that we see in the observations. For example, if we see the sun rise every day for many years, this is a pattern in our experience. It leads us to conjecture the law that “the sun rises every day” – where the law extends beyond the range of our experience and observations. This is just a guess, based on patterns we see in the data. A regression model is an excellent example of an empirical model. It identifies patterns in the data, without any concern for the underlying realities. For example, a regression of Australian consumption per capita on China’s GDP gives an excellent fit –

Australian Consumption =  a + b Chinese GDP per capita + error (high R-squared, significant t-stats)

This shows us that there is a pattern in the data – increases in China’s GDP go along with increases in Australian consumption. The regression cannot answer the question of why there is this pattern. Any two series of data can display correlation – time series measuring numbers of sunspots sighted on the sun’s surface can correlate with a wide variety of economic phenomena. The regression model which picks up this relationship has nothing to say about the reasons for the correlation. Given any kind of data, we can always find some regression relationship. For example, here is a regression which has a strong fit, but no meaning.

Pakistani Consumption = a + b Survival rate to age 65 of Females + c Pollution Levels by Carbon Monoxide + error

In terms of classification – we can find many different kinds of patterns in any arbitrary set of data. Whether or not the patterns have meaning depend on the real-world processes which generate these patterns. This is something which Real Models are meant to explore.

Real (Structural) Models: The empirical models look at the surface structure, the appearances, the data that is based on observations. Structural models try to explore the hidden structure underneath the appearances. Consider for example a regression of consumption per capita on GNP per capita

C = a + b Y + epsilon

From the point of view of an empirical model, this is a pattern in the data. The names of the variables do not matter. If the consumption is Australian and the GNP is Chinese, the pattern is the same as if both variables belong to the same country. The names of the variable, and the relationships between them, matter only when we think about real structural models. For example, if we think that consumers earn incomes, and then spend some proportion of the income on consumer goods, this is a real structural relationship which explains why we see the pattern in the regression relationship. This structure justifies regressing Australian consumption on Australian GDP, but not on Chinese GDP. Also, if the determinants of GDP are the production processes, we cannot reverse the variables and run a regression of GDP on Consumption. Consumption is not a determinant of GDP. For an empirical model, C on Y and Y on C are the same patterns. Correlations are symmetric, but causal relationship are one directional. Real Structural Models attempt to find hidden real variables which cause the patterns that we see. For example, the tendency of consumers to consume a proportion of their income is the hidden cause for the surface data relationship between consumption and income within a country.

Mental Models: A pattern in the data is just a pattern – there is no explanation for it. This is the Baconian model of science. If we see a pattern in the data, we deduce that a law holds which generates this model. Any pattern that we see could be a law. A mental model imagines a structure of reality which could be an explanation for the reality. For example, an aggregate consumption function can arise from individual consumers who optimize utility derived from consumption bundles subject to budget constraints. It could also arise from consumers who make completely random consumption decisions, while staying within their budget. Any imaginary structure of reality which leads to observations which match what is actually observed is a mental model.

Originally, mental models were designed by thinking about what the nature of hidden reality could be, and then trying to build a mental model to match that hidden structure. However, post-Kant, the main idea became different. Trying to match hidden reality was abandoned, and instead, the goal of the model became to create a match to the observations. As a result, many concepts which are of vital importance to modelling reality were abandoned or misunderstood. For example, the idea of causation is of great importance in understanding reality. Rubbing a match against sulfur on the matchbox causes the match to burn. Learning about causation is of extreme important in learning to navigate the world we live in.    Our mental models are supposed to be representations of reality. For complicated reasons, economists FORGOT this basic idea about the nature of mental models, that they are supposed to capture the hidden real mechanisms which generate the observations. This has been an empiricist tendency starting from Hume. The idea that we cannot talk about hidden unknown realities has deep roots in Western intellectual rejection of God and religion. As already discussed, Kant suggested that we can create a Copernican revolution in philosophy by changing the focus of our inquiry into the world. The following diagram explains the current Empiricist or Kantian views about models and reality. All that matters about mental models is that they should provide a match to the observations. It does not matter whether or not they match the true structures of reality which produce the observations.

Philosophers have thought for ages about the problem of how we can find out if our mental models match the reality, the hidden unknown structures. But this is the WRONG question (according to Kant and the empiricists). We can never find out the answer, because the true hidden structures of reality will NEVER be observable. So, we should abandon this ancient question. Instead, we should focus on the question of how our mind organized the observations into a coherent picture of apparent reality. The diagram below shows the Kantian shift of focus. Traditional philosophy is concerned with the question of whether or not our mental models MATCH the hidden structures of the real world. This is the question of whether or not our models are TRUE. Kant and the empiricists said that this was impossible to know. We should only be concerned about whether or not our empirical models provide a good fit for the observations. So, the question itself was changed. Instead of asking if models match reality (and hence, whether or not they are TRUE), we ask whether the output of the models provides a match to the observations.

Another useful analogy is to consider the theory of vision: How we see the world. What we REALLY see is a pair of flat 2-D upside down images of the world on our retina. Our brain has the complex task of reconstructing the real 3-D world out there from this imperfect scan of it on our retina. For more detailed discussions of this process, see “How Our Eyes See Everything Upside Down”, or “Introduction to the Science of Vision”Introduction to the Science of Vision”.  Simple minds just EQUATE the mental image with the reality out there — this is the easiest model. But there are many optical illusions which can be used to show that our methods for interpreting images on our retina do not always reproduce the real world out there – A mirage, an illusion of water, being the simplest of them. The Kantian question is to consider how our mind creates a 3-D image of reality from the imperfect 2-D pair of images available as observations. The Real Philosophers ask about the match between our mental image and the complex reality.

NEXT POST:  Unrealistic Mental Models 6 

Previous Posts in this sequence on Models and Reality: Mistaken Methodologies of Science 1, Models and Realities 2, Thinking about Thinking 3, Errors of Empiricism 4,

 

 

 

Errors of Empiricism 4

Empiricism holds that observations are all that we have. We cannot penetrate through the observations to the hidden reality which generates these observations. Here is a picture which illustrates the empiricist view of the world:

The wild and complex reality generates signals which we observe using our five senses. The aspects of reality which we can observe are the only things that we can know about reality. The true nature of hidden reality, as it really is, independent of our observations, is unknown and can never be known to us. Influential philosopher Kant calls it the “thing-in-itself”. Noumena is the wild reality, and Phenomena is what we can perceive/observe about the reality. Quote from Encyclopedia Britannica:

Noumenon, plural Noumena, in the philosophy of Immanuel Kant, the thing-in-itself (das Ding an sich) as opposed to what Kant called the phenomenon—the thing as it appears to an observer. Though the noumenal holds the contents of the intelligible world, Kant claimed that man’s speculative reason can only know phenomena and can never penetrate to the noumenon.

Kant was enormously influential in de-railing the philosophy of science (See  Kant’s Blunder). Prior to Kant, philosophers understood science in the way we have explained in the beginning: science is about looking through the appearances in order to understand the hidden reality. However, Kant argued that this was an impossible task. All we have is appearances (phenomena), and we cannot look through them to get at the underlying hidden realities (noumena). He proposed that instead of studying the relation between appearances and reality, we should study the relationship between our thought process and the observations of the real world:

It is important to understand Kant, because his way of thinking is at the heart of Economic Modeling today. To get a deeper understanding of Kant, we provide several arguments favoring his views. Think about how a simple computer camera looks at the world. The area being looked at by the camera is represented as a square two-dimensional patch which is say 1000 x 1000 pixels. At each pixel, if the camera detects light, it puts a 1 and if it does not, it puts a 0. So, we end up with a picture of reality which is a 1000 x 1000 matrix of 1’s and 0’s. This is the OBSERVATION. Now how can we translate these observations into a picture of reality? This is the basic problem of computer vision – taking a stream of numerical inputs from the camera and translating it into a picture of reality. For example, a particular stream of 1’s and 0’s may be interpreted as a picture of a tree, by a computer vision program. As human beings, we face a similar problem. We don’t actually see the world out there. What we see is a reflection of the world within our eyes. Our minds process the image on our retina into a picture of the external reality.  Before Kant, most people thought that the image in our minds matched the external reality. What Kant said was that we have no way of knowing this. We have no way of knowing the external reality. All we can see is the image of it on our retina, and the interpretation of it in our minds. A Kantian model, which we will label a mental model later, explains how we convert streams of 0’a and 1’s into an image of reality.

 

For understanding the nature of models, we will need to keep these three things in mind. Reality generates observations. And our minds interpret observations as a picture of reality. Most of us think that the picture in our minds is exactly what the reality is. When I look at a tree, I do not say that my mind has interpreted an image on my retina as a tree. I say that there is a tree out there in external reality which I am seeing. However this is an over-simplified understanding. For example, when I see a mirage, I interpret the image on my retina as water, but in fact there is no water in external reality. Similarly, a fly has a compound eye, and sees the world in way which is very different from how we see it.

As opposed to Kant, traditional philosophy is concerned with the question of how the image we have formed relates to external reality (not to the bitstream of observations). Traditional philosophy would ask: which is the “correct” picture of external reality? What the fly sees or what we see? What Kant says is that there is no way to learn the answer to this question. We have no separate access to external reality apart from our observations. So instead of thinking about whether our mental pictures match true reality, we should think about how we process the stream of sensations we receive into an image (a model) of the world.  Favoring Kant, Evolutionary biologists argue that the picture that we see of the world tends to highlight those aspects which matter for our survival, and ignore or neglect those aspects which don’t. This means that the representation of reality that is captured by our senses has less to do with the true external reality, and more to do with our own survival. The point of all this is that the naïve idea that what we see is just a true picture of reality is not necessarily correct.

This idea of Kant, that we can and should abandon looking for truth – the true picture of reality – has had a powerful effect on the philosophy of science today. Especially in economics, models that we build have no relation to reality. Rather the models in use are ways of organizing our own thoughts about reality. This has led to models which are hopelessly bad. Furthermore, the IDEA that we do not need to try to match reality, has led to the impossibility of correcting bad models to make them better. All that happens is that bad models are replaced by more complex models which are even worse. To understand this better, we now discuss three types of models –

Next Post: Three Types of Models 5 

Previous Posts in this sequence:  Mistaken Methodologies of Science 1,  Models and Realities 2,  Thinking about Thinking 3,

Thinking about Thinking 3

When we think about epistemology (theory of knowledge), then we are doing meta-thinking. That is, we are thinking about thoughts people have, which they think is “knowledge”.  Because there are many many wrong ideas, and very few right ideas, we must learn to think critically. Unless we do so, our thoughts will be captured by the enormous amounts of fake news which circulates on social media these days. Thinking about thinking, or Meta-Thought, is very different from the standard education which students receive. Instead of asking about the “models” in use, and assessing adequacy or failure of their “assumptions”, at the meta-level we ask how economists began to use these models instead of others, what kind of thoughts are promoted by such models, and what kinds of thoughts are blocked, because the models are incapable of expressing such ideas. This kind of higher-level thinking is completely missing from conventional textbooks.

To highlight the differences, we consider as an illustrative example, how Martin Osborne begins his textbook on game theory, and explains what game theory is about:

GAME THEORY aims to help us understand situations in which decision-makers interact. … the range of situations to which game theory can be applied: firms competing for business, political candidates competing for votes, jury members deciding on a verdict  etc. etc. etc. .

Next, note what famous game-theorist Ariel Rubinstein has to say about this issue: “Nearly every book on game theory begins with the sentence: ‘Game theory is relevant to …’ and is followed by an endless list of fields, such as nuclear strategy, financial markets, the world of butterflies and flowers, and intimate situations between men and women. Articles citing game theory as a source for resolving the world’s problems are frequently published in the daily press. But after nearly forty years of engaging in this field, I have yet to find even a single application of game theory in my daily life”  (see Quotes Critical of Economics).

There is a strong conflict here. Martin Osborne tells us that game theory helps us to understand a huge variety of different situations. However, Ariel Rubinstein tells us that he has not been able to find even one useful application of game theory in forty years. Which of these two thoughts is correct? How can we tell who is wrong and who is right? We need to compare and evaluate these two thoughts, which required meta-thinking. Also, we are concerned with evaluating knowledge claims – what do we know, and what we do not know. So, this is a topic in epistemology.

Of central importance to us in resolving these issues is the concept of a “MODEL”. What is a model, and how does it relate to reality? Here is what Martin Osborne writes: “Like other sciences, game theory consists of a collection of models. A model is an abstraction we use to understand our observations and experiences. What “understanding” entails is not clear-cut.” This last sentence is revealing. Economists do not understand what a model is, and how it helps us to understand the real world. You will find often repeated assertions that “models are simplifications of reality” and that “models are always false”. These maxims are not helpful in understanding models. Actually the “simplification” that models perform is of a very special type == models set out for us “what matters” and also exclude “what does not matter”. The variables and descriptors we use are the ones which matter. Anything which does not enter into the theory does not matter. This despite textual assertions to the contrary — what we are being taught in the economics textbooks does not lie in the words that are written — it is contained in the words that are not written. By not writing about compassion for the hungry, and social responsibility, we are told that these are not relevant concepts for the economic system – these things do not matter. So one function of models, not explicitly mentioned, is to tell us what is important and to separate these variables from the large numbers of variables which do not matter. The second function is to specify the chains of causation. Consumers have incomes and they make consumption decisions. Investors borrow money to invest. Firms make production decisions. All of these theories provide a strong causal sequencing about how things happen, what happens first, what happens next as a consequence. This actually sets up the exogenous variables and the endogenous variables, again without any explicit mention of causality. A third aspect of models sets up superstructures as well as constructing RULES which are used to evaluate models.  These we will discuss later, when we discuss the three major categories of models, in the next section.  Also of essential importance is the question of how models “explain” – how they “help us to understand” – reality. This will also be discussed in greater detail later.

RELATED POSTS: On the Central Importance of a Meta-Theory for Economics. and  Meta-Theory and Pluralism in the Methodology of Polanyi   This is the 3rd post in a sequence; The first two are:  Mistaken Methodologies of Science 1  and  Models and Realities 2  The NEXT post is  Errors of Empiricism 4 

Models and Realities 2

Foundations for modern social sciences were laid in the early twentieth century, and were strongly influenced by logical positivism. The central idea of positivism is that science is true and valid because it deals (principally) with observables, while religion is false and invalid because it deals (principally) with unobservables. For a detailed discussion, see “Logical Positivism and Islamic Economics“.   Later, logical positivism had a spectacular collapse. It became clear to philosophers of science that the idea that we can base science purely on observables was seriously mistaken. Even those who were very strong proponents of positivism admitted that the philosophy was wrong. Strangely, this did not lead to rebuilding of the foundations for the social sciences. Especially in economics, the wrong philosophies about nature of human knowledge, which translate into bad theoretical models, continue to be used. A recent survey by Hands (2009) showa that the economists continue to believe in positivist philosophy, without any conscious awareness of this. The disastrously bad methodology in use by economists has led to models which failed to predict the Global Financial Crisis. What is worse, these positivist models continue to be used, even after the crisis. For reasons to be explained in detail, current methodology makes it impossible to learn from experience. THIS is the real source of the problem. Having a wrong theory is never a problem; it can happen to the best of scientists. The real problem is refusal to modify theories in light of experience, which makes it impossible to learn and improve. But this analysis is not sufficiently deep — precisely what is it about the methodology that makes it impossible for economists to learn from experience and to modify and improve theories in light of experience? Explaining this is the main goal of this sequence of posts. But, before proceeding to do this, it is worth documenting the stubborn resistance of economists to mere facts.

Keynes: Economists are unmoved by lack of correspondence between their theories and facts.

Sitglitz: Economists frequently make claims in conflict with easily observable facts, because economics is a religion, not a science.

Paul Romer: Macroeconomic theorists ignore mere facts by feigning an obtuse ignorance.

Olivier Blanchard: DSGE (are based on) assumptions profoundly at odds with what we know about consumers and firms.”

The full quotations from these and many other economists can be found in my blog post  “Quotes Critical of Economics.”  While most quotes are general condemnations of the discipline, the particular four quotes picked up above point to a specific problem — the failure of economists to respond to empirical rejections of theory by modifying the theory. Economists do not follow Feynman’s methodological princple: “It doesn’t matter how beautiful your theory is, it doesn’t matter how smart you are. If it doesn’t agree with experiment, it’s wrong“. Theories flatly contradicted by empirical evidence continue to be used. For example, Stiglitz states that “ Ricardian equivalence  is taught in every graduate school in the country. It is also sheer nonsense.” Further strong evidence of this methodological failure is found in  Romer’s Trouble With Macro . Romer discusses the anti-scientific attitude of leading theorists, refusal to learn from empirical evidence, and the retrograde progress in macro, leading to loss of precious earned knowledge –Again, we need to know exactly how a methodology PERMITS this loss of knowledge — how can good theories be replaced by worse theories while everyone is watching?

To understand this failure, we need to explore the concepts of “models”, “explanation”, and “reality”. As we will see, there are many different types of models, many different concepts of what it means to explain, and also many different approaches to the nature of hidden (unobservable) reality.

In remaining posts, we hope to explain how economists gradually slipped from a valid concept of models and how they help us to understand reality, to  Friedman’s Folly  which insulates, protects, and advances the cause of crazy models. NEXT POST:Thinking about Thinking 3.