Real Statistics (4/4) Illusion of Objectivity

[bit.do/azrs4] 4th of a 4-Part Lecture about an online course: Real Statistics: An Islamic Approach.  The course is free, open enrollment, and allows student to go through a sequence of lessons at their own pace. To register for the first module of this course, on general principles of an Islamic education, fill in the form: Registration: PIE. Previous three parts of this lecture were: Part I – Fundamental of an Islamic Approach {bit.do/azrs1} Part II: Teaching Statistics as an Act of Worship {bit.do/azrs2} Part III: Statistics as Rhetoric {bit.do/azrs3}. This is the final Part IV: How Subjective Comparisons are made to Appear Objective {bit.do/azrs4}. The original talk at Univ of Baluchistan was in urdu. The four parts of the urdu talk are available on YouTube via the following links: RSIA (1/4) Islamic Fundamentals, RSIA (2/4) Teaching as Worship, RSIA (3/4) Statistics as Rhetoric, RSIA (4/4) Illusion of Objectivity.   The 18min English Video Lecture is linked below:

Comparing Two Numbers – (3000 Word Summary)

What could be simpler than comparing two numbers? If is number is 5.6% and the other is 3.5% then 5.6% is the bigger of the two. This is objective, factual, independent of any observer bias, and does not require any specialized knowledge or expertise about the real world. Appearances are deceiving. This simple act of comparison is actually one the key deceptive strategies of conventional statistics. When we look at the numbers alone, then it is indeed true that the comparison is objective and independent of observers bias or value judgments. However, when numbers represent measures of underlying real world factors, the comparison creates an objective fact out of a subjective value judgment – a powerful rhetorical strategy. By making the comparison appear to objective, discussion, arguments, values are bypassed. We give an example of how this process works; consider admissions or selection of two faculty members X and Y

  1. X has 10 publications, Y has 3
  2. X graduated with CGPA of 3.9, Y with 3.6
  3. X scored 8/10 in his interview, while Y scored 6/10.

When we look at the NUMBERS, the comparisons are obvious, objective, and trivial. But when we look beyond the numbers to the underlying realities, then serious questions arise about the validity of these comparisons. Suppose that we learn that all 10 publications of X are in a journal published by his own university, where X is the Chief Editor, while the 3 publications of Y are in international, well-established, highly ranked journals in the field. Similarly, for the other two numbers, learning more about the real-world process which generated the numbers can lead us to change the obvious comparison which is based on the objective numbers.

The essential point conveyed by the adjective “Real” in the title of the course, is that numbers cannot be analyzed in isolation from the reality which generated the numbers. Conventional statistical methodology can be represented by the following diagram:

AppTheorySplit

The hidden reality manifests itself in various observables. These observables can often be measured on some way to create numbers. The numbers can then be subjected to statistical analysis. Conventional statistics is based on the idea that the Applied and the Theoretical portions of the analysis can be separated. The theoretical statistician comes in after the numbers have been generated, and does a purely objective analysis of the numbers themselves, without concern about the complex and unobservable real-world processess which generate the observations, and the methodologies used to convert observations into numbers. It is this separation that we reject in an Islamic approach. Instead, the Islamic approach requires us to look at the theory and the practice together:

NoSplit

Instead of restricting the analysis to numbers alone, the Statistician must look at all aspects of the process, starting from the hidden real world phenomena, to the observations, to the process of quantification, to the statistical analysis. Different parts of the analysis cannot be separated from each other. After understanding this, we can now present the central difference between “Real Statistics” and Conventional Statistics in terms of different understanding of scientific methodology:

Islamic Approach to Science: Science involves grasping the unseen hidden reality by looking at the observable manifestations.

The most important example of this is learning to recognize our Creator by looking at the signs (ayat) which surround us. These signs are present in the external reality, and in our inner souls. For complicated reasons, Western understanding of science, and scientific reasoning, became deeply distorted – see “The Emergence of Logical Positivism” for more details about this. As a result, what we are trained to understand as “rationality” by a Western education, is actually the height of folly. Because of deeply immoral and inhumane behavior by Christian leaders, Europeans came to regard religion as superstition, and made science into their new religion. In the process of transition from Christianity to Science, they came to the wrong conclusion that the distinguishing feature of science was that it deals with the observables, present before us, while religion dealt with unobservables, hidden realities. A fundamental principle of Western epistemology became the rejection of the UNSEEN (God, Angles, Afterlife, etc.). Religion became regarded as superstition – belief in things for which there was no empirical evidence. REASON was defined as believing only in what could be observed. Note the strong opposition between this European definition, which may be called “Enlightenment Rationality” or E-rationality, and the Quran. The Quran starts by explaining that it contains guidance for the God-Conscious, those who believe in the unseen.  

This creates a radical difference between Western and Islamic epistemology (theory of knowledge).  The theory of knowledge dominant in the West today is based on “Kant’s Blunder” — an apparently deep and sophisticated idea, which turns out to be a big mistake. Kant said that philosophers have been trapped in a false quest — they want to use the observables to reach behind them, and grasp the nature of hidden reality (Exactly what I have defined as the Islamic approach to science earlier). According to Kant, this is a big mistake. This is because the nature of hidden reality is impossible for us to learn – we ONLY have access to our observations (by definition) so we can never hope to learn MORE than what we can observe. The “thing-in-itself” – the nature of external reality as it exists independent of our observation — is impossible for us to grasp. So, instead of trying to solve the impossible problem of finding the nature of external reality, we should focus on how we use observations to create a picture of the external reality (a MODEL of reality) in our minds. We should abandon the attempt to assess whether our models of reality actually MATCH reality, because this task is impossible to accomplish.

This argument of Kant became widely accepted, and continues to command widespread support among Western intellectuals.  Superficially, it appears sensible and plausible — even deep. Widespread acceptance of this idea has led to downplay of the “real” underlying hidden phenomena, and over-emphasis on the observables. Once we deny the importance and relevance of the real and unobservable phenomena — like quality of research — then observations and quantification is all that remains. Then it becomes plausible to argue that we can analyze numbers independently of the complex real phenomena which generated the numbers. As a consequence of this, the “Nominalist” approach to science in general became dominant. For a more detailed explanation of how this has led to a disastrously wrong approach to econometrics, see “A Realist Approach to Econometrics“. If we stop at the surface, looking only at numbers, and not going beyond the numbers to the complex real world processes which generate the numbers, then it becomes impossible to distinguish between correlations and causations. All of conventional modern econometrics is based on failure to adequately distinguish between the two, due to a defective methodology — for an example of the type of errors we can make because of this, see “Spurious Correlations

As Jack Goody has documented in “The Theft of History”, Europeans stole discoveries of other civilizations, suppressed the true origins, and claimed credit for themselves; see “Is Science Western in Origin?” by CK Raju for a detailed discussion. The consequences of this theft have been enormous.  The misconception that science originated in the West, led some authors to the belief that Europeans are uniquely capable of rational thought, and science is a unique characteristic of the West. For a rebuttal, see  “Islamic Origins of Science“, which explains how the light of Islam from Al-Andalus ended the dark ages of Europe. Even though Europeans imported science from the Islamic Civilization, they never understood the methodology of science – from the start with Bacon’s idea of induction, to the present day, where a popular textbook asks “What is This Thing Called Science?” and cannot come up with an answer to the question posed. The reason for this failure is that acceptance of Kant’s Blunder blocks the route to the understanding of science.

This focus on comparison of numbers, and refusal to look at the complex and unquantifiable realities creates the disastrously wrong approach to statistics currently prevalent throughout the world. To explain this abstraction in more concrete terms, assume that Household Income and Expenditure Survey (HIES) takes a random sample of 100 people in some area X and find that average income of the 100 is PKR 5000. One year later, another random sample of another 100 people leads to an average income of PKR 6000. Looking at the numbers alone, we may conclude that average nominal income has gone up by 20%. This is all that a statistician can say, and what implications it has about reality are left for the field expert. The comparison of two numbers is simple. However, the job of Islamic Statistician starts where this comparison ends. All parties to a transaction of Riba are held equally responsible for the deed. So we must ask the question: Who is making the argument that growth has occurred? Why is he making this argument? Is it the case that a politician wants to justify success of his policies? What were these policies? Were they actually beneficial? Did they have any causal effects on the change that we observe in the two samples? More generally, we cannot do an isolated and detached analysis of numbers, without looking at the context where our analysis will be used.

Real Statistics involves looking through the data to the underlying realities which the data attempts to capture. Our goal is ALWAYS to make statements about the reality, and NEVER to make statements purely about numbers. The analysis of numbers is ALWAYS a means to an end, and never an end in itself. To illustrate this general principle, we reconsider the HIES random sample of 100 people in District X. Looking at the sample, we find that 99 people had income of PKR 1000, and 1 Person – big landlord – had income of PKR 401,000 – Total income for 100 people is PKR 500,000 – average income is PKR 5000. We see that this income DOES not correctly represent ANYBODY in the sample! 99 people have income of only PKR 1000 which is only 20% of the “average”, while one person is enormously rich, and PKR 5000 is a vast misrepresentation of his income. The median income of PKR 1000 would more properly represent the incomes of the 99 poor people in the sample. However, in a different real world context, different types of analysis would be relevant. For example, if a company is drilling for oil and these numbers represent the income it gets from the holes, then the average would be correct description of its profits from the enterprise. So the analysis of numbers cannot be done in isolation from the real world context from which the numbers emerge.

In the original scenario, when average income increases by 20% from PKR 5000 to PKR 6000, this is an objective fact about the numbers. But when we look behind the numbers to the underlying realities, we may find that:

  1. In 2016, incomes of 99 people reduced to PKR 500 each, but rich man income doubled.
  2. All incomes doubled, but price of food tripled and price of luxuries remained the same.
  3. HIES Sample was not representative of population.

In all such cases, the meaning of 20% growth of the average is an objective fact about some numbers, but does not provide any relevant or useful information about the real world phenomena which are pictured by these numbers. Real Statistics requires us to look through the numbers to the real world processes generating the numbers.

Building on the idea of Statistics as Rhetoric requires us to EXPAND the focus of our inquiry. We should ask the question of WHY data on incomes is being gathered and analyzed?   Sometime ago, the rise of USA to global leadership led to GNP per capita becoming the most important criterion to measure wealth of nations. Previous Empires had other definitions, more suited to establishing their superiority — See “Re-Defining Development” for a more complete discussion of this issue. There is a wide range of criteria we can use for defining development — putting down a criterion and creating consensus on it creates a powerful instrument for guiding economic policy. There are strong arguments which show that use of the GDP per capita measure strongly favors the interests of the rich and powerful, and works against the interests of the poorer labor classes (see “ET1%: The Economic Theory of the top 1%“). When we meekly submit to analyzing GDP per capita as a measure of prosperity, not participating in the larger discussion of whether or not this is a suitable measure of prosperity, then we implicitly agree to to a huge number of false value judgments about social welfare.

The conventional Western statistician is content to analyze numbers in isolation from the larger debate. Indeed, there is an insistence that statisticians should stick to objective analysis of facts, and leave value judgments and policies to practitioners. Instead, an Islamic approach must look through the numbers to the arguments taking place in the real world which make use of these numbers. Mahbubul Haq actually did what I am advocating here with powerful effects; I will explain this historical episode in greater detail, since it illustrated perfectly the concept underlying our course of “Real Statistics”.

Mahbubul-Haq personally experienced the effects of theory and policy focused on growth rates. Implementing these policies led to a concentration of wealth, and actually had adverse effects on the poor. He was wise enough to realize that policies were driven by statistics. Politicians were able to argue that we achieved great growth rates in GNP per capita, and they would be forgiven for all since on developmental fronts. To counter this statistical strategy, he invented the concept of the Human Development Index. He added life expectancy and education to the standard measure of GNP per capita. Amartya Sen opposed the HDI initially on the grounds that it was too crude — actually including concerns about poverty would require substantially greater detail. Mahbubul Haq understood the limitations of the HDI very well but felt, correctly, that this was a simple and small change which would be acceptable in the global arena. This was a rhetorical strategy which proved highly successful. The HDI achieved widespread acceptance and changed the nature of the development discourse. For the first time, concerns of the poor were introduced as targets for policy makers and planners. Previously, the spell of GDP per capita had created the illusion that increasing aggregate wealth would automatically create prosperity for all via the infamous trickle-down effect. The more sophisticated but correspondingly more difficult to sell theories of multidimensional poverty still face an uphill battle in terms of acceptability within mainstream economic theory. The opening that now exists for these theories has been created by the HDI gambit of Mahbubul Haq. This is an illustration of how paying attention to how statistics are used in the real world, and rhetorical strategies and counter-strategies, have effects on the lives of millions of the poor.

Summary and Conclusions: Statistics are NEVER produced in a vacuum. We do not measure or calculate anything unless it is relevant and important to someone. Huge amounts of money are spent, and vast amounts of resources are utilized in carrying out censuses and surveys to calculate population numbers and to get measures of aggregate wealth. This is because the dominant economic theories place great emphasis on this as a measure of progress and prosperity.

Consider a thought experiment. Suppose that, based on Islamic teachings, we were to make “hunger” the primary criterion for progress (see “First Fundamental Economic Problem: Feeding the Hungry“). Suppose we define ‘civilization’ to be the ability to take care of needs of the citizens (rather than aggregate wealth in hands of citizens). Suppose we take as a measure of development the number of hungry people, the number of suicides, the number of people with major unfulfilled basic needs, infant mortality, access to health, education, and social services. Consensus on these criteria as measures of progress would create a revolution. Political Leaders would be held accountable for improvements along these dimensions. The USA would show up as very low on these rankings, and many countries with smaller GDP per capita would rank higher. There would be political pressure on the USA to regain world leadership by improving the criteria and programs would be developed and money spent to increase these rankings. When targets are created by social consensus, then human effort is devoted to meeting these targets. The current targets created by GNP per capita are deadly to the welfare of the poor, and changing these targets involves intervening in the statistical rhetoric, the arguments being used to justify this measure. The Muslim statistician cannot avoid responsibility by saying that this is not my job – I only analyze the numbers that are given to me. The broader debate about whether or not this is a suitable measure of development is for others to do, not me.  Whenever we look at HOW statistical arguments are being used, we will find a HUGE role for Islamic teachings, since there will always be normative issues involved about which Islam provides us with definitive guidance. “Real Statistics” means statistics which goes beyond numbers to look at the real world contexts within which numbers are used for persuasion.

This entry was posted in econometrics, methodology, Real Statistics by Asad Zaman. Bookmark the permalink.

About Asad Zaman

BS Math MIT (1974), Ph.D. Econ Stanford (1978)] has taught at leading universities like Columbia, U. Penn., Johns Hopkins and Cal. Tech. Currently he is Vice Chancellor of Pakistan Institute of Development Economics. His textbook Statistical Foundations of Econometric Techniques (Academic Press, NY, 1996) is widely used in advanced graduate courses. His research on Islamic economics is widely cited, and has been highly influential in shaping the field. His publications in top ranked journals like Annals of Statistics, Journal of Econometrics, Econometric Theory, Journal of Labor Economics, etc. have more than a thousand citations as per Google Scholar.

8 thoughts on “Real Statistics (4/4) Illusion of Objectivity

  1. Pingback: Real Statistics: An Islamic Approach | An Islamic WorldView

  2. My reflections on the first four lectures

    Allah has created human beings to achieve higher goals in life as directed in Qur’an. However, we are taught and trained to become a human resource to earn higher and higher income to satisfy material needs. In this way, a multidimensional and a multipurpose human being is turned into a unidimensional and a single-goal human resource. By suppressing this complex nature and identity of human beings, it creates a false image of human behavior governed by self-interest. This leads to the narrow way of looking and analyzing the events. In contrast, Qur’an teaches us to use a holistic and realistic approach to understand and analyze problems.

    The primary target of Qur’anic teachings is ‘heart’. When heart transforms to goodness, it creates a ripple effect on family, society, and institutions, and hence the whole system works towards uplifting the moral and ethical values, which in turn, strengthen the heart. A real transformation takes place due to this ‘heart-centered’ approach. And this is exactly what happened in the state of Madinah.

    Since western knowledge does not enter the heart, it corrupts and spoils heart and hence leads to evil. It is a ‘head-centered’ or ‘utility-centric’ approach- whatever is appealing and fascinating to mind is acceptable and applicable. Whether knowledge transforms our hearts or not leads to the Islamic distinction of useful and useless knowledge. However, it does not mean that the mind or reason has no place in Islam. Rather, it greatly emphasizes reasoning and use of intellect but governed by the fear of Allah (Taqwa) and our intentions which reside in our hearts.

    The discussion above clearly shows that the nature of human beings is multidimensional and complex. However, it reduced to a unidimensional entity suppressing the heterogeneity of human beings. It is easy to analyze its behavior by traditional statistical approach since it uses tools that apply average or typical behavior by combining various dimensions assuming homogeneity. This leads to our distinction between factual and fictional numbers. Most of the statistics deal with fictional numbers because of the nature of the unidimensional assumption about human behavior. These tools fail to analyze qualitative and unobservable human characteristics since they cannot be averaged to a single fictional number. Therefore, the scope of conventional statistics is very limited in understanding various phenomena around us. The purpose of the Islamic approach to statistics is to rectify this limitation by studying linkages between hidden realities and their observable and measurable counterparts in unison.

    Wassalam

    Hamid Hasan

  3. Pingback: Q & A: Islamic Approach to Statistics | An Islamic WorldView

  4. Pingback: Kelvin’s Blunder: Measure Everything | An Islamic WorldView

  5. Pingback: Linguistic Paradoxes & Limits of Human Knowledge | An Islamic WorldView

  6. Pingback: The Struggle for Knowledge | An Islamic WorldView

  7. Pingback: AZ Students

  8. Pingback: Understanding Statistical Distributions | WEA Pedagogy Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s