Introduction to Real Statistics

This introductory Chapter 0 presents some motivational material explaining why we need to launch a revolution in statistics. To register for the upcoming online course, signup on Google Form http://bit.ly/RSRA000

Preface:

“A momentous revolution occurred in the Western intellectual tradition during the 1930s that shattered the spine of knowledge systems that humanity had preserved and transmitted across generations since the dawn of knowledge itself. This was the catastrophic movement called “logical positivism” in which “science” was classified as the sole source of knowledge, while all other types of human knowledge were reduced in value to meaningless noise.”  In her book entitled “The Making of the Modern University: Intellectual Transformation and the Marginalization of Morality”, Julie Reuben provides the historical details and documentation of this transition, sketched in the following quote:

In 1884 Harvard officials … had inherited a world view that strongly associated truth and religion. The term truth encompassed all “correct” knowledge; religious doctrines, common-sense beliefs, and scientific theories were all judged by the same cognitive standards. Religious truth was the most important and valuable form of knowledge because it gave meaning to mundane knowledge. Religion transformed abstract knowledge into “moral” truths—truths that guided individuals’ daily actions and explained their ultimate destiny.

The Harvard officials’ views about truth represented the beliefs of most educated Americans at that time. In the late nineteenth century intellectuals assumed that truth had spiritual, moral, and cognitive dimensions. By 1930, however, intellectuals had abandoned this broad conception of truth. They embraced, instead, a view of knowledge that drew a sharp distinction between “facts” and “values.” They associated cognitive truth with empirically verified knowledge and maintained that by this standard, moral values could not be validated as “true.” In the nomenclature of the twentieth century, only “science” constituted true knowledge. Moral and spiritual values could be “true” in an emotional or nonliteral sense, but not in terms of cognitively verifiable knowledge. The term truth no longer comfortably encompassed factual knowledge and moral values.

As has been detailed by Julie Reuben, and many other authors, it was the emergence of the philosophy of logical positivism that created this schism: science was classified as the sole source of knowledge, while all other types of human knowledge were reduced in value to meaningless noise. The logical positivists argued that “… it is impossible to find a criterion for determining the validity of ethical judgements … because they have no objective validity whatsoever . . . They are pure expressions of feeling and as such do not come under the category of truth and falsehood.” As a philosophy, logical positivism had a spectacular crash in mid 20th Century. Unfortunately, the schism in the foundations of knowledge created by it was never healed. Even though the philosophy has been rejected, the methodologies of the social sciences based upon it continue to dominate the universities, and shape the minds of the seekers of knowledge.

Islam strongly supports the idea of unity of knowledge. The value of knowledge, and the honor accorded to seekers of knowledge, is unique to Islamic teachings. More details on this issue are provided in the first chapter. For this preface, we note that it is a unique feature of Islam that human beings are honored because of knowledge. From the beginning, Allah T’aala gave knowledge to Adam AS, and then the angels were ordered to prostrate. In the first revelation to our Prophet Mohammed SAW, Allah Táala introduces Himself as a TEACHER, who will give to mankind a knowledge which they do not have. This knowledge transformed the ignorant and backwards early Muslims into the leaders of the world.

The main thesis of this textbook is that the knowledge revealed to mankind is still just as revolutionary as it was 1440 years ago. However, the emergence of this split in the body of knowledge, the elevation of “science” and the rejection of religion, has led to an unwarranted and unjustified fragmentation of knowledge. Knowledge is unified by purpose. When we make our goal the acquisition of knowledge for the purpose of recognition and glorification of God, and for providing service to the creation of God, this changes both the substance of knowledge and the method of approach to study. This also implies that all human knowledge is built on moral foundations. The idea of “secular knowledge” is an oxymoron. No aspect of human knowledge can be separated from the fundamental knowledge of the existence of the ONE God, which unites all of the existent. These central ideas about the nature of knowledge will be illustrated by this course on “statistics”. It is the use of these Islamic methodological perspectives which has led to a radically different approach to the subjects, which rejects and replaces the approach initiated by Sir Ronald Fisher a century ago.

In this preface, we mention some of the novel ideas which form part of this new approach to the discipline of statistics. It should be emphasized that this is not at all just conventional statistics with a sprinkling of Quran and Hadeeth thrown in for flavor. Sir Ronald Fisher is known as the father of statistics. He created the approach to statistics currently in use in the discipline all over the world. This approach is based on making certain simplifying assumptions about the data which permit handling large data sets with minimal computational capabilities. We reject this approach in toto, for two reasons. One is that computational capabilities today allow us to avoid making unjustified assumptions. The second is deeper: the ability to make unjustifiable assumptions as part of the standard statistical methodology allows one to pursue agendas and hide them within the assumptions. That allows one to easily “lie with statistics”. This capability can be, and has been, used to dramatically misrepresent ground realities, and paint the outcomes of enormously unjust and exploitative economic systems as remarkable achievements. This accounts for the famous saying that “there are lies, damned lies, and statistics”, Our hope in developing this alternative approach is to keep closer track of the relationships between the statistics and the ground realities they represent, so that the Islamic imperatives of telling the truth are better served by this discipline.  

Chapter 0: Introduction

A: Preliminaries

Sir Ronald Fisher created an approach to statistics which has been built upon for a century to create modern statistics. This textbook advocates abandoning this approach, and starting afresh, redefining the subject matter of statistics, and developing a new methodology. This introduction provides a sketch of the ideas which led to the creation of this approach. Very briefly, rejection of Christianity as a basis for public life in Europe led to the creation of a way-of-life (deen) which may be termed secular modernity. At the heart of this deen is the idea that objective knowledge is possible – that we can arrive at uncontestable truths about this world, and our lives, by using reason alone. This textbook starts from the opposite premise of epistemic humility advocated by Islam: we human beings have been given very little knowledge. All the knowledge that we have is subjective and limited by the range of our experiences. We cannot arrive at universal truths using reason alone. The remaining sections of the introduction expand upon these brief remarks, but a full exposition of these ideas would require a separate book. Our current goal is only to provide some motivation and explanation for the approach to statistics taken in this textbook. We rely on the maxim that the proof of the pudding is in the eating. We believe that the approach to statistics adopted in this text will stand or fall according to how well it holds up in real-world applications, and not based on the philosophies which led to its creation.

One of the central premises of this textbook is that knowledge must be built on moral foundations. This is directly opposite of a fundamental epistemological premise of the West: morality cannot be derived from observations and logic, and hence is not part of human knowledge. The fact that moral foundations are never mentioned when conventional statistics is taught does not mean that these do not exist. The way of thinking and living known as “secular modernity” arose in the West after the rejection of Christianity. Since there is no God, afterlife, judgment, it makes perfect sense to concentrate on maximizing pleasure, power, and profits on this planet. In complex and concealed ways, these are the moral foundations on which the stock of knowledge built by the West over the past few centuries. This course was born out the attempt to replace these toxic moral foundations by those furnished by the teachings of Islam.

Where does this approach place non-Muslim students? It is an Islamic teaching that our intentions for an action determine its value. In a secular modern approach, students study statistics for personal benefits: career, wealth, knowledge. An Islamic approach is based on seeking useful knowledge for serving the creation of God, for the sake of the love of God. Any student who seeks to learn statistics for the sake of serving mankind will be comfortable with the approach adopted, since this goal is aligned with Islamic goals for pursuit of knowledge. It is worth emphasizing the secular modernity is itself a religion which replaced Christianity in the West. We do not a choice between an objective and impartial approach in contrast with an biased and subjective religion based approach. All approaches to study are based on moral foundations – we can only choose between the moral foundations of secular modernity, or those of some other religion. The ones furnished by Islam are adapted to the nature with which human beings are born, so non-Muslims would find them naturally appealing.      

B: The Puzzles Posed by Human History

Looking at the broad patterns of history from an Islamic perspective raises several major puzzles. More than fourteen centuries ago, a powerful message revealed by God to mankind dramatically changed the course of world history. It transformed ignorant and backwards tribes into world leaders, and created a civilization which enlightened the world for more than a thousand years. One of the unique features of Islam was the imperative to seek knowledge: the ink of the scholars was considered as more precious than the blood of the martyrs. This led the Muslims to gather knowledge from all over the globe.

It is useful to tag 1492 as the year when the world changed radically[1]. The successful voyage of Columbus to the Americas opened up vast treasures for exploitation by the Europeans. Completion of the reconquest of Al-Andalus led to the acquisition of the treasures of knowledge in the millions of books in the libraries of the defeated Muslims. This influx of knowledge ended the dark ages of Europe. The third critical event of 1492 was the purchase of the Papacy by Rodrigo Borgia, under the title of Alexander VI. This extreme corruption of the papacy eventually led to the breakup of the Catholic Church, followed by centuries of fratricidal warfare among European nations. Acquisition of knowledge, gold, and loss of morality which resulted from rejection of Christianity, led to the European scramble for wealth and power which have shaped the modern world.  The amazing success of Europeans at colonizing nearly 90% of the globe, and enriching themselves while impoverishing the rest of the planet, raises several questions from an Islamic perspective:

  1. What was the substance of the message which revolutionized the lives of the early Muslims, and changed the tides of history?
  2. After a thousand years of success what were the causes of the decline of the Islamic Civilization, leading to the current backwardness of the Islamic lands?
  3. What were the causes of the rise of West over the past few centuries, and its current global hegemony?
  4. All over the globe, ignorance and poverty dominate the Islamic lands, resembling the pre-Islamic times. What is the remedy for these troubles?
  5. In particular, of special importance, has the final message of God to mankind lost the power that it displayed 14 centuries ago? Can it no longer lead mankind from darkness to light?

There exists an enormous amount of literature presenting answers to these questions. A large number of movements for revival of the Islamic Civilization are based on assumptions about the right answers to these questions. The most common and widely accepted set of answers attributes the rise of the West to the development of science and technology, which placed them ahead of the Islamic Civilization on the frontiers of knowledge. Accordingly, the remedy is to acquire this Western knowledge, by developing institutions to spread education among the ignorant masses in the Islamic countries. The Ghazali Project is based on the diametrically opposite view. Western education is the source of disease in Islamic societies. It is built on toxic moral foundations created by the rejection of Christianity. It also inculcates a worldview which glorifies the West and places zero value on the accomplishments of the Islamic Civilization. Nonetheless, a Western education also provides training in skills essential for the modern world. The only solution requires the development of an Islamic alternative to a Western education, which teaches all the skills required for the modern world within an Islamic framework. The Ghazali Project can also be thought of as a variant of project of the “Islamization of Knowledge”.  

C: The Central Importance of Knowledge

 The dominance of the Western civilization has spread a materialistic perspective throughout the globe. When we think of rise and fall of empires, we think in terms of wealth, power, armies, weapons, trade, etc. – that is, in terms of material causes. A far more insightful perspective emerges when we put knowledge at the center. The rise of the Islamic Civilization occurred because of a special knowledge which was given to the Muslims, as promised in the first Wahy: (God) taught man that which he did not know. At the center of the rise of Islam was the extraordinary emphasis given to the search for knowledge in Islam. This led the Muslims to seek knowledge from around the globe, as the lost property of the Mo’min. The rise of the Islamic Civilization can then be seen from the perspective of an increase in knowledge, as per Quran: “My Lord, increase me in knowledge”.

Putting knowledge, instead of the “wealth of nations”, at the center leads to an interesting and unusual perspective on history. The dark ages of Europe started with the burning of the library of Alexandria in the 5th Century, and ended with the translations of books in the library of recaptured Toledo in the 13th Century. The burning of the library of Baghdad in the 13th Century was a major blow to the Islamic Civilization. However, the real damage occurred during the colonization of the Islamic world from the 18th to the early 20th Century.   18th to the early 20th Century. Again, we should think of colonization as the conquest of knowledge. Indigenous systems for production of knowledge throughout the Islamic world were destroyed, as a matter of policy, by the colonizers. These were replaced by Western educational systems, designed to teach lessons of Western superiority and create contempt and hatred for indigenous knowledge traditions.   

This is the central problem facing the Islamic Civilization. A Western education teaches us only about the knowledge created by European intellectual over the past few centuries. This automatically creates the (false) impression that this is the only type of worthwhile knowledge which exists. In particular, this implies that the knowledge given to the early Muslims, the complete and perfect revelation from God, is of no value today, at least in terms of dealing with our worldly problems. This is an illusion. However, when we make the claim that the Quran does provide guidance for today, we are bound to produce evidence for this. Currently, this evidence, an Islamic alternative to a Western education, which provides skills required for the modern world, without the accompanying indoctrination into a Eurocentric worldview, does not exist.

The challenge facing Muslims today is to devise an alternative to a Western education. This requires understanding the foundational defects in the body of knowledge created by the West over the past few centuries. These defects stem from the rejection of Christianity, and the attempt to create knowledge without any moral foundations. Since the usefulness of knowledge can only be evaluated with reference to a goal, and useful goals depend on a conception of human welfare, a body of useful knowledge can only be built on moral foundations. Shifting from the Christian conception of welfare based on judgment and afterlife to a purely worldly conception based on pleasure, power, and profits led to a radical change in the type of knowledge which was judged to be useful. The type of knowledge created over the past few centuries by the West is extremely useful for the pursuit of wealth and power, but harmful for learning how to be the best of the creations of God. It is up to us to show that if we seek knowledge according to Islamic guidelines, this will also create, as a side-effect, beneficial worldly knowledge.

This textbook on Statistics is a proof-of-concept. It shows how the entire body of modern statistics is defective because it has been built on the wrong moral foundations. It also shows how Islamic perspectives on knowledge help us to create a superior alternative.  

D: Rebuilding Knowledge on Islamic Foundations

Whoever is kept away from the Fire and admitted to the Garden will have triumphed.  The present world is only an illusory pleasure.” (Q 3:185)

We human beings have only been created for worship. The central question faced by a Muslim student is: how to ensure that my study is an act of worship? If we can ensure this, then the ink of our pens will have greater weight than the blood of the martyrs, on the day of Judgment. Countless verses of the Quran, many Ahadeeth, and books of scholars, testify to the central importance of pursuit of knowledge in the way of life (Deen) given by Islam. But, there is a crucial distinction between useful and useless knowledge, which has been lost from sight and vastly misunderstood, in modern times. Our prophet Mohammad SAW prayed for useful knowledge, and sought the protection of God from useless knowledge. So, when we study statistics, or any other subject, we must begin by asking what kind of knowledge this is. If it is useless knowledge, then we must stay away from it:

They learned what harmed them, not what benefited them, knowing full well that whoever gained [this knowledge] would lose any share in the Hereafter. (Q 2:102)

Knowledge which enters the heart, and brings us closer to God, is useful. Knowledge which hinders our progress towards the recognition of God, and darkens our heart, is useless or harmful. How can we learn whether or not statistics (or other subjects) are useful or useless? We must have clarity on the answer to this question prior to beginning our study.

The answer is, on the face of it, obvious. Modern statistics was created by Sir Ronald Fisher and followers, to support the vastly harmful and evil ideology of Eugenics, which held that white races were superior to others. The technique of correlation was invented to determine extent to which traits of fathers were passed onto sons. Analysis of variance was created to separate out the genetic influence from the environmental one. Can this body of knowledge, built on such foul moral foundations, be “Islamized”? The vast majority of Muslims answer “yes” – even though statistics was developed for evil purposes, useful techniques for data analysis were developed, and these can be deployed for good purposes, as part of an Islamization of knowledge project. This textbook is built on the idea is that the answer is “no” – the techniques which have been developed are seriously deficient technically, for reasons to be explained in greater detail later. These technical flaws make it necessary for us to reject most of the developments in statistics over the past century, and rebuild the discipline from the ground up. Numerous fundamental innovations, to be described in greater detail later, are part of this new course. However, in this introductory section, we wish to address deeper issues, regarding the nature of “knowledge” itself.

Morality is the roadmap for the path towards recognition of God. Good actions advance us on this path, while bad actions move in the opposite direction. All useful knowledge is built on moral foundations. This Islamic conception is starkly opposed to the modern Western intellectual tradition, which holds that morality is not knowledge at all. Furthermore, knowledge is purely “secular” – that is, ethically neutral. From an Islamic perspective “secular knowledge” is an oxymoron: if knowledge does not teach us about the path to God, it is useless and harmful. Christianity offers exactly the same perspective about knowledge. However, historical events in Europe led European intellectuals to discard these ideas and develop a different approach. Over a century of devastation caused by religious wars between Catholics and Protestants placed European intellectuals in an impossible position. Any approach to understanding and regulating social behavior inevitably involves morality. On the other hand, sharp disagreements between Christian factions deprived them of the natural religious basis for morals. The solution that ultimately won widespread acceptance was the idea of secular knowledge —   build politics, economics, and all social sciences on purely objective and value-neutral grounds, which would be equally acceptable to all rational human beings.  European intellectuals were forced to strive for an impossible goal: the creation of an ethically neutral body of knowledge about human beings and societies. They accomplished this by fraud. Moral assumptions were hidden in the framework and foundations, and labeled by misleading names such as “rationality” and “science”. This concealment created the appearance of an objective and ethically neutral body of knowledge. The name “Social Science” uses the word ”science” to make a false and deceptive claim to knowledge. Similar strategies were used in the foundation of statistics, where concealed assumptions about the data were added to create the possibility of using any data to prove any preconception. Many people have taken note of the ability of modern statistics to support deception. The most popular textbook in statistics is called “How to Lie with Statistics”, which has sales more than the combined sales of all other statistics textbooks. The aphorism that “There are lies, damned lies, and Statistics” is widely known and appreciated.

The entire body of “secular” knowledge created by European intellectuals, and followers, is based on deeply flawed and concealed moral foundations. This body of knowledge must be rebuilt on the solid moral foundations provided by the teachings of Islam. This textbook of statistics is an illustration – a proof of concept. It shows how one apparently objective and neutral subject is saturated with false and misleading assumptions about the nature of human knowledge. Recognizing and removing them leads to a radically different approach to the same subject. These are grand claims, and cannot be fully supported in this brief introduction. Some indicative and sketchy arguments, together with more detailed references, will be provided in here. We begin by noting a deep confusion created by the common use of the label “Science” for both Physical (Natural) Science and Social Science. These two domains of knowledge are dramatically different. Even though both domains require rebuilding on moral foundations, the approach to this reconstruction of knowledge will be very different for the two areas. Our concern here is not with the Physical Sciences, but with rebuilding the Western Social Sciences. However, we provide a brief paragraph of explanation for how the Physical Sciences must be rebuilt.

The modern Western Physical Sciences were created for the purpose of control and domination of the world, and they have served this purpose excellently. Islam teaches us that the worth of an action depends on the intentions. The evil intentions behind this body of knowledge are revealed in the damage and destruction they have wrought upon the planet, leading to a climate catastrophe which threatens all human life on the planet. The last century has seen the largest destruction of innocent human lives in the history of mankind, thanks to the power of the technology created and deployed for warfare. The fact that there have been some beneficial side-effects of the technologies developed for warfare should not distract us from seeing that this science has placed the power to control billions of lives in the hands of a handful of people. Rebuilding the physical sciences requires explicit recognition of the close connection between power and knowledge. Nuclear Physicists should be trained to understand their ethical responsibilities for the consequences of their discoveries. Molecular biologists who develop terminating seeds for productive varieties of food grains should understand that they are contributing to corporate profits by creating poverty and hunger. Moral foundations shape the nature of the research, and resultant technological developments – physicists who are aware of their moral responsibility would not do the research required for building more deadly bombs. The current pretense of ethical neutrality of science deceives us into separating technological developments from their use. Building all knowledge on moral foundations requires an enormously different approach to teaching the sciences. We must start from the premise that knowledge of nature is to be used for the recognition of the signs of God, and for the service of the creation of God. Models of this approach currently do not exist, and are an essential requirement for the education of the Ummah.

In this textbook, we provide a demonstration of how an alternative approach, based on moral foundations provided by Islam, lead to radical changes in both methodology and the subject matter of statistics. This is a practical demonstration – we will simply show how it is done. There is a theory which is behind the construction, but the proof of the pudding is in the eating. We will briefly discuss the theoretical foundations on which this alternative approach to statistics was constructed. First, is statistics a social science at all? Should it not be considered as a part of mathematics, which has no apparent moral foundations? To answer this question, we must consider how statistics are used in our daily lives. Upon reflection, it seems clear that statistics are used to evaluate different types of policies, and to judge relative efficacy of different actions. This evaluative role is primarly a moral one: when we decide that a certain policy is useful, we are judging between different possible social conditions according to an implicit or explicit moral framework. A number of methodological strategies are used to hide the moral framework in the conventional approach to statistics. One of these is the nominalist methodology: according to this, we only need to look at the appearances, captured and measured by the numbers. We need not go beyond the numbers to the reality being imperfectly measured by these numbers. The “REAL” in the title of the course expresses firm opposition to this idea. We cannot understand numbers except in context of the real world; numbers cannot be meaningfully analyzed in isolation. The second toxic idea is the separation of theory and practice. The idea that the statistician can extract information from numbers and present it to the practitioners is wrong. Methods for analyzing data depend crucially on the real-world context; theory and practice cannot be separated. Whenever we use data analysis to compare and evaluate social conditions, a normative framework is inevitably involved.

We can conclude and summarize this lecture as follows. Knowledge is useful if it provides us with guidance on how to build better lives and improve human societies. This inevitable involves a normative framework for judging between better and worse. Knowledge which does not provide us with any such guidance is irrelevant and useless. Islam teaches us to seek protection from such knowledge. The entire body of “secular” knowledge created by the West over the past few centuries makes the opposite epistemological claims. It is claimed that knowledge is objective and value-free, while morality and value-judgments are not part of knowledge at all. This perverse position leads to absurd conclusions, permitting teachers to educate students the technology to build atom bombs, without accepting moral responsibility for the consequences of such knowledge. The idea that we can teach skills without discussing morality is itself a toxic moral judgment, leading to experts who deploy weapons to destroy millions of lives, without any sense of responsibility. Today, these Western educational methods are globally dominant, and result in moral stunting of students all over the world. There is an urgent need for creating an alternative. This textbook on statistics is a demonstration of how this can be done. It results from an attempt to create knowledge on an explicit moral foundation: the service of the creation of God. Having an explicit moral framework changes the methodology, approach, and the substance of Statistics, as we will demonstrate in this textbook.

E: Innovations of this textbook

In the early 20th Century, Sir Ronald Fisher initiated an approach to statistics which he characterized as follows: : “… the object of statistical methods is the reduction of data. A quantity of data, which usually by its mere bulk is incapable of entering the mind, is to be replaced by re latively few quantities which shall adequately represent the whole …” As he clearly indicates, we want to reduce the data because our minds cannot comprehend large amounts of data. Therefore, we want to summarize the data in a few numbers which adequately represent the whole data set.

It should be obvious from the start that this is an impossible task. One cannot reduce the information contained in 1000 points of data to two or three numbers. There must be loss of information in this process. Fisher developed a distinctive methodology, which is still at the heart of conventional statistics. The central element of this methodology was an ASSUMPTION – the data is a random sample from a larger population, where the larger population is characterized by a few key parameters. Under these assumptions, the key parameters which characterized the larger population would be sufficient to characterize the data set at hand. Under such assumptions, Fisher showed that there were “sufficient statistics” – a small set of numbers which captured all of the information available in the data. Thus, once in possession of the sufficient statistics, the data analyst could actually throw away the original data, as all relevant information from the data set had been captured in the sufficient statistics. Our goal in this section is to explain how this methodology works, why it was a brilliant contribution of Fisher at his time, and why this methodology is now obsolete, and a handicap to progress in statistics.  

In the raw data, each data point is unique and informative. But Fisher’s approach anonymizes all of the data by making them all equally representative of a population. This  actually has parallels to our real approach – we think of the data as informing us about the real world which is hidden. But the problem is that Fisher uses an imaginary world from which the data comes, whereas we are interested in the real world. According to conventional statistical methodology, the statistician is free to make up a class of imaginary populations from which the data is treated as being a representative sample. Using this freedom, the statistician can restrict the imaginary populations to satisfy some desired prerequisite or bias. Then statistical inference will confirm this bias, making it appear as if the data is providing us with this information, when in fact, it is the bias has been built into the assumptions, and all data sets will confirm this bias.

At this introductory stage, it is hard to provide a deep and detailed discussion of all the innovations, both methodological and substantive, in this textbook. We therefore provide a bullet point list, which highlights the innovations chapter by chapter:

  1. The first chapter provides a more detailed discussion of the Islamic approach to pedagogy which lies at the heart of this textbook.
  2. The second chapter shows that even the simplest of operations – comparing two numbers to see which one is larger – requires considerations of the real-world context from which these numbers emerge. In contrast, conventional statistics methodology confines attention to the numbers.
  3. The theme of this book is that statistics must be learnt within context of real-world applications. The third chapter discusses computation and analysis of life expectancies. It shows how assumptions go into the manufacture of numbers which are presented as objective and concrete. It also illustrates the use of some basic statistical tools like the histogram.
  4. The fourth chapter discusses an issue about which conventional statistical methodology has created enormous confusion: index numbers. When objects – like universities, automobiles, research productivity – are ranked, an “index” number must be created to enable such ranking. A little-known fact is that there is no way of creating an objective index number. This means that there is no objective way of deciding which university is best, or which author has the highest research productivity, or which student has the highest overall performance. Even though rankings are done routinely, all of them necessarily incorporate subjective judgments about the relative worth of different dimensions of performance.
  5. The fifth chapter provides a detailed discussion of Sir Ronald Fisher’s approach to statistics, and the biases that it inherited from his racist agenda.
  6. The sixth chapter illustrates how we can use statistics to compare infant mortality across time and across countries. The discussion introduces basic tools for such comparisons which differ substantially from conventional tools which are based on assuming that the data is normal.
  7. The seventh chapter introduces basic probability contexts using a binomial distribution. It provides a new non-positivist definition of probability. This is different from the frequentist and the Bayesian approaches, both of which are based on positivist ideology.
  8. Chapter 8 introduces causality, while Chapter 9 discusses associations (termed correlation in conventional statistics).
  9. Chapter 10 discusses several real-world applications and shows how to differentiate between correlation and causation.
  10. Chapter 11 and 12 provide a deeper discussion of causation, and the technical tools required for its analysis. These provide basic foundations for understanding causation, something which is not currently available in conventional textbooks of statistics. More advanced discussion of causation is left for a later book.

[1] See “1492: The Year the World Began” by Felipe Fernández-Armesto.

Three Foundational Flaws in Statistics

{bit.ly/3FFStat} In a previous posts Preface to Radical Statistics and Why an Islamic Approach to Statistics?, I have explained how we can develop a new approach to statistics, which rejects a century of developments based on a methodology created by Sir Ronald Fisher, father of modern statistics. It is my hope that this will be a disruptive technology – it will eventually sweep away the entire structure of knowledge which has been built up under this name, and replace it with a radically different alternative. This structure of statistics (which I studied in detail and depth at the Ph.D. Statistics program at Stanford University), is currently being taught in universities around the globe. This structure is deeply deficient for a number of reasons. The most important among these is the logical positivist philosophy which was used to create the foundations of this discipline in the early 20th Century. We briefly describe the key problems created by this. The 3 minute video is followed by a brief writeup

Three Foundational Flaws of Statistics

First, a very brief introduction to the Logical Positivist philosophy. Over a century of brutal and massively destructive religious warfare between Christian factions made it necessary for European intellectuals to find an alternative to religion on which to create a secular political science. Bitter experience had shown them that Christianity was highly unsuitable for this purpose. Ultimately, over the period of more than two centuries, this led to the creation of a whole body of knowledge which forms the foundations of secular modernity. Given the warfare among Christian factions, it was a necessity for secular knowledge to command consensus without reference to an agreed upon body of religious or moral knowledge. Therefore, modern secular knowledge claimed to be a body of objective knowledge, universally applicable to all human societies, and derived on basis of publicly confirmable observations and uncontestable logic alone. This theory of secular knowledge received its final polished form in the early 20th Century with the development of the philosophy of logical positivism (LP). According to LP, valid knowledge could be based only on observations and logic. Science was based on observations and logic and was the only form of valid knowledge. Religion was based on unobservables like God, afterlife, Day of Judgment, angels, etc. and hence not a form of valid knowledge. For a more detailed account, see The Emergence of Logical Positivism.

In fact, the task facing the European intellectuals was an impossible one. Any body of knowledge which informs us about appropriate political, economic, and social institutions and regulations for a human society, must necessarily be built on moral foundations. But violent disagreements among Christian factions made it impossible to find suitable moral foundations on which consensus could be achieved. This led to the use of “reason” as a code-word to develop knowledge — reason could be re-defined in ways suitable to achieve desired epistemological goals. Thus Kant used reason to arrive at Newton’s Law of Universal Gravitation, and also to devise a universal moral code of behavior, applicable to all “rational” human beings (which he identified as white males: see Kant: Misogynist & Racist). Continuing in this fashion, economists today define “rationality” in a peculiar way to endorse as rational only behavior which seeks to maximize worldly pleasure — belief in the unseen is considered irrational. Also, the pleasure created by loving others and being loved, is not publicly observable, and hence not factored into economic theory. The punchline is that secular modern knowledge claims to be objective, universal, and applicable to all human societies, but it conceals a large base of Eurocentric assumptions. For a more detailed account of what has been concealed beneath the façade of “reason”, see The Puzzle of Western Social Science and Origins of Western Social Sciences.

At the heart of Logical Positivism is a rejection of unobservables as a basis for knowledge. This is in dramatic contrast to the message of the Quran, which opens by explaining that Taqwa (God-consciousness) requires faith in the unseen. It is this rejection of unobservables that leads to serious problems at the heart of modern statistics. The concept of probability refers to unobservable events which might have happened, instead of what actually occurred. Statistics is based on flawed foundations because of Logical Positivist efforts to remove these unobservables, in accordance with the misconception that science is based purely on observables, In probability, the awkward and incoherent frequency definition, and the equally awkward and incoherent Bayesian definitions are in use in textbooks around the world. Neither refer to the concept of probability as being about what might have happened, because such terminology is meaningless according to LP. For more discussion, see A New Definition of Probability. Equally serious and fundamental is the problem that causality is never observable. Only correlations are observable. So despite clarity on the fact that correlations are not the same as causation, statistician continue to equate the two, because they have not satisfactory theory of causation — see Causality as Child’s Play. Within the philosophical framework of LP, no satisfactory theory of probability, or causation, can ever be developed, because both concepts relate fundamentally to unobservables.

In addition, a third fundamental defect in modern statistics was created by lack of computing capabilities. Since large data sets were difficult to handle, Sir Ronal Fisher defined statistics to be about the reduction of data, to small number of “sufficient statistics”. The technique he devised for this purpose was to imagine that the data is a random sample from a hypothetical parent population. A well-chosen parent population can be characterized by a few parameters, and modern statistics became the art of learning about these few parameters from the data. Modern textbooks continue to base statistics on this methodology created by Fisher, even though advances in computational capabilities have rendered it obsolete. By careful choice of the imaginary parent population, an expert statistician can make any data set appear to have any pre-selected characteristics desired. With modern computers, there is no need to reduce the data, since we can analyze enormously large data sets directly, without imposing simplifying assumptions upon them. This simple fact has not been noticed, and statistics continues to be taught in the Fisherian mold.

These fundamental defects in the foundations of modern statistics render it ripe for a revolution. This is what my new textbook in Statistics entitled: Real Statistics: A Radical Approach attempts to accomplish. In a previous post (Preface to Radical Statistics) I have described some of the key features of the new textbook, and provided links to the first draft. The first draft basically compiles a large number of lectures which were created independently, without serious consideration of coherence and sequencing. These posts contain repetition, and are written in an informal manner. I an now in process of created the second draft, which should be a nearly final and polished draft, ready for publication. As a I go through the revisions, I will be putting up the revised sections up for comments and feedback, on a number of social media channels. Teachers and students of statistics are especially encouraged to participate in this venture by following along and offering comments on the posts regarding clarity, coherence, or any other relevant issue. It is worth noting that this textbook will be published for the purpose of spreading knowledge, and will be made available freely, or at minimal cost to cover expenses, to as large an audience as possible.

Let me end with a brief discussion of the political power/knowledge issues which will arise when attempting to use a disruptive technology – one which renders previous approaches obsolete, and calls for a revolution in how statistics is taught around the world. After examining the fate of the heterodoxy within the heartlands of capitalism, it is my considered opinion that a revolution in statistics cannot be launched within the West. The opposition is too strong. Using conventional statistical methodology, one can prove anything with any data, by imagining that the data is a random sample from a suitably chosen parent population. This capability which enables “How to Lie with Statistics“, is essential to sustain the mirage that capitalism has been of great benefit to humanity. We will be laughing and singing as our boat approaches Niagara Falls. Beneficiaries of a revolution in statistics will be those who are exploited by this manipulation of data — see for example, the Confessions of an Economic Hitman. Even among this audience, within the third world, the rich and powerful are beneficiaries of the system. Also, those educated in conventional statistics, within the academia, will seek to protect their investment. Launching a revolution will require sacrifices, as it will oppose power structures supported by the fake knowledge created by artificial statistics. To provide the required energy, an alternative ideology is required, which will create the desired commitment and vision. This is why my primary target for my upcoming online course is teachers of introductory statistics in the Islamic world. I am now actively looking for teachers of introductory statistics courses throughout the Islamic world. If you would like to learn a revolutionary approach to teaching statistics, based on Islamic epistemological principles, please sign up on the Google Document: https://forms.gle/FhbmVcoZkQTT9wku5

Online Course: Real Statistics: A Radical Approach

{bit.ly/ocRSRA} Wikipedia article about “How To Lie With Statistics” shows that this is the most popular statistics textbook, with larger sales than that of all other statistics textbooks combined!! WHY?

Three Major Defects in Modern Statistics: These defects are common to the entire structure of knowledge created in the West over the past few centuries, and these make it easy to lie with statistics:

  • Ontology: What exists
  • Epistemology: What can we know about the world (and what exists)
  • Methodology: What are the correct methods to acquire knowledge

All three were created in early 20th Century, based on the philosophy of Logical Positivism. This philosophy had a spectacular crash in mid 20th Century. BUT: the foundations of statistics were not revisited!! Planned course rebuilds statistics by replacing these foundations.

Historical Background of European Intellectual Developments

The philosophy of Logical Positivism is a theory of knowledge which arose in the West due to peculiar historical circumstances outlined below:

  1. Europeans reconquer advanced but decaying Islamic Spain (Al-Andalus).
  2. Translations of millions of books in the libraries of Al-Andalus ends the dark ages of Europe.
  3. Influx of new knowledge creates a battle between “science” and religion which lasts for two centuries, and splits Christianity.
  4. Bloody fratricidal battles between Christian factions necessitate building knowledge on secular grounds – equally acceptable to all factions
Continue reading

Islamic Statistics: A Disruptive Technology

Disruptive technology is an innovation that significantly alters the way that consumers, industries, or businesses operate. A disruptive technology sweeps away the systems or habits it replaces because it has attributes that are recognizably superior. I am now actively looking for teachers of introductory statistics courses throughout the Islamic world. If you would like to learn a revolutionary approach to teaching statistics, based on Islamic epistemological principles, please sign up on the Google Document: https://forms.gle/FhbmVcoZkQTT9wku5

In a previous posts Preface to Radical Statistics and Why an Islamic Approach to Statistics?, I have explained how we can develop a new approach to statistics, which rejects a century of developments based on a methodology created by Sir Ronald Fisher, father of modern statistics. It is my hope that this will be a disruptive technology – it will eventually sweep away the entire structure of knowledge which has been built up under this name, and replace it with a radically different alternative. This structure of statistics (which I studied in detail and depth at the Ph.D. Statistics program at Stanford University), is currently being taught in universities around the globe. This structure is deeply deficient for a number of reasons. The most important among these is the logical positivist philosophy which was used to create the foundations of this discipline in the early 20th Century. We briefly describe the key problems created by this.

First, a very brief introduction to the Logical Positivist philosophy. Over a century of brutal and massively destructive religious warfare between Christian factions made it necessary for European intellectuals to find an alternative to religion on which to create a secular political science. Bitter experience had shown them that Christianity was highly unsuitable for this purpose. Ultimately, over the period of more than two centuries, this led to the creation of a whole body of knowledge which forms the foundations of secular modernity. Given the warfare among Christian factions, it was a necessity for secular knowledge to command consensus without reference to an agreed upon body of religious or moral knowledge. Therefore, modern secular knowledge claimed to be a body of objective knowledge, universally applicable to all human societies, and derived on basis of publicly confirmable observations and uncontestable logic alone. This theory of secular knowledge received its final polished form in the early 20th Century with the development of the philosophy of logical positivism (LP). According to LP, valid knowledge could be based only on observations and logic. Science was based on observations and logic and was the only form of valid knowledge. Religion was based on unobservables like God, afterlife, Day of Judgment, angels, etc. and hence not a form of valid knowledge. For a more detailed account, see The Emergence of Logical Positivism.

In fact, the task facing the European intellectuals was an impossible one. Any body of knowledge which informs us about appropriate political, economic, and social institutions and regulations must necessarily be built on moral foundations. But violent disagreements among Christian factions made it impossible to find suitable moral foundations on which consensus could be achieved. This led to the use of “reason” as a code-word to develop knowledge — reason could be re-defined in ways suitable to the desired epistemological goals. Thus Kant used reason to arrive at Newton’s Laws of Gravity, and also to devise a universal moral code of behavior, applicable to all “rational” human beings. Continuing in this fashion, economists today define “rationality” in a peculiar way to endorse as rational only behavior which seeks to maximize worldly pleasure — belief in the unseen is considered irrational. The punchline is that secular modern knowledge claims to be objective, universal, and applicable to all human societies, but it conceals a large base of Eurocentric assumptions. For a more detailed account of what has been concealed beneath the facade of “reason”, see The Puzzle of Western Social Science and Origins of Western Social Sciences.

At the heart of Logical Positivism is a rejection of unobservables as a basis for knowledge. This is in dramatic contrast to the message of the Quran, which opens by explaining that Taqwa (God-consciousness) requires faith in the unseen. It is this rejection of unobservables that leads to serious problems at the heart of modern statistics. The concept of probability refers to unobservable events which might have happened, instead of what actually occurred. Statistics is based on flawed foundations because of Logical Positivist efforts to remove these unobservables, in accordance with the misconception that science is based purely on observables, In probability, the awkward and incoherent frequency definition, and the equally awkward and incoherent Bayesian definitions are in use in textbooks around the world. Neither refer to the concept of probability as being about what might have happened, because such terminology is meaningless according to LP. Equally serious and fundamental is the problem that causality is never observable. Only correlations are observable. So despite clarity on the fact that correlations are not the same as causation, statistician continue to equate the two, because they have not satisfactory theory of causation — see Causality as Child’s Play. Within the philosophical framework of LP, no satisfactory theory of probability, or causation, can ever be developed, because both concepts relate fundamentally to unobservables.

In addition, a third fundamental defect in modern statistics was created by lack of computing capabilities. Since large data sets were difficult to handle, Sir Ronal Fisher defined statistics to be about the reduction of data, to small number of “sufficient statistics”. The technique he devised for this purpose was to imagine that the data is a random sample from a hypothetical parent population. A well-chosen parent population can be characterized by a few parameters, and modern statistics became the art of learning about these few parameters from the data. Modern textbooks continue to base statistics on this methodology created by Fisher, even though advances in computational capabilities have rendered it obsolete. By careful choice of the imaginary parent population, an expert statistician can make any data set appear to have any pre-selected characteristics desired. With modern computers, there is no need to reduce the data, since we can analyze enormously large data sets directly, without imposing simplifying assumptions upon them. This simple fact has not been noticed, and statistics continues to be taught in the Fisherian mold.

These fundamental defects in the foundations of modern statistics render it ripe for a revolution. This is what my new textbook in Statistics entitled: Real Statistics: A Radical Approach attempts to accomplish. In a previous post (Preface to Radical Statistics) I have described some of the key features of the new textbook, and provided links to the first draft. The first draft basically compiles a large number of lectures which were created independently, though with a view to coherence and sequencing. Nonethless, these posts contain repetition, and are written in an informal manner. I an now in process of created the second draft, which should be a nearly final and polished draft, ready for submission for publication. As a I go through the revisions, I will be putting up the revised sections up on this blog for comments and feedback. Teachers and students of statistics are especially encouraged to participate in this venture by following along and offering comments on the posts regarding clarity, coherence, or any other relevant issue. It is worth noting that this textbook will be published for the purpose of spreading knowledge, and will be made available freely, or at minimal cost to cover expenses, to as large an audience as possible. In the next post, I will discuss the political power/knowledge issues which will arise when attempting to use a disruptive technology – one which renders previous approaches obsolete, and calls for a revolution in how statistics is taught around the world.

Why an Islamic Approach to Statistics?

{bit.ly/WaIAtS} On first appearance, the idea of an Islamic approach to statistics seems absurd and ridiculous. After all, 2+2=4, regardless of whether you are a Buddhist, Christian, Atheist, or Muslim. Statistics is presented as a method for objective analysis of numbers, which would give the same results regardless of your religious orientation, or lack thereof. However, this appearance is an illusion. Actually, statistical analyses contain lot of hidden assumptions, which can project the biases of the researcher, while making them appear to be objective facts conveyed by the data. The most widely sold book on statistics is entitled “How to Lie with Statistics” – at last count, it had sales greater than that of all other statistics textbooks combined!  Why is it that statistics has an appearance of objectivity, while the flexible methodology and variety of assumptions allow the same set of numbers to be used to convey radically different messages? This video lecture (and the writeup below the video) provides a brief sketch of an answer to this deep and difficult question. For further details, see previous post on Preface to Radical Statistics

The Power of the Quranic Message

The message of the Quran transformed ignorant and backwards Bedouin into world leaders. It created a civilization which enlightened the world for more than a thousand years. The Quran mentions that God will provide knowledge to mankind which they did not have.This leads to the following questions:

Continue reading

Preface to Radical Statistics

I have recently complete a new textbook on elementary statistics which rejects the century long developments based on the approach created by Sir Ronald Fisher in the early 20th Century. It advocates rebuilding the discipline on new foundations. In particular, statistics should be treated as a modern branch of rhetoric: the art of persuasion using data. The video/blogpost below provides an introduction to the ideas that went into the creation of this new approach. Because it is in early draft form, the entire course is currently available online for beta-testing from https://portal.alnafi.com/courses/descriptive-statistics. I would request especially teachers of introductory statistics in the Islamic world to go through this course and provide me with feedback directly or on the website.

The Ghazali Project is based on the idea that a Western education imparts a Eurocentric worldview, as well as toxic moral ideas, to all students. But it also imparts essential skills for the modern world. This makes it essential to develop alternatives which impart the necessary skills but are built on Islamic moral foundations, and framed within an Islamic worldview. Understanding why a secular modern education causes damage to Muslim youth requires a study of European history of thought, very briefly sketched below.

Radical Transition in European Epistemology

  1. Over a century of wars between Christian factions led to the necessity to build political science on secular foundations.
  2. This first step eventually led to the creation of secular modernity, built on “objective” knowledge, derived purely from reasons and observations, without any moral inputs.
  3. The idea of “objective knowledge” is an illusion. ALL useful human knowledge is necessarily built on moral foundations.
  4. Knowledge created by European intellectuals over the past few centuries pretends to be objective and rational, but is built on concealed toxic moral foundations

The Ghazali Project

  1. Standard Diagnosis of Ills of the Muslim Ummah: We have fallen far behind the West in acquisition of knowledge, commanded by God.
  2. Standard Solution: Acquire Western knowledge, build schools & universities.
  3. Ghazali Diagnosis: The Western education being provided all over the globe is the SOURCE of the disease, because it is built on toxic moral foundations. Nonetheless, it also imparts skills essential for today,
  4. Ghazali Solution: Build an Islam alternative to Western education which provides the required skills, but builds them on Islam moral foundations.

Proof-of-Concept:  A Statistics Textbook

Continue reading

Central Ideas Part 2

{bit.ly/Ghazali2} Video is followed by a Summary of Part 1: Central Ideas of the Ghazali Project, and a writeup of Part 2. The Ghazali Project is about the central importance of the intellectual challenge currently facing the Islamic Civilization, and how to respond to this challenge.

Summary of Part 1 (Central Ideas of the Ghazali Project – http://bit.ly/Ghazali1 ):

Islam and West offer us two diametrically opposite worldviews. According to Islamic teachings, the final message of God to mankind, delivered to our Prophet Mohammad SAW more than 1400 years ago, launched a revolution in this world. This message is complete and perfect, and transformed ignorant and backwards Bedouin into world leaders. For a detailed explanation of this worldview, see Syed Abul Hassan Ali Nadwi’s book on What the World Lost Due to the Decline of the Islamic Civilization. In dramatic contrast, Western education teaches us that the world was in ignorance and darkness until European Enlightenment. Revolutionary advances in science, technology, and governance, based on use of reason ended the dark ages of Europe. The advances in knowledge created by the Europeans dazzled the world and continue shape every aspect of our lives today.  

Today, across the globe, Muslim youth receive a Western education which concentrates on European intellectual achievements, and makes no mention of any other contributions to the body of knowledge which is the intellectual heritage of mankind. This naturally creates the impression that the knowledge revealed by God to the Muslims is now obsolete, and not relevant for the modern world.

The central conflict is epistemological: Islam claims that the most important and valuable knowledge is the final revelation of God to mankind, while the West claims the same role for knowledge developed by European intellectuals and their heirs over the past three centuries. The goal of the Ghazali project is to resolve this conflict by proving the central importance of the revelation to our lives today. As described in greater detail earlier in The Ghazali Project: Indonesia Chapter , the project is in three stages.

Continue reading

Central Ideas of the Ghazali Project

{bit.ly/Ghazali1} The Ghazali Project (discussed in detail in a sequence of 10 lectures linked in How to Launch an Islamic Revival?) is based on complex analysis and diagnosis of the current difficulties facing the Ummah. This lecture is a part of a sequence of Ghazalian Webinars delivered on Sundays to the Jakarta Chapter of the Ghazali Project, headed by Lisa Listiana. The YouTube Video provides a recording of the complete lecture. It is followed by a detailed writeup of the first part of the talk. I hope to be able to writeup the second part later.

Zoom Recording: Central Ideas of Ghazali Project

Given below is a writeup of first part of this talk articulates the issues made in greater detail:

Stating the Problem

The Message of Islam – the complete and perfect final message of God to mankind – led ignorant and backwards Bedouin to world leadership, and created a magnificent civilization which enlightened the world for over a thousand years. Today, Muslims all over the globe are ignorant and backwards, and our civilization is far behind others on many apparent measures of success.

  • DIAGNOSIS: What is the reason for our current backwardness?
  • CURE: What is the solution – the path to progress?
Continue reading

A First Lecture: Motivating Students

Urdu Lecture linked below is a generic first lecture, suitable as the opening lecture for any class. The teachings of Islam created the thirst for knowledge among Muslims. Seeking knowledge from around the globe as part of the religious duty, Muslims created a civilization which enlightened the world for more than a thousand years. The decline of Muslims corresponds to a decline in the desire for knowledge. This lecture explains the reasons for this decline, and how to counter it. It encourages students to become seekers of knowledge, and to invest time, effort, and energy, into the acquisition of knowledge, as the highest form of worship. Obstacles to this goal are also discussed. The Urdu Podcast is linked below. This podcast is also available from ANCHOR: Students 2: علم کی طلب. This is followed by a brief outline of the talk in English.

https://podcasters.spotify.com/episode/2TbHZxmntia1gTgR8lygjG

English Summary and Outline of Talk:

Continue reading

Tribute to Rajani Kanth

I was invited to speak at a special session entitled: “Celebrating Rajani Kanth, and His Life’s Work” held on Sat Jan 8th, at the annual meetings of the American Economic Association. For Full 2:30 hr session, see: https://www.youtube.com/watch?v=SJTq6N-Lnc4. The work of Rajani Kanth can be summarized as “reverse anthropology”. It is not well known that Immanuel Kant lectures on anthropology far outnumber his lectures on philosophy. He provided an inscrutable philosophical basis for the idea that white races were human while all other skin colors were markers of subhuman races, incapable of thinking rationally, and hence suitable for colonization, conquest, and enslavement. See “Kant: Misogynist and Racist” (http://bit.ly/AZKant). Orientalism by Edward Said argues that central to the power of Europeans is their ability to describe and characterize non-Europeans, without interference from these other races. Thus, anthropology is a tool of power, where Europeans study the eccentricities, weakness, and failings of non-Europeans. Reverse anthropology is the study of Europeans by non-Europeans, an attempt to counter the power of orientalism. In this area, Kanth’s work is of great importance. My brief 5m tribute in the 2:30 hour session is outlined below:

Females, Families, & Tribes Against Eurocentric Modernity

Continue reading