Shubham Agarwal - What comes to your mind when I say the term data? And what do you think when I say story? Now, while your mind throws up a gazillion things about the two terms, come back. We're going to be discussing that in today's episode exactly. Is the data more important, or is it the story?

So, cast your vote in your mind, and just in case your vote changes by the end of this episode, let us know in the comments. Welcome to the Counterpoint Podcast by Vector Consulting Group. For the conversation today, we have Malik with us, Partner with Vector Consulting Group, who's led many breakthrough solutions over the years with clients. So, let's bring him in and discuss more.

Hi, Malik. Welcome to the Counterpoint podcast. How are you?

Mohammad Malik Rafi - I'm good. Thank you.

Shubham Agarwal - I'm fine. Thank you. So, Malik, before we go into the discussion and, you know, deep dive into what's more important, data or story, I want to first go back in time as much as possible and discuss about the origins of data and story. Obviously, stories have been there from time immemorial. But what about data? When did data become a part of the everyday life and everyday conversations that we have?

Mohammad Malik Rafi - So, it's a very interesting question for our listeners, what they'll have to understand is that storytelling has been an innate part of humanity from time immemorial. In fact, go back to the famous stories you remember, even the epics sort of that we have in India, the Ramayana, the Mahabharata, and even globally. So, you'll see that there is a chain of stories that sort of is common across civilizations.

In fact, anthropologists have studied stories that have been around for generations across cultures. And these stories have transcended cultural boundaries. For instance, the famous story of Noah's Ark and the Flood is found in different permutations across cultures. There are stories that have to do with what happened in the past.

For instance, you will see cave paintings of people trying to make sure that what they've lived through is immortalized. And you will see new cave paintings getting discovered now and then. There are famous caves in France, where cave paintings were created about 10,000 years ago or even earlier than that. So, the innate need to tell stories has always existed in human beings. And that's what's driven us throughout these ages.

Shubham Agarwal - That's how we've taken the teachings ahead.

Mohammad Malik Rafi - That's how we've taken teachings.

Shubham Agarwal - That's how we have grown and evolved.

Mohammad Malik Rafi - Even before writing. Even before writing, stories were common. As soon as language evolved, people were passing on their knowledge and wisdom through stories. That's how it's been. And as civilization developed, as people started farming, collecting together, forming tribes, and settling down, the need for counting, the need for making sure that you know how many people are there in your tribe, how much land you have to cultivate, how many animals do you have, the need for basic data, what we call 'data'. It was simply about survival back then. So, in a village, if you have 100 to 200 people, how much food do you need, how much land do you need, and how many animals do you need? This need started evolving; therefore, numbers began coming up, and the number system started evolving. So, it evolved. Stories and data have sort of evolved with civilization.

Then, obviously, came the age of scientific enlightenment, where in the Middle Ages, people started looking up to the stars and started figuring out how many stars are there.

Are we at the centre of the universe? How far are we from the sun? How far are we from the moon? How big is the sun? How big is the moon? How do the of galaxies move? How do the stars move? How does the sun and the moon move? So, as soon as people started asking these questions, the need for measuring started coming up, you will see measuring instruments getting evolved. You will see the numerical system getting evolved. You will also see algebra getting developed. You will see the decimal system getting developed. All of these things started developing. But at the core of it was humanity's need to understand the world around them. Where are we? Who are we? Where are we in the centre of the universe?

Even with counting, your grandmother would have told you a lot of stories and you would still remember them. And it's only recently that a lot of sociological research has told us that knowledge gets disseminated faster and more effectively through stories.

Shubham Agarwal - So, Malik, it is interesting because you are right, how stories and data go back in time. And obviously, stories have been so powerful in communicating wisdom, learning, or teachings over the centuries or, you know, since... yeah, in fact, millennia, you're right. And so is data. Like you rightly said, who we are, where we are, to measure everything. So, as far as the number system goes, data also goes. However, over the last few years, data has become extremely important. A lot of focus means a lot of impact on data. Why that? How did that shift happen?

Mohammad Malik Rafi - Okay, so we'll have to go back a couple of centuries for this. There was a major shift in the way humans lived.

For most of our existence, we've been agrarian people. We've sort of settled in a place. And for most of humanity, if you were born in a certain place, 80%-90% of the time, you will live there and you will die there.

But in the early 1700s and early 1800s, as industrial evolution started picking up as new discoveries started impacting human life, you would see industries come up. You saw productivity rates go up. Now, you no longer had to have 100 people employed in a field to generate X yield. You could have sort of done away with, say, ten people. So, what about the 90 people who are now free? So, the age of exploration started.

Which new lands can we explore? Can we settle there? Can we find new stuff there? Can we search for gold? So, you will see these stories coming up. So, as soon as these exploratory ventures started coming up, and colonialism is sort of a pursuit of that, you will see that risk-taking enterprises started going up. As soon as risk started coming in more and more into human life, you would see people pooling risk, so you would see corporations forming. The East India Company was the largest corporation in the world at the time. And it's a completely different story, but it's very interesting. India, which contributed 40% of the world's GDP, was conquered by not an army, not a nation, but by a corporation.

So, the East India Company was a corporation. So, what was the East India Company? It was a risk-pooling enterprise. So, as soon as you have these risk-pooling enterprises coming up, you need to know how much money has been deployed, how many people are employed, and what the risk of the investment. So, you would see industries like banking, insurance, and the stock market, these institutions started building. As soon as you see these institutions start building, you see the need for counting, collecting data, storing data, and analysing the data going up. Suppose you've sent 100 people on a voyage. Now, you need to know approximately what percentage will survive and what is the risk of the voyage succeeding or failing.

Therefore, you started seeing people looking into probability, for instance. Probability, the theory of probability, Bayes' theorem, and the whole idea of probability started taking shape at this point in time. The whole insurance industry was, in fact, an experiment, a lab in statistics and probability at that point in time. So, that's where we've sort of built our knowledge from. As these enterprises started growing, so did the need for collecting data, and the need for understanding the nuances of the data started going up.

And I think that's mainly why you started seeing a shift towards the data side of things more, okay. But there was also a strand of scientists or people who were interested in knowing ‘why’, who sort of rebelled against this. And there has been this debate continuously for the last 200 to 250 years. You would see statisticians swear by data. You will see them saying that the data reveals all, and if the data says something, that's the truth. And then, on the other hand, you have, for instance, theoretical physicists, or you have people studying diseases who would sort of swear by causality, who would say that the data needs to follow causality, not the other way around. You need to understand the world, and then the data needs to validate or invalidate your theory. One of the biggest proponents of this worldview, the causal worldview, as Albert Einstein. So, you would see a patent clerk sitting in his office who's just graduated in physics at 24 years of age, trying to sort of disprove everything we've understood about reality and writing a very simple paper on the Theory of Relativity.

And with very limited empirical evidence, he didn’t do experiments, there we no large instruments used by Einstein, it was all thought experiments. So, this causal revolution in physics also exploded. Until then, physics had become an empirical enterprise. You measure something, and then you prove it. If it's measured, hence proven. But he flipped it around, and there were a lot of people who started doing this again. So, this tussle between measuring and trying to understand the world through numbers versus trying to first understand the world through cause and effect and then sort of trying to see if the numbers fit into the theory has always been there.

Shubham Agarwal - Interesting that you mentioned Albert Einstein because you'd probably think for someone like Einstein to never really have a causal view, as in he would be, in your mind, it would be someone who would first look at data and then try to prove something. But, very interesting that there was a complete group of physicists or people who were trying to prove things, and they have an hypothesis first and then prove it. So, do you feel the other side of the story, the side that thinks that data is more important and the databased approach is a more prudent approach? Do you think they have taken it too far? Do you think they have started to abuse data a little? Do you see that happening?

Mohammad Malik Rafi - No, I wouldn't say that. Let's take a story.

Shubham Agarwal - Sure. Interestingly, you take a story to prove whether the data is right.

Mohammad Malik Rafi - So, it will prove a lot of points. In the mid-1800s, around the 1850s, there was a gentleman called John Snow in the UK, back then God's Own Country. And he was a doctor, he was trying to understand why the city of London, and back then London was not the financial capital of the world.

Sort of the India...

Shubham Agarwal - They're still going and developing.

Mohammad Malik Rafi - Yeah, so India was the financial capital of the world back then.

London was a very a desolate, not desolate, I'd say it was a shanty town. It was a dirty place. And a lot of people clubbed together, a lot of people working in factories, in small apartments, and a lot of dirt lying around on the streets. And, you know, the typical third world, that's what London was in the mid-1800s. Improper drainage systems. A lot of diseases were prevalent then, and this one mysterious disease suddenly came up, and a lot of people started dying. And if people contracted that disease, young, old, children alike, they would die in four hours to a day.

So, this obviously was a mystery, and you will see a lot of caricatures and drawings from that time showing people doing weird stuff to try and prevent themselves from catching this disease. And this doctor was obsessed with finding out why this was happening. It was not as if other people were not trying. His approach was completely different, and that's what makes it interesting. A lot of people were counting how many people were dying, where they were dying, and what their age was. So, a lot of data was available. But no one was able to explain why this was happening.

Shubham Agarwal - There was no insight in the data.

Mohammad Malik Rafi - There was no insight in the data. You could plot a map and figure out that in this locality, in this neighbourhood, so many people have died in this age group, male and female. But why? Why is this happening? So, he was obsessed with finding out ‘why’.

His obsession led him to challenge the prevailing understanding of why this disease was spreading. The prevailing understanding was that there is a 'miasma' in the air, and this miasma spreads across the air in a certain population, with some conditions being in place. The 'miasma' causes this disease, and people die.

Now, a lot of questions came up in the doctor's mind. If this is happening, then the rate of death should be constant. Across the city, it should be similar, right? And it should not be happening in pockets. There were clusters where people were dying more and where people were dying less. More interestingly, in the clusters where people were dying, there were small pockets where people were not dying.

Shubham Agarwal - Within clusters, there were mini clusters.

Mohammad Malik Rafi - So, suppose there is a colony and 100 deaths, and you would see within the colony a building or a couple of buildings where people are not dying, and they're living there in the same locality. This intrigued the doctor, and he started thinking ‘why?’.

Shubham Agarwal - I can see how I can relate to data that we analyze daily. We have data groups, and then there are clusters within those data.

Mohammad Malik Rafi - This curiosity led him to build a hypothesis that air is not the problem, because everyone is breathing the same air. Something else must be happening. So he built a hypothesis that the water might be the problem, the water that they're drinking.

Then he started analyzing water samples and realized that there was no problem with the water in this area. So, he stuck to his hypothesis, and he started seeing a pattern. He started seeing that around certain water pumps. And in London back then, it was so poor that people didn't have running water in their houses. You had to have common water pumps. There used to be wells that were covered, and there used to be a water pump wherein people would collect, take the water from there, and then use it in their homes. There were extremely high clusters of deaths around a particular water pump.

So, obviously, now, this was extremely intriguing. And within the same locality, there were clusters where people were not dying, and there were clusters where people were dying. And the locality was near this water pump. So, he realized that in one building, which was very close to the water pump, they had their own water supply, and they were not drinking from this water pump. And even more interesting, there was a locality which was, say, two, three miles away from this particular water pump, wherein a couple of people had died. Why? Because some people had taken the water from this pump and gone there.

So, this sort of started validating his understanding, and he requested the authorities to stop the water supply from this pump. So, they uprooted the water pump, and you see a sudden decline in the number of deaths. So, it sort of proves his theory, and they were still not able to understand why, but a few years down the line, they realized that the contamination, the dirt on the streets, was sort of seeping into the well in that water pump, because of the structure of the kind of reinforcements that they had done. And in the other water pumps, the reinforcements were better. And this proved his theory. So, now you see the interplay between causality and data.

Shubham Agarwal - Had I gone by data, I doubt if I were to reach that conclusion.

Mohammad Malik Rafi - Yes, you would not have been able to reach this conclusion. You would have still continued with the existing theory. And John Snow has been immortalized now. This sort of sparked how disease studies or epidemiology, as they call it, are done today. So, the theory is built first, and the data sort of validates or invalidates the theory.

Shubham Agarwal - Which brings me to an earlier conversation that we were having around data and story, all this discussion that we had. You used a term, a very specific term called as 'spurious co-relations'. That data often has spurious correlations. Can you please explain that term, and why do you say that? Because I think that's interesting to bring it in here.

Mohammad Malik Rafi - Okay, so as the computer age has set in, so has the amount of data that is available for us to analyse. I mean, computers can crunch out gazillion bytes of data now, and you've got the age of ‘big data’. Big data is just a fancy word for saying you cannot analyze that data on an Excel spreadsheet. And you need something else to analyse it, that's big data. So, as this has evolved, so has the need to crunch this data because you have something in front of you, and you're itching to sort of try and use it and try and find insights. So, there are a lot of funny blogs on the internet, and I'll refer you to some later on where you see correlations like moon landings and consumption of cheese in a population that is extremely related. You see ice cream sales and murder rates being extremely related. You see, these random events are extremely related, either positively correlated or negatively correlated.

Shubham Agarwal - And they've shown data around that.

Mohammad Malik Rafi - There is data around it and a high correlation. This might sound like a joke, but you will see that business decisions are taken on correlations. There was this famous study done in, and there is an article in The Guardian on eBay in 2013 or 2014, where eBay was trying to find out if their advertisement budget is being used well. So, they hired consultants, God's gift to mankind, and the consultant said, "Yes, your advertisement budget is being used well."

Shubham Agarwal - We are not advocating consulting, but…ok

Mohammad Malik Rafi - And they said your budgets are going well, and you should keep spending on digital advertising. But some curious souls in the company stood up and said, "This is wrong. Let us prove this to you." So, they requested the eBay leadership allow them to do an experiment.

So, what they did is they requested that a particular region where these sorts of digital ads were being pushed to people, the ad spending be stopped for two months. And then let's see what happens on sales. Is there a change in behaviour? Are consumers changing? Is the click rate changing?

So, nothing changed after two months. No impact on sales. And you know why? Because the team had a hypothesis that the people who were clicking on these ads were the people who had pre-decided that they wanted to go on eBay.

Shubham Agarwal - Oh, that happens today also. I mean, I checked for something on Google, and the first link comes up as an ad.

Mohammad Malik Rafi - As an ad.

Shubham Agarwal - You click the ad.

Mohammad Malik Rafi - Instead of clicking the organic link, you click the ad.

Shubham Agarwal - But I was anyway going to be.

Mohammad Malik Rafi - You were going to click on the ad. So, the hypothesis was that the majority of people who were clicking on the ads were the people who had pre-decided they had put in the search term.

Mohammad Malik Rafi - So, there was no point of spending that money.

Mohammad Malik Rafi - So, it started with a hypothesis. But if you look at data, you will see a high correlation between sales and ad spends. So, you could have continued spending. So, there are funny examples available, but actually people take decisions based on these correlations. So, this is a spurious correlation. It seems that there is some link between A and B. But there is none.

Shubham Agarwal - I don't know if you know, but I think spurious correlations might be coming out because of how I project that data in my mind, probably because a lot of these times, we patternize things, you know, in life. Our minds turn up and tell us that this is why this is happening, and that is why that is happening. Probably. But I want to jump the other way around, and I want to then ask what the value that data is adding to our life is. I mean, why should I look at data in that case? Because if I want to have a hypothesis and then just prove it right or wrong, why should I even look at data in the first place? Then what is the value that you think data has brought into our life? Because I'm sure if the world is using data, there must be some value. So what is what is that?

Mohammad Malik Rafi - So that's a very, very important question. Even Albert Einstein needed experiments to prove his theory or disprove his theory. So, Einstein was famous for writing at the end of his papers, that, I was not quoting him, but "Can someone please check if what I'm saying is valid or not?" Can someone please experiment because he was not an experimenter, he was a theoretical physicist and a very good one at that. So, you will see a lot of Einstein's predictions being proven by people who have been experimenting.

One of his famous predictions was light would bend around the source of strong gravity. So, it took about five to ten years, I guess, for someone to find a solar eclipse and find a place that was right and try and figure out if this is actually true or not. But when it got proven, Einstein became a celebrity. But you have to prove the theory. And that's where data comes in.

Shubham Agarwal - Lazy guy, I guess.

Mohammad Malik Rafi - We are better off with him.

Shubham Agarwal - But what comes out is that he's at the cusp of that story and data problems. But I understand what you're saying that data is important to prove some of the hypotheses.

Mohammad Malik Rafi - Prove or disprove.

Shubham Agarwal - or disprove.

Mohammad Malik Rafi - In a way, data is a mirror to our reality. But a mirror, you cannot look at a mirror and judge reality. You have to understand why something is happening. Then, maybe the mirror can give you a better understanding of what is happening, but not the other way around.

Shubham Agarwal - Interesting.

So, the reason that we decided on this topic was because, in life in general, we want to build strong narratives. Isn't it? Every conversation is probably a negotiation in a way, trying to convince the other person of something or, a theory or some process that you're trying to implement. Even in consulting, for that matter, when you go to clients, we're trying to prove something and take a new course of action in the future. And for that, we build narratives. Now, we sometimes use data as a narrative, while at other times, we use the story as a narrative.

And I asked the listeners at the beginning of the episode to cast their vote. What do you and what's your vote on that and why?

Mohammad Malik Rafi - So, we do a lot of discussions internally within our firm. And we've realized that in today's day and age, the people who are working on a problem, say a client, they know more about their environment, about the data, the raw data of their environment than an outsider.

And if you try to explain stuff to them using their own data, they will have a thousand questions to ask you. Because they've analysed it, sliced it and diced it in every way possible. But if you are able to connect the dots and tell them this is why it is happening and this is why this should be happening and the 'a-ha' moment wherein you predict something that if this is your environment, these are the ten different things that must be happening. Something we call 'predicted effects', which I think one of our members from one of your podcast series would have talked about, Satya would have talked about that at last.

So, you will see that if you are able to build what you call a narrative or a story or what I call 'causal stories', wherein if A happens and B happens, therefore, C must be happening. This brings a lot of credibility to your pitch to your client or the person you're talking to because then they understand. And you have to understand that human beings are innately causal creatures. They want to understand 'why' something is happening. So, if you explain the ‘why’ and then you sort of show data to prove it or disprove it, it creates a very strong buy-in. So, we've seen this happen with our clients, and we've seen this in our normal conversations we have internally as well. So, it actually makes business sense to try to understand the ‘why’ and then use data to

Shubham Agarwal - PS, someone please prove it.

Mohammad Malik Rafi - Yes, please prove it.

Shubham Agarwal - So, then an obvious question. Can you tell us how to build such strong narratives that can drive success, that can drive compelling conversation, compelling buy-ins in organizations?

Mohammad Malik Rafi - Okay, that's a very good question. It's an art and a science.

Shubham Agarwal - So, we'll talk about the science first.

Mohammad Malik Rafi - Let's talk about the science.

Shubham Agarwal - Practice and make it better.

Mohammad Malik Rafi - So, I'll give our listeners one tip. Don't look at the data, before you start analysing a problem. Try and meet the people who are involved in the problem. Try and understand why they are doing what they're doing. Try and build a model of why something is happening and then look at the data. That will give you a much more clearer understanding of why the data is looking the way it is, and it will also give you an understanding of the exceptions.

Shubham Agarwal - The outliers.

Mohammad Malik Rafi - The outliers. And the outliers, actually, we ignore outliers. Usually, we ignore outliers. Even in data, we ignore outliers. But the outliers explain the truth much more than the sort of inliers.

Shubham Agarwal - The normal falsifiability is easily proven through outliers.

Mohammad Malik Rafi - Yes. So, you need to understand why there is an outlier. And that sort of explains why, it gives you a much clearer understanding of reality. And this is true in science, this is true in business, this is true in life. So, I think that would be the one single tip that we should let our listeners go with.

Shubham Agarwal - So, I think it was a great discussion, Malik. I want to end on a slightly personal note. Because I know across the years that we have worked together and at Vector also, I know that you have sort of transitioned from a very data-heavy, extensive data approach to a very story-based and narrative-based approach as we have discussed. What has been your personal experience through this journey? What has been that one learning that was more like a realization for you and that hit you very hard?

Mohammad Malik Rafi - That's a very good question.

So, it came from my work that we've been doing here. If you look at our educational system, how do we learn in our universities or our schools? We've got these ready-made textbooks with ready-made formulas and the ready-made questions and answers. And you just have to do the grunt work. And even the high-quality stuff that we learn, such as trigonometry, algebra, calculus, advanced maths, and advanced physics.

It's almost like we've been there and done that. People have done it, and you just have to absorb it and just apply some thinking to it for a slightly different problem and just give the answer.

The application of this knowledge starts when you start working or build your career, or interact with the real world. That's when the application begins. And as soon as the application begins, you will start seeing failures. You will start seeing stuff that is not working. There is a theory, there is a tool, or there is a solution that's been built for a particular problem, and you see it doesn't work. You've done everything under your power, and you see it doesn't work. And this is where you sort of have a moment where you have a crisis of faith.

What is it I'm doing? Why has someone told me this? Because it doesn't work here. And this is when you start questioning the boundaries of the solution or the assumptions behind the solution. And that's where you start going under the hood, sort of, of what you've been taught or what you've been told. And then you start thinking about why this works and why it doesn't work. And it's actually a reality that pushes you in this direction. And what I've seen is people who've been working in the industry for years, they've sort of imbibed a system. The system has been working. There are problems, they know that these problems are there, it exists, this is how the industry operates. And they live with it.

Very rarely do you see someone questioning, breaking that mould, and questioning. So, a great example is SpaceX. NASA builds rockets for a billion dollars, and they've been able to build it at one-tenth the cost. Someone has to question why it costs so much. And it's not a simple answer. I mean, NASA being NASA, they are literally rocket scientists. So, they know how to build rockets.

So, it's not easy to question that paradigm. But then you have to start challenging assumptions. So, for instance, why should a rocket go up and not come back? Why can't I reuse it? If it can come back, I can reuse it multiple times over a lifetime and I can save.

So, this thinking needs to be applied. These kinds of fundamental assumptions need to be challenged. But it's when reality hits you that you start questioning these assumptions. Otherwise, you've sort of got the data. You've got what kind of sales you're doing, what kind of products you're selling. And you're sort of there. You know how the industry operates. So, the normal prevailing conditions, if you're used to the prevailing conditions and you're sort of comfortable with the problems, you will never ask these questions.

Shubham Agarwal - It's your curiosity.

Mohammad Malik Rafi - Yes, yes. So, generally if someone is uncomfortable with something happening or not happening, that's when you start questioning. That's when you start challenging the data and the narrative, and that's where you start questioning the assumptions and the boundary conditions. And within our firm, you will also see these conversations happening a lot. And I think that's where this curiosity sort of kicked in, and the approach changed to understanding the narrative first and the data next. So, that's how it began, I guess.

Shubham Agarwal - Wonderful. I think a lovely end to a discussion is when you know something that is very personal, and you have shared that very candidly. So, thanks for that. And thanks for this wonderful conversation overall. Thank you. Thanks! Great. So, I think a wonderful discussion.

A lot of thoughts around data and stories and narratives and what is more important. So, I leave the thinking part to you. I think Malik helped us a lot in terms of understanding both worlds way better, I think, for me personally also. So, I leave the word to you, and we'll wait to listen from you, hear from you in the comments.

Until then, this is Shubham signing off. Bye-bye. --- this last line is missing in the audio.