“you are smarter than your data. Data do not understand causes and effects; humans do.”&nbsp;― Judea Pearl, The Book of Why: The New Science of Cause and EffectUnderstanding the ‘why’ to navigate ambiguities in decision-makingIn the complex world of business, the decision-making process can often feel like trying to find your way through a fog. It's hard to be sure you're choosing the right path when everything seems unclear. This article emphasizes a powerful tool to cut through that fog: understanding the ‘why’ behind things, which is a bit like using a map in unknown territory. By looking closely at the reasons behind business challenges, just like scientists do when they're solving mysteries, we can make better decisions. Through real-life examples and stories, we'll see how figuring out the ‘why’ can light up the way forward in business.A case study: Beyond sales team ‘right-sizing’Consider a case of one of our clients (operating in a B2B environment). They faced with the dilemma of a ‘bloated’ sales team that was trained for years to handle client servicing (everything from order delays to payment follow ups). The market was projected to grow slowly at 7% p.a. However, the client wanted to grow faster and profitably. A simple approach would be to cut down on excess salespeople and improve profitability. Right?Wrong! Analyzing this problem with existing data risked taking actions detrimental to the existence of the company! The sales team, in this case, played a crucial role in ensuring that the client’s customers were served on time, every time in a highly competitive market.The understanding we built here was ‘if the market is highly competitive, then order servicing is a business-critical action and cannot be diluted’; having the sales team involved in order servicing end-to-end was a feature, not a bug!Hence, what needed to be done was a rethink of the market the client was operating in, and not ‘right-sizing’ the sales team. We worked out a strategy to retain the customer servicing skills of the existing sales team and reskilling top performers for a high growth market segment that was untapped. We then used data from the untapped market to figure out how to build a moat for the client, thus solving for profitable growth.It worked! The client is in the process of re-designing their entire sales strategy. Taking a step back allowed us to prevent an action that could have jeopardized a business-critical feature of the sales operations!The quest for ‘why’This way of working on business problems (taking a step back and building a deep understanding of the causes at play) is inspired by science. We call it Causal Thinking (you can call it – the quest for ‘Why’). There are many pitfalls to not following this approach. Here’s one example of the many disasters in the business world where decision making has been purely based on data without understanding the ‘Why’.The 2008 Financial CrisiA devastating blow to the global economy was exacerbated by an overreliance on sophisticated algorithms used by rating agencies and trading desks at major banks. These algorithms, intended to assess the risk associated with various financial instruments, were critically flawed. They mistakenly rated complex mortgage-backed securities, many of which were essentially junk bonds, as AAA—the highest possible credit rating. This misrepresentation of risk encouraged a widespread investment in these toxic assets, leading to a massive bubble. When the bubble burst, it triggered a cascade of financial failures, as the true value of these securities became apparent. This crisis highlighted the dangers of relying too heavily on algorithmic judgments in financial markets, underscoring the need for more robust oversight and a deeper understanding of the risks involved in complex financial instruments. The Movie The Big Short illustrates how some predicted the crash by digging deep into story behind mortgage-backed securities, unlike the major banks that missed the signs (or chose to ignore).David Spiegelhalter, in his book, The Art of Statistics: Learning from Data writes, ‘Senior managers simply did not realize the frail basis on which these models were built, losing track of the fact that models are simplifications of the real world–they are the maps, not the territory. The result was one of the worst global economic crises in history.’Contrast this with the lifesaving work done by Dr. John Snow amidst the Cholera outbreak of the 1850s in London.Dr. John Snow, an English physician, is often considered one of the fathers of modern epidemiology thanks to his pioneering work during the Cholera outbreaks in London in the mid-19th century. His most famous research is associated with the 1854 Cholera outbreak in the Broad Street area of Soho, London.&nbsp;Snow did not believe in the then-dominant miasma theory, which suggested that diseases such as Cholera spread through the air as a form of ‘bad air.’ Instead, he proposed that Cholera was spreading through contaminated water – a new theory.Dr. Snow had evidence contrary to the miasma theory which allowed him to propose a new one. For instance, if the ‘miasma’ spread through air, why were there pockets of no disease within a cluster of high disease prevalence? How could air be causing the disease if everyone was breathing the same contaminated air in the same cluster and prominent pockets were completely healthy?Thus began Dr. Snow’s meticulous case tracking and mapping of Cholera cases. He identified a public water pump on Broad Street as the outbreak's epicentre.By gathering data on where the victims of Cholera lived and where they got their water, Dr. Snow was able to convincingly argue that those who drank water from the contaminated Broad Street pump were much more likely to develop and die from Cholera and those who were drinking from safe supplies in the same vicinity as the Broad Street pump, were safe, even though they were in the vicinity. &nbsp;He famously had the handle of the pump removed, and the outbreak quickly subsided.Dr. Snow's work didn't immediately change public health policies, but over time, his theory, backed by a strong evidence-based approach, helped shift the scientific community's perspective towards understanding the importance of sanitation, clean water, and the germ theory of disease. His mapping and data analysis method remains foundational in Epidemiology today, illustrating the importance of building a sound theory, gathering empirical evidence and the application of scientific methods to public health.The key point is Dr. Snow started with a causal framework, a theory of ‘Why’ and then worked his way to meticulously investigated with data to determine whether it was correct - NOT the other way around!<figure class="image"><img style="aspect-ratio:1692/1030;" src="https://new-stage-be.vectorconsulting.in/storage/editor_uploads//screenshot-2024-05-13-at-44453-pm_1715598908.png" width="1692" height="1030"></figure>Data as a Compass, Not the MapJust like Dr. John Snow used data to trace the cause of Cholera back to a single water pump, data can illuminate the actual reasons behind what we observe, helping us validate or invalidate the causal story. A striking illustration from science is how observations during a solar eclipse validated one of Einstein's revolutionary predictions—that gravity can bend light. This concept reshaped our understanding of the universe, but it wasn't universally accepted until data from a solar eclipse showed stars' light curving around the sun, precisely as Einstein had predicted. This historical moment underscores a critical lesson: data is most powerful when used to test our theories and hypotheses, not when viewed in isolation. Just gathering data without a clear idea of what you're looking for can be like finding pieces of different puzzles mixed—you might have a lot of pieces, but they don't help you see the whole picture.In business, this means, before diving into data, we should start with a theory or a hypothesis about why things are happening. Then, use data to see if our theory holds up, guiding us toward the right solutions and decisions. This approach ensures that we're not just collecting data for data's sake but using it as a tool to challenge and refine our understanding, leading to clearer insights and better outcomes.A simple guide to building a causal viewpoint:Let’s circle back to our case on Sales Team ‘Right Sizing’ and analyze it using simple causal analysis steps:1. Clarify the problem you’re trying to fix (an exact definition of the problem) –<ul><li>Good precision: “Growth has slowed down to the market rate of ~7% while the cost of sales (including sales team opex) has gone up 1% point”</li><li>Poor Precision: ”We have a large sales force”</li></ul>2. Do NOT explain the problem – let the problem stand on its own<ul><li>‘Cost of sales has gone up because we’ve hired people faster that market growth’</li></ul>3. Start with a Hypothesis - a simple explanation of ‘why this might be happening’<ul><li>&nbsp;‘Share of business at regular customers is growing at market rate while the share of business at irregular customers is stagnant or declining as these customers are switching to other types of input raw materials (this is where the bulk of new sales recruits have been deployed)’</li></ul>&nbsp;4. Try to invalidate the hypothesis by counter-examples or any other variable impacting the outcome<ul><li>Switching to other raw materials is irrelevant, as this is true only for a few customers</li><li>Import of input raw material has increased impacting the client’s customer base</li><li>Overall consumption of the input raw material still remains steady at 7% for ‘irregular’ customers as well</li></ul>5. If the hypothesis still stands, check for effects of the hypothesis being true, if not build a new hypothesis.The hypothesis does not stand the test of invalidation in this case. A new hypothesis is needed!<ul><li>New Hypothesis: Imported raw material is cheaper and of similar quality, allowing ‘irregular’ customers to switch</li><li>Check for effects if the new hypothesis is true:<ul><li>‘Irregular’ customers are still consuming the same raw material at a steady pace</li><li>Switched customers are fine with increased lead time of supply from imports</li><li>Sales team’s efforts are rejected primarily on the price differential</li><li>If the sales teamis able to match price with imports, some of the irregular customers switch back</li></ul></li></ul>6. Build a direction of solutionThe new hypothesis stands validated. It is now time to brainstorm solutions.a. Since competing on price alone is recipe for continuous margin pressure, this direction is ruled out.b. There will always be imported raw materials flooding the domestic market for regular SKUs; there is no way of stopping thisc. Customers already prefer our raw material for non-regular SKUsd. The ONLY way to sustain existing demand and grow FASTER than the market is to DEVELOP more consumption occasions – i.e. convert users of other types of raw materials! A tall ask, but its possible. Our client, therefore, started re-deploying the sales team, and the ‘bloated’ team no longer needed to be ‘right-sized’. In fact, we might need to hire more!These simple steps need rigor of analysis and can be quite challenging to implement without practice. It’s easy to find quick fixes using existing data. However, building a habit of thinking causally can have immense bottom line benefits for companies – by avoiding erroneous decision-making and by ignoring irrelevant variables. This approach allows focussing on the right problem and the right explanation for the problem. So, happy thinking!

“you are smarter than your data. Data do not understand causes and effects; humans do.”&nbsp;― Judea Pearl, The Book of Why: The New Science of Cause and EffectUnderstanding the ‘why’ to navigate ambiguities in decision-makingIn the complex world of business, the decision-making process can often feel like trying to find your way through a fog. It's hard to be sure you're choosing the right path when everything seems unclear. This article emphasizes a powerful tool to cut through that fog: understanding the ‘why’ behind things, which is a bit like using a map in unknown territory. By looking closely at the reasons behind business challenges, just like scientists do when they're solving mysteries, we can make better decisions. Through real-life examples and stories, we'll see how figuring out the ‘why’ can light up the way forward in business.A case study: Beyond sales team ‘right-sizing’Consider a case of one of our clients (operating in a B2B environment). They faced with the dilemma of a ‘bloated’ sales team that was trained for years to handle client servicing (everything from order delays to payment follow ups). The market was projected to grow slowly at 7% p.a. However, the client wanted to grow faster and profitably. A simple approach would be to cut down on excess salespeople and improve profitability. Right?Wrong! Analyzing this problem with existing data risked taking actions detrimental to the existence of the company! The sales team, in this case, played a crucial role in ensuring that the client’s customers were served on time, every time in a highly competitive market.The understanding we built here was ‘if the market is highly competitive, then order servicing is a business-critical action and cannot be diluted’; having the sales team involved in order servicing end-to-end was a feature, not a bug!Hence, what needed to be done was a rethink of the market the client was operating in, and not ‘right-sizing’ the sales team. We worked out a strategy to retain the customer servicing skills of the existing sales team and reskilling top performers for a high growth market segment that was untapped. We then used data from the untapped market to figure out how to build a moat for the client, thus solving for profitable growth.It worked! The client is in the process of re-designing their entire sales strategy. Taking a step back allowed us to prevent an action that could have jeopardized a business-critical feature of the sales operations!The quest for ‘why’This way of working on business problems (taking a step back and building a deep understanding of the causes at play) is inspired by science. We call it Causal Thinking (you can call it – the quest for ‘Why’). There are many pitfalls to not following this approach. Here’s one example of the many disasters in the business world where decision making has been purely based on data without understanding the ‘Why’.The 2008 Financial CrisiA devastating blow to the global economy was exacerbated by an overreliance on sophisticated algorithms used by rating agencies and trading desks at major banks. These algorithms, intended to assess the risk associated with various financial instruments, were critically flawed. They mistakenly rated complex mortgage-backed securities, many of which were essentially junk bonds, as AAA—the highest possible credit rating. This misrepresentation of risk encouraged a widespread investment in these toxic assets, leading to a massive bubble. When the bubble burst, it triggered a cascade of financial failures, as the true value of these securities became apparent. This crisis highlighted the dangers of relying too heavily on algorithmic judgments in financial markets, underscoring the need for more robust oversight and a deeper understanding of the risks involved in complex financial instruments. The Movie The Big Short illustrates how some predicted the crash by digging deep into story behind mortgage-backed securities, unlike the major banks that missed the signs (or chose to ignore).David Spiegelhalter, in his book, The Art of Statistics: Learning from Data writes, ‘Senior managers simply did not realize the frail basis on which these models were built, losing track of the fact that models are simplifications of the real world–they are the maps, not the territory. The result was one of the worst global economic crises in history.’Contrast this with the lifesaving work done by Dr. John Snow amidst the Cholera outbreak of the 1850s in London.Dr. John Snow, an English physician, is often considered one of the fathers of modern epidemiology thanks to his pioneering work during the Cholera outbreaks in London in the mid-19th century. His most famous research is associated with the 1854 Cholera outbreak in the Broad Street area of Soho, London.&nbsp;Snow did not believe in the then-dominant miasma theory, which suggested that diseases such as Cholera spread through the air as a form of ‘bad air.’ Instead, he proposed that Cholera was spreading through contaminated water – a new theory.Dr. Snow had evidence contrary to the miasma theory which allowed him to propose a new one. For instance, if the ‘miasma’ spread through air, why were there pockets of no disease within a cluster of high disease prevalence? How could air be causing the disease if everyone was breathing the same contaminated air in the same cluster and prominent pockets were completely healthy?Thus began Dr. Snow’s meticulous case tracking and mapping of Cholera cases. He identified a public water pump on Broad Street as the outbreak's epicentre.By gathering data on where the victims of Cholera lived and where they got their water, Dr. Snow was able to convincingly argue that those who drank water from the contaminated Broad Street pump were much more likely to develop and die from Cholera and those who were drinking from safe supplies in the same vicinity as the Broad Street pump, were safe, even though they were in the vicinity. &nbsp;He famously had the handle of the pump removed, and the outbreak quickly subsided.Dr. Snow's work didn't immediately change public health policies, but over time, his theory, backed by a strong evidence-based approach, helped shift the scientific community's perspective towards understanding the importance of sanitation, clean water, and the germ theory of disease. His mapping and data analysis method remains foundational in Epidemiology today, illustrating the importance of building a sound theory, gathering empirical evidence and the application of scientific methods to public health.The key point is Dr. Snow started with a causal framework, a theory of ‘Why’ and then worked his way to meticulously investigated with data to determine whether it was correct - NOT the other way around!<figure class="image"><img style="aspect-ratio:1692/1030;" src="https://new-stage-be.vectorconsulting.in/storage/editor_uploads//screenshot-2024-05-13-at-44453-pm_1715598908.png" width="1692" height="1030"></figure>Data as a Compass, Not the MapJust like Dr. John Snow used data to trace the cause of Cholera back to a single water pump, data can illuminate the actual reasons behind what we observe, helping us validate or invalidate the causal story. A striking illustration from science is how observations during a solar eclipse validated one of Einstein's revolutionary predictions—that gravity can bend light. This concept reshaped our understanding of the universe, but it wasn't universally accepted until data from a solar eclipse showed stars' light curving around the sun, precisely as Einstein had predicted. This historical moment underscores a critical lesson: data is most powerful when used to test our theories and hypotheses, not when viewed in isolation. Just gathering data without a clear idea of what you're looking for can be like finding pieces of different puzzles mixed—you might have a lot of pieces, but they don't help you see the whole picture.In business, this means, before diving into data, we should start with a theory or a hypothesis about why things are happening. Then, use data to see if our theory holds up, guiding us toward the right solutions and decisions. This approach ensures that we're not just collecting data for data's sake but using it as a tool to challenge and refine our understanding, leading to clearer insights and better outcomes.A simple guide to building a causal viewpoint:Let’s circle back to our case on Sales Team ‘Right Sizing’ and analyze it using simple causal analysis steps:1. Clarify the problem you’re trying to fix (an exact definition of the problem) –<ul><li>Good precision: “Growth has slowed down to the market rate of ~7% while the cost of sales (including sales team opex) has gone up 1% point”</li><li>Poor Precision: ”We have a large sales force”</li></ul>2. Do NOT explain the problem – let the problem stand on its own<ul><li>‘Cost of sales has gone up because we’ve hired people faster that market growth’</li></ul>3. Start with a Hypothesis - a simple explanation of ‘why this might be happening’<ul><li>&nbsp;‘Share of business at regular customers is growing at market rate while the share of business at irregular customers is stagnant or declining as these customers are switching to other types of input raw materials (this is where the bulk of new sales recruits have been deployed)’</li></ul>&nbsp;4. Try to invalidate the hypothesis by counter-examples or any other variable impacting the outcome<ul><li>Switching to other raw materials is irrelevant, as this is true only for a few customers</li><li>Import of input raw material has increased impacting the client’s customer base</li><li>Overall consumption of the input raw material still remains steady at 7% for ‘irregular’ customers as well</li></ul>5. If the hypothesis still stands, check for effects of the hypothesis being true, if not build a new hypothesis.The hypothesis does not stand the test of invalidation in this case. A new hypothesis is needed!<ul><li>New Hypothesis: Imported raw material is cheaper and of similar quality, allowing ‘irregular’ customers to switch</li><li>Check for effects if the new hypothesis is true:<ul><li>‘Irregular’ customers are still consuming the same raw material at a steady pace</li><li>Switched customers are fine with increased lead time of supply from imports</li><li>Sales team’s efforts are rejected primarily on the price differential</li><li>If the sales teamis able to match price with imports, some of the irregular customers switch back</li></ul></li></ul>6. Build a direction of solutionThe new hypothesis stands validated. It is now time to brainstorm solutions.a. Since competing on price alone is recipe for continuous margin pressure, this direction is ruled out.b. There will always be imported raw materials flooding the domestic market for regular SKUs; there is no way of stopping thisc. Customers already prefer our raw material for non-regular SKUsd. The ONLY way to sustain existing demand and grow FASTER than the market is to DEVELOP more consumption occasions – i.e. convert users of other types of raw materials! A tall ask, but its possible. Our client, therefore, started re-deploying the sales team, and the ‘bloated’ team no longer needed to be ‘right-sized’. In fact, we might need to hire more!These simple steps need rigor of analysis and can be quite challenging to implement without practice. It’s easy to find quick fixes using existing data. However, building a habit of thinking causally can have immense bottom line benefits for companies – by avoiding erroneous decision-making and by ignoring irrelevant variables. This approach allows focussing on the right problem and the right explanation for the problem. So, happy thinking!&nbsp;&nbsp;

Discover the power of causality in solving business challenges. Learn how understanding 'why' can drive effective decision-making and sustainable growth strategies.

Add your comment

Let's Empower Growth