How Dashboards can mislead

Read an interesting article from John Shapiro professor at Northwestern Kellog on how dashboards can mislead executives and I cannot agree more. To be honest, I love visualization of data and have pushed my data architects and report writers to give me snapshots of various measures but how often the rich data didn't mean anything as it did not align with organizational goals. Even more, what information is important to me is not necessarily relevant to other executives in the organization.  Data analytics visualized on dashboards typically describe existing measures on past phenomena, some better ones predict future events and past data and the best one prescribe a course of corrective or strategic actions.

Shapiro talks about three types of traps executives can fall for:

1. The Context Trap:  We equate empirical data to the objective. I have blatantly used the cliche "numbers don't lie." But this belief can be dangerous because we can track wrong measures or metrics and make wrong conclusions. An example in the article the author shares was when tracking sales leads or touchpoints without correlating data points that show actual and expected we are unable to make intelligent contextual decisions.

2. The Causality Trap: When data is displayed in groupings on a dashboard managers interpret it as causative when there may not be one. A sound knowledge of statistics is needed for critical thinkers who are managers and executives to not equate correlation to causality. Statistical tests calculate correlations between variables and there are tests like Granger causality or convergent cross mapping techniques that verify the hypothesis of causal relationships in correlational data.

3. The Importance Trap:  Dashboards are built on assumptions made by data or IT architects, or sometimes using deployed dashboard software that has pre-defined measures. This does not necessarily mean that the data views of the dashboard align with the goals and the business model of the companies. Hence context of people involved in frontlines, articulation, and alignment of business strategy are important inputs for defining the lead and lag measures.

Thoughts,

Sam Kurien