A lot of the work our team does involves extensive use of statistics. A lot of these are from company report and accounts and can usually be trusted (although fraud and human error are risks to be aware of). However, we also come across data and particularly graphs from a number of other sources. And this is where the fun really begins. All reports we read are written for a reason and this reason tends to create a bias. A trade body might be trying to put its numbers in the best light, an investment bank analyst might be trying to prove a point – positive or negative - and an ‘independent’ research house will have a company or companies sponsoring its work. This often encourages the reader to jump to an incorrect conclusion or to give excessive credit to analysis conducted with the use of flaky assumptions.
So welcome to the chartists’ wall of shame…
The classic is to use a graph with no scale. Here’s one from Tesco a few years ago (Fig. 1).
Clearly (sic) the company was illustrating that a significant opportunity existed to trade better versus their competitors, but the chart provides the reader with absolutely no guidance whether these differences are massive or miniscule.
Sainsbury also offer some charts which are a challenge for investors to interpret. Here’s one encouraging investors to believe that the reduction in promotions has helped win market share. Without two scales the chart is meaningless (Fig. 2).
When a scale is used it can mislead. Often it does not start at zero (thus exaggerating the change). In Figure 3, Sainsbury is encouraging investors to appreciate the big lead it has in customer satisfaction over its closest competitors and how customer satisfaction itself has been growing. However, the scale, by starting at 70, significantly overstates the data. A fairer interpretation of the chart is that most people are satisfied with their shopping experience at the Big 4 and there has been, given the margin for error in this type of survey, no meaningful change in responses over the last 3 years.
Often time periods used on charts appear arbitrary. To quote a colleague, ‘is that performance since launch or lunch’. For example, to put equity valuations in context should one use the last 100 years (which would include many periods of low valuations) or use, say the last (more highly rated) 30 years?
Pie charts can also be used in manipulative ways. For example, 3D pie charts make the items closer to the reader appear larger than they are in reality compared with competing pies. Here’s a ready-made pie from those helpful folks at Wikipedia. (Fig. 4)
Sometimes headings are used to convince the reader of an attribute that is far from obvious…(Fig. 5)
…or just completely incorrect. +0% appears to be considered ‘strong’ at fertiliser company Yara…(Fig. 6)
And whilst not strictly a graph, my colleague Steve felt that the sports equipment manufacturer Asics was worthy of a call-out for its efforts to convince readers that a tick is equivalent to a success…even though sales in Americas ‘significantly decreased’. (Fig. 7)
And if all else fails perhaps it is useful to create a chart which is completely unintelligible (Fig. 8)
Perhaps, the most worrying charts are those which use data which are total fantasy. Usually one would imagine that this concern would cover forecast data but it often includes historic data too. For example, in Figure 9, in 2011 IDC, a provider of market intelligence for a variety of consumer technology markets downgraded historic data by around two thirds compared with the data it had used in 2010. (The line with blue circles was replaced with the line in red triangles).
What lessons can we draw from this? Graphs are often created to prove a biased conclusion using data which has been manipulated, incorrectly calculated or simply made up. Handle with care.