Study Asks, How Deceptive are Deceptive Visualizations?

Data visualization is a powerful communication tool to support arguments with numbers in a way that is accessible and engaging. Professionals in a variety of fields are using data to shape informative, convincing narratives supported by visualizations.

It’s safe to say that overall the popularity of data visualization has skyrocketed over the past decade thanks in part to user-friendly software, like Infogram. More people than ever before are making their own charts and infographics, which is presenting a unique problem. Despite the availability of some great charting resources, we are witnessing an influx of poorly-designed misleading or downright deceptive data visualizations.

This is why New York University’s School of Law and School of Engineering teamed up to explore the concept of deception in the context of data visualization and popular distortion techniques.

What Makes a Visualization Deceptive?

Deceptive Visualization: “a graphical depiction of information, designed with or without an intent to deceive, that may create a belief about the message and/or its components, which varies from the actual message.”

What do deceptive visualizations look like? Below you will find some examples collected from the media involving notorious distortions, such as manipulation of axis orientation/scale (a,c), use of disproportionate sizes (f), incorrect representation (d) and non-linear scales (b,e).

The Experiment

In the study, researchers selected a set of common misrepresentation techniques from the classes they identified as frequent and created deceptive and non-deceptive versions of the same charts. Then they ran a crowdsourced user study to identify the deceptiveness of those visualizations.

The results show that deceptive charts have a major impact on how people interpret a message. The four distortion techniques they tested are listed below.

Distortion Techniques

Truncated Axis (Message Exaggeration)

‘In the truncated axis visual distortion, one or more of the axes of a chart are altered by changing the minimum and maximum values presented on the scale. Such alteration of the axis range leads to exaggeration or understatement of the quantities presented, thus directly affecting the user’s response to the “how much” type of questions.’

Area as Quantity (Message Exaggeration)

‘Encoding quantitative data with size has faced serious criticisms in the visualization community and is a process that requires careful mapping of data with graphics. Although no guidelines are available about how to map the actual data with graphical area, it is believed that a one-to-one mapping between the data and the graphical area is least prone to distortion.’

Aspect Ratio (Message Exaggeration)

‘This type of distortion primarily affects line charts as it directly impacts the rate of increase or decrease of one quantity over another. While one may argue that aspect ratio may impact other visualizations such as bar charts, they apply this distortion to only line charts as they appear more frequently.’

Inverted Axis (Message Reversal)

‘Human beings relate direction with trends. This directional interpretation makes inverted axis one the most common distortion techniques that leads to reversal of the message and makes the users susceptible to drawing false conclusions. In other words, here deception occurs due to reversal of the message instead of exaggeration or understatement.’

Results

The research team recruited 330 unique participants to take part in the final user study. They studied two types of deception: Message Exaggeration and Message Reversal by applying relevant distortion techniques to commonly used visualizations and asking “How much” and “What” types of questions.

The results confirm that these techniques do lead to major misinterpretation from the reader’s side and that the effects are also rather large. Click here to download the study and explore the results for yourself!

This study was conducted by Anshul Vikram Pandey, Oded Nov, and Enrico Bertini from NYU’s Polytechnic School of Engineering, along with Katharina Rall and Margaret L. Satterthwaite from NYU’s School of Law.

Now that you know what deceptive visualizations are and how much of an impact they can have – it’s time to craft a proper visualization of your own. Download our free eBook for simple data visualization techniques to make your charts 110% better.