Lost in Translation: Media Data Representation
Creating research campaigns for an array of different clients and activities, can often be an exercise in painting in shades of grey. Not so many ‘yes’ and ‘no’ choices, more open-ended wording to tailor and generate thought-provoking answers.
However, that doesn’t mean there’s a clear line in the sand whereby the monochromatic scale of consciousness becomes distinctly binary.
There’s a commonly recognised thesis to the stuff you read in newspapers and online; it’s been there for quite some time and involves pinches of salt that vary in amount depending on your choice of outlet.
This past Monday, just nine days removed from the Paris attacks that claimed over 130 lives and with a lot of the western world still on high security alerts, The Sun took the decision to run with the headline ‘1 in 5 Brit Muslims’ sympathy for jihadis’. You can see the front page in question (in all its ‘world issues juxtaposed with reality star shower pics’ glory) HERE.
Since its publication, the headline has drawn a record number of complaints to the Independent Press Standards Organisation (IPSO), with a total of 450 recorded within roughly the first 24 hours.
In fact, even the company that initially created the survey, Survation, has distanced itself from the headline, saying they neither ‘support nor endorse’ the story, nor any of the intentions, which were built with their work.
The exact key sentence in Survation’s riposte is thus: “Survation does not support or endorse the way in which this poll’s findings have been interpreted.”
Integrally, the statement, attributed to chief executive Damien Lyons Lowe, says that they do not deny the survey’s findings, they disagree with the publicised representation of their data.
It’s worth noting that the poll surveyed both British Muslim and non-Muslim opinions, The Sun chose to run with just the one strand of questioning. They go on to say that their preferred method of representation is ‘in the proper context alongside a comparable sample of non-Muslims’.
Which, when you bring it down to brass tacks, is as straightforward as it gets – clarity and transparency. Yet, we live in a world where people are just as interested in looking behind the curtain as they are in looking at the stage show itself.
Which is why, when Sky News published their own set of findings a few months ago regarding the same subject, they showed that 30% of non-Muslim Brits had some degree of sympathy for young Muslims fighting in Syria and the whole story went largely noticed.
One of the most important things to bear in mind when analysing these results, and all polls of its ilk, is the interpretation of a specific word in questioning. In this context, it’s the word ‘sympathy’.
‘Sympathy’, with this story, could as easily be interpreted as The Sun is seemingly intending – that the ‘1 in 5’ are behind the same cause as the ‘jihadis’ and therefore are a danger to public safety. Or, in a slightly more humanistic approach, that they are sympathetic towards the very idea of being sent from the UK to Syria, or similar, in the first place.
It could also mean an array of things in-between those two, but it boils down to personal consumer interpretation and, as we all know by now, you can’t keep everyone happy all the time.
“If a question is vague then it shouldn’t be used,” says Tim Furdui, our Senior Research Analyst, “using vague questions is one of the most common mistakes in surveying. All surveys should be based in objectivity and must provide the respondent with all possible outcomes. Failing to do so will invalidate the research and make the results completely unbelievable.”
The Sun’s biggest flaw in this whole affair seems to be assuming people won’t question their results, or at least that they wouldn’t come under such scrutiny. To be objective for a second, it wouldn’t be too far away from The Sun’s form-book to suggest they were being purposefully ‘eye-catching’, shall we say.
“If we were to ask the question ‘Do you drink a lot?’” Tim says, “’a lot’ is vague and could possibly mean two to three units for one person and 10-20 for someone else.”
Regardless of intent, the most worrying aspect of data misrepresentation remains. Whether those behind the headline will admit it or not, the basic message on their front page is a derogatory and provocative statement built on (what we now know to be) morally questionable application of the truth.
Bearing in mind that certain aspects of the question line are open to interpretation, it becomes abundantly clear how easily a false account of a near-truth can be published on the front of a national newspaper.
By Stuart Buchanan