For months, SABC’s Interface programme called on viewers to vote in an SMS poll. The results were then delivered by the presenter during the show, in percentages — like “40% agree with the statement, and 60% disagree”.
Whether a particular percentage was of a universe of 10 SMS messages — or 10 000, we weren’t told. Of course, it makes quite a difference as to whether the tallies represented four “yes” SMSs, and six “no’s”, versus a total of 4 000 and 6 000 votes.
Worse still, the results were also presented as representing those viewers who did not send in SMSs. So you would squirm on your sofa as you heard the announcer blithely advise that “40% of our viewers agree … ”. That’s despite the fact that even 10 000 SMS votes would still not necessarily be representative of the show’s total audience.
Finally, we never heard whether the SABC disqualified more than one SMS coming from the same number — in other words, the extent to which possible vote-rigging was rooted out.
That particular abuse of statistics has now been corrected. But Business Day delivered equally badly on reporting statistics last week. That paper’s readers were met with a story-brief headlined: “Whites, coloureds have little confidence in Zuma — survey”.
The text that followed this cue was taken directly from the top paragraphs of a Sapa story and sourced this political finding to “the latest Ipsos-Markinor survey”. The study, it continued, was released on February 26.
Now, read on, to see what’s wrong with that. It’s a problem that popped up in two other papers as well:
- On Saturday, The Citizen used the same Ipsos-Markinor survey to underpin its editorial “Most ANC voters believe JZ guilty”. That piece said that the statistics on Zuma’s guilt emerge from a “new survey”. It added that the result “flies in the face of the prevailing perception” that the party is 100% for JZ.
- If you turn to Friday’s Sowetan, you get the same study being reported under the headline “Confidence in Zuma low — survey”. It’s the same Sapa story as Business Day, but in more length. Five paragraphs into it, you discover — finally — that the survey was actually conducted way back in October last year.
At least the same piece also includes the information that the research sample was 3 500 people, which helps put additional perspective on the findings. But you wouldn’t know either the date of the survey or its sample size from skimming the headlines of the three papers.
If a day can be a lifetime in politics, then old survey findings — predating so many recent developments — can be expected to contrast with what The Citizen calls the “prevailing perception”.
The basic problem is presenting the Ipsos-Markinor research findings as if they have current resonance. Interesting, if you examine Friday’s Daily Sun, you find the survey’s status reported accurately. This tabloid paper has the headline: “Mbeki’s shadow over the ANC. In this poll, he was still the man voters liked!”
Note the phrase “was still”, which points to the fact that the report is about history, not the present. And right up front, in the second paragraph, the Sun’s writer tells us that the information dates to October. His fifth paragraph explicitly cautions: “ … the nationwide research was carried out five months ago when the situation was a lot different to today”. When the country’s biggest tabloid reports statistics properly, it makes you wonder what makes for misleading reporting in the “serious” press.
My guess is that it’s incompetence. But many people would say the mainstream media is quick to make a meal, even when quarter-baked, of statistics that suggest problems for Zuma’s leadership. For them, journalists have a political agenda against the new ANC.
Perhaps wish-fulfilment does come into the picture — which is probably why the mainstream missed predicting the extent of the Zuma victory at Polokwane.
Ultimately, the misuse of statistics is probably caused by a combination of reasons, yet even politically partisan journalism needs to do better if it wants to carry some credibility.
A case making the point by negative example was the Sunday Times this weekend. Seven young people’s mug shots and political views were presented on page four. That paltry sample was deemed sufficient for the headline to declare: “Gauteng youth give the ANC a wide berth”.
Readers are now supposed to believe that seven people reflect “Gauteng youth”?
Business Day, The Citizen, Sowetan and the Sunday Times — four newspapers that need to pull up their statistical socks. And it’s the leading tabloid paper that avoids tabloid treatment of statistics.
The Markinor survey was conducted in October, but the press is labeling it as ‘new’ because it was only released last week. It’s not the Citizen’s fault that Markinor took three months to publish it.
Agreed about the ‘Wide Berth’ story…
How true! The same approach seems to be used even when reporting on what is supposed to be “breaking News” about a wide range of issues like crime, aids, poverty and of course Zuma. This is a clear indication of lack of quality, commitment and professionalism on the part of our media in general. The media has shown disrespect to the audience, which they are busy brain washing with information that is reported as fact yet the fact finding exercises themselves are flawed. Themba Dlomo
When SAFM conducted a survey some weeks ago – I smsed in exactly that point (how did they know that nutcases were not sending multiple calls from the same cellphone). The presenter read half way through my sms – and then fudged it!
*nods in agreement*
You would think Polokwane would have taught the “mainstream” media about avoiding that trap.
In all my life, I have never been interviewed by any of these polling companies on any of the issues they produced stats on.
But then, I also have never won the lottery.
Just unlucky? Or are polls by definition flawed?
you are correct. They are commiting the same mistake they did in Polokwane. on top of that the study does not tell us whether they asked for the membership of the ANC as they claim that those are members of the ANC.
It is depressing to read newspapers these days. i regret the day i subscribed to Sunday times.
I agree with you Prof…and journalists are not friends with stats. But how do we remedy this situation, how do we report stats in an accurate way, or shud we just stay away from stats and figures when reporting? Which is the best and accurate w2ay of reporting stats?
I despair on a weekly (sometimes daily) basis at the quality of financial / econommic journalism in even the so-called ‘serious’ papers (Business Day is a prime culprit).
I have a brother working for one of Markinor’s direct competitors, and I trust his experience and expertise when it comes to gathering and analysing primary data. Thanks to him, I have some insight into the resources needed to conduct a representative survey.
Seven people is an opinion piece, not a survey. I agree with Themba that this kind of shoddy journalism is an insult to readers.
It is no incompitence that leads the liberal mainstream media into spewing these absurdities and lies. They have deemed the new ANC leadership under Zuma unfit, and have since gone out of their way to discredit it. Everything is reported transcontextually. But what the media does not seem to realise is that it does not enjoy any legitimacy with the MAJORITY of our compatriots, thus their attepts at destroying the ANC will fail dismally. Sunday Times,702, eTV, Citizen, City Press, Star are the worst. The Mail and Guardian has sadly joined the choire of late- see lasy Friday’s mischivious headline. We hear nothing about the undemocratic nature of COPE, petronizing paternal racism of Zille, blatant racism of the FF. All we read about is the ‘corrupt and incompitent’ ANC.
But we will win these elections, because we were NOT made by the media. And when one points out the inaccuracies and lies, they tell us we are feeling sorry for ourselves!
Benzol i am with you on that one. I have never been interviewed and i feel left out
As you say, it is incompetence. Perhaps a course in elementary statistics should be a prerequisite for Rhodes’ journalism degrees?
It’s certainly not incompetence. It’s pursuing an agenda. Mondli Makhanya et al are not stupid journalists. But they will fail. ANC under Zuma will emerge victorious wheteher they like it or not. I didn’t renew my Sunday Time subscptin recently and never will. Under Mondli, Sund Times is a joke. e.g. Fist page headline that some ANC heavyweights joinning COPE, yet evry journalist knows Macozoma et al hav bin working 4 Cpe since January. Whats breaking news here e-TV, Sunday Times? Since when has Phumzile become ANC heavyweight? Media analysts/experts will never 4give ANC 4 proving them wrong all the time
I listened to Metro FM yesterday on this very issue and I was disappointed with defensive comments from some journalists. Like most of you on this forum I am yet to be interviewed on these surveys, but I don’t think that’s the issue as we have about 48m people in the country. However the “Wide berth” headline on Sunday Times takes the cake. I mean how is it possible to use 7 random people who for some reason happen to support either DA or COPE. Let alone that Park Station, EC,Chartsworth were full of supporters of ANC recently which is a total opposite to what that survey was saying. On Markinor surveys, I honestly think in most cases they are useless and are only used by certain section of the media to support their propaganda.
Advice in stats: “never publish or draw conclusions from stats in percentages if you do not have access to the absolute figures”
The reason why media in general is dangerous to the mind, is because of their claim of objectivity. i used to read the Sunday Times until i realised that their views are so lopsided, that the only people who read the their drivel are those who like their right-wing nonsense and those who read it to compalin about the rubbish.I have decided to stop spoiling my sunday mornings.
I respect people’s right to hold different views from mine, but i take offence to agenda-based reporting masquerading as objective opinion at my financial expense.
Themba
Surveys must be paid for – who commissioned this one from Markinov? I doubt that it was the media.
And the ANC is the worst at manipulating stats- they do it all the time!
We are told the Public Works Programme has created 1 million jobs – but we are not told that only 160,000 were permanent, and that the rest were temporary – at less than R80 per day!
And do you really believe the latest stats from SA Stats that unemployment has gone down, because of new employment in construction? More temporary jobs at less than R80 per day?
We reap what we sow, under Apartheid the liberal media were probably worse in their attack against the evil ruling authority and their bent on ‘making the news’ not reporting it. So why shoud they change now??
Brent
Lyndall, you are right about manipulation of stats by politicians. The examples u quoted are classical and there are many more. Agreed Lyndall. But sholdnt we expect something better frm our media, especially the so called serious ones: e TV, Sunday Times, Bus Day? I am sure they take lementary statistics at Journalism Courses/Academies, but even withou stats, this is common sense/reasoning..isn’y it, Lyndall. I think our newspaper need some serious introspection and 2 rise out of the gutter. Mondli ( he’s my homeboy & I have nothing against him) must be the worst Sunday Times editor 24 me since I started reading the paper as a 5 year old in Durban. Even racist liberals and concervatives who ran/edited the paper were more scrupulous and less naive, treated their readers wi some respect ( not this ourageous political bis at the xpense of the truth). I dont want these analysts/xperts/editors 2 agree wit me,but their wishes/desires must not be confused with political analysis.
i guess we will all remember president jacob zuma himself saying: the media is out of touch with the society it reports on.
of course, it was either ignored or dismissed as self-defence on his part just because he is the face of the anc.
however, most of what these researchers, analysts, reports and commentators have to say is just fignment of their own imagination. they are simply expressing an opinion, trying to make a name and, above, making money to keep their businesses running.
it is a sad day, indeed, that research has become so unreliable. who will guard the guardians when the guardians are fed drivel themselves?
71.25 percent of all statistics are made up on the spot…