Were 88% of Conservative ads ‘misleading’?

First Draft News claimed 88% of Conservative ads were ‘misleading’, compared to 7% for Labour.

During the latest General Election campaign, political parties and campaigners often had their claims examined by independent fact-checkers.

An organisation called First Draft News claimed that for Conservative party advertisements on Facebook: “nearly 90% of the ads posted in the first days of December push figures already challenged by Full Fact”.

Their article initially claimed that no Labour ads had been similarly challenged. This summary was widely shared, including by the BBC, ITV, the Independent, and on social media. My article will examine this claim.

Image for post
This image was shared by Richard Burgon MP, currently the Shadow Justice Secretary. (Image: Facebook)

First Draft News method

This is the process that First Draft News undertook to make their calculation:

  1. Draw data from the Facebook Ads Library API on ads posted by the main accounts of major political parties, between 1st and 4th December 2019;
  2. Look at the caption, image and link of each Facebook advert;
  3. Check any claims made in these ads against those independent fact-checker Full Fact had already published;
  4. Counting each duplicate and variation of each advertisement separately, calculate what proportion of ads contained a falsifiable claim which had been challenged by Full Fact.

First Draft News had originally claimed — under this method — that 88% of Conservatives ads were ‘misleading’, versus none for Labour. Their article has since been amended the latter figure to 7%, highlighting some adverts were missing within the data-file:

seven Labour Party ads from December 1 to 4 link to webpages which contain figures Full Fact says are “inaccurate” or “need more context”.

Methodological issues

There are numerous methodological problems with this calculation.

The time period is specifically chosen because it is not representative of the whole campaign, following an increase in Conservative ad expenditure:

With 12 days until polling day, the Conservative Party massively stepped up its ad campaign on Facebook.

First Draft News only look at advertisements which started during that four-day period, not the ads that were live at that time. As an example, 48 versions of the same ‘fair tax’ advert from Labour are not included:

Image for post
This ad was seen on Facebook and Instagram. (Image: Facebook Ad Library)

This ad was posted on 28th November and was active until 5th December. Full Fact had, as part of its check on the Labour manifesto, stated:

But that doesn’t mean Labour plans no tax rises whatsoever for these people, because there’s more to tax than just VAT, income tax and National Insurance.

Their article then highlighted various tax increases which will affect people earning less than £80,000 a year.

There is the additional problem in assuming Full Fact — or any other fact-checker — can have total oversight of all political advertising. A false or misleading claim may appear, but go unchecked.

The categorisation method heavily relies on ‘misleading’ claims being contained within linked articles:

At least 54% (3,646) of the total ads served link to a webpage carrying the misleading claims.

This is concerning, since click-through rates on Facebook advertisements are generally low, averaging less than 1% in the United States.

Image for post
Legal ads and retail industries had the highest click-through rates. (Image: Wordstream)

Finally, First Draft News counts every single instance of an advertisement separately. The Conservatives posted around 2,500 duplicates of the same advert — a video — with different target audiences, all posted between 1st and 4th December:

Image for post
ORDER! (Image: Facebook Ads Library)

To see the issue with this calculation method, imagine if there are two parties:

  • first party: posted nine duplicate ‘misleading’ adverts with very small audiences, plus one other ad which was not labelled as ‘misleading’;
  • second party: posted one ‘misleading’ advert which it pushed widely, and nine non-misleading smaller ads.

The first party would have 90% of its adverts labelled ‘misleading’, and the second party scored at 10%. This is despite the second party having a wider audience for its ‘misleading’ claims.

Due to the calculation method, this one advert was about 37% of the Conservative ads that First Draft News looked at. The advert itself contains no falsifiable claims, urging people to vote Conservative to “end the chaos and uncertainty”.

First Draft News labels this ad as ‘misleading’ because the linked article (a Conservative campaign landing page) contains a ten minute video just above the bottom of the page. That embedded YouTube video is mostly of a speech by the Prime Minister on 6th November: at around four and a half minutes in, Johnson mentions “40 new hospitals, by the way”.

Categorisation problems and false sourcing

Whilst First Draft News uses the label ‘misleading’, Full Fact may not say the same. As a fact-checking organisation, they do not rate the veracity of claims. For instance, Full Fact did not state the Conservative manifesto pledge of “50,000 more nurses” was “misleading”. Their manifesto fact-check stated:

The Conservative manifesto promises 50,000 more nurses, although it doesn’t say when it aims to deliver this by or whether they’ll all be full-time.

This pledge needed clarification, which has since been provided: the Conservatives are intending on increasing (in full-time equivalent terms) the number of nurses in NHS England from around 280,000 to 330,000, by 2025. Describing this pledge as ‘misleading’ does not appear justified.

The categorisation of “40 more hospitals” rests on whether a long-term pledge (exceeding the lifetime of the Parliament) is inherently misleading. Full Fact do not use this term in their articles.

The Conservatives exaggerated their own policies and misrepresented their opposition in their Facebook ads. One Conservative video advert was on the ‘cost of Corbyn’ — valued at “an additional £2,400 bill” — which had over 600 duplicates in the Facebook ad library.

Image for post
This is an inaccurate evaluation of Labour policy. (Image: Facebook)

Full Fact stated: “problems with the calculation make this claim largely meaningless.”

When this pair of figures was shared on social media, the claim was sometimes attributed to Full Fact itself. This is false sourcing. Full Fact have published their own article about party election ads. On the suggestion ‘Full Fact have claimed 0% of Labour ads were misleading’, Abbas Panjwani writes:

That’s not true: Full Fact wouldn’t make such a statement, and Labour definitely has released ads that contain claims we’ve disputed.

Distinct approaches

The two main parties had distinct approaches to Facebook election ads.

Image for post

As Panjwani (Full Fact) has summarised, the Conservatives — beyond the slogan of ‘Get Brexit Done’ — often exaggerated their own platform, and misrepresented their opposition. Their advertisement strategy focused on many duplicates and variants, for analytical purposes.

In contrast, Labour spent larger sums on individual adverts, making more general attacks, pushing its own pledges and getting out the vote — whilst making some claims that are false or likely to be misleading.

Claiming 88% of Conservative ads were misleading versus 0% for Labour requires correction (to 7% for Labour) and severe qualification:

  • it looks at ads posted — not active — during a specific period, and was not intended to be a representative record of the election campaign;
  • the calculation method counts every instance of each advert, including all duplicates and variants, separately;
  • it is based on categorisation by First Draft News, heavily dependent on contested claims appearing in advert links — which readers may not see;
  • it wrongly suggests Full Fact has considered and declared every claim as “misleading”, when the fact-checker has not.

These figures only consider Facebook paid advertisements, rather than misinformation pushed by parties through ads on other digital platforms, posted on Facebook without ad boosts, or non-digital means.

There is not a single objective measure of misinformation: such statistics are dependent on methodological choices and subjective judgements.

Georgina Lee (Channel 4 Fact Check) and Christopher Snowdon have also written about this matter.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store