The Threat of Fake Academic Research

Interesting analysis of the possibility, feasibility, and efficacy of deliberately fake scientific research, something I had previously speculated about.

Posted on August 27, 2019 at 5:14 AM • 27 Comments

Comments

WinterAugust 27, 2019 6:18 AM

We have good examples of successful fake science campaigns:
- Creationism

- Tobacco industry delaying confirmation of the smoking-cancer link by decades

- Climate Change "skeptics"

and many more, see Merchants of doubt
https://www.theguardian.com/books/2010/aug/08/merchants-of-doubt-oreskes-conway

https://www.theguardian.com/environment/planet-oz/2015/mar/05/doubt-over-climate-science-is-a-product-with-an-industry-behind-it

This is actually a genuine US industry geared against anything that might impede commerce in the name of public health or environmental protection.

Science as an activity has implemented many safeguards against manipulation. Anyone with contacts to the tobacco industry is a scientific pariah by now. Big farma is suspected, results have to be double checked by now. Creationists are simply ignored, as they do not even pretend to do science. Anti-Vaxxers and Homeopaths have to come up with very, very solid data before anyone even wants to look. CC Skeptics have yet to come up with any data, credible or not.

Sadly, common media outlets let themselves be duped time and again. But they do learn.

IvanAugust 27, 2019 6:33 AM

We have good examples of successful fake political campaigns: Winter (above). 100% Demshevik koolaid.

Petre Peter August 27, 2019 8:02 AM

Fake news relies on 'research shows', 'experts say' as a way to boost confidence even though research and expert are difficult to verify. Fake research tries to become real news by relying on the fact that it doesn't matter what the truth is--what matters is if we believe it.

WinterAugust 27, 2019 8:14 AM

"Like the former Dr, Andrew Wakefield and his report on vaccines causing autism?"

Indeed, that was a very successful fake science attack.

The drop in vaccination coverage since his publication is sizeable, "For several years, however, global coverage with the first dose of measles vaccine has stalled at 85 percent."
https://www.who.int/immunization/newsroom/measles-data-2019/en/

US Measles Cases in Just 5 Months of 2019 Have Exceeded Any Full Year Since 1992
https://www.sciencealert.com/us-measles-cases-in-first-5-months-of-2019-most-since-1992

In 2017 alone, 110,000 people died of measles (globally).

WinterAugust 27, 2019 8:15 AM

@Ivan
"We have good examples of successful fake political campaigns"

How can a political campaign be "fake"?

RenatoAugust 27, 2019 9:53 AM

@Winter "Anti-Vaxxers and Homeopaths have to come up with very, very solid data".

Heh, you almost got me, there. :D

parabarbarianAugust 27, 2019 10:05 AM

Skepticism is an important part of keeping science honest. Real science -- as opposed to the "consensus" variant -- requires open access to data and, when necessary, source code to see if the results are reproducible and if the statistical models are biased or not. So, a first step in fighting this kind of crap is to ask to see the data and the source code. If the alleged "scientist" refuses (or sues you) it's a good bet it's bull**** science.

David LeppikAugust 27, 2019 11:22 AM

As described here, I see this as akin to hardware/firmware attacks. It could be a powerful technique, but difficult and expensive, and not worth pursuing when disinformation is so much easier to spread other ways. What made the autism/vaccines bad science dangerous was not that the scientific community didn't catch it (it did) but that it was done along with a powerful public campaign.

That said, this is not a theoretical problem, nor a new one. In the 1960s, the sugar industry paid scientists to blame fat for the risks of high-calorie diets. This altered the course of nutrition advice for decades.

More pernicious is the way that funding has always directed science to ultimately put profits before the common good, while remaining just hands-off enough to not kill the golden goose. This was the case in the "Age of Exploration" where finding new lands and natural wonders went hand-in-hand with bringing gold to European royalty, and it's the case today where science funding goes largely toward discovering expensive drugs to treat diseases, with little or no interest in cheap or simple techniques to cure diseases. (Except now for the trend toward "cost effective" personalized drugs that can be priced to be slightly less than a lifetime supply of current medications.)

Scientists are trained to be skeptical, and are surrounded by threats to their integrity, so I'm not concerned about anti-science sleeper agents as something beyond run of the mill frauds. The real threats are to science funding and science reporting, which are softer and more lucrative targets.

WilliamAugust 27, 2019 11:57 AM

In many ways, this is the defunding of education and the resultant decrease in literacy throughout our culture coming home to roost.

Sergey BabkinAugust 27, 2019 12:22 PM

Nothing really new here. The same Russia supports the anti-fracking research/"research" in an attempt to remove competition for its oil and gas business. And the "global warming" alarmism is the same kind of fake research, on a grand scale.

Ross SniderAugust 27, 2019 3:05 PM

@Thunderbird

It describes an information campaign from the Soviet Union, which no longer exists, not Russia. That's not pedantry.

Certainly intelligence communities target information attacks by aligning them bodies of authority (science is one example), and the KGB, CIA, FSB, M6, Mossad, whoever have all participated in this. I bet there are some examples of Russia either fabricating or encouraging studies that benefit its interests.

The new thing (at least in Western intelligence) - I don't know much about the current Near East - is to encourage/discourage and influence the voices of "influencers": for example religious authorities in Middle East whose interests somewhat align with Western intelligence objectives or panelists and journalists whose perspective reinforces domestic perception of Western legitimacy and justness.

Anyhow, the best recent examples of information attacks I can attribute are from corporate sources - the oil industry spreading disinformation about global warming for example. I suspect, but can't prove, that much of the growing public distrust (Western world again) of science has its roots in these attacks and abuses of authority.

ThunderbirdAugust 27, 2019 4:03 PM

Mr. Snider, Russia/USSR--potato/potahto. Okay, Russia is run by an ex-KGB guy so it seems reasonable to me to have tagged it thus, but I understand your objection. I would, however, call it a *little* pedantic.

Not sure what happened to my post. Guess it was nuked for some reason. Maybe one of those "computer bugs" I've heard about.

Alyer Babtu August 27, 2019 4:08 PM

@parabarbarian

requires open access to data

And that applies to climate science data, and code, especially, given the intensely topical character, and that it’s ultimately tax money at work.

In a more fundamental way, modern science from Bacon/Descartes on is closed, namely to truth. The original (and continuing) primary focus was to obtain material benefits, hence a demand for control and practical mastery, not understanding. The path of questioning since then is only incidentally scientific. Practical quantitative models are the focus. Such models are formally indifferent to real understanding of nature.

And the funny thing is it’s a very inefficient appoach even for its own aims. The models are inevitably only comprehensible in their mathematical expression, and quantities that arise from the mathematics are taken as real for the model, whether they truly exist or not. This guarantees the modeller believes in a lot of fictions. Then one day some actuality accidentally intervenes and the model is changed, and scientific revolution is declared. It’s a joke. Sleepwalkers, sleepwalking backwards.

The modeling habit does nothing to make its practioners and society intellectually acute, so it’s little wonder we are pathologically susceptible to phoniness.

meAugust 27, 2019 4:29 PM

MULTIPLEX EVERYTHING

I am delayed in reading several articles of this site.
I am responding only to the summary thesis statement.

Nonetheless, propaganda is certainly nothing at all new. And pseudoscience never went away as far as I've been familiar with.

It's a slight relief for this topic to be at hand, however. Thanks for bringing this topic up.

I find that there are more highly+visible ersatz elements to modern society than originals.

"Be a false negative of yourselves." --> me

Ross SniderAugust 27, 2019 10:40 PM

@Thunderbird

Russia is not the USSR. *Shrug* Dunno what else we can say about it. I guess let's end the conversation there.

I think your message was taken down by moderators (in addition to my first message, looks the the original exchange is gone). I don't know about you but the majority of my messages are taken off the moderated internet (Reddit, Facebook, etc) nowadays.

WinterAugust 28, 2019 1:48 AM

@parabarbarian
"requires open access to data"

If you had published in a scientific journal in the past decade you would have noticed that access to the data actually is a requirement for publication nowadays.

For instance, climate data are found here:
http://www.realclimate.org/index.php/data-sources/

Most "skeptics", from creationists and homeopaths to anti-vaxxers and CC deniers are basically opposed to science that is not ideologically driven to give the "right", desired, answers.

In this sense, this goes beyond "Fake science" and tries to destroy science by denying that science is even possible. All facts are labelled ideology and all scientists are considered propagandists.

For examples, see comments on this blog post.

Clive RobinsonAugust 28, 2019 6:19 AM

@ Bruce,

The trend towards "Fraud in the Journal Industry" has been going on for years, and if you look back on this blog you will find I've been warning about it for some time now.

Part of the problem is that the journals themselves quite deliberatly encorage the fraud by their business practices.

As for the article I'm very surprised by the lack of actual research, or ignoring of some important facts. For instance in India several university courses had the stipulation that to receive the qualification they had to have had one or two articles published in journals...

The result a whole industry of faux journals, faux conferances and large kickbacks/profits to those academics and administrators who put the stipulation for published papers in.

For the student the cost of getting a paper published and attending the conferances is compared to the average income inordinately high, thus represents a significant barrier to those not already of "privilege".

Not that this "milk the student" policy is new[1]. When you look at "Professional Societies" in regulated domains such as Law and Medicine you will find events the student has to not just attend but pay for as well as considerably higher fees to be a Society member.

Such behaviours are at best parasitic, and this encorages criminality of all forms.

For instance as I've mentioned before there are certain publishing houses of scientific books. They actively pursue academics in Universiries to write books for them. However the books are priced in the high hundreds and are almost never advertised or made available via general publishing outlets. They are pushed at University libraries only.

Thus when you look at these publishers and their business model it is to make the books as cheaply as possible, and sell at the highest price possible and thus take a large slice of the research and funding budgets from Universities.

With such levels of questionable if not criminal activity going on around scientific publishing, I would be very surprised if manipulating the process for geopolitical reasons had not occurred to many in various government entities.

But actually after a little thought it becomes clear that actually manipulating scientific publishing for geopolitical reasons is slow and ineffective and there are much better ways to go about it.

Part of the reason is the asymmetric relationship between science and it's declaimers. Science looks for broad evidence on which to make a limited claim, once this is accepted then more specific yet still broad evidence is used. At each step things are tested in many ways. Science declaimers however actively seek out things that arr very very specific and then claim they are contradictory to the broad evidence based claims of science. Usually they move on like butterflys fluttering from point to point not settling on any one argument for very long. The reason is in general where the declaimers have sufficient visability one or more scientists will take the time to disprove the declaimers claims. Such things take time, which is what the declaimers rely upon so that they flit from argument to argument staying ahead of science disproving their false claims.

But even science has it's own way of attacking ideas it does not like which is "literature reviews"[2]. This is basically somebody gathering up their chosen collectiin of old papers and statisticaly analysing their results then claiming not unexpectedly that the literature does not support the idea they are attacking. These "reviews" can be manipulated in so many ways they can be used to show anything you want. The important thing to note is that the papers are "old" and almost certainly do not actually have the same or often even similar research goals, thus paper selection and the statistical methods used in both the papers and the review are very critical to the outcome of a literature review. But due to the fact such reviews take time, literature reviews are not the tool of choice for declaimers.

History shows that the way many corporate declaimers work is to set up well funded institutions with laudable sounding names, that fund research. What is not obvious even to many that they select those who receive their grants based on the way the researchers previous work has supported or not the aims of the institutions funders. Three well known offenders in this area have already been mentioned above by others.

The next step up from such institutes are the "think tanks" many of which have clear political aims, but are in essence run by declaimers and thus produce biased often very biased reports[3]. Then above those are various preasure groups that have political influance.

As has been seen from recent investigations, preasure groups can be infiltrated by "Agents of a foreign power". This process has been made ridiculously easy by rules on the anonymity of funding for various types of donations. When you look behind who lobbied for such anonymity you again find certain types of people, who shall we say you would not invite into your home... So they find other ways in via News and Media, and by distancing themselves via cut outs that such anonymous donation systems allow.

Perhaps one of the worst tricks in the book is legislation for "Balanced Reporting". In the UK for instance a political party forced legislation onto the British Broadcasting Corporation (BBC) where by they had to provide supposed balance in news and similar reporting. Unfortunatly what this did was give an unwarranted platform for declaimers and their pet arguments. Thus people get two viewpoints on a subject they know little or nothing about, thus it becomes a question not of truth or false but who sounds most credible. This unfortunatly gives an advantage to those that favour "excess today, with extinction tommorow" behaviours.

Whilst geopolitical interests have an interest in manipulating science, it's rarely without an existing extensive network of declaimers already in place, that can be "nudged" or "assisted". Such networks exist from either corporate sponsoring[4] or minority view points.

In the case of minority view points, those who have them will not usually respond to rational argument or debate, because to them it is their belief system. Thus they respond favourably only to those who confirm their beliefs. Which makes them fairly easy to manipulate.

People that are easy to manipulate, are the basic prey stock for all sorts of criminals. One worrying factor is that this has made the extreamist behaviours seen between the two world wars resurface and become rather more prevelant today. Which is increasing political instability, which in turn makes for more belief polarisation. Thus forms a degenerative spiral, which provides significant opportunities to those that wish to manipulate the issue to their benifit.

[1] It's one of the reasons I ceased to be a member of two Engineering Societies when they started pushing for "Professional recognition under law". Another was the Societies increasing bias towards politics and large businesses we now know had corrupt practices including bribary and tax evasion.

[2] Literature Reviews have a proper place in science, in that they can be used to find where research has not been done, changes over time, etc etc. Whilst they appear to be very blunt instruments they can if used properly provide laser like highlighting of defects in testing methodology that have skewed previous research results.

[3] Such organisations can usually be spotted by the fact the reports were written by now obviously politically oriented individuals. For instance look at the works of Boloton and Barr, and find the commonality of the think tanks.

[4] Corporate manipulation of "false science" can be easily seen by following the trail of "Pure White and Deadly" which started with Dr Ancel Keys[5] and to this day is still killing millions of people a year in the West by inflicting various diseases on them. Worse others deliberatly manipulate the secondary and tertiary markets that arise from these diseases. Western Governments with very short term thinking actually encorage these behaviours for various reasons.

[5] Dr Ancel Keys is known as the inventor of the US Military K-Ration. Which unfortunatly during WWII became a "staple ration". Even though it was known at the time due to historic and more exacting research that the composition of the K-Ration would lead to your death due to starvation even though it had a surfeit of calories. In more recent times Dr Keys research findings have been found not to agree with his actual research data and that his later behaviours showed that he was probably very well aware of it.

WinterAugust 28, 2019 9:02 AM

@me
"I think you will find that carbon taxes are a well studied way of attempting to internalize the externality of pollution."

Indeed, but that is utterly independent of the science behind CC. If you do not like carbon taxes, then you can use something else. That does nothing to the underlying science on CC.

But what I see most often happening is that people do not like certain policies and then attack the scientists instead of the policies.

WinterAugust 28, 2019 10:05 AM

@MarkH
" I will note in passing that the "counter-argument" cited by RealFakeNews has been thoroughly disposed of for a long time:"

I can only cite Clive Robinson here:


Science looks for broad evidence on which to make a limited claim, once this is accepted then more specific yet still broad evidence is used. At each step things are tested in many ways. Science declaimers however actively seek out things that arr very very specific and then claim they are contradictory to the broad evidence based claims of science. Usually they move on like butterflys fluttering from point to point not settling on any one argument for very long. The reason is in general where the declaimers have sufficient visability one or more scientists will take the time to disprove the declaimers claims. Such things take time, which is what the declaimers rely upon so that they flit from argument to argument staying ahead of science disproving their false claims.

The persistence of such disproved claims is not an accident. These claims are there to confuse the unwary and to sap the time of those who actually know the science behind it. They are repeated by people who do not check their sources, or do not care whether they are true or not.

Clive RobinsonAugust 28, 2019 11:40 AM

@ Winter,

That is not up to date anymore. The current practice is systematic reviews

They are the same thing or they should be... and the fact they had to be renamed is where the problem became to obvious.

In essence it is an acknowledgment of the failure of people to carry out literature reviews in an objective fashion.

As I noted there were two main issues with such reviews the first is the way the old papers are selected, the second the statistical methods used to analyze the results of the selected old papers.

The idea of "Systemic Reviews" was mainly to tighten up the paper selection process. However even though there might be a stated plan/methodology it might well be massaged/falsified to obtain the desired results.

The reason is the more you limit a data set in general the more likelyhood there is that two apparently independent selection methods of candidates from a set are to produce coincidental or near coincidental results (there a bunch of mathmatics behind it but it's fairly intuative to understand it).

Thus to cheat the system, you first find all those papers that support your argument, those that don't, and those that do neither as graded sets. Your first step is to find an apparently innocuous way to eliminate the neutral set but ensure it also takes away more of the anti rather than pro set. Often the selection of something basic like journals or date ranges will do this. This is akin to the statistical trick of changing your sampling rate and start point to those that most favours your argument. You then go on and look for other ways to change the papers selected, these should likewise appear inocuous but do the hidded desired selection process.

Back in the 1980's I was peripherally involved with the design of tools that would test such rules. The Managing Director of the company discovered by chance that there was a viable rule linking the phase of the moon to the gold price. Thus by using a four week sample period (not unusual) and an apparently random starting point you could align your sampling without it being obvious over relatively short time periods.

The point is there are all sorts of tools available to find ways of adjusting the sizes of the selection sets such that you can hide your intent and achive it. It would not be that dificult to automate the process either. As some one I know quite dryly observed many years ago,

    There will always be some part of Pi that will match your desired selection process. The hard part is finding the starting point.

Obviously similar tricks apply with the statistical analysis stage as well.

The moral is "Where there is the freedom to chose, there is the opportunity to cheat".

WinterAugust 29, 2019 3:54 AM

@Clive
"The idea of "Systemic Reviews" was mainly to tighten up the paper selection process. However even though there might be a stated plan/methodology it might well be massaged/falsified to obtain the desired results."

This holds for statistics and data collection. The solution chosen by science is to make all "choices" made in the research transparent.

Yes, statistics can be massaged, but the rules are such that you can see when this has been done. The same for a systematic reviews. As the search terms and procedures have to be spelt out, it is easy to see where "lucky", and suspicious, choices have been made.

It is clear that any human activity can be corrupted by fraud. The solution is to make actions transparent and visible and to allow peers to review what was done.

Mike D.August 29, 2019 12:57 PM

"Usually they move on like butterflys fluttering from point to point not settling on any one argument for very long."

More formally known as the Gish Gallop.

RealFakeNewsAugust 30, 2019 4:47 AM

It seems my post has been removed. @Moderator @Bruce I wasn't aware it broke any blog rules???

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Sidebar photo of Bruce Schneier by Joe MacInnis.