Free Republic
Browse · Search
News/Activism
Topics · Post Article

Skip to comments.

Most scientific papers are probably wrong
newscientist.com ^ | 02:00 30 August 2005 | Kurt Kleiner

Posted on 08/31/2005 12:09:18 AM PDT by dila813

Most published scientific research papers are wrong, according to a new analysis. Assuming that the new paper is itself correct, problems with experimental and statistical methods mean that there is less than a 50% chance that the results of any randomly chosen scientific paper are true.

John Ioannidis, an epidemiologist at the University of Ioannina School of Medicine in Greece, says that small sample sizes, poor study design, researcher bias, and selective reporting and other problems combine to make most research findings false. But even large, well-designed studies are not always right, meaning that scientists and the public have to be wary of reported findings.

"We should accept that most research findings will be refuted. Some will be replicated and validated. The replication process is more important than the first discovery," Ioannidis says.

In the paper, Ioannidis does not show that any particular findings are false. Instead, he shows statistically how the many obstacles to getting research findings right combine to make most published research wrong.

Massaged conclusions

Traditionally a study is said to be "statistically significant" if the odds are only 1 in 20 that the result could be pure chance. But in a complicated field where there are many potential hypotheses to sift through - such as whether a particular gene influences a particular disease - it is easy to reach false conclusions using this standard. If you test 20 false hypotheses, one of them is likely to show up as true, on average.

Odds get even worse for studies that are too small, studies that find small effects (for example, a drug that works for only 10% of patients), or studies where the protocol and endpoints are poorly defined, allowing researchers to massage their conclusions after the fact.

Surprisingly, Ioannidis says another predictor of false findings is if a field is "hot", with many teams feeling pressure to beat the others to statistically significant findings.

But Solomon Snyder, senior editor at the Proceedings of the National Academy of Sciences, and a neuroscientist at Johns Hopkins Medical School in Baltimore, US, says most working scientists understand the limitations of published research.

"When I read the literature, I'm not reading it to find proof like a textbook. I'm reading to get ideas. So even if something is wrong with the paper, if they have the kernel of a novel idea, that's something to think about," he says.

Journal reference: Public Library of Science Medicine (DOI: 10.1371/journal.pmed.0020124)


TOPICS: Crime/Corruption
KEYWORDS: globalwarming
Navigation: use the links below to view more comments.
first 1-2021-38 next last
well well well......
1 posted on 08/31/2005 12:09:19 AM PDT by dila813
[ Post Reply | Private Reply | View Replies]

To: dila813

Bush's fault.


2 posted on 08/31/2005 12:16:49 AM PDT by kb2614 ("Speaking Truth to Power" - What idiots say when they want to sound profound!!)
[ Post Reply | Private Reply | To 1 | View Replies]

To: dila813
Assuming that the new paper is itself correct, problems with experimental and statistical methods mean that there is less than a 50% chance that the results of any randomly chosen scientific paper are true.

Perfect example of a post by someone who does not understand science.

Science is the business of putting forth a statement (theory) and inviting everyone else to shoot it down. Therefore, its not surprising there are a lot of disproved theories around.

But the odd part of this post, (and one the poster does not even seem to recognize) is that the only way a scientific paper can be proven wrong is by ... you guessed it, more scientific papers. So the denigration of scientific papers requires the very thing being denigrated.

3 posted on 08/31/2005 12:21:29 AM PDT by konaice
[ Post Reply | Private Reply | To 1 | View Replies]

To: PatrickHenry

Exaggerated ping


4 posted on 08/31/2005 12:29:20 AM PDT by AdmSmith
[ Post Reply | Private Reply | To 1 | View Replies]

To: konaice
Science is the business of putting forth a statement (theory) and inviting everyone else to shoot it down.

Try telling that to the environmentalist left, which constantly uses "scientific papers" as some sort of fact that global warming- apparently caused by us- is going to destroy us all. These people have no concept of "theory."

5 posted on 08/31/2005 12:31:04 AM PDT by SunnyD1182
[ Post Reply | Private Reply | To 3 | View Replies]

To: dila813

This is why every year you have medical studies which contradict one anothr. One paper says vitamin C can lower cancer risk, one says it can't, one says sunlight is good for you, one says it isn't.

The problem with most science studies is that they find pieces of data which seem to match but probably do not.

Kind of like the famous study which showed that bread caused crime, cancer, AIDS etc because it was found it most people systems who had commited crimes or had these diseases.


6 posted on 08/31/2005 12:33:08 AM PDT by JimDingle (Give Dingle a Jingle)
[ Post Reply | Private Reply | To 1 | View Replies]

To: dila813

Why does this finding not surprise me.

There are lies, d-mned lies and statistics.

The vast number of papers prepared depend, in some form, on the use of statistics. And the alternate name for statistics is SWAG.

Scientific wild-ass guess.

A veneer of "science" through appling mathematical formulae to otherwise unrelated and poorly quantifiable data seems to "make sense" to the researcher.

Rather like looking for "patterns" in the random placement of stars in the night sky.

The "dots" sometimes exist in so many dimensions that no honest relationship can be established.

But since the conclusion has already been made BEFORE the research was completed, the data will just have to conform.


7 posted on 08/31/2005 12:36:34 AM PDT by alloysteel ("Master of the painfully obvious.....")
[ Post Reply | Private Reply | To 1 | View Replies]

To: SunnyD1182
Try telling that to the environmentalist left

You think the religious right is better? You think scripture is free from error and bias? The world is hard to understand.

8 posted on 08/31/2005 12:37:35 AM PDT by liberallarry
[ Post Reply | Private Reply | To 5 | View Replies]

To: liberallarry
You think the religious right is better? You think scripture is free from error and bias? The world is hard to understand.

Dear Troll:

I didn't say anything about religion- or even being religious- nor did I say anything about scripture, so thank you for bringing up something that had nothing to do with anything.

9 posted on 08/31/2005 12:42:16 AM PDT by SunnyD1182
[ Post Reply | Private Reply | To 8 | View Replies]

To: SunnyD1182
so thank you for bringing up something that had nothing to do with anything

Thank you. The point is that all approaches to gaining knowledge and understanding are flawed and full of error.

10 posted on 08/31/2005 12:44:45 AM PDT by liberallarry
[ Post Reply | Private Reply | To 9 | View Replies]

To: dila813

"and it must be repeatable" one of the basic laws of the scientific method


11 posted on 08/31/2005 12:45:05 AM PDT by kublia khan (absolute war brings total victory)
[ Post Reply | Private Reply | To 1 | View Replies]

To: dila813

What an idiot! If less than 50% are probably true, then odds are, he's wrong!

But if he's wrong, then more than 50% are probably true, which means he's right... but if he's right, then less than 5....


My head hurts.


12 posted on 08/31/2005 12:49:50 AM PDT by djf (Government wants the same things I do - MY guns, MY property, MY freedoms!)
[ Post Reply | Private Reply | To 1 | View Replies]

To: kublia khan

Most errors are repeatable... And often are... ;-)


13 posted on 08/31/2005 12:52:16 AM PDT by DB ()
[ Post Reply | Private Reply | To 11 | View Replies]

To: dila813

Excellent. Now I don't feel so bad about the "D" I got in Science class when I was a junior in HS. ;o)


14 posted on 08/31/2005 12:52:31 AM PDT by Liberty Valance (Keep a simple manner for a happy life :o)
[ Post Reply | Private Reply | To 1 | View Replies]

To: dila813
I think that this is especially common among medical research as these authors have a very rudimentary knowledge about statistics and use hypothesis testing from a "cook book" without knowing the whys and whats.

Naturally, I am speaking in a statistical sense ;-)
15 posted on 08/31/2005 12:53:36 AM PDT by AdmSmith
[ Post Reply | Private Reply | To 1 | View Replies]

To: liberallarry
...all approaches to gaining knowledge and understanding are flawed and full of error.

Would that include your absolutist statement? Could the 'absolute' point of your statement be one big error? Food for thought.

16 posted on 08/31/2005 12:56:46 AM PDT by Step_Into_the_Void (Don't take my money and don't hire the government to take if from me for you. You thief.)
[ Post Reply | Private Reply | To 10 | View Replies]

To: konaice
But the odd part of this post, (and one the poster does not even seem to recognize) is that the only way a scientific paper can be proven wrong is by ... you guessed it, more scientific papers. So the denigration of scientific papers requires the very thing being denigrated.

May the circle, go unbroken......

This article = bizarre maunder
17 posted on 08/31/2005 12:56:54 AM PDT by Jaysun (Democrats: We must become more effective at fooling people.)
[ Post Reply | Private Reply | To 3 | View Replies]

To: DB

Funny, but it has nothing to do with the scientific method


18 posted on 08/31/2005 12:57:41 AM PDT by kublia khan (absolute war brings total victory)
[ Post Reply | Private Reply | To 13 | View Replies]

To: Step_Into_the_Void
Would that include your absolutist statement?

No. :) Bertrand Russel and others made their reputations considering problems like this.

19 posted on 08/31/2005 1:18:39 AM PDT by liberallarry
[ Post Reply | Private Reply | To 16 | View Replies]

To: dila813

Science is junk. Bees can not fly. Computers will never work. Splitting the Atom is joke. Atomic fusion is even more of a joke. DNA stands for "Do Not Ask." Science is junk.


20 posted on 08/31/2005 1:35:14 AM PDT by Jeff Gordon (Recall Barbara Boxer)
[ Post Reply | Private Reply | To 1 | View Replies]


Navigation: use the links below to view more comments.
first 1-2021-38 next last

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
News/Activism
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson