Free Republic
Browse · Search
News/Activism
Topics · Post Article

Skip to comments.

Most scientific papers are probably wrong
New Scientist ^ | 8/30/05 | Kurt Kleiner

Posted on 08/30/2005 10:29:44 AM PDT by LibWhacker

Most published scientific research papers are wrong, according to a new analysis. Assuming that the new paper is itself correct, problems with experimental and statistical methods mean that there is less than a 50% chance that the results of any randomly chosen scientific paper are true.

John Ioannidis, an epidemiologist at the University of Ioannina School of Medicine in Greece, says that small sample sizes, poor study design, researcher bias, and selective reporting and other problems combine to make most research findings false. But even large, well-designed studies are not always right, meaning that scientists and the public have to be wary of reported findings.

"We should accept that most research findings will be refuted. Some will be replicated and validated. The replication process is more important than the first discovery," Ioannidis says.

In the paper, Ioannidis does not show that any particular findings are false. Instead, he shows statistically how the many obstacles to getting research findings right combine to make most published research wrong.

Massaged conclusions

Traditionally a study is said to be "statistically significant" if the odds are only 1 in 20 that the result could be pure chance. But in a complicated field where there are many potential hypotheses to sift through - such as whether a particular gene influences a particular disease - it is easy to reach false conclusions using this standard. If you test 20 false hypotheses, one of them is likely to show up as true, on average.

Odds get even worse for studies that are too small, studies that find small effects (for example, a drug that works for only 10% of patients), or studies where the protocol and endpoints are poorly defined, allowing researchers to massage their conclusions after the fact.

Surprisingly, Ioannidis says another predictor of false findings is if a field is "hot", with many teams feeling pressure to beat the others to statistically significant findings.

But Solomon Snyder, senior editor at the Proceedings of the National Academy of Sciences, and a neuroscientist at Johns Hopkins Medical School in Baltimore, US, says most working scientists understand the limitations of published research.

"When I read the literature, I'm not reading it to find proof like a textbook. I'm reading to get ideas. So even if something is wrong with the paper, if they have the kernel of a novel idea, that's something to think about," he says.


TOPICS: News/Current Events
KEYWORDS: bias; conclusions; creationping; data; massaged; papers; scientific; statistics; wodlist; wrong
Navigation: use the links below to view more comments.
first 1-2021-4041-6061-80 ... 141-158 next last

1 posted on 08/30/2005 10:29:45 AM PDT by LibWhacker
[ Post Reply | Private Reply | View Replies]

To: LibWhacker
I don't believe this article.
2 posted on 08/30/2005 10:33:12 AM PDT by inquest (FTAA delenda est)
[ Post Reply | Private Reply | To 1 | View Replies]

To: LibWhacker

It has always been important to have reproducible results.


3 posted on 08/30/2005 10:33:58 AM PDT by knuthom
[ Post Reply | Private Reply | To 1 | View Replies]

To: LibWhacker
Most scientific papers are probably wrong...

Well, duh.

When a person, scientist or not, sets out to prove or disprove a theory, they saddle up with the bias of their intent.

Searching for the truth is a completely different thing.

4 posted on 08/30/2005 10:34:25 AM PDT by Dark Skies ("The sleeper must awaken!")
[ Post Reply | Private Reply | To 1 | View Replies]

To: LibWhacker
Some will be replicated and validated. The replication process is more important than the first discovery," Ioannidis says.

The problem in science is, despite all the talk of replication, that replication doesn't get you anything. Replication is considered low. Only "first discovery" is rewarded with more grant money. There is no incentive to try and repeat another's experiment for verification.

5 posted on 08/30/2005 10:34:39 AM PDT by Rokurota (.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: LibWhacker
Assuming that the new paper is itself correct, problems with experimental and statistical methods mean that there is less than a 50% chance that the results of any randomly chosen scientific paper are true.

I am left to wonder what statistical method the author used. :-)
6 posted on 08/30/2005 10:37:36 AM PDT by ARCADIA (Abuse of power comes as no surprise)
[ Post Reply | Private Reply | To 1 | View Replies]

To: LibWhacker

It was bound to happen-a study about studies. It's really just common sense. If you notice how much of what passes itself off as 'science' just happens to validate some leftist group's political agenda, like global warming, you start to wonder about the validity of the study. Political correctness is torturing history exactly the same way. The earth is flat and don't you dare question it. Especially on a college campus.


7 posted on 08/30/2005 10:39:02 AM PDT by Spok (Est omnis de civilitate.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: LibWhacker
Alrigty then, ditch science. We know how well it worked without it in the DARK AGES! </sarc>
8 posted on 08/30/2005 10:39:03 AM PDT by CarrotAndStick (The articles posted by me needn't necessarily reflect my opinion.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: DaveLoneRanger

"Science-as-settled-fact" ping


9 posted on 08/30/2005 10:39:35 AM PDT by My2Cents ("It takes a nation of candyasses to hold this military back.")
[ Post Reply | Private Reply | To 1 | View Replies]

Comment #10 Removed by Moderator

To: LibWhacker

Most income tax returns are "wrong" too. What's the articles point?


11 posted on 08/30/2005 10:42:06 AM PDT by narby (There are Bloggers, and then there are Freepers.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: LibWhacker; gobucks; mikeus_maximus; MeanWestTexan; JudyB1938; isaiah55version11_0; bondserv; ...
(((Creationist Ping)))



You have been pinged because of your interest in matters of Creation vs. Evolution, Creation trumping evolution, and evolutionary fraud. Freep-mail me if you want on/off this list.

Colossians 1:16 "For by him all things were created: things in heaven and on earth, visible and invisible, whether thrones or powers or rulers or authorities; all things were created by him and for him."



Do not assume this is true...just remember that everything we are told by the governing science elites who know so much more than we could possibly fathom should be taken with a grain of salt. Just remember it when you see evolutionists post huge reams of spam links from sophisticated scientific journals...just because they have PhD doesn't mean they're infallible.
12 posted on 08/30/2005 10:43:08 AM PDT by DaveLoneRanger (Be on your guard; stand firm in the faith; be men of courage; be strong. (1 Corinthians 16:13))
[ Post Reply | Private Reply | To 1 | View Replies]

To: inquest

LOL!


13 posted on 08/30/2005 10:43:30 AM PDT by <1/1,000,000th%
[ Post Reply | Private Reply | To 2 | View Replies]

To: narby
What's the articles point?

I believe the point is... don't believe everything you read.

14 posted on 08/30/2005 10:43:43 AM PDT by rhombus
[ Post Reply | Private Reply | To 11 | View Replies]

To: TaxRelief

But I thought "science" was always right!


15 posted on 08/30/2005 10:44:12 AM PDT by Tax-chick (Oklahoma is the cultural center of the universe ... take me back to Tulsa!)
[ Post Reply | Private Reply | To 1 | View Replies]

To: LibWhacker
Traditionally a study is said to be "statistically significant" if the odds are only 1 in 20 that the result could be pure chance.

In physics, the standard is 3-sigma (about a chance in 400) to claim evidence for something, and 5-sigma (about a chance in 1.8 million) to claim discovery. I'm constantly appalled at the weak statistical cases that are bandied about in the press as the gospel truth, particularly when it comes to medical studies.

Science can handle a poor signal-to-noise ratio, but public policy cannot.

16 posted on 08/30/2005 10:44:13 AM PDT by Physicist
[ Post Reply | Private Reply | To 1 | View Replies]

To: bobbdobbs

Only in philosophy where there never is a check with reality can one bad idea build on another without ever getting corrected.

I would include education also.


17 posted on 08/30/2005 10:44:23 AM PDT by PeterPrinciple (Seeking the truth here folks.)
[ Post Reply | Private Reply | To 10 | View Replies]

To: ARCADIA
I think the article refers to less than 50% of the papers providing conclusions stand the test of time, and or rigorous open peer review.

Results that are not reproducible are not to be believed, a classic is global warming where we have one study, the Mann study, that all the libs like to latch onto to prove man caused global warming, while at the same time denying respectability to contrary views.
18 posted on 08/30/2005 10:44:31 AM PDT by Tarpon
[ Post Reply | Private Reply | To 6 | View Replies]

To: bobbdobbs
Most scientists regard published results with scepticism. And of course any follow on work based on flawed papers will come up with bad results.

Ideally this is true, but it is extremely rare to find a paper published that directly and specifically refutes another.

And when you do find this, often the first paper is in a "high class" journal but the refutation will only get published in a "low class" journal. Which means that often the refutation is ignored.

19 posted on 08/30/2005 10:44:34 AM PDT by Rokurota (.)
[ Post Reply | Private Reply | To 10 | View Replies]

To: LibWhacker
Most scientific papers are probably wrong

After a few decades of experience with research in psychology, like the commentator, I don't take any published study on a stand-alone basis, but accept things, if ever, only after they have been replicated by the researcher's opponent.

Even if several studies are reported in the media with a certain finding, they are not trustworthy because one remarkable result generates a bandwagon for uncritical acceptance of further findings of the same sort.

There is no way to sort out the truth, but only to enlarge the context we have for understanding things.

20 posted on 08/30/2005 10:44:53 AM PDT by Marylander
[ Post Reply | Private Reply | To 1 | View Replies]


Navigation: use the links below to view more comments.
first 1-2021-4041-6061-80 ... 141-158 next last

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
News/Activism
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson