Posted on 05/10/2018 9:09:20 AM PDT by DeweyCA
Don’t believe the latest study you read in the headlines, chances are, it could be wrong, according to a new report by the National Association of Scholars that delves into what it calls the “use and abuse of statistics in the sciences.”
The report broke down the issue of irreproducibility, or the problem that a lot of scientific research cannot be reproduced. The report took aim at unverifiable climate science, but also critiqued medical studies, behavioral research and other fields.
The 72-page report took the matter a step further in calling the issue a politicization of science.
“Not all irreproducible research is progressive advocacy; not all progressive advocacy is irreproducible; but the intersection between the two is very large. The intersection between the two is a map of much that is wrong with modern science,” the report states.
Co-authored by David Randall and Christopher Wesler, “The Irreproducibility Crisis of Modern Science: Causes, Consequences, and the Road to Reform” focused on the irreproducibility of recent scientific studies.
It references a study performed by researchers at Amgen in 2012. For this study, researchers tried to reproduce the results of “53 landmark studies in oncology and hematology.” Researchers were only able to replicate the results of six studies.
“People have found similar results in psychology and economics. Different fields are affected different amounts,” Randall told The College Fix. “As a rule of thumb, fields that use statistics intensively are more likely to have troubles than fields that don’t.”
The report hypothesized that there are a number of different reasons for irreproducibility that include such things as “flawed statistics, faulty data, deliberate exclusion of data, and political groupthink,” among other reasons. “Actual fraud on the part of researchers appears to be a growing problem,” the report also states.
“‘Stereotype threat’ as an explanation for poor academic performance? Didn’t reproduce. ‘Social priming,’ which argues that unnoticed stimuli can significantly change behavior? Didn’t reproduce that well … Tests of implicit bias as predictors of discriminatory behavior? The methodology turned out to be dubious, and the test of implicit bias may have been biased itself,” the report states.
The report also alludes multiple times to the notion that climate science is on shaky ground.
“Climate science has significant work to do to make its data and its statistical procedures properly reproducible,” Randall said.
Randall cited Judith Curry, a world-renowned climatologist, who has warned that the climate science field is heavily affected by groupthink, a collective way of thinking that has been known to stop individuals from questioning widely accepted theories.
Randall said he believes that climate change data needs to be reproducible because it is “more than usually intrusive into the lives of Americans.”
To provide the public with accurate statistical information, the report endorses the expansion of the Secret Science Reform Act of 2015 to cut down on irreproducible data used to back public policy.
When asked what the average person could do in order to make sure that the information that is backing public policy is credible, Randall recommended: “Always ask ‘has this study been reproduced? Did this study have pre-registered research protocols? Does it support an unpopular belief?’ If the answer to any of these is no, suspend judgment. Don’t disbelieve blindly, but don’t believe blindly either.”
I first remember being taught the basics of the Scientific Method in fourth grade (late 60s) and it was beaten into me all the way through my Masters program in Physics (mid 80s). Even in the 80s nincompoops were publishing garbage about Global Cooling using junk science and bad statistics.
I think it really became prevalent in the mid to late 90s.
| Science = Reproducable Not Reproducable = Not Science |
Good article. I have read something like this before, and I am also quite familiar with John Ionnnaitis and his work on the failure of replicating much scientific work. Journals are not interested in publishing “null tests.” They are not nearly as “sexy” as studies that purport to have discovered some amazing new insight. People forget that scientists are “fallen” people, just like the rest of us. As such, they are susceptible to the same temptations of pursuing fame and fortune as the rest of us.
“Many studies can’t be replicated, study says.”
Used to have a subscription.
I saw this first hand in 1967. I was a lab technician in a products research lab. The company required “blind” testing before approval of any new product introduction. A very sweet food product was being advocated by a product manager and I got assigned to the “blind” panel set up. They used church groups to insure no contact between the product people and the actual subjects. The study was cancelled twice and went forward only when it was assigned to a large, mostly black church. I later learned that women of color are about twice as likely to prefer “very sweet” to “somewhat sweet.” Hence rigging of the “blind” test.
The product was well liked by the church ladies, was introduced with much fanfare, but ended up a complete flop.
So, even private industry has been rigging the statistics for a very long time.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.