Posted on 04/27/2020 7:00:58 AM PDT by Kaslin
The decisions that are being made during this crisis are far too important and complex to be based on such imprecise data and with such unreliable results.
There’s been a lot of armchair analysis about various models being used to predict outcomes of COVID-19. For those of us who have built spatial and statistical models, all of this discussion brings to mind George Box’s dictum, “All models are wrong, but some are useful”—or useless, as the case may be.
The problem with data-driven models, especially when data is lacking, can be easily explained. First of all, in terms of helping decision makers make quality decisions, statistical hypothesis testing and data analysis is just one tool in a large tool box.
It’s based on what we generally call reductionist theory. In short, the tool examines parts of a system (usually by estimating an average or mean) and then makes inferences to the whole system. The tool is usually quite good at testing hypotheses under carefully controlled experimental conditions.
For example, the success of the pharmaceutical industry is, in part, due to the fact that they can design and implement controlled experiments in a laboratory. However, even under controlled experimental procedures, the tool has limitations and is subject to sampling error. In reality, the true mean (the true number or answer we are seeking) is unknowable because we cannot possibly measure everything or everybody, and model estimates always have a certain amount of error.
Simple confidence intervals can provide good insight into the precision and reliability, or usefulness, of the part estimated by reductionist models. With the COVID-19 models, the so-called “news” appears to be using either the confidence interval from one model or actual estimated values (i.e., means) from different models as a way of reporting a range of the “predicted” number of people who may contract or die from the disease (e.g., 60,000 to 2 million).
Either way, the range in estimates is quite large and useless, at least for helping decision makers make such key decisions about our health, economy, and civil liberties. The armchair analysts’ descriptions about these estimates show how clueless they are of even the simplest of statistical interpretation.
The fact is, when a model has a confidence interval as wide as those reported, the primary conclusion is that the model is imprecise and unreliable. Likewise, if these wide ranges are coming from estimated means of several different models, it clearly indicates a lack of repeatability (i.e., again, a lack of precision and reliability).
Either way, these types of results are an indication of bias in the data, which can come from many sources (such as not enough data, measurement error, reporting error, using too many variables, etc.). For the COVID-19 models, most of the data appears to come from large population centers like New York. This means the data sample is biased, which makes the entire analysis invalid for making any inferences outside of New York or, at best, areas without similar population density.
It would be antithetical to the scientific method if such data were used to make decisions in, for example, Wyoming or rural Virginia. While these models can sometimes provide decision makers useful information, the decisions that are being made during this crisis are far too important and complex to be based on such imprecise data. There are volumes of scientific literature that explain the limitations of reductionist methods, if the reader wishes to investigate this further.
Considering the limitations of this tool under controlled laboratory conditions, imagine what happens within more complex systems that encompass large areas, contain millions of people, and vary with time (such as seasonal or annual changes). In fact, for predicting outcomes within complex and adaptive and dynamic systems, where controlled experiments are not possible, data is lacking, and large amounts of uncertainty exist, the reductionists’ tool is not useful.
Researchers who speak as if their answers to such complex and uncertain problems are unquestionable and who politicize issues like COVID-19 are by definition pseudo-scientists. In fact, the scientific literature (including research from a Nobel Prize winner) shows that individual “experts” are no better than laymen at making quality decisions within systems characterized by complexity and uncertainty.
The pseudo-scientists want to hide this fact. They like to simplify reality by ignoring or hiding the tremendous amount of uncertainty inherent in these models. They do this for many reasons: it’s easier to explain cause/effect relationships, it’s easier to “predict” consequences (that’s why most of their predictions are wrong or always changing), and it’s easier to identify “victims” and “villains.”
They accomplish this by first asking the wrong questions. For COVID-19, the relevant question is not, “How many people will die?” a divisive and impossible question to answer, but “What can we do to avoid, reduce, and mitigate this disease without destroying our economy and civil rights?”
Secondly, pseudo-scientists hide and ignore the assumptions inherent in these models. The assumptions are the premise of any model; if the assumptions are violated or invalid, the entire model is invalid. Transparency is crucial to a useful model and for building trust among the public. In short, whether a model is useful or useless has more to do with a person’s values than science.
The empirical evidence is clear: what’s really needed is good thinking by actual people, not technology, to identify and choose quality alternatives. Technology will not solve these issues and should only be used as aids and tools (and only if they are transparent and reliable as possible).
What is needed, and what the scientific method has always required but is nowadays often ignored, is what is called multiple working hypotheses. In laymen’s terms, this simply means that we include experts and stakeholders with different perspectives, ideas, and experiences.
The type of modeling that is needed to make quality decisions for the COVID-19 crisis is what we modelers call participatory scenario modeling. This method uses Decision Science tools like Bayesian networks and Multiple Objective Decision Analysis that explicitly link data with the knowledge and opinions of a diverse mix of subject matter experts. The method uses a systems, not a reductionist, approach and seeks to help the decision maker weigh the available options and alternatives.
The steps are: frame the question appropriately, develop quality alternatives, evaluate the alternatives, and plan accordingly (i.e., make the decision). The key is participation from a diverse set of subject-matter experts from interdisciplinary backgrounds working together to build scenario models that help decision makers assess the decision options in terms of probability of the possible outcomes.
Certain models, such as COVID-19, require a diverse set of experts, whereas climate change models require participation from stakeholders and experts. The participatory nature of the process makes assumptions more transparent, helps people better understand the issues, and builds trust among competing interests.
For COVID-19, we likely need a set of models for medical and economic decisions that augment final decision-support models that help the decision makers weigh their options. No experienced decision maker would (or should) rely on any one model or any one subject-matter expert when making complex decisions with so much uncertainty and so much at stake.
Pseudo-scientists only allow participation from subject-matter experts who agree with their agenda. In other words, they often rig the participatory models. I’m not saying this is occurring with COVID-19, but it has happened before and could happen again.
Models are/were unreliable because the data was/is incomplete/corrupted.
Because model are terrible at predicting anything. Now, let’s discuss the models that are predicting catastrophic global warming.
Because anyone who points this out is called a heretic in the religion of science.
Meanwhile those who have no science, are basically ignorant of things involving science, parade around flaunting their ignorance
Because it’s the same folks doing climate models?
I haven’t noticed the studies or the models being available for inspection by others, even though actual science requires results and the methodologies to be testable and repeatable. Just more wild ass guesses.
And those who think that science is absolute just screwed the world big time. The models were wrong, wrong, wrong. No excuses. You all had your chance. You f-—ed up!!!!!
No more chances. You failed!
The so called modelers hid away like thieves in the night. We should find out who they are.
60,000 - 80,000 deaths seems to be realistic, slightly worse than some of the bad flu death years.
Now that deaths are in decline, and we know that UV light kills the virus well (although the media is too hung up on lying about what Trump said to report the actual news)- our economy can begin to open, and it is.
Most if not all of the economy will reopen over the next 30 days, despite recalcitrant Dem Governors.
You are correct sir, and the fact that certain “scientists” don’t want to share their data or methods is usually an indication that they’re a fraud.
A good example of that is Michael Mann and his “hockey stick” graph.
Notice that Bill Gates’ IHME and the WHO both don’t share their data or methods with others ... very secretive
...Because anyone who points this out is called a heretic in the religion of science.
Science is not the religion, Earth/Universe worship is
Science is a series of testable theories, but never portrays itself as “Truth”
The closest Science can get to “truth” is assertions for which there are no known exceptions.
Conservation of Energy, Conservation of Momentum.
Science can NEVER say why the assertions are “True”, only that they are, as best we can tell.
To understand truth one has to look at the “Why” of the Universe. To do that requires looking from the outside, in.
We can’t do that because we are bound inside the Universe.
It is the fundamental precept of Metaphysics
The original models cited at the first President’s Task Force press conferences were based on data obtained from Italy’s experience. None of the conditions in Italy were consistent with the US and therefore those models were a fraud.
Public vilification, scorn, tar and feathers are in order. A public hanging is probably too harsh but warranted in my opinion.
Children, what have we learned today? That science is not perfect as many would have us believe. And when we bet our lives on what these fools predict and they are wrong, we may suffer greatly.
Worst of all, they were presented as accurate. Whomever created the models should have the degrees and credentials stripped from them and should be forced clean toilets and Grand Central Station for the rest of their lives.
It seems that way too many of these models are designed to show a desired outcome. If the data goes the wrong way, just let the software nudge it back to the appropriate direction.
Political polls are very much the same thing.
...None of the conditions in Italy were consistent with the US and therefore those models were a fraud.
They would only be a fraud if they did not list their sources and the limitations of same. Scientific articles should always list their references. I do not remember whether the original discussion did this.
A better word might be flawed, instead of fraud. All models are flawed to greater or lesser degrees, as even at their best, they do not account for all data points, and are skewed by uncertainty. Also observer bias plays a large part.
Bias is everywhere, including FreeRepublic. I just happen to think it is less here than in many other sites. YMMV
To some, it is a religion.
Here is the most important point: NO ONE EVER EXPLAINED THAT THE PREDICTIONS MIGHT BE FALWED AND BY HOW MUCH!!!! They and all who were silent are complicit and guilty by omission; by their silence.
Our top Dem health idiot here in Virginia is saying schools cannot open for TWO YEARS!
Of course, IDIOT should be capitalized.........
...To some, it is a religion.
That is called Atheism. The belief that the Universe represents “All There Is” and there is nothing else. The study of science, to them, becomes the pursuit of ultimate “Truth”
The are trapped inside a Hall of Mirrors
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.