From the oft-cited and live-updated COVID-19 map released by John Hopkins University to the countless “flatten the curve” epidemic models, data and resulting projections can provide a great deal of information and insight for decision-making on both a personal and public level. But models are often conflicting (think: the recent leak from the White House) and aren’t always accurate.
But scientists, public health professionals, policymakers and everyday citizens are looking at them for daily decision-making. Should we still be staying home? How long will the pandemic last? Is social distancing working?
So how should we understand these models? Dr. Reza Yaesoubi, a decision scientist and professor at Yale’s School of Public Health, works on mathematically simulating pandemic models in order to make informed decisions on policy.
“Ideally, we want to make sure that the way that we are responding [to the pandemic is] informed by the data,” he said.
ARE MODELS ACCURATE?
While states’ reopening and national policy decisions heavily rely on data scientists’ modeling, it’s important to understand that they are tools, not exact predictions of the future. In fact, no model can ever be 100% accurate because it cannot take into account every single factor and moment-to-moment change in those factors.
“All models are wrong, but some are useful. It is because, by definition, these models are essentially a prototype of a real system,” Yaesoubi said. “You’re trying to create a prototype of what is happening in real life, and that’s something very challenging because you have to make a lot of assumptions.”
The most accurate models, he said, take years to develop. As new policy decisions are being made every day, however, COVID-19 pandemic modeling doesn’t have that kind of time.
Currently, data is being collected on a massive scale, including variables such as population density, disease prevalence and social distancing laws that can all be used as inputs for modeling. And more data means potentially more accurate modeling.
SO HOW EXACTLY ARE MODELS USED?
Many state policies for social distancing and stay-at-home orders were driven by the “flatten the curve” model, which reflected the severity of taking no action against COVID-19 by displaying two potential scenarios: In one, no policies were enforced, leading to a giant spike in cases and overloaded hospitals, while the other showed a milder, flatter peak indicating a reduction of cases and a more manageable hospital load.
As in this example, models are often used to show what may happen under differing policies or actions.
“We develop these models […] to inform decisions and to do what-if analysis. The main purpose of these models [is] not to project [the] future; it is more to understand what happens if we take action A versus action B,” Yaesoubi said.
In this case, these “flattening the curve” scenarios and models found themselves on poster boards in press conferences and trending across social media under the hashtag #FlattenTheCurve as a way to educate the public on the importance of social distancing and careful protocol. The result? Nearly every state enacted some sort of stay-at-home policy and mobility nationwide decreased dramatically.
Models are also vulnerable to misinterpretation. When enacted policies successfully avert predicted worst-case scenarios, it becomes easier to dismiss the severity of the dangers originally presented.
For example, when confirmed COVID-19 cases, hospitalizations and deaths didn’t reach the projected highs, some saw this as proof that the measures were an overreaction. The irony, however, is that models themselves can influence both policy and behavior, thus changing their own trajectory (hopefully for the better).
“What is being missed is that we are back to normal because of the extreme measures that we took; so now if you reopen everything back up again, there are many people who are still susceptible and then, very quickly, you’ll see a rebound,” he said.
WHAT MAKES A GOOD MODEL AND HOW SHOULD WE READ IT?
Every model that we see during this pandemic will have a certain level of uncertainty, especially as they try to keep up and respond to constantly-changing variables.
“[N]one of these models are perfect because [the] future is uncertain, and also what we have observed in the past is uncertain and incomplete,” Yaesoubi said.
Many models process factors such as population density, location and social distancing to project a potential trajectory for figures such as COVID-19 deaths. Modelers may update assumptions and parameters on their models to match different scenarios caused as a result of new policies, changes in people’s behavior and other unforeseeable variables.
“I should come and look whether I was successful in my projections and, if not, that means that maybe there is something that [was] not captured. So we should again revise some of the assumptions or some of the structure of the model to improve it,” he said.
When looking at models, it is important to look at multiple outcomes and scenarios and to understand the modelers’ assumptions. In these upcoming weeks, many models will be continually developed to reflect the quickly-changing and ever-present crisis as we also try to understand where we are and what to do next. The key? Stay informed and give room for uncertainty.
“I think the key is to make informed decisions. Ideally, our goal should be [that] I’m making this decision after accounting for many of these complexities that we are dealing with, and I recognize that this might not be the best decision because [the] future is uncertain. But at least we should be open about what factors can be account[ed] for when we are recommending to close everything or when we [are] recommending to go back to normal,” he said.
While we are consulting experts in the field to get answers to important questions during this crisis, new information and studies come out almost every day and much remains unknown regarding COVID-19. Midstory encourages everyone to follow all public health and safety protocols and exercise extreme caution.
