UMass lab cited for precision forecasting COVID-19 deaths

AMHERST — University of Massachusetts researchers on the Amherst campus are being credited with the most accurate short-term projections of COVID-19 deaths in the United States, developing a tool that has been used by the U.S. Centers for Disease Control and Prevention.

A recent scientific journal article shows that the methods used by the Reich Laboratory in creating the COVID-19 Forecast Hub in April 2020, with funding from the CDC, have been most precise for calculating deaths from the viral illness at both the state and national levels over the first 18 months of the pandemic.

“A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021,” writes lead author Estee Cramer, an epidemiological candidate in the School of Public Health and Health Sciences.

The Proceedings of the National Academies of Science article about the Forecast Hub, which is directed by Nicholas Reich and Evan Ray, faculty in the School of Public Health, was co-authored by 292 researchers whose modeling groups submit data to the UMass site. Reich and Ray are also co-authors of the article.

Cramer said in a statement that these forecasts, available at https://covid19forecasthub.org/, have been critical for resource allocation, such as health care staffing and medical supplies, and for making decisions related to school closures.

“Including a variety of models in the ensemble contributes to its robustness and ability to overcome individual model biases,” Cramer said. “This is a really important consideration for public health agencies when using forecasts to inform policies during an outbreak of any size.”

The Forecast Hub synthesizes data from 90 sources, including universities and scientific industry partners and researchers without an affiliation. The hub then collects numerous points of data and factors in potential policy decisions by politicians and public health professionals and behavioral changes by the public.

The study doesn’t examine forecasts for COVID-19 cases or hospitalizations.

Reich, a biostatistician, explains that the paper shows how data, collected in real time across the entire pandemic, can be used to better understand which modeling approaches worked and which did not, and why that was the case.

“It’s going to take many years to unpack all of the lessons of the last few years. In some ways, this is just the beginning,” Reich said.

Though accurate, even the ensemble model had difficulties during various waves of the pandemic, and also when trying to determine longer-term numbers.

Reich compares that to weather predictions.

“This work shows that the accuracy of forecasts for deaths is pretty good for the next four weeks, but at horizons of six weeks or more, the accuracy is typically substantially worse,” Reich said.