Model: COVID-19 Forecast Hub - COVIDhub-trained_ensemble

    Name: trained_ensemble
    Abbreviation: COVIDhub-trained_ensemble
    Owner: covid19hub
    Team name: COVID-19 Forecast Hub
    Description: A weighted ensemble, or model average, of submitted forecasts to the COVID-19 Forecast Hub.
    Contributors: Evan L. Ray, Estee Cramer, Aaron Gerding, Nicholas Reich <nick@schoolph.umass.edu>
    License: Creative Commons Attribution 4.0
    Notes: other
    Citation: Missing
    Methods: This is a weighted combination of the component model forecasts. All component models that provide forecasts at all required quantile levels for all forecasts at 1 to 4 week ahead horizons (or 1 to 28 day ahead for hospitalizations) are included. From 2021-02-10 to 2021-03-01, ensemble forecasts at each quantile level are a weighted linear combination of the component model forecasts at that quantile level. Weights are estimated separately for each combination of geographic scale (county, state, or national) and prediction target variable (inc case, inc hosp, inc death, cum death). Model weights are estimated by optimizing the mean weighted interval score (WIS) of the ensemble forecasts in past weeks, subject to the constraints that the weights are non-negative and sum to 1, and the predictive quantiles are properly ordered. We select an ensemble specification with best mean WIS across all past weeks in a retrospective evaluation; a separate specification may be selected for each combination of geographic scale and prediction target variable. We consider variations on training that use a single shared weight across all quantile levels for each model, separate weights for each quantile level, or three groups of quantile levels (the four in the lower tail, the 15 in the middle, and the 4 in the upper tail). We also consider varying ranges for how many past weeks of forecasts are used in weight estimation, from 3 weeks to 10 weeks (except at county level, where we only consider window sizes of 3 weeks). For the national and state level, we also consider estimation jointly across those geographic units. During the weight estimation process, any missing forecasts are mean-imputed; the estimated weights are then redistributed to account for missingness patterns (so that models with missing forecasts in the training window will have their estimated weight discounted). From 2021-05-03 to 2021-05-17, ensemble forecasts are an equally weighted median of the forecasts from the top-performing individual forecasters. Top performing forecasters are selected based on the relative WIS, which measures the average performance of each model relative to all other models on the subset of forecasts that each pair of models has in common (see https://www.medrxiv.org/content/10.1101/2021.02.03.21250974v1 for more detail). This relative WIS ranking is updated each week based on a rolling window of recent forecasts. The window size and number of component forecasters to include are selected separately for each target variable using time series cross validation to select values leading to strong forecast skill. From 2021-05-24 to present, ensemble forecasts are a weighted median of the forecasts from the top-performing individual forecasters. Top performing forecasters are selected based on the relative WIS, which measures the average performance of each model relative to all other models on the subset of forecasts that each pair of models has in common (see https://www.medrxiv.org/content/10.1101/2021.02.03.21250974v1 for more detail). This relative WIS ranking is updated each week based on a rolling window of recent forecasts. The window size and number of component forecasters to include are selected separately for each target variable using time series cross validation to select values leading to strong forecast skill. The component forecasters are assigned weights w_m = exp(-theta * relative WIS model m) / sum_j exp(-theta * relative WIS model j). The parameter theta is estimated via a grid search optimizing the ensemble WIS over a window of recent weeks. The same window size is used for model selection and estimating theta.
    Home: https://covid19forecasthub.org/
    Auxiliary data: https://github.com/reichlab/covid19-forecast-hub

    Forecasts (26)

    Timezero Data Source Upload Time Issued at Version
    2020-05-05 (No data)
    2019-12-30 (No data)
    2020-03-15 (No data)
    2020-05-16 (No data)
    2020-05-15 (No data)
    2020-05-06 (No data)
    2020-05-07 (No data)
    2020-08-01 (No data)
    2020-07-31 (No data)
    2020-08-02 (No data)
    2020-05-10 (No data)
    2020-05-23 (No data)
    2020-05-22 (No data)
    2020-05-27 (No data)
    2020-06-26 (No data)
    2020-07-11 (No data)
    2020-07-18 (No data)
    2020-07-25 (No data)
    2020-08-04 (No data)
    2020-07-28 (No data)
    2020-08-05 (No data)
    2020-08-06 (No data)
    2020-08-07 (No data)
    2020-08-08 (No data)
    2020-08-09 (No data)
    2020-08-11 (No data)
    2020-08-12 (No data)
    2020-08-13 (No data)
    2020-08-14 (No data)
    2020-08-15 (No data)
    2020-08-16 (No data)
    2020-08-18 (No data)
    2020-08-19 (No data)
    2020-08-20 (No data)
    2020-08-21 (No data)
    2020-08-22 (No data)
    2020-08-23 (No data)
    2020-08-25 (No data)
    2020-08-26 (No data)
    2020-08-27 (No data)
    2020-08-30 (No data)
    2020-08-29 (No data)
    2020-09-02 (No data)
    2020-09-01 (No data)
    2020-09-03 (No data)
    2020-09-04 (No data)
    2020-09-05 (No data)
    2020-09-06 (No data)
    2020-09-08 (No data)
    2020-09-13 (No data)
    2020-09-19 (No data)
    2020-03-23 (No data)
    2020-03-30 (No data)
    2020-04-06 (No data)
    2020-04-13 (No data)
    2020-04-20 (No data)
    2020-04-27 (No data)
    2020-05-04 (No data)
    2020-05-11 (No data)
    2020-05-18 (No data)
    2020-05-25 (No data)
    2020-06-01 (No data)
    2020-06-08 (No data)
    2020-06-15 (No data)
    2020-06-22 (No data)
    2020-06-29 (No data)
    2020-07-06 (No data)
    2020-07-13 (No data)
    2020-07-20 (No data)
    2020-07-27 (No data)
    2020-08-03 (No data)
    2020-08-10 (No data)
    2020-08-17 (No data)
    2020-08-24 (No data)
    2020-08-31 (No data)
    2020-09-07 (No data)
    2020-09-14 (No data)
    2020-09-21 (No data)
    2020-09-28 (No data)
    2020-10-05 (No data)
    2020-10-12 (No data)
    2020-10-19 (No data)
    2020-10-26 (No data)
    2020-11-02 (No data)
    2020-11-09 (No data)
    2020-11-16 (No data)
    2020-11-23 (No data)
    2020-11-30 (No data)
    2020-12-07 (No data)
    2020-12-14 (No data)
    2020-12-21 (No data)
    2020-12-28 (No data)
    2020-04-12 (No data)
    2020-04-16 (No data)
    2020-04-19 (No data)
    2020-04-23 (No data)
    2020-04-26 (No data)
    2020-04-15 (No data)
    2020-04-17 (No data)
    2020-04-18 (No data)
    2020-04-21 (No data)
    2020-04-22 (No data)
    2020-04-24 (No data)
    2020-04-25 (No data)
    2020-04-28 (No data)
    2020-04-29 (No data)
    2020-04-30 (No data)
    2020-05-01 (No data)
    2020-05-02 (No data)
    2020-05-03 (No data)
    2020-03-22 (No data)
    2020-03-29 (No data)
    2020-04-05 (No data)
    2020-04-08 (No data)
    2020-03-27 (No data)
    2020-03-31 (No data)
    2020-04-01 (No data)
    2020-04-07 (No data)
    2020-04-09 (No data)
    2020-01-13 (No data)
    2020-01-20 (No data)
    2020-01-27 (No data)
    2020-02-03 (No data)
    2020-02-10 (No data)
    2020-02-17 (No data)
    2020-02-24 (No data)
    2020-03-02 (No data)
    2020-03-09 (No data)
    2020-01-06 (No data)
    2020-03-16 (No data)
    2020-05-08 (No data)
    2020-05-09 (No data)
    2020-05-14 (No data)
    2020-05-12 (No data)
    2020-05-17 (No data)
    2020-05-13 (No data)
    2020-05-21 (No data)
    2020-05-24 (No data)
    2020-05-20 (No data)
    2020-05-19 (No data)
    2020-08-28 (No data)
    2020-09-09 (No data)
    2020-05-28 (No data)
    2020-04-14 (No data)
    2020-05-29 (No data)
    2020-05-31 (No data)
    2020-05-26 (No data)
    2020-05-30 (No data)
    2020-06-02 (No data)
    2020-06-03 (No data)
    2020-06-04 (No data)
    2020-06-07 (No data)
    2020-06-05 (No data)
    2020-06-06 (No data)
    2020-06-09 (No data)
    2020-06-10 (No data)
    2020-06-11 (No data)
    2020-06-12 (No data)
    2020-06-13 (No data)
    2020-06-14 (No data)
    2020-06-16 (No data)
    2020-06-17 (No data)
    2020-06-18 (No data)
    2020-06-19 (No data)
    2020-06-20 (No data)
    2020-06-21 (No data)
    2020-06-23 (No data)
    2020-06-24 (No data)
    2020-06-25 (No data)
    2020-06-27 (No data)
    2020-06-28 (No data)
    2020-06-30 (No data)
    2020-07-01 (No data)
    2020-07-02 (No data)
    2020-07-03 (No data)
    2020-07-04 (No data)
    2020-07-05 (No data)
    2020-07-07 (No data)
    2020-07-08 (No data)
    2020-07-09 (No data)
    2020-07-12 (No data)
    2020-07-10 (No data)
    2020-07-14 (No data)
    2020-07-15 (No data)
    2020-07-16 (No data)
    2020-07-17 (No data)
    2020-07-19 (No data)
    2020-07-21 (No data)
    2020-07-22 (No data)
    2020-07-23 (No data)
    2020-07-24 (No data)
    2020-07-26 (No data)
    2020-09-10 (No data)
    2020-09-12 (No data)
    2020-09-11 (No data)
    2020-07-29 (No data)
    2020-07-30 (No data)
    2020-09-15 (No data)
    2020-09-16 (No data)
    2020-09-17 (No data)
    2020-09-18 (No data)
    2020-09-20 (No data)
    2020-09-22 (No data)
    2020-09-23 (No data)
    2020-09-24 (No data)
    2020-09-27 (No data)
    2020-09-26 (No data)
    2020-09-25 (No data)
    2020-09-29 (No data)
    2020-09-30 (No data)
    2020-10-01 (No data)
    2020-10-03 (No data)
    2020-10-02 (No data)
    2020-10-04 (No data)
    2020-10-08 (No data)
    2020-10-11 (No data)
    2021-06-29 (No data)
    2021-07-01 (No data)
    2021-07-05 2021-07-05-COVIDhub-trained_ensemble.csv 2021-07-07 00:42:01 UTC 2021-07-07 00:42:01 UTC
    2021-07-06 (No data)
    2021-07-11 (No data)
    2021-07-13 (No data)
    2021-07-14 (No data)
    2021-07-15 (No data)
    2021-07-18 (No data)
    2021-07-20 (No data)
    2021-07-21 (No data)
    2021-07-25 (No data)
    2021-07-24 (No data)
    2021-07-30 (No data)
    2021-07-31 (No data)
    2021-08-02 2021-08-02-COVIDhub-trained_ensemble.csv 2021-08-03 18:37:21 UTC 2021-08-03 18:37:21 UTC
    2021-08-03 (No data)
    2021-08-05 (No data)
    2021-08-07 (No data)
    2021-08-09 2021-08-09-COVIDhub-trained_ensemble.csv 2021-08-11 18:27:16 UTC 2021-08-11 18:27:16 UTC
    2021-08-12 (No data)
    2021-08-18 (No data)
    2021-08-22 (No data)
    2021-08-23 2021-08-23-COVIDhub-trained_ensemble.csv 2021-08-24 18:26:23 UTC 2021-08-24 18:26:23 UTC
    2021-08-24 (No data)
    2021-08-26 (No data)
    2021-08-27 (No data)
    2021-08-30 2021-08-30-COVIDhub-trained_ensemble.csv 2021-09-02 00:26:43 UTC 2021-09-02 00:26:43 UTC
    2021-08-31 (No data)
    2021-09-01 (No data)
    2021-09-03 (No data)
    2021-09-06 2021-09-06-COVIDhub-trained_ensemble.csv 2021-09-08 20:17:20 UTC 2021-09-08 20:17:20 UTC
    2021-09-10 (No data)
    2021-09-12 (No data)
    2021-09-15 (No data)
    2021-09-18 (No data)
    2021-09-19 (No data)
    2021-09-21 (No data)
    2020-10-15 (No data)
    2020-10-16 (No data)
    2020-10-18 (No data)
    2021-07-04 (No data)
    2021-07-02 (No data)
    2021-07-07 (No data)
    2021-07-12 2021-07-12-COVIDhub-trained_ensemble.csv 2021-07-13 12:32:03 UTC 2021-07-13 12:32:03 UTC
    2021-07-16 (No data)
    2021-07-22 (No data)
    2021-08-01 (No data)
    2021-08-08 (No data)
    2021-08-10 (No data)
    2021-08-16 2021-08-16-COVIDhub-trained_ensemble.csv 2021-08-17 18:24:07 UTC 2021-08-17 18:24:07 UTC
    2021-08-19 (No data)
    2021-08-21 (No data)
    2021-08-28 (No data)
    2021-09-02 (No data)
    2021-09-04 (No data)
    2021-09-13 2021-09-13-COVIDhub-trained_ensemble.csv 2021-09-14 18:28:16 UTC 2021-09-14 18:28:16 UTC
    2021-09-16 (No data)
    2020-10-22 (No data)
    2020-10-25 (No data)
    2020-10-29 (No data)
    2020-11-01 (No data)
    2020-11-04 (No data)
    2020-11-08 (No data)
    2020-11-05 (No data)
    2020-11-12 (No data)
    2020-11-15 (No data)
    2020-11-17 (No data)
    2020-11-18 (No data)
    2020-11-19 (No data)
    2020-11-22 (No data)
    2020-11-21 (No data)
    2020-11-24 (No data)
    2020-11-26 (No data)
    2020-11-29 (No data)
    2020-12-01 (No data)
    2020-12-03 (No data)
    2020-12-06 (No data)
    2020-12-05 (No data)
    2020-12-08 (No data)
    2020-12-10 (No data)
    2020-12-11 (No data)
    2020-12-13 (No data)
    2020-12-15 (No data)
    2020-12-16 (No data)
    2020-12-17 (No data)
    2020-12-18 (No data)
    2020-12-19 (No data)
    2020-12-20 (No data)
    2020-12-22 (No data)
    2020-12-24 (No data)
    2020-12-23 (No data)
    2020-12-27 (No data)
    2020-12-30 (No data)
    2021-01-03 (No data)
    2021-01-04 (No data)
    2021-01-05 (No data)
    2021-01-06 (No data)
    2021-01-10 (No data)
    2021-01-09 (No data)
    2021-01-11 (No data)
    2021-01-13 (No data)
    2021-01-14 (No data)
    2021-01-17 (No data)
    2021-01-15 (No data)
    2021-01-16 (No data)
    2021-01-18 (No data)
    2021-01-19 (No data)
    2021-01-21 (No data)
    2021-01-22 (No data)
    2021-01-23 (No data)
    2021-01-24 (No data)
    2021-01-25 (No data)
    2021-01-27 (No data)
    2021-01-28 (No data)
    2021-01-31 (No data)
    2021-01-29 (No data)
    2021-02-01 2021-02-01-COVIDhub-trained_ensemble.csv 2021-04-17 20:36:46 UTC 2021-02-02 12:00:00 UTC
    2021-02-02 (No data)
    2021-02-04 (No data)
    2021-02-05 (No data)
    2021-02-06 (No data)
    2021-02-07 (No data)
    2021-02-08 2021-02-08-COVIDhub-trained_ensemble.csv 2021-02-09 14:18:07 UTC 2021-02-09 12:00:00 UTC
    2021-02-11 (No data)
    2021-02-12 (No data)
    2021-02-13 (No data)
    2021-02-14 (No data)
    2021-02-15 2021-02-15-COVIDhub-trained_ensemble.csv 2021-02-16 18:24:10 UTC 2021-02-16 12:00:00 UTC
    2021-02-16 (No data)
    2021-02-18 (No data)
    2021-02-17 (No data)
    2021-02-20 (No data)
    2021-02-21 (No data)
    2021-02-22 2021-02-22-COVIDhub-trained_ensemble.csv 2021-02-23 18:30:56 UTC 2021-02-23 12:00:00 UTC
    2021-02-23 (No data)
    2021-02-25 (No data)
    2021-02-24 (No data)
    2021-02-27 (No data)
    2021-02-28 (No data)
    2021-03-01 2021-03-01-COVIDhub-trained_ensemble.csv 2021-03-03 00:36:55 UTC 2021-03-02 12:00:00 UTC
    2021-03-02 (No data)
    2021-03-03 (No data)
    2021-03-04 (No data)
    2021-03-05 (No data)
    2021-03-06 (No data)
    2021-03-07 (No data)
    2021-03-08 (No data)
    2021-03-10 (No data)
    2021-03-09 (No data)
    2021-03-11 (No data)
    2021-03-12 (No data)
    2021-03-13 (No data)
    2021-03-14 (No data)
    2021-03-15 (No data)
    2021-03-21 (No data)
    2021-03-20 (No data)
    2021-03-19 (No data)
    2021-03-17 (No data)
    2021-03-18 (No data)
    2021-03-22 (No data)
    2021-03-24 (No data)
    2021-03-25 (No data)
    2021-03-27 (No data)
    2021-03-26 (No data)
    2021-03-28 (No data)
    2021-03-29 (No data)
    2021-03-30 (No data)
    2021-03-31 (No data)
    2021-04-01 (No data)
    2021-04-02 (No data)
    2021-04-03 (No data)
    2021-04-04 (No data)
    2021-04-05 (No data)
    2021-04-08 (No data)
    2021-04-10 (No data)
    2021-04-11 (No data)
    2021-04-12 (No data)
    2021-04-13 (No data)
    2021-04-14 (No data)
    2021-04-15 (No data)
    2021-04-16 (No data)
    2021-04-18 (No data)
    2021-04-17 (No data)
    2021-04-19 (No data)
    2021-04-20 (No data)
    2021-04-21 (No data)
    2021-04-22 (No data)
    2021-04-23 (No data)
    2021-04-25 (No data)
    2021-04-24 (No data)
    2021-04-26 (No data)
    2021-04-27 (No data)
    2021-04-28 (No data)
    2021-04-29 (No data)
    2021-04-30 (No data)
    2021-05-01 (No data)
    2021-05-02 (No data)
    2021-05-03 2021-05-03-COVIDhub-trained_ensemble.csv 2021-05-04 04:16:26 UTC 2021-05-04 12:00:00 UTC
    2021-05-04 (No data)
    2021-05-05 (No data)
    2021-07-03 (No data)
    2021-07-08 (No data)
    2021-07-17 (No data)
    2021-07-19 2021-07-19-COVIDhub-trained_ensemble.csv 2021-07-20 18:32:12 UTC 2021-07-20 18:32:12 UTC
    2021-07-23 (No data)
    2021-05-06 (No data)
    2021-05-07 (No data)
    2021-05-08 (No data)
    2021-05-09 (No data)
    2021-05-10 2021-05-10-COVIDhub-trained_ensemble.csv 2021-05-11 06:30:23 UTC 2021-05-11 12:00:00 UTC
    2021-05-11 (No data)
    2021-05-12 (No data)
    2021-05-13 (No data)
    2021-05-14 (No data)
    2021-05-15 (No data)
    2021-05-16 (No data)
    2021-05-17 2021-05-17-COVIDhub-trained_ensemble.csv 2021-05-18 01:12:43 UTC 2021-05-17 12:00:00 UTC
    2021-05-19 (No data)
    2021-05-20 (No data)
    2021-05-21 (No data)
    2021-05-23 (No data)
    2021-05-24 2021-05-24-COVIDhub-trained_ensemble.csv 2021-05-25 18:43:04 UTC 2021-05-25 18:43:04 UTC
    2021-05-22 (No data)
    2021-07-26 2021-07-26-COVIDhub-trained_ensemble.csv 2021-07-27 18:28:15 UTC 2021-07-27 18:28:15 UTC
    2021-05-25 (No data)
    2021-05-26 (No data)
    2021-05-27 (No data)
    2021-05-28 (No data)
    2021-05-29 (No data)
    2021-05-30 (No data)
    2021-05-31 2021-05-31-COVIDhub-trained_ensemble.csv 2021-06-01 15:22:37 UTC 2021-06-01 15:22:37 UTC
    2021-06-01 (No data)
    2021-06-03 (No data)
    2021-06-02 (No data)
    2021-06-06 (No data)
    2021-06-05 (No data)
    2021-06-07 2021-06-07-COVIDhub-trained_ensemble.csv 2021-06-08 18:57:39 UTC 2021-06-08 18:57:39 UTC
    2021-06-08 (No data)
    2021-06-09 (No data)
    2021-06-10 (No data)
    2021-06-11 (No data)
    2021-06-13 (No data)
    2021-06-14 2021-06-14-COVIDhub-trained_ensemble.csv 2021-06-15 12:29:59 UTC 2021-06-15 12:29:59 UTC
    2021-06-16 (No data)
    2021-06-18 (No data)
    2021-06-17 (No data)
    2021-06-19 (No data)
    2021-06-20 (No data)
    2021-06-21 2021-06-21-COVIDhub-trained_ensemble.csv 2021-06-22 11:13:26 UTC 2021-06-22 11:13:26 UTC
    2021-06-22 (No data)
    2021-08-15 (No data)
    2021-08-11 (No data)
    2021-08-14 (No data)
    2021-08-17 (No data)
    2021-08-20 (No data)
    2021-06-23 (No data)
    2021-06-24 (No data)
    2021-06-25 (No data)
    2021-06-27 (No data)
    2021-06-28 2021-06-28-COVIDhub-trained_ensemble.csv 2021-06-29 16:02:05 UTC 2021-06-29 16:02:05 UTC
    2021-08-25 (No data)
    2021-08-29 (No data)
    2021-09-05 (No data)
    2021-09-20 2021-09-20-COVIDhub-trained_ensemble.csv 2021-09-21 15:16:31 UTC 2021-09-21 15:16:31 UTC