Project score data for “CDC Real-time Forecasts”

    Scores (all models)

    Name Description Last Update # Score Values
    Absolute Error The absolute value of the truth value minus the model's point estimate. Lower is better. Nov. 14, 2019, 11:56 a.m. 82445
    Error The the truth value minus the model's point estimate. Nov. 14, 2019, 11:50 a.m. 82445
    Log score (multi bin) This is calculated by finding the natural log of probability assigned to the true and a few neighbouring bins. Higher is better. Nov. 14, 2019, 12:03 p.m. 82445
    Log score (single bin) Natural log of probability assigned to the true bin. Higher is better. Nov. 14, 2019, 12:01 p.m. 82445
    Probability Integral Transform (PIT) The probability integral transform (PIT) is a metric commonly used to evaluate the calibration of probabilistic forecasts. Nov. 14, 2019, 12:05 p.m. 82445
    (Cached file updated 2019-11-20 03:01:06)

    Score Counts (per model)

    Model Score # Score Values
    Basis Regression (epiforecast package defaults) Absolute Error 3135
    Basis Regression (epiforecast package defaults) Error 3135
    Basis Regression (epiforecast package defaults) Log score (multi bin) 3135
    Basis Regression (epiforecast package defaults) Log score (single bin) 3135
    Basis Regression (epiforecast package defaults) Probability Integral Transform (PIT) 3135
    Bayesian Model Averaging Absolute Error 3080
    Bayesian Model Averaging Error 3080
    Bayesian Model Averaging Log score (multi bin) 3080
    Bayesian Model Averaging Log score (single bin) 3080
    Bayesian Model Averaging Probability Integral Transform (PIT) 3080
    Delta Density (epiforecast package defaults) Absolute Error 3135
    Delta Density (epiforecast package defaults) Error 3135
    Delta Density (epiforecast package defaults) Log score (multi bin) 3135
    Delta Density (epiforecast package defaults) Log score (single bin) 3135
    Delta Density (epiforecast package defaults) Probability Integral Transform (PIT) 3135
    Dynamic Bayesian Model plus Absolute Error 1595
    Dynamic Bayesian Model plus Error 1595
    Dynamic Bayesian Model plus Log score (multi bin) 1595
    Dynamic Bayesian Model plus Log score (single bin) 1595
    Dynamic Bayesian Model plus Probability Integral Transform (PIT) 1595
    Dynamic Bayesian Model with a hierarchical discrepancy, 2017/2018 version Absolute Error 1540
    Dynamic Bayesian Model with a hierarchical discrepancy, 2017/2018 version Error 1540
    Dynamic Bayesian Model with a hierarchical discrepancy, 2017/2018 version Log score (multi bin) 1540
    Dynamic Bayesian Model with a hierarchical discrepancy, 2017/2018 version Log score (single bin) 1540
    Dynamic Bayesian Model with a hierarchical discrepancy, 2017/2018 version Probability Integral Transform (PIT) 1540
    Dynamic Harmonic Model with ARIMA errors Absolute Error 1595
    Dynamic Harmonic Model with ARIMA errors Error 1595
    Dynamic Harmonic Model with ARIMA errors Log score (multi bin) 1595
    Dynamic Harmonic Model with ARIMA errors Log score (single bin) 1595
    Dynamic Harmonic Model with ARIMA errors Probability Integral Transform (PIT) 1595
    Empirical Bayes (conditioning on past four weeks only, 2017/2018 version) Absolute Error 1540
    Empirical Bayes (conditioning on past four weeks only, 2017/2018 version) Error 1540
    Empirical Bayes (conditioning on past four weeks only, 2017/2018 version) Log score (multi bin) 1540
    Empirical Bayes (conditioning on past four weeks only, 2017/2018 version) Log score (single bin) 1540
    Empirical Bayes (conditioning on past four weeks only, 2017/2018 version) Probability Integral Transform (PIT) 1540
    Empirical Bayes (epiforecast package defaults, 2017/2018 version) Absolute Error 1540
    Empirical Bayes (epiforecast package defaults, 2017/2018 version) Error 1540
    Empirical Bayes (epiforecast package defaults, 2017/2018 version) Log score (multi bin) 1540
    Empirical Bayes (epiforecast package defaults, 2017/2018 version) Log score (single bin) 1540
    Empirical Bayes (epiforecast package defaults, 2017/2018 version) Probability Integral Transform (PIT) 1540
    Empirical Futures (epiforecast package defaults) Absolute Error 3135
    Empirical Futures (epiforecast package defaults) Error 3135
    Empirical Futures (epiforecast package defaults) Log score (multi bin) 3135
    Empirical Futures (epiforecast package defaults) Log score (single bin) 3135
    Empirical Futures (epiforecast package defaults) Probability Integral Transform (PIT) 3135
    Empirical Trajectories (epiforecast package defaults) Absolute Error 3135
    Empirical Trajectories (epiforecast package defaults) Error 3135
    Empirical Trajectories (epiforecast package defaults) Log score (multi bin) 3135
    Empirical Trajectories (epiforecast package defaults) Log score (single bin) 3135
    Empirical Trajectories (epiforecast package defaults) Probability Integral Transform (PIT) 3135
    Ensemble Adjustment Kalman Filter SEIRS Absolute Error 3135
    Ensemble Adjustment Kalman Filter SEIRS Error 3135
    Ensemble Adjustment Kalman Filter SEIRS Log score (multi bin) 3135
    Ensemble Adjustment Kalman Filter SEIRS Log score (single bin) 3135
    Ensemble Adjustment Kalman Filter SEIRS Probability Integral Transform (PIT) 3135
    Ensemble Adjustment Kalman Filter SIRS Absolute Error 3135
    Ensemble Adjustment Kalman Filter SIRS Error 3135
    Ensemble Adjustment Kalman Filter SIRS Log score (multi bin) 3135
    Ensemble Adjustment Kalman Filter SIRS Log score (single bin) 3135
    Ensemble Adjustment Kalman Filter SIRS Probability Integral Transform (PIT) 3135
    Ensemble Kalman Filter SEIRS Absolute Error 3190
    Ensemble Kalman Filter SEIRS Error 3190
    Ensemble Kalman Filter SEIRS Log score (multi bin) 3190
    Ensemble Kalman Filter SEIRS Log score (single bin) 3190
    Ensemble Kalman Filter SEIRS Probability Integral Transform (PIT) 3190
    Ensemble Kalman Filter SIRS Absolute Error 3190
    Ensemble Kalman Filter SIRS Error 3190
    Ensemble Kalman Filter SIRS Log score (multi bin) 3190
    Ensemble Kalman Filter SIRS Log score (single bin) 3190
    Ensemble Kalman Filter SIRS Probability Integral Transform (PIT) 3190
    Ensemble of dynamic harmonic model and historical averages Absolute Error 1595
    Ensemble of dynamic harmonic model and historical averages Error 1595
    Ensemble of dynamic harmonic model and historical averages Log score (multi bin) 1595
    Ensemble of dynamic harmonic model and historical averages Log score (single bin) 1595
    Ensemble of dynamic harmonic model and historical averages Probability Integral Transform (PIT) 1595
    Kernel Conditional Density Estimation Absolute Error 3135
    Kernel Conditional Density Estimation Error 3135
    Kernel Conditional Density Estimation Log score (multi bin) 3135
    Kernel Conditional Density Estimation Log score (single bin) 3135
    Kernel Conditional Density Estimation Probability Integral Transform (PIT) 3135
    Kernel Density Estimation Absolute Error 3190
    Kernel Density Estimation Error 3190
    Kernel Density Estimation Log score (multi bin) 3190
    Kernel Density Estimation Log score (single bin) 3190
    Kernel Density Estimation Probability Integral Transform (PIT) 3190
    Markovian Delta Density (epiforecast package defaults) Absolute Error 3135
    Markovian Delta Density (epiforecast package defaults) Error 3135
    Markovian Delta Density (epiforecast package defaults) Log score (multi bin) 3135
    Markovian Delta Density (epiforecast package defaults) Log score (single bin) 3135
    Markovian Delta Density (epiforecast package defaults) Probability Integral Transform (PIT) 3135
    Rank Histogram Filter SEIRS Absolute Error 3190
    Rank Histogram Filter SEIRS Error 3190
    Rank Histogram Filter SEIRS Log score (multi bin) 3190
    Rank Histogram Filter SEIRS Log score (single bin) 3190
    Rank Histogram Filter SEIRS Probability Integral Transform (PIT) 3190
    Rank Histogram Filter SIRS Absolute Error 3190
    Rank Histogram Filter SIRS Error 3190
    Rank Histogram Filter SIRS Log score (multi bin) 3190
    Rank Histogram Filter SIRS Log score (single bin) 3190
    Rank Histogram Filter SIRS Probability Integral Transform (PIT) 3190
    SARIMA model with seasonal differencing Absolute Error 3135
    SARIMA model with seasonal differencing Error 3135
    SARIMA model with seasonal differencing Log score (multi bin) 3135
    SARIMA model with seasonal differencing Log score (single bin) 3135
    SARIMA model with seasonal differencing Probability Integral Transform (PIT) 3135
    SARIMA model without seasonal differencing Absolute Error 3135
    SARIMA model without seasonal differencing Error 3135
    SARIMA model without seasonal differencing Log score (multi bin) 3135
    SARIMA model without seasonal differencing Log score (single bin) 3135
    SARIMA model without seasonal differencing Probability Integral Transform (PIT) 3135
    Statistical Ensemble (using the eight submitted components, with no backcasting or nowcasting, 2017/2018 version) Absolute Error 1540
    Statistical Ensemble (using the eight submitted components, with no backcasting or nowcasting, 2017/2018 version) Error 1540
    Statistical Ensemble (using the eight submitted components, with no backcasting or nowcasting, 2017/2018 version) Log score (multi bin) 1540
    Statistical Ensemble (using the eight submitted components, with no backcasting or nowcasting, 2017/2018 version) Log score (single bin) 1540
    Statistical Ensemble (using the eight submitted components, with no backcasting or nowcasting, 2017/2018 version) Probability Integral Transform (PIT) 1540
    Subtype weighted historical average model Absolute Error 1595
    Subtype weighted historical average model Error 1595
    Subtype weighted historical average model Log score (multi bin) 1595
    Subtype weighted historical average model Log score (single bin) 1595
    Subtype weighted historical average model Probability Integral Transform (PIT) 1595
    UTA - Empirical dynamic model, 2017/2018 version Absolute Error 825
    UTA - Empirical dynamic model, 2017/2018 version Error 825
    UTA - Empirical dynamic model, 2017/2018 version Log score (multi bin) 825
    UTA - Empirical dynamic model, 2017/2018 version Log score (single bin) 825
    UTA - Empirical dynamic model, 2017/2018 version Probability Integral Transform (PIT) 825
    Uniform Distribution Absolute Error 3135
    Uniform Distribution Error 3135
    Uniform Distribution Log score (multi bin) 3135
    Uniform Distribution Log score (single bin) 3135
    Uniform Distribution Probability Integral Transform (PIT) 3135
    constant-weights Absolute Error 3135
    constant-weights Error 3135
    constant-weights Log score (multi bin) 3135
    constant-weights Log score (single bin) 3135
    constant-weights Probability Integral Transform (PIT) 3135
    equal-weights Absolute Error 3080
    equal-weights Error 3080
    equal-weights Log score (multi bin) 3080
    equal-weights Log score (single bin) 3080
    equal-weights Probability Integral Transform (PIT) 3080
    target-region-weights Absolute Error 3080
    target-region-weights Error 3080
    target-region-weights Log score (multi bin) 3080
    target-region-weights Log score (single bin) 3080
    target-region-weights Probability Integral Transform (PIT) 3080
    target-type-weights Absolute Error 3135
    target-type-weights Error 3135
    target-type-weights Log score (multi bin) 3135
    target-type-weights Log score (single bin) 3135
    target-type-weights Probability Integral Transform (PIT) 3135
    target-weights Absolute Error 3135
    target-weights Error 3135
    target-weights Log score (multi bin) 3135
    target-weights Log score (single bin) 3135
    target-weights Probability Integral Transform (PIT) 3135