Project score data for “CDC Retrospective Forecasts”

    Scores (all models)

    Name Description Last Update # Score Values
    Absolute Error The absolute value of the truth value minus the model's point estimate. Lower is better. Nov. 22, 2019, 12:34 p.m. 260425
    Error The the truth value minus the model's point estimate. Nov. 22, 2019, 12:01 p.m. 260425
    Log score (multi bin) This is calculated by finding the natural log of probability assigned to the true and a few neighbouring bins. Higher is better. Nov. 22, 2019, 1:18 p.m. 260425
    Log score (single bin) Natural log of probability assigned to the true bin. Higher is better. Nov. 22, 2019, 1:09 p.m. 260425
    Probability Integral Transform (PIT) The probability integral transform (PIT) is a metric commonly used to evaluate the calibration of probabilistic forecasts. Nov. 22, 2019, 1:27 p.m. 260425
    (Cached file updated 2019-12-06 04:01:11)

    Score Counts (per model)

    Model Score # Score Values
    Basis Regression (epiforecast package defaults, 2018/2019 version) Absolute Error 13475
    Basis Regression (epiforecast package defaults, 2018/2019 version) Error 13475
    Basis Regression (epiforecast package defaults, 2018/2019 version) Log score (multi bin) 13475
    Basis Regression (epiforecast package defaults, 2018/2019 version) Log score (single bin) 13475
    Basis Regression (epiforecast package defaults, 2018/2019 version) Probability Integral Transform (PIT) 13475
    Bayesian Model Averaging Absolute Error 13475
    Bayesian Model Averaging Error 13475
    Bayesian Model Averaging Log score (multi bin) 13475
    Bayesian Model Averaging Log score (single bin) 13475
    Bayesian Model Averaging Probability Integral Transform (PIT) 13475
    Cheetah (Ensemble of dynamic harmonic model and historical averages) Absolute Error 13475
    Cheetah (Ensemble of dynamic harmonic model and historical averages) Error 13475
    Cheetah (Ensemble of dynamic harmonic model and historical averages) Log score (multi bin) 13475
    Cheetah (Ensemble of dynamic harmonic model and historical averages) Log score (single bin) 13475
    Cheetah (Ensemble of dynamic harmonic model and historical averages) Probability Integral Transform (PIT) 13475
    Delta Density (epiforecast package defaults, 2018/2019 version) Absolute Error 13475
    Delta Density (epiforecast package defaults, 2018/2019 version) Error 13475
    Delta Density (epiforecast package defaults, 2018/2019 version) Log score (multi bin) 13475
    Delta Density (epiforecast package defaults, 2018/2019 version) Log score (single bin) 13475
    Delta Density (epiforecast package defaults, 2018/2019 version) Probability Integral Transform (PIT) 13475
    Dynamic Bayesian Model with a hierarchical discrepancy + Google-based nowcasts Absolute Error 12650
    Dynamic Bayesian Model with a hierarchical discrepancy + Google-based nowcasts Error 12650
    Dynamic Bayesian Model with a hierarchical discrepancy + Google-based nowcasts Log score (multi bin) 12650
    Dynamic Bayesian Model with a hierarchical discrepancy + Google-based nowcasts Log score (single bin) 12650
    Dynamic Bayesian Model with a hierarchical discrepancy + Google-based nowcasts Probability Integral Transform (PIT) 12650
    Empirical Futures (epiforecast package defaults, 2018/2019 version) Absolute Error 13475
    Empirical Futures (epiforecast package defaults, 2018/2019 version) Error 13475
    Empirical Futures (epiforecast package defaults, 2018/2019 version) Log score (multi bin) 13475
    Empirical Futures (epiforecast package defaults, 2018/2019 version) Log score (single bin) 13475
    Empirical Futures (epiforecast package defaults, 2018/2019 version) Probability Integral Transform (PIT) 13475
    Empirical Trajectories (epiforecast package defaults, 2018/2019 version) Absolute Error 13475
    Empirical Trajectories (epiforecast package defaults, 2018/2019 version) Error 13475
    Empirical Trajectories (epiforecast package defaults, 2018/2019 version) Log score (multi bin) 13475
    Empirical Trajectories (epiforecast package defaults, 2018/2019 version) Log score (single bin) 13475
    Empirical Trajectories (epiforecast package defaults, 2018/2019 version) Probability Integral Transform (PIT) 13475
    Ensemble Adjustment Kalman Filter SEIRS Absolute Error 12650
    Ensemble Adjustment Kalman Filter SEIRS Error 12650
    Ensemble Adjustment Kalman Filter SEIRS Log score (multi bin) 12650
    Ensemble Adjustment Kalman Filter SEIRS Log score (single bin) 12650
    Ensemble Adjustment Kalman Filter SEIRS Probability Integral Transform (PIT) 12650
    Ensemble Adjustment Kalman Filter SIRS Absolute Error 12650
    Ensemble Adjustment Kalman Filter SIRS Error 12650
    Ensemble Adjustment Kalman Filter SIRS Log score (multi bin) 12650
    Ensemble Adjustment Kalman Filter SIRS Log score (single bin) 12650
    Ensemble Adjustment Kalman Filter SIRS Probability Integral Transform (PIT) 12650
    Ensemble Kalman Filter SEIRS Absolute Error 12650
    Ensemble Kalman Filter SEIRS Error 12650
    Ensemble Kalman Filter SEIRS Log score (multi bin) 12650
    Ensemble Kalman Filter SEIRS Log score (single bin) 12650
    Ensemble Kalman Filter SEIRS Probability Integral Transform (PIT) 12650
    Ensemble Kalman Filter SIRS Absolute Error 12650
    Ensemble Kalman Filter SIRS Error 12650
    Ensemble Kalman Filter SIRS Log score (multi bin) 12650
    Ensemble Kalman Filter SIRS Log score (single bin) 12650
    Ensemble Kalman Filter SIRS Probability Integral Transform (PIT) 12650
    Kernel Density Estimation Absolute Error 12650
    Kernel Density Estimation Error 12650
    Kernel Density Estimation Log score (multi bin) 12650
    Kernel Density Estimation Log score (single bin) 12650
    Kernel Density Estimation Probability Integral Transform (PIT) 12650
    Kudu (Subtype weighted historical average model) Absolute Error 13475
    Kudu (Subtype weighted historical average model) Error 13475
    Kudu (Subtype weighted historical average model) Log score (multi bin) 13475
    Kudu (Subtype weighted historical average model) Log score (single bin) 13475
    Kudu (Subtype weighted historical average model) Probability Integral Transform (PIT) 13475
    Markovian Delta Density (epiforecast package defaults, 2018/2019 version) Absolute Error 13475
    Markovian Delta Density (epiforecast package defaults, 2018/2019 version) Error 13475
    Markovian Delta Density (epiforecast package defaults, 2018/2019 version) Log score (multi bin) 13475
    Markovian Delta Density (epiforecast package defaults, 2018/2019 version) Log score (single bin) 13475
    Markovian Delta Density (epiforecast package defaults, 2018/2019 version) Probability Integral Transform (PIT) 13475
    Rank Histogram Filter SEIRS Absolute Error 12650
    Rank Histogram Filter SEIRS Error 12650
    Rank Histogram Filter SEIRS Log score (multi bin) 12650
    Rank Histogram Filter SEIRS Log score (single bin) 12650
    Rank Histogram Filter SEIRS Probability Integral Transform (PIT) 12650
    Rank Histogram Filter SIRS Absolute Error 12650
    Rank Histogram Filter SIRS Error 12650
    Rank Histogram Filter SIRS Log score (multi bin) 12650
    Rank Histogram Filter SIRS Log score (single bin) 12650
    Rank Histogram Filter SIRS Probability Integral Transform (PIT) 12650
    SARIMA model with seasonal differencing Absolute Error 12650
    SARIMA model with seasonal differencing Error 12650
    SARIMA model with seasonal differencing Log score (multi bin) 12650
    SARIMA model with seasonal differencing Log score (single bin) 12650
    SARIMA model with seasonal differencing Probability Integral Transform (PIT) 12650
    SARIMA model without seasonal differencing Absolute Error 12650
    SARIMA model without seasonal differencing Error 12650
    SARIMA model without seasonal differencing Log score (multi bin) 12650
    SARIMA model without seasonal differencing Log score (single bin) 12650
    SARIMA model without seasonal differencing Probability Integral Transform (PIT) 12650
    Springbok (Dynamic Harmonic Model with ARIMA errors) Absolute Error 13475
    Springbok (Dynamic Harmonic Model with ARIMA errors) Error 13475
    Springbok (Dynamic Harmonic Model with ARIMA errors) Log score (multi bin) 13475
    Springbok (Dynamic Harmonic Model with ARIMA errors) Log score (single bin) 13475
    Springbok (Dynamic Harmonic Model with ARIMA errors) Probability Integral Transform (PIT) 13475
    Uniform Distribution Absolute Error 12650
    Uniform Distribution Error 12650
    Uniform Distribution Log score (multi bin) 12650
    Uniform Distribution Log score (single bin) 12650
    Uniform Distribution Probability Integral Transform (PIT) 12650