Table 2 Forecast evaluation for Germany and Poland (incidence scale, based on RKI/MZ data).

From: National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021

Germany
	1 wk ahead cases				2 wk ahead cases				1 wk ahead deaths				2 wk ahead deaths
Model	AE	WIS	C_0.5	C_0.95	AE	WIS	C_0.5	C_0.95	AE	WIS	C_0.5	C_0.95	AE	WIS	C_0.5	C_0.95
epiforecasts-EpiExpert	9252	5415	0.25	1.00	20,233	13,607	0.42	0.50	300	204	0.50	0.92	509	323	0.33	0.92
epiforecasts-EpiNow2	9676	6644	0.67	0.83	29,348	21,478	0.58	0.67	300	188	0.75	0.92	581	417	0.67	0.75
FIAS_FZJ-Epi1Ger	10,218	6294	0.33	0.92	25,662	16,621	0.08	0.58	436	336	0.17	0.25	655	475	0.17	0.58
IHME-CurveFit									516				656
Imperial-ensemble2									*193	*136	0.80	0.90
itwm-dSEIR	6905	4644	0.42	0.75	18,935	13,626	0.42	0.75	483	326	0.58	0.83	534	354	0.50	0.83
ITWW-county_repro	15,223	12,418	0.08	0.25	31,836	25,851	0.00	0.17	564	527	0.00	0.00	286	236	0.08	0.25
Karlen-pypm	18,532	13,629	0.50	0.92	35,010	25,385	0.25	0.83	380	232	0.42	0.92	628	394	0.08	0.83
LANL-GrowthRate	12,623	10,542	0.75	1.00	15,797	13,945	0.75	1.00	338	222	0.50	1.00	425	265	0.58	1.00
LeipzigIMISE-SECIR	9161	6376	0.17	0.67	26,650	19,185	0.17	0.58	370	281	0.58	1.00	874	636	0.33	0.75
MIT_CovidAnalytics-DELPHI	*11,910	* 8277	0.55	0.91	*22,734	*16,006	0.36	0.73	803	490	0.42	0.92	773	451	0.58	1.00
SDSC_ISG-TrendModel	7861								436
USC-SIkJalpha	13,766	9001	0.25	0.83	25,730	17,681	0.17	0.58	381	255	0.50	0.83	568	348	0.25	0.83
KIT-baseline	12,756	7953	0.42	0.92	23,785	17,330	0.17	0.58	411	277	0.58	0.92	780	525	0.17	0.67
KIT-extrapolation_baseline	8823	5715	0.50	1.00	22,858	14,679	0.33	0.75	456	269	0.33	1.00	806	490	0.33	0.83
KIT-time_series_baseline	15,583	10,281	0.25	0.75	32,306	22,026	0.25	0.67	406	263	0.67	1.00	851	601	0.50	0.92
KITCOVIDhub-inverse_wis_ensemble	8586	5294	0.58	1.00	22,000	13,824	0.50	0.83	216	149	0.75	1.00	307	207	0.75	1.00
KITCOVIDhub-mean_ensemble	8377	5277	0.75	1.00	21,825	13,662	0.50	0.92	220	152	0.58	1.00	346	219	0.75	1.00
KITCOVIDhub-median_ensemble	7344	4660	0.67	1.00	19,296	12,734	0.42	0.83	232	150	0.75	1.00	376	225	0.58	1.00

Poland
	1 wk ahead cases				2 wk ahead cases				1 wk ahead deaths				2 wk ahead deaths
Model	AE	WIS	C_0.5	C_0.95	AE	WIS	C_0.5	C_0.95	AE	WIS	C_0.5	C_0.95	AE	WIS	C_0.5	C_0.95
epiforecasts-EpiExpert	7500	4553	0.50	0.92	25,316	17,408	0.17	0.67	208	137	0.33	0.92	287	181	0.50	0.83
epiforecasts-EpiNow2	7928	5906	0.58	0.92	29,762	22,098	0.42	0.83	184	119	0.58	1.00	340	228	0.58	1.00
ICM-agentModel	*23,011	*15,824	0.27	0.91	*26,694	*18,098	0.73	1.00	*488	*294	0.73	1.00	*605	*507	0.82	1.00
IHME-CurveFit									374				520
Imperial-ensemble2									*188	*138	0.30	0.70
ITWW-county_repro	20,054	17,364	0.17	0.25	36,651	31,445	0.17	0.50	589	551	0.00	0.00	784	711	0.00	0.00
LANL-GrowthRate	8129	5787	0.83	1.00	23,269	15,240	0.58	0.92	229	137	0.17	0.83	347	216	0.33	0.92
MIMUW-StochSEIR	5705	4028	0.33	0.83	17,642	15,347	0.17	0.33	237	224	0.17	0.17	288	267	0.00	0.00
MIT_CovidAnalytics-DELPHI	*22,344	*12,912	0.20	0.90	*49,687	*33,033	0.10	0.70	*393	*244	0.55	1.00	*520	*296	0.36	1.00
MOCOS-agent1	5173	4978	0.42	0.67	15,022	11,380	0.25	0.67	158	132	0.75	1.00	203	149	0.83	1.00
SDSC_ISG-TrendModel	6323								265
USC-SIkJalpha	10,404	6919	0.33	0.83	32,822	24,436	0.17	0.50	206	133	0.33	0.92	266	168	0.42	0.92
KIT-baseline	16,407	9736	0.42	0.83	32,182	22,709	0.08	0.42	258	167	0.42	0.92	416	275	0.17	0.67
KIT-extrapolation_baseline	9448	5992	0.50	0.92	29,638	22,165	0.25	0.58	269	190	0.58	0.83	404	284	0.42	0.83
KIT-time_series_baseline	10,784	7787	0.75	0.83	30,359	21,510	0.50	0.75	300	232	0.67	0.67	467	362	0.50	0.58
KITCOVIDhub-inverse_wis_ensemble	7319	4689	0.50	1.00	23,418	15,580	0.42	0.83	150	111	0.75	1.00	197	144	0.75	1.00
KITCOVIDhub-mean_ensemble	6866	4784	0.75	1.00	23,673	15,573	0.33	0.83	141	114	0.75	1.00	173	152	0.92	1.00
KITCOVIDhub-median_ensemble	7130	4403	0.58	1.00	23,027	16,241	0.33	0.83	162	103	0.67	1.00	193	137	0.75	1.00

Summaries are based on 12 weekly forecasts per target.
Abbreviations: C_0.5, C_0.95: coverage rates of the 50 and 95% prediction intervals, AE: mean absolute error, WIS: mean weighted interval score.
^*Asterisks mark entries where scores were imputed for at least one week. Weighted interval scores and absolute errors were imputed with the worst (largest) score achieved by any other forecast for the respective target and week. Models marked thus received a pessimistic assessment of their performance. If a model covered less than two-thirds of the evaluation period, results are omitted.

Back to article page

Table 2 Forecast evaluation for Germany and Poland (incidence scale, based on RKI/MZ data).

Search

Quick links