MOS vs NWS Statistics Page

Link to paper (submitted to Weather & Forecasting, 9/13/04) describing the MOS vs. NWS Statistics
Link to paper presented at 2004 AMS annual meeting describing the MOS vs. NWS Statistics
Link to 2004 AMS annual meeting MOS vs. NWS Statistics Presentation .ppt file

Forecasts:
NWS National Weather Service
GMOS GFS MOS (prior to 12/15/03: AVN MOS)
EMOS Eta MOS
NMOS NGM MOS
CMOS Average of AMOS, EMOS and NMOS
CMOS-GE Average of GMOS and EMOS
WMOS Weighted average of AMOS and EMOS and NMOS

Temperature | Precipitation

MAE (all stations), 25-Jul-2003 - 03-Mar-2009
MAX-T,
day-1
(period 1)		 n MIN-T,
day-2
(period 2)		 n MAX-T,
day-2
(period 3)		 n MIN-T,
day-3
(period 4)		 n
NWS 2.20 54177 2.44 54620 2.71 54032 2.78 54452
CMOS 2.32 54177 2.45 54620 2.80 54032 2.79 54452
CMOS_GE 2.30 54177 2.37 54620 2.72 54032 2.64 54452
WMOS 2.22 54177 2.36 54620 2.68 54032 2.67 54452
GMOS 2.51 54177 2.50 54620 2.91 54032 2.74 54452
EMOS 2.56 54177 2.56 54620 3.12 54032 2.93 54452
NMOS 2.92 54177 3.04 54620 3.54 54032 3.60 54452
BIAS (all stations), 25-Jul-2003 - 03-Mar-2009
MAX-T,
day-1
(period 1)		 n MIN-T,
day-2
(period 2)		 n MAX-T,
day-2
(period 3)		 n MIN-T,
day-3
(period 4)		 n
NWS -0.03 54177 -0.20 54620 -0.17 54032 -0.31 54452
CMOS 0.07 54177 -0.25 54620 -0.12 54032 -0.34 54452
CMOS_GE 0.20 54177 0.20 54620 0.06 54032 0.16 54452
WMOS 0.10 54177 -0.15 54620 -0.06 54032 -0.23 54452
GMOS -0.10 54177 -0.11 54620 -0.17 54032 -0.21 54452
EMOS 0.00 54177 0.01 54620 -0.20 54032 0.03 54452
NMOS 0.28 54177 -0.67 54620 0.02 54032 -0.83 54452
Temperature departure from climatology >= +/- 20° F, MAE (all stations), 25-Jul-2003 - 03-Mar-2009
MAX-T,
day-1
(period 1)		 n MIN-T,
day-2
(period 2)		 n MAX-T,
day-2
(period 3)		 n MIN-T,
day-3
(period 4)		 n
NWS 3.53 1526 3.92 1228 5.19 1446 5.03 1248
CMOS 4.25 1526 4.53 1228 5.84 1446 5.84 1248
CMOS_GE 3.98 1526 3.93 1228 5.22 1446 4.85 1248
WMOS 4.13 1526 4.36 1228 5.67 1446 5.46 1248
GMOS 4.53 1526 4.17 1228 5.72 1446 4.93 1248
EMOS 3.95 1526 4.21 1228 5.25 1446 5.31 1248
NMOS 5.23 1526 5.67 1228 7.39 1446 7.79 1248
1-Day Temperature change >= +/-10° F, MAE (all stations), 25-Jul-2003 - 03-Mar-2009
MAX-T,
day-1
(period 1)		 n MIN-T,
day-2
(period 2)		 n MAX-T,
day-2
(period 3)		 n MIN-T,
day-3
(period 4)		 n
NWS 3.21 8851 3.33 7835 3.79 8757 3.89 7805
CMOS 3.34 8851 3.43 7835 3.90 8757 4.11 7805
CMOS_GE 3.24 8851 3.28 7835 3.72 8757 3.77 7805
WMOS 3.24 8851 3.31 7835 3.78 8757 3.91 7805
GMOS 3.52 8851 3.51 7835 3.91 8757 3.82 7805
EMOS 3.42 8851 3.37 7835 4.09 8757 4.11 7805
NMOS 4.04 8851 4.20 7835 4.87 8757 5.34 7805
Number of days forecast is most accurate (all stations), 25-Jul-2003 - 03-Mar-2009
MAX-T,
day-1
(period 1)		 MIN-T,
day-2
(period 2)		 MAX-T,
day-2
(period 3)		 MIN-T,
day-3
(period 4)
NWS 20938 20121 18692 18759
CMOS 46495 46525 43613 44083
CMOS_GE 13949 13705 12195 12251
WMOS 47952 49659 45075 46859
GMOS 15002 16665 13900 16010
EMOS 47124 49067 45056 47380
NMOS 14644 14223 12718 12846
Number of days forecast is least accurate (all stations), 25-Jul-2003 - 03-Mar-2009
MAX-T,
day-1
(period 1)		 MIN-T,
day-2
(period 2)		 MAX-T,
day-2
(period 3)		 MIN-T,
day-3
(period 4)
NWS 13595 16190 12750 15364
CMOS 38343 41466 35053 38019
CMOS_GE 4623 5251 3384 4188
WMOS 41717 44751 36806 39813
GMOS 7971 9522 6303 7865
EMOS 41440 42567 36555 37835
NMOS 5141 5736 3879 4668
Number of days |forecast - obs| > 5° F (all stations), 25-Jul-2003 - 03-Mar-2009
MAX-T,
day-1
(period 1)		 MIN-T,
day-2
(period 2)		 MAX-T,
day-2
(period 3)		 MIN-T,
day-3
(period 4)
NWS 3599 4452 6445 6471
CMOS 4114 4409 6754 6527
CMOS_GE 4029 4003 6316 5611
WMOS 3705 3965 6173 5817
GMOS 5162 4861 7575 6218
EMOS 5418 5071 8638 7399
NMOS 6443 7389 10215 11021

Temperature | Precipitation

Brier Scores (all stations), 25-Jul-2003 - 03-Mar-2009
Day 1
(period 1)		 n Day 2
(period 2)		 n Day 2
(period 3)		 n Day 3
(period 4)		 n
NWS 0.084 51037 0.087 51710 0.098 50927 0.098 51433
CMOS 0.082 51037 0.084 51710 0.095 50927 0.098 51433
CMOS_GE 0.080 51037 0.083 51710 0.093 50927 0.095 51433
WMOS 0.081 51037 0.083 51710 0.094 50927 0.097 51433
GMOS 0.084 51037 0.085 51710 0.095 50927 0.096 51433
EMOS 0.084 51037 0.088 51710 0.100 50927 0.104 51433
NMOS 0.092 51037 0.095 51710 0.109 50927 0.115 51433
Number of days each forecast is least accurate (all stations), 25-Jul-2003 - 03-Mar-2009
Day 1
(period 1)		 Day 2
(period 2)		 Day 2
(period 3)		 Day 3
(period 4)
NWS 12828 12338 12006 10880
CMOS 51037 51710 50927 51433
CMOS_GE 2607 2407 1834 1692
WMOS 51037 51710 50927 51433
GMOS 7015 6947 5693 4991
EMOS 51037 51710 50927 51433
NMOS 2821 2545 1985 1785
Brier Scores, precip >= 3X daily climo value, (all stations), 25-Jul-2003 - 03-Mar-2009
Day 1
(period 1)		 n Day 2
(period 2)		 n Day 2
(period 3)		 n Day 3
(period 4)		 n
NWS 0.136 4042 0.170 3742 0.210 3860 0.251 3719
CMOS 0.143 4042 0.171 3742 0.217 3860 0.256 3719
CMOS_GE 0.133 4042 0.152 3742 0.195 3860 0.221 3719
WMOS 0.142 4042 0.169 3742 0.214 3860 0.251 3719
GMOS 0.145 4042 0.161 3742 0.204 3860 0.222 3719
EMOS 0.136 4042 0.162 3742 0.207 3860 0.246 3719
NMOS 0.174 4042 0.227 3742 0.279 3860 0.356 3719
Brier Scores, departure from climatology >= 0.20", (all stations), 25-Jul-2003 - 03-Mar-2009
Day 1
(period 1)		 n Day 2
(period 2)		 n Day 2
(period 3)		 n Day 3
(period 4)		 n
NWS 0.126 2947 0.148 2713 0.193 2840 0.225 2696
CMOS 0.131 2947 0.147 2713 0.199 2840 0.229 2696
CMOS_GE 0.121 2947 0.127 2713 0.176 2840 0.191 2696
WMOS 0.130 2947 0.145 2713 0.196 2840 0.223 2696
GMOS 0.131 2947 0.134 2713 0.182 2840 0.188 2696
EMOS 0.125 2947 0.138 2713 0.191 2840 0.219 2696
NMOS 0.163 2947 0.206 2713 0.264 2840 0.337 2696
Brier Scores, precip >= 0.00", (all stations), 25-Jul-2003 - 03-Mar-2009
Day 1
(period 1)		 n Day 2
(period 2)		 n Day 2
(period 3)		 n Day 3
(period 4)		 n
NWS 0.083 40198 0.085 41484 0.096 40145 0.096 41280
CMOS 0.080 40198 0.082 41484 0.093 40145 0.095 41280
CMOS_GE 0.079 40198 0.081 41484 0.091 40145 0.093 41280
WMOS 0.079 40198 0.081 41484 0.092 40145 0.094 41280
GMOS 0.082 40198 0.084 41484 0.093 40145 0.094 41280
EMOS 0.083 40198 0.087 41484 0.098 40145 0.102 41280
NMOS 0.090 40198 0.093 41484 0.107 40145 0.112 41280
Brier Scores, precip >= 0.01", (all stations), 25-Jul-2003 - 03-Mar-2009
Day 1
(period 1)		 n Day 2
(period 2)		 n Day 2
(period 3)		 n Day 3
(period 4)		 n
NWS 0.089 9129 0.094 8602 0.104 9085 0.107 8533
CMOS 0.088 9129 0.091 8602 0.102 9085 0.107 8533
CMOS_GE 0.086 9129 0.089 8602 0.100 9085 0.103 8533
WMOS 0.087 9129 0.090 8602 0.102 9085 0.105 8533
GMOS 0.090 9129 0.092 8602 0.102 9085 0.103 8533
EMOS 0.090 9129 0.095 8602 0.107 9085 0.114 8533
NMOS 0.099 9129 0.103 8602 0.118 9085 0.126 8533
Number of days |forecast - obs| > 50.00 (all stations), 5-Jul-2003 - 03-Mar-2009 Number of days |forecast - obs| > 0.50 (all stations),
Day 1
(period 1)		 Day 2
(period 2)		 Day 2
(period 3)		 Day 3
(period 4)
NWS 4906 5234 5729 6020
CMOS 5676 5890 6769 6936
CMOS_GE 5567 5756 6537 6726
WMOS 5585 5840 6663 6826
GMOS 5832 6031 6846 6756
EMOS 5887 6171 7061 7312
NMOS 6547 6820 7897 8332