Combine mvgam forecasts into evenly weighted ensembles
Source:R/ensemble.R
ensemble.mvgam_forecast.RdGenerate evenly weighted ensemble forecast distributions from mvgam_forecast objects
Arguments
- object
listobject of classmvgam_forecast. Seeforecast.mvgam()- ...
More
mvgam_forecastobjects.- ndraws
Positive integer specifying the number of draws to use from each forecast distribution for creating the ensemble. If some of the ensemble members have fewer draws than
ndraws, their forecast distributions will be resampled with replacement to achieve the correct number of draws
Value
An object of class mvgam_forecast containing the ensemble predictions. This
object can be readily used with the supplied S3 functions plot and score
Details
It is widely recognised in the forecasting literature that combining forecasts
from different models often results in improved forecast accuracy. The simplest way to create
an ensemble is to use evenly weighted combinations of forecasts from the different models.
This is straightforward to do in a Bayesian setting with mvgam as the posterior MCMC draws
contained in each mvgam_forecast object will already implicitly capture correlations among
the temporal posterior predictions.
Examples
# \donttest{
# Simulate some series and fit a few competing dynamic models
set.seed(1)
simdat <- sim_mvgam(n_series = 1,
prop_trend = 0.6,
mu = 1)
plot_mvgam_series(data = simdat$data_train,
newdata = simdat$data_test)
m1 <- mvgam(y ~ 1,
trend_formula = ~ time +
s(season, bs = 'cc', k = 9),
trend_model = AR(p = 1),
noncentred = TRUE,
data = simdat$data_train,
newdata = simdat$data_test)
#> Compiling Stan program using cmdstanr
#>
#> In file included from stan/lib/stan_math/stan/math/prim/prob/von_mises_lccdf.hpp:5,
#> from stan/lib/stan_math/stan/math/prim/prob/von_mises_ccdf_log.hpp:4,
#> from stan/lib/stan_math/stan/math/prim/prob.hpp:359,
#> from stan/lib/stan_math/stan/math/prim.hpp:16,
#> from stan/lib/stan_math/stan/math/rev.hpp:16,
#> from stan/lib/stan_math/stan/math.hpp:19,
#> from stan/src/stan/model/model_header.hpp:4,
#> from C:/Users/uqnclar2/AppData/Local/Temp/RtmpuihtV8/model-49f869091ff5.hpp:2:
#> stan/lib/stan_math/stan/math/prim/prob/von_mises_cdf.hpp: In function 'stan::return_type_t<T_x, T_sigma, T_l> stan::math::von_mises_cdf(const T_x&, const T_mu&, const T_k&)':
#> stan/lib/stan_math/stan/math/prim/prob/von_mises_cdf.hpp:194: note: '-Wmisleading-indentation' is disabled from this point onwards, since column-tracking was disabled due to the size of the code/headers
#> 194 | if (cdf_n < 0.0)
#> |
#> stan/lib/stan_math/stan/math/prim/prob/von_mises_cdf.hpp:194: note: adding '-flarge-source-files' will allow for more column-tracking support, at the expense of compilation time and memory
#> Start sampling
#> Running MCMC with 4 parallel chains...
#>
#> Chain 1 Iteration: 1 / 1000 [ 0%] (Warmup)
#> Chain 2 Iteration: 1 / 1000 [ 0%] (Warmup)
#> Chain 3 Iteration: 1 / 1000 [ 0%] (Warmup)
#> Chain 4 Iteration: 1 / 1000 [ 0%] (Warmup)
#> Chain 3 Iteration: 100 / 1000 [ 10%] (Warmup)
#> Chain 2 Iteration: 100 / 1000 [ 10%] (Warmup)
#> Chain 3 Iteration: 200 / 1000 [ 20%] (Warmup)
#> Chain 4 Iteration: 100 / 1000 [ 10%] (Warmup)
#> Chain 2 Iteration: 200 / 1000 [ 20%] (Warmup)
#> Chain 3 Iteration: 300 / 1000 [ 30%] (Warmup)
#> Chain 4 Iteration: 200 / 1000 [ 20%] (Warmup)
#> Chain 3 Iteration: 400 / 1000 [ 40%] (Warmup)
#> Chain 2 Iteration: 300 / 1000 [ 30%] (Warmup)
#> Chain 4 Iteration: 300 / 1000 [ 30%] (Warmup)
#> Chain 3 Iteration: 500 / 1000 [ 50%] (Warmup)
#> Chain 2 Iteration: 400 / 1000 [ 40%] (Warmup)
#> Chain 3 Iteration: 501 / 1000 [ 50%] (Sampling)
#> Chain 4 Iteration: 400 / 1000 [ 40%] (Warmup)
#> Chain 1 Iteration: 100 / 1000 [ 10%] (Warmup)
#> Chain 3 Iteration: 600 / 1000 [ 60%] (Sampling)
#> Chain 2 Iteration: 500 / 1000 [ 50%] (Warmup)
#> Chain 4 Iteration: 500 / 1000 [ 50%] (Warmup)
#> Chain 2 Iteration: 501 / 1000 [ 50%] (Sampling)
#> Chain 3 Iteration: 700 / 1000 [ 70%] (Sampling)
#> Chain 4 Iteration: 501 / 1000 [ 50%] (Sampling)
#> Chain 2 Iteration: 600 / 1000 [ 60%] (Sampling)
#> Chain 3 Iteration: 800 / 1000 [ 80%] (Sampling)
#> Chain 4 Iteration: 600 / 1000 [ 60%] (Sampling)
#> Chain 1 Iteration: 200 / 1000 [ 20%] (Warmup)
#> Chain 2 Iteration: 700 / 1000 [ 70%] (Sampling)
#> Chain 3 Iteration: 900 / 1000 [ 90%] (Sampling)
#> Chain 4 Iteration: 700 / 1000 [ 70%] (Sampling)
#> Chain 1 Iteration: 300 / 1000 [ 30%] (Warmup)
#> Chain 2 Iteration: 800 / 1000 [ 80%] (Sampling)
#> Chain 3 Iteration: 1000 / 1000 [100%] (Sampling)
#> Chain 3 finished in 5.2 seconds.
#> Chain 2 Iteration: 900 / 1000 [ 90%] (Sampling)
#> Chain 1 Iteration: 400 / 1000 [ 40%] (Warmup)
#> Chain 4 Iteration: 800 / 1000 [ 80%] (Sampling)
#> Chain 2 Iteration: 1000 / 1000 [100%] (Sampling)
#> Chain 2 finished in 5.7 seconds.
#> Chain 1 Iteration: 500 / 1000 [ 50%] (Warmup)
#> Chain 1 Iteration: 501 / 1000 [ 50%] (Sampling)
#> Chain 4 Iteration: 900 / 1000 [ 90%] (Sampling)
#> Chain 1 Iteration: 600 / 1000 [ 60%] (Sampling)
#> Chain 4 Iteration: 1000 / 1000 [100%] (Sampling)
#> Chain 4 finished in 6.2 seconds.
#> Chain 1 Iteration: 700 / 1000 [ 70%] (Sampling)
#> Chain 1 Iteration: 800 / 1000 [ 80%] (Sampling)
#> Chain 1 Iteration: 900 / 1000 [ 90%] (Sampling)
#> Chain 1 Iteration: 1000 / 1000 [100%] (Sampling)
#> Chain 1 finished in 7.0 seconds.
#>
#> All 4 chains finished successfully.
#> Mean chain execution time: 6.0 seconds.
#> Total execution time: 7.1 seconds.
#>
m2 <- mvgam(y ~ time,
trend_model = RW(),
noncentred = TRUE,
data = simdat$data_train,
newdata = simdat$data_test)
#> Compiling Stan program using cmdstanr
#>
#> In file included from stan/lib/stan_math/stan/math/prim/prob/von_mises_lccdf.hpp:5,
#> from stan/lib/stan_math/stan/math/prim/prob/von_mises_ccdf_log.hpp:4,
#> from stan/lib/stan_math/stan/math/prim/prob.hpp:359,
#> from stan/lib/stan_math/stan/math/prim.hpp:16,
#> from stan/lib/stan_math/stan/math/rev.hpp:16,
#> from stan/lib/stan_math/stan/math.hpp:19,
#> from stan/src/stan/model/model_header.hpp:4,
#> from C:/Users/uqnclar2/AppData/Local/Temp/RtmpuihtV8/model-49f8254fb78.hpp:2:
#> stan/lib/stan_math/stan/math/prim/prob/von_mises_cdf.hpp: In function 'stan::return_type_t<T_x, T_sigma, T_l> stan::math::von_mises_cdf(const T_x&, const T_mu&, const T_k&)':
#> stan/lib/stan_math/stan/math/prim/prob/von_mises_cdf.hpp:194: note: '-Wmisleading-indentation' is disabled from this point onwards, since column-tracking was disabled due to the size of the code/headers
#> 194 | if (cdf_n < 0.0)
#> |
#> stan/lib/stan_math/stan/math/prim/prob/von_mises_cdf.hpp:194: note: adding '-flarge-source-files' will allow for more column-tracking support, at the expense of compilation time and memory
#> Start sampling
#> Running MCMC with 4 parallel chains...
#>
#> Chain 1 Iteration: 1 / 1000 [ 0%] (Warmup)
#> Chain 2 Iteration: 1 / 1000 [ 0%] (Warmup)
#> Chain 3 Iteration: 1 / 1000 [ 0%] (Warmup)
#> Chain 3 Iteration: 100 / 1000 [ 10%] (Warmup)
#> Chain 4 Iteration: 1 / 1000 [ 0%] (Warmup)
#> Chain 3 Iteration: 200 / 1000 [ 20%] (Warmup)
#> Chain 3 Iteration: 300 / 1000 [ 30%] (Warmup)
#> Chain 3 Iteration: 400 / 1000 [ 40%] (Warmup)
#> Chain 3 Iteration: 500 / 1000 [ 50%] (Warmup)
#> Chain 3 Iteration: 501 / 1000 [ 50%] (Sampling)
#> Chain 3 Iteration: 600 / 1000 [ 60%] (Sampling)
#> Chain 3 Iteration: 700 / 1000 [ 70%] (Sampling)
#> Chain 3 Iteration: 800 / 1000 [ 80%] (Sampling)
#> Chain 2 Iteration: 100 / 1000 [ 10%] (Warmup)
#> Chain 3 Iteration: 900 / 1000 [ 90%] (Sampling)
#> Chain 3 Iteration: 1000 / 1000 [100%] (Sampling)
#> Chain 3 finished in 1.4 seconds.
#> Chain 1 Iteration: 100 / 1000 [ 10%] (Warmup)
#> Chain 2 Iteration: 200 / 1000 [ 20%] (Warmup)
#> Chain 2 Iteration: 300 / 1000 [ 30%] (Warmup)
#> Chain 2 Iteration: 400 / 1000 [ 40%] (Warmup)
#> Chain 1 Iteration: 200 / 1000 [ 20%] (Warmup)
#> Chain 1 Iteration: 300 / 1000 [ 30%] (Warmup)
#> Chain 1 Iteration: 400 / 1000 [ 40%] (Warmup)
#> Chain 1 Iteration: 500 / 1000 [ 50%] (Warmup)
#> Chain 1 Iteration: 501 / 1000 [ 50%] (Sampling)
#> Chain 2 Iteration: 500 / 1000 [ 50%] (Warmup)
#> Chain 2 Iteration: 501 / 1000 [ 50%] (Sampling)
#> Chain 2 Iteration: 600 / 1000 [ 60%] (Sampling)
#> Chain 2 Iteration: 700 / 1000 [ 70%] (Sampling)
#> Chain 4 Iteration: 100 / 1000 [ 10%] (Warmup)
#> Chain 1 Iteration: 600 / 1000 [ 60%] (Sampling)
#> Chain 1 Iteration: 700 / 1000 [ 70%] (Sampling)
#> Chain 1 Iteration: 800 / 1000 [ 80%] (Sampling)
#> Chain 2 Iteration: 800 / 1000 [ 80%] (Sampling)
#> Chain 2 Iteration: 900 / 1000 [ 90%] (Sampling)
#> Chain 4 Iteration: 200 / 1000 [ 20%] (Warmup)
#> Chain 4 Iteration: 300 / 1000 [ 30%] (Warmup)
#> Chain 4 Iteration: 400 / 1000 [ 40%] (Warmup)
#> Chain 4 Iteration: 500 / 1000 [ 50%] (Warmup)
#> Chain 4 Iteration: 501 / 1000 [ 50%] (Sampling)
#> Chain 4 Iteration: 600 / 1000 [ 60%] (Sampling)
#> Chain 1 Iteration: 900 / 1000 [ 90%] (Sampling)
#> Chain 1 Iteration: 1000 / 1000 [100%] (Sampling)
#> Chain 2 Iteration: 1000 / 1000 [100%] (Sampling)
#> Chain 4 Iteration: 700 / 1000 [ 70%] (Sampling)
#> Chain 4 Iteration: 800 / 1000 [ 80%] (Sampling)
#> Chain 1 finished in 1.8 seconds.
#> Chain 2 finished in 1.8 seconds.
#> Chain 4 Iteration: 900 / 1000 [ 90%] (Sampling)
#> Chain 4 Iteration: 1000 / 1000 [100%] (Sampling)
#> Chain 4 finished in 1.9 seconds.
#>
#> All 4 chains finished successfully.
#> Mean chain execution time: 1.7 seconds.
#> Total execution time: 2.1 seconds.
#>
# Calculate forecast distributions for each model
fc1 <- forecast(m1)
fc2 <- forecast(m2)
# Generate the ensemble forecast
ensemble_fc <- ensemble(fc1, fc2)
# Plot forecasts
plot(fc1)
#> Out of sample DRPS:
#> 53.167638
plot(fc2)
#> Out of sample DRPS:
#> 50.908428
plot(ensemble_fc)
#> Out of sample DRPS:
#> 45.89366292
# Score forecasts
score(fc1)
#> $series_1
#> score in_interval interval_width eval_horizon score_type
#> 1 0.8890420 1 0.9 1 crps
#> 2 0.4492823 1 0.9 2 crps
#> 3 1.5185570 1 0.9 3 crps
#> 4 0.8989273 1 0.9 4 crps
#> 5 1.8055245 1 0.9 5 crps
#> 6 3.9993945 0 0.9 6 crps
#> 7 3.4538378 1 0.9 7 crps
#> 8 2.4664340 1 0.9 8 crps
#> 9 1.3232315 1 0.9 9 crps
#> 10 2.5988078 1 0.9 10 crps
#> 11 2.2445383 1 0.9 11 crps
#> 12 0.5999212 1 0.9 12 crps
#> 13 0.8349463 1 0.9 13 crps
#> 14 0.7671573 1 0.9 14 crps
#> 15 0.8676385 1 0.9 15 crps
#> 16 1.7038020 1 0.9 16 crps
#> 17 4.3611705 1 0.9 17 crps
#> 18 1.2247887 1 0.9 18 crps
#> 19 3.0207840 1 0.9 19 crps
#> 20 6.0145475 1 0.9 20 crps
#> 21 3.5215620 1 0.9 21 crps
#> 22 5.4205837 1 0.9 22 crps
#> 23 1.0779020 1 0.9 23 crps
#> 24 1.3068293 1 0.9 24 crps
#> 25 0.7984283 1 0.9 25 crps
#>
#> $all_series
#> score eval_horizon score_type
#> 1 0.8890420 1 sum_crps
#> 2 0.4492823 2 sum_crps
#> 3 1.5185570 3 sum_crps
#> 4 0.8989273 4 sum_crps
#> 5 1.8055245 5 sum_crps
#> 6 3.9993945 6 sum_crps
#> 7 3.4538378 7 sum_crps
#> 8 2.4664340 8 sum_crps
#> 9 1.3232315 9 sum_crps
#> 10 2.5988078 10 sum_crps
#> 11 2.2445383 11 sum_crps
#> 12 0.5999212 12 sum_crps
#> 13 0.8349463 13 sum_crps
#> 14 0.7671573 14 sum_crps
#> 15 0.8676385 15 sum_crps
#> 16 1.7038020 16 sum_crps
#> 17 4.3611705 17 sum_crps
#> 18 1.2247887 18 sum_crps
#> 19 3.0207840 19 sum_crps
#> 20 6.0145475 20 sum_crps
#> 21 3.5215620 21 sum_crps
#> 22 5.4205837 22 sum_crps
#> 23 1.0779020 23 sum_crps
#> 24 1.3068293 24 sum_crps
#> 25 0.7984283 25 sum_crps
#>
score(fc2)
#> $series_1
#> score in_interval interval_width eval_horizon score_type
#> 1 0.6874878 1 0.9 1 crps
#> 2 1.6471578 1 0.9 2 crps
#> 3 3.0363160 1 0.9 3 crps
#> 4 1.7709870 1 0.9 4 crps
#> 5 1.7643665 1 0.9 5 crps
#> 6 1.9566545 1 0.9 6 crps
#> 7 1.1733258 1 0.9 7 crps
#> 8 1.7089578 1 0.9 8 crps
#> 9 1.3833360 1 0.9 9 crps
#> 10 1.2870628 1 0.9 10 crps
#> 11 2.6310383 1 0.9 11 crps
#> 12 2.1334693 1 0.9 12 crps
#> 13 1.7572383 1 0.9 13 crps
#> 14 1.8325305 1 0.9 14 crps
#> 15 1.6366635 1 0.9 15 crps
#> 16 1.5081563 1 0.9 16 crps
#> 17 3.9940062 1 0.9 17 crps
#> 18 2.3000615 1 0.9 18 crps
#> 19 2.2467020 1 0.9 19 crps
#> 20 1.6397138 1 0.9 20 crps
#> 21 1.7773023 1 0.9 21 crps
#> 22 2.4609862 1 0.9 22 crps
#> 23 1.9270123 1 0.9 23 crps
#> 24 3.8463365 1 0.9 24 crps
#> 25 2.8015595 1 0.9 25 crps
#>
#> $all_series
#> score eval_horizon score_type
#> 1 0.6874878 1 sum_crps
#> 2 1.6471578 2 sum_crps
#> 3 3.0363160 3 sum_crps
#> 4 1.7709870 4 sum_crps
#> 5 1.7643665 5 sum_crps
#> 6 1.9566545 6 sum_crps
#> 7 1.1733258 7 sum_crps
#> 8 1.7089578 8 sum_crps
#> 9 1.3833360 9 sum_crps
#> 10 1.2870628 10 sum_crps
#> 11 2.6310383 11 sum_crps
#> 12 2.1334693 12 sum_crps
#> 13 1.7572383 13 sum_crps
#> 14 1.8325305 14 sum_crps
#> 15 1.6366635 15 sum_crps
#> 16 1.5081563 16 sum_crps
#> 17 3.9940062 17 sum_crps
#> 18 2.3000615 18 sum_crps
#> 19 2.2467020 19 sum_crps
#> 20 1.6397138 20 sum_crps
#> 21 1.7773023 21 sum_crps
#> 22 2.4609862 22 sum_crps
#> 23 1.9270123 23 sum_crps
#> 24 3.8463365 24 sum_crps
#> 25 2.8015595 25 sum_crps
#>
score(ensemble_fc)
#> $series_1
#> score in_interval interval_width eval_horizon score_type
#> 1 0.7546147 1 0.9 1 crps
#> 2 0.8195527 1 0.9 2 crps
#> 3 2.1283856 1 0.9 3 crps
#> 4 1.2097069 1 0.9 4 crps
#> 5 1.7376459 1 0.9 5 crps
#> 6 2.8649248 1 0.9 6 crps
#> 7 1.8178390 1 0.9 7 crps
#> 8 1.5547210 1 0.9 8 crps
#> 9 1.1941840 1 0.9 9 crps
#> 10 1.7886700 1 0.9 10 crps
#> 11 2.3734945 1 0.9 11 crps
#> 12 1.1552394 1 0.9 12 crps
#> 13 0.9968969 1 0.9 13 crps
#> 14 0.9847705 1 0.9 14 crps
#> 15 1.0202320 1 0.9 15 crps
#> 16 1.4975177 1 0.9 16 crps
#> 17 4.1781860 1 0.9 17 crps
#> 18 1.6047802 1 0.9 18 crps
#> 19 2.0475912 1 0.9 19 crps
#> 20 3.0501099 1 0.9 20 crps
#> 21 2.3956491 1 0.9 21 crps
#> 22 3.7320900 1 0.9 22 crps
#> 23 1.4265935 1 0.9 23 crps
#> 24 2.2222251 1 0.9 24 crps
#> 25 1.3380422 1 0.9 25 crps
#>
#> $all_series
#> score eval_horizon score_type
#> 1 0.7546147 1 sum_crps
#> 2 0.8195527 2 sum_crps
#> 3 2.1283856 3 sum_crps
#> 4 1.2097069 4 sum_crps
#> 5 1.7376459 5 sum_crps
#> 6 2.8649248 6 sum_crps
#> 7 1.8178390 7 sum_crps
#> 8 1.5547210 8 sum_crps
#> 9 1.1941840 9 sum_crps
#> 10 1.7886700 10 sum_crps
#> 11 2.3734945 11 sum_crps
#> 12 1.1552394 12 sum_crps
#> 13 0.9968969 13 sum_crps
#> 14 0.9847705 14 sum_crps
#> 15 1.0202320 15 sum_crps
#> 16 1.4975177 16 sum_crps
#> 17 4.1781860 17 sum_crps
#> 18 1.6047802 18 sum_crps
#> 19 2.0475912 19 sum_crps
#> 20 3.0501099 20 sum_crps
#> 21 2.3956491 21 sum_crps
#> 22 3.7320900 22 sum_crps
#> 23 1.4265935 23 sum_crps
#> 24 2.2222251 24 sum_crps
#> 25 1.3380422 25 sum_crps
#>
# }