one factor grouping / section 10.7 / accuracy command on grouped structure

[lchenin]

i reproduced the following chunk of code to check the optimal for "Total", "State", "Legal", "Gender", "Bottom", "All series" but could not assemble the corresponding grouped structure, as I got only "Total" for levels = 0. For levels =0:1, of course I got the states, but ungrouped. Any hint, please ?

data <- window(prison.gts, start = 2005, end = 2015)
test <- window(prison.gts, start = 2015)
fcasts <- forecast(data, h = 8, method = "bu", fmethod = "arima")

a1 <- accuracy.gts(fcasts, test, levels = 0)

fcasts1 <- forecast(data, h = 8, method = "comb", weights ="ols")

a2 <- accuracy.gts(fcasts1, test, levels = 0)

fcasts2 <- forecast(data, h = 8, method = "comb", weights ="wls")

a3 <- accuracy.gts(fcasts2, test, levels = 0)

fcasts3 <- forecast(data, h = 8, method = "comb", weights ="nseries")

a4 <- accuracy.gts(fcasts3, test, levels = 0)

fcasts4 <- forecast(data, h = 8, method = "comb", weights ="mint", covariance="shr")

a5 <- accuracy.gts(fcasts4, test, levels = c("Total", "State", "Legal", "Gender"))

Also for "Total", the optimal is method = "comb" , weights = "mint" , and covariance = "shr" , not
method = "comb" and weights = "wls"

# |method |    bu|     ols|    wls|  nseries|   mint|
# |:----------|------:|-------:|-------:|----------:|--------:|
# |MAPE   | 3.78|  1.52| 1.214|  1.1461|  1.076|
# |MASE   | 1.28|  0.49| 0.401|  0.3764|  0.345|

---

^{Referred here by Forecasting: Principles and Practice, by Rob J Hyndman and George Athanasopoulos}

[robjhyndman]

Here is the code we use in the book to create Table 10.1 in https://otexts.com/fpp2/reconciliation.html. You can adapt it to cover other possible methods.

library(fpp2)
library(hts)

prison.gts <- gts(prison / 1e3,
  characters = c(3, 1, 9),
  gnames = c(
    "State", "Gender", "Legal",
    "State*Gender", "State*Legal",
    "Gender*Legal"
  )
)

train <- window(prison.gts, end = c(2014, 4))
test <- window(prison.gts, start = 2015)

fcsts.opt <- forecast(train, h=8, method="comb",
                      weights="wls", fmethod="ets")
fcsts.bu <- forecast(train, h=8, method="bu",
                     fmethod="ets")

tab <- matrix(NA, ncol = 4, nrow = 6)
rownames(tab) <- c("Total", "State", "Legal status", "Gender", "Bottom", "All series")
colnames(tab) <- c("MAPE", "MASE", "MAPE", "MASE")
tab[1, ] <- c(
  accuracy(fcsts.bu, test, levels = 0)[c("MAPE", "MASE"), "Total"],
  accuracy(fcsts.opt, test, levels = 0)[c("MAPE", "MASE"), "Total"]
)
j <- 2
for (i in c(1:3, 7)) {
  tab[j, ] <- c(
    mean(accuracy(fcsts.bu, test, levels = i)["MAPE", ]),
    mean(accuracy(fcsts.bu, test, levels = i)["MASE", ]),
    mean(accuracy(fcsts.opt, test, levels = i)["MAPE", ]),
    mean(accuracy(fcsts.opt, test, levels = i)["MASE", ])
  )
  j <- j + 1
}
tab[6, ] <- c(
  mean(accuracy(fcsts.bu, test)["MAPE", ]),
  mean(accuracy(fcsts.bu, test)["MASE", ]),
  mean(accuracy(fcsts.opt, test)["MAPE", ]),
  mean(accuracy(fcsts.opt, test)["MASE", ])
)
tab
#>                   MAPE     MASE      MAPE     MASE
#> Total         5.319487 1.836202  3.083556 1.064423
#> State         7.587158 1.875263  7.623787 1.845028
#> Legal status  6.404279 1.756171  4.319457 1.143369
#> Gender        8.619322 2.684583  8.722777 2.744076
#> Bottom       15.822023 2.233101 15.247794 2.157624
#> All series   12.412906 2.157424 12.024966 2.079179

^{Created on 2020-06-07 by the reprex package (v0.3.0)}

[lchenin]

Thank you very much ! it is really fine to compare various forecast.