Is there a problem with this forecast model?

[level]

Hello all,

I ran this script as part of a forecasting project, but I got some odd results especially with the MAPE values. Can anyone tell me if this report is accurate or if I missed something?

Thanks,

# Author: Sal Meza
# Data Science 489/ Spring
# Data: 2/11/19
# last modified: 2/11/19

library(ggplot2)
library(forecast)
library(tseries)
library(reprex)

terror <- tibble::tribble(
  ~imonth, ~iyear, ~monthly,
  1,   2015,     1534,
  2,   2015,     1295,
  3,   2015,     1183,
  4,   2015,     1277,
  5,   2015,     1316,
  6,   2015,     1168,
  7,   2015,     1263,
  8,   2015,     1290,
  9,   2015,     1107,
  10,   2015,     1269,
  11,   2015,     1172,
  12,   2015,     1091,
  1,   2016,     1162,
  2,   2016,     1153,
  3,   2016,     1145,
  4,   2016,     1120,
  5,   2016,     1353,
  6,   2016,     1156,
  7,   2016,     1114,
  8,   2016,     1162,
  9,   2016,     1045,
  10,   2016,     1140,
  11,   2016,     1114,
  12,   2016,      923,
  1,   2017,      879,
  2,   2017,      879,
  3,   2017,      961,
  4,   2017,      856,
  5,   2017,     1081,
  6,   2017,     1077,
  7,   2017,      994,
  8,   2017,      968,
  9,   2017,      838,
  10,   2017,      805,
  11,   2017,      804,
  12,   2017,      749
)


# aggregated data
terror_byMonth_Train = ts(data = terror$monthly,
                          start = c(2015,1), 
                          end = c(2016,12),
                          frequency=12) 

terror_byMonth_Test = ts(data = terror$monthly,
                         start = c(2017,1), 
                         end = c(2017,12),
                         frequency=12)

# arima instead of exp smooth
m_arima <- auto.arima(terror_byMonth_Train)
#> Warning in value[[3L]](cond): The chosen test encountered an error, so no
#> seasonal differencing is selected. Check the time series data.

# fit exp smooth model
m_ets = ets(terror_byMonth_Train) 

# Get length of terror_byMonth_Test set
size <- length(terror_byMonth_Test)

# forecast for 2017 using multiple forecast (Davis Style)
f_arima_multi <- m_arima %>%
  forecast(h = size)

f_arima_multi %>%
  autoplot()

# forecast ARIMA 2017 (Orininal Style)
f_arima<-forecast(m_arima,h=12)
f_arima %>%
  autoplot()

# forecast ETS 2017
f_ets = forecast(m_ets, h=12) 
f_ets %>%
  autoplot()

# check accuracy ETS
acc_ets <- accuracy(m_ets) 

#check accuracy ARIMA, between train and test sets
acc_arima_TrainVSTest <- accuracy(f_arima_multi, x = terror_byMonth_Test)

# check accuarcy ARIMA
acc_arima <- accuracy(f_arima)

# MAPE(ETS)= 20.03 < MAPE(ARIMA) = 22.05
# ETS model chosen 

# Compair to acctually 2017 data
accuracy(f_ets, terror_byMonth_Test)
#>                     ME      RMSE       MAE       MPE      MAPE      MASE
#> Training set -14.30982  90.08823  70.06438 -1.606862  5.900178 0.5790445
#> Test set     303.53575 316.03133 303.53575 23.986363 23.986363 2.5085599
#>                       ACF1 Theil's U
#> Training set  0.0008690031        NA
#> Test set     -0.2148651254  2.356116

^{Created on 2019-02-13 by the reprex package (v0.2.1)}

[level]

It turned out that the problem was with the way I set the "data" value in ts(). I put "data=terror$monthly" and then set the start and end variables. However, this does not specify what part of "data" to use. What I should have used was "data=tail(terror$monthly, 12)" This would have specified to only use the last 12 elements found in terror$monthly startarting with 2017,1 and ending with 2017,12.

Wrong:

@level

terror_byMonth_Test = ts(data = terror$monthly, start = c(2017,1), end = c(2017,12), frequency=12)

Should be:

@level

terror_byMonth_Test = ts(data = tail(terror$monthly, 12), start = c(2017,1), end = c(2017,12), frequency=12)