I am working my way through the Forecasting: Principles and Practice book: https://otexts.com/fpp3/
In section 5.8, the book describes how to evaluate point forecast accuracy. It says that you can only calculate this when applying the model to data which wasn't used when fitting the model.
However, the example given creates a training dataset which is a subset of the main dataset. The accuracy of the model is calculated using the main dataset although the main dataset contains data that was used to fit the model. I am confused as to why this is being done and why isn't separate train and test datasets being produced. Would it be best to train the model using a completely separate train dataset and then evaluate the model using a test dataset?
The accuracy() function will only use the elements of the data set that correspond to the test period. If only the test data is passed, the same error measures are obtained.
All columns are identical apart from MASE and RMSSE. In those two cases, the training data is needed to calculate the scaling factor, so when only the test data is included they cannot be computed.