Plotting values from a function

General
1

Hello. I have found the following code:

L = length(x_test)
scaler = Scaled$scaler
predictions = numeric(L)

for(i in 1:L){
  X = x_test[i]
  dim(X) = c(1,1,1)
  yhat = model %>% predict(X, batch_size=batch_size)
  # invert scaling
  yhat = invert_scaling(yhat, scaler,  c(-1, 1))
  # invert differencing
  yhat  = yhat + MORTGAGE30US_3_$MORTGAGE30US[(n+i)]
  # store
  predictions[i] <- yhat
 
}

I need to print/plot the prediction values. But when I do:

  yhat = invert_scaling(yhat, scaler,  c(-1, 1))
  # invert differencing
  yhat  = yhat + MORTGAGE30US_3_$MORTGAGE30US[(n+i)]
  # store
  predictions[i] <- yhat
 print (predictions)

**OR**

plot(predictions)
}

Generally nothing happens.
Could you,please, tell me, how to plot these values directly from a function?

2

Hi Maxim,
So it's easier to understand your code, could your post a fully reproducible example? The goal is to have someone else be able to take the code you posted and run it in their own R session, without it depending on any other objects, etc. that only you have access to. For example, when I tried to run the example you posted, I couldn't because you haven't defined the object x_test or the object Scaled.

For more information on how to create a reproducible example, you can see this guide: FAQ: What's a reproducible example (`reprex`) and how do I do one?

3

Generally, the fully reproducible example is shown here:

The desciption of what is being done is also presented alongside the code.
More or less I understand the author's code. I dont't understand how to plot the last picture (with the forecasted values). That's why I'm asking for help.

4

So, I'm still a little unsure of what you're asking. It looks like what's happening in the code snipped you included is that the author is using a for-loop to fill up a numeric vector, one element at a time. The line predictions[i] <- yhat assigns the predicted value to the ith element of the vector predictions.

To make a plot like the one shown at the end of that blog post, I would call plot(predictions) outside of the for-loop.

It seems like you're trying to plot predictions inside the for-loop. Can you explain why you want to do that? You'd be plotting an incomplete vector with each iteration of the loop.

1 Like
5

I've tried plot(predictions) outside of the for-loop as well as inside the for-loop.
Nothing works.

6

What are the contents of the vector predictions? Like, what happens if you just type predictions or head(predictions)?

7

You've omitted the key element of what you are doing, just exactly at where it differs from the tutorial you follow...
You say you want the code in a function, but you show no function definition, so we can't identify what you may have done wrong with the function you created.
Probably you simply want to return predictions at the end of your function. Then if you assign the value of the function return to a name like predictions or what have you, its a cinch to call plot on it.

1 Like
8

When I type plot(predictions) within a function, I get the following picture:
image

This doesn't look like the last picture in the webpage, link on which I've provided before.
Moreover, the function still works and the picture changes a bit.

When I type head(predictions) within a for-loop I get nothing at all.

9

Above I've provided a link on the webpage with the source code and the description of anything, that is being done. I take everything from here.

10

It's normal that you wouldn't get any output from head(predictions) inside the for-loop--you would need to call print() on that if you actually wanted it to print from inside the loop. I don't know why you want to do that, though; why not just look at the predictions object after it's been created, outside the for-loop?

I agree with @nirgrahamuk that you need to provide the function code you're actually using. You've said that the blog post you linked to is a fully reproducible example, but it isn't--the blog author didn't include the code that they used to produce that plot.

Can you explain what you mean by "when I type plot(predictions) within a function"? What function are you talking about? You keep referring to the fact that you're trying to plot values from inside a function, but like @nirgrahamuk pointed out, you haven't actually shown us what function that is, so we don't know how to help you.

1 Like
11

That's why I have asked this question. Maybe someone, who saw the source code from the blog, could help me to reproduce the last picture. Code for the last picture is omitted in the authors code.

"when I type plot(predictions) within a function"

I mean the last for-loop:

for(i in 1:L){
  X = x_test[i]
  dim(X) = c(1,1,1)
  yhat = model %>% predict(X, batch_size=batch_size)
  # invert scaling
  yhat = invert_scaling(yhat, scaler,  c(-1, 1))
  # invert differencing
  yhat  = yhat + MORTGAGE30US_3_$MORTGAGE30US[(n+i)]
  # store
  predictions[i] <- yhat

When I did this, I got about 2000 of NaN values.

12

@Maxim one of the problems here is that the code in the tutorial post you've cited is incomplete. In order to copy his approach and make a function that works, you will need to fill in the gaps in the code that he has left. For example, he says that he's obtained data on US interest rates. But he hasn't shown how he did that, or what that data looks like. So my question is, what data are you using? What does it look like?

You have to show us or we can't help you. If you just use the code on his page, it will not give you the output you want, because it is incomplete itself.

13

The interest rate data can be found on https://fred.stlouisfed.org/graph/?g=NUh

Then you need to press Download button on the webpage (it should be downloaded in csv format)
And then:
library(readr)
Series<-read_csv("~/Downloads/MORTGAGE30US (3).csv")

14

or you could save us the trouble and do

dput(Series)

and post the output of that.

1 Like
15 
dput(MORTGAGE30US)
3.98, 3.91, 3.94, 3.93, 3.84, 3.89, 3.9, 3.91, 3.86, 3.85, 
3.76, 3.82, 3.79, 3.76, 3.87, 3.98, 3.97, 3.95, 3.93, 3.95, 3.97, 
3.96, 4.01, 3.97, 3.92, 3.81, 3.79, 3.72, 3.65, 3.65, 3.62, 3.64, 
3.68, 3.73, 3.71, 3.71, 3.59, 3.58, 3.59, 3.66, 3.61, 3.57, 3.58, 
3.64, 3.66, 3.6, 3.54, 3.56, 3.48, 3.41, 3.42, 3.45, 3.48, 3.43, 
3.45, 3.43, 3.43, 3.46, 3.44, 3.5, 3.48, 3.42, 3.42, 3.47, 3.52, 
3.47, 3.54, 3.57, 3.94, 4.03, 4.08, 4.13, 4.16, 4.3, 4.32, 4.2, 
4.12, 4.09, 4.19, 4.19, 4.17, 4.15, 4.16, 4.1, 4.21, 4.3, 4.23, 
4.14, 4.1, 4.08, 3.97, 4.03, 4.02, 4.05, 4.02, 3.95, 3.94, 3.89, 
3.91, 3.9, 3.88, 3.96, 4.03, 3.96, 3.92, 3.93, 3.9, 3.89, 3.86, 
3.82, 3.78, 3.78, 3.83, 3.83, 3.85, 3.91, 3.88, 3.94, 3.94, 3.9, 
3.95, 3.92, 3.9, 3.94, 3.93, 3.94, 3.99, 3.95, 3.99, 4.04, 4.15, 
4.22, 4.32, 4.38, 4.4, 4.43, 4.46, 4.44, 4.45, 4.44, 4.4, 4.42, 
4.47, 4.58, 4.55, 4.55, 4.61, 4.66, 4.56, 4.54, 4.62, 4.57, 4.55, 
4.52, 4.53, 4.52, 4.54, 4.6, 4.59, 4.53, 4.51, 4.52, 4.54, 4.6, 
4.65, 4.72, 4.71, 4.9, 4.85, 4.86, 4.83, 4.94, 4.94, 4.81, 4.81, 
4.75, 4.63, 4.62, 4.55, 4.51, 4.45, 4.45, 4.45, 4.46, 4.41, 4.37, 
4.35, 4.35, 4.41, 4.31, 4.28, 4.06, 4.08, 4.12, 4.17, 4.2, 4.14, 
4.1, 4.07, 4.06, 3.99, 3.82, 3.82, 3.84, 3.73, 3.75, 3.75, 3.81, 
3.75, 3.75, 3.6, 3.6, 3.55, 3.58, 3.49, 3.56, 3.73, 3.64, 3.65, 
3.57, 3.69, 3.75, 3.78, 3.69, 3.75, 3.66, 3.68, 3.68, 3.73, 3.73, 
3.74, 3.72, 3.64, 3.65, 3.6, 3.51, 3.45, 3.47, 3.49, 3.45, 3.29, 
3.36, 3.65, 3.5, 3.33, 3.33, 3.31, 3.33, 3.23, 3.26, 3.28, 3.24, 
3.15, 3.18, 3.21, 3.13, 3.13, 3.07, 3.03, 2.98, 3.01, 2.99
16

@Maxim thanks for that.

Do you use RStudio? Do you have a Github account?

You could post your whole code up on a Gist on github.com.
Even better, use RStudio Connect to publish an RMarkdown output of your code. If you can't or won't provide a proper reprex here.

My apologies - I meant RPubs not RStudio Connect

Even a tiny difference from what Richard Wanjohi wrote can make a big difference to the output.

17

Well, I'm not familiar with that. Where can I find, how to work with it?

I can provide you with the whole code here:

library(keras)
library(tensorflow)


  
  Series = df$Value  # your time series 
  
  # transform data to stationarity
  diffed = diff(Series, differences = 1)
  
  
  # create a lagged dataset, i.e to be supervised learning
  
  lags <- function(x, k){
    
    lagged =  c(rep(NA, k), x[1:(length(x)-k)])
    DF = as.data.frame(cbind(lagged, x))
    colnames(DF) <- c( paste0('x-', k), 'x')
    DF[is.na(DF)] <- 0
    return(DF)
  }
  supervised = lags(diffed, k)
  
  
  ## split into train and test sets
  
  N = nrow(supervised)
  n = round(N *0.66, digits = 0)
  train = supervised[1:n, ]
  test  = supervised[(n+1):N,  ]
  
  
  ## scale data
  normalize <- function(train, test, feature_range = c(0, 1)) {
    x = train
    fr_min = feature_range[1]
    fr_max = feature_range[2]
    std_train = ((x - min(x) ) / (max(x) - min(x)  ))
    std_test  = ((test - min(x) ) / (max(x) - min(x)  ))
    
    scaled_train = std_train *(fr_max -fr_min) + fr_min
    scaled_test = std_test *(fr_max -fr_min) + fr_min
    
    return( list(scaled_train = as.vector(scaled_train), scaled_test = as.vector(scaled_test) ,scaler= c(min =min(x), max = max(x))) )
    
  }
  
  
  ## inverse-transform
  inverter = function(scaled, scaler, feature_range = c(0, 1)){
    min = scaler[1]
    max = scaler[2]
    n = length(scaled)
    mins = feature_range[1]
    maxs = feature_range[2]
    inverted_dfs = numeric(n)
    
    for( i in 1:n){
      X = (scaled[i]- mins)/(maxs - mins)
      rawValues = X *(max - min) + min
      inverted_dfs[i] <- rawValues
    }
    return(inverted_dfs)
  }
  
  
  Scaled = normalize(train, test, c(-1, 1))
  
  y_train = Scaled$scaled_train[, 2]
  x_train = Scaled$scaled_train[, 1]
  
  y_test = Scaled$scaled_test[, 2]
  x_test = Scaled$scaled_test[, 1]
  
  ## fit the model
  
  dim(x_train) <- c(length(x_train), 1, 1)
  dim(x_train)
  X_shape2 = dim(x_train)[2]
  X_shape3 = dim(x_train)[3]
  batch_size = 1
  units = 1
  
  model <- keras_model_sequential() 
  model%>%
    layer_lstm(units, batch_input_shape = c(batch_size, X_shape2, X_shape3), stateful= TRUE)%>%
    layer_dense(units = 1)
  
  
  
  model %>% compile(
    loss = 'mean_squared_error',
    optimizer = optimizer_adam( lr= 0.02 , decay = 1e-6 ),  
    metrics = c('accuracy')
  )
  
  
  
  summary(model)
  
  nb_epoch = Epochs   
  for(i in 1:nb_epoch ){
    model %>% fit(x_train, y_train, epochs=1, batch_size=batch_size, verbose=1, shuffle=FALSE)
    model %>% reset_states()
  }
  
  
  L = length(x_test)
  dim(x_test) = c(length(x_test), 1, 1)
  
  scaler = Scaled$scaler

  predictions = numeric(L)
  for(i in 1:L){
    X = x_test[i , , ]
    dim(X) = c(1,1,1)
    # forecast
    yhat = model %>% predict(X, batch_size=batch_size)
    
    # invert scaling
    yhat = inverter(yhat, scaler,  c(-1, 1))
    
    # invert differencing
    yhat  = yhat + Series[(n+i)] 
    
    # save prediction
    predictions[i] <- yhat
  }
1 Like
18

There's no value for k, so the code as written will fail.

1 Like
20

There's also no value for df

21

Where? In function lags?

[quote="Maxim, post:17, topic:83604"]

lags <- function(x, k){
    
    lagged =  c(rep(NA, k), x[1:(length(x)-k)])
    DF = as.data.frame(cbind(lagged, x))
    colnames(DF) <- c( paste0('x-', k), 'x')
    DF[is.na(DF)] <- 0
    return(DF)
  }
  supervised = lags(diffed, k)

Generally I ran the whole code and it worked.