Simplest keras hidden layer model with relu doesn't work

I have tried to create a very simple model to introduce the concept of a hidden layer in keras. I cannot get it to work reliably. Can anyone spot my mistake please?


dataset <- mtcars %>% select(disp,mpg)

maxdisp <- max(dataset$disp)
x_data <- dataset$disp/maxdisp # This scaling is for numerical stability
y_data <- dataset$mpg
#Create the model

model  <-  keras_model_sequential() %>% 
  layer_dense(units=3, activation = "relu",bias_initializer='ones', input_shape=1,name="HiddenLayer") %>% 
  layer_dense(units=1, activation = 'linear', name="RegressionLayer")
#Display the model
#Compile the model
# Use mean squared error (mse) for the loss function 
#  and stochastic gradient descent (SGD) for the optimizer
SGD = optimizer_sgd(lr=0.005)
compile(model,loss='mse', optimizer=SGD) # This makes changes directly to model
# Learn
history <- fit(model,x_data,y_data,epochs=6000,verbose=0)
#Plot Loss history
LossHistory <-$params$epochs,loss=history$metrics$loss))
# Compute model outputs
y_predicted <- predict(model,x_data)
newdataset <- dataset
newdataset$y_predicted <- y_predicted
# Display the result
rawplot <- ggplot(dataset,mapping=aes(disp,mpg))+geom_point()
rawplot +geom_point(mapping=aes(x=disp,y=y_predicted),data=newdataset,colour="red")
# I was expecting a piecewise linear function with at least one noticeable kink.
# Instead I get a linear relationship. If I re-run the code several times,
# it will sometimes work as expected.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.