Console and Rmd output differ. Same Program used but the calculation gives a different result!

[HAHR]

Using the .Rmd / knit process to generate a PDF file, the "same" routine calculates a different result! This is the program:

my_model <- train(diagnosis ~ ., data = small_train, method = "glm")
pred_mymodel <- predict(my_model, newdata= small_test)
myConfmtx <- confusionMatrix(pred_mymodel, small_test$diagnosis)
print(myConfmtx)

Accuracy : 0.6463 (console or source()) vs Accuracy : 0.7561 (.Rmd produced)

Any clue of what is going on?

I load the libraries :

library(caret)
library(AppliedPredictiveModeling)
library(Hmisc)

I am using latest available versions of R (4.0.0 pckg) and R Studio (1.3.959) on a MacBook with High Sierra.

[HanOostdijk]

No solution but some thoughts:

• random numbers: do you or the packages explicitly set a random number seed?
• the normal way for the knit is to use a new environment but the program in the console would use the existing environment. Could that bite you?
• in both cases you load the same libraries in the same order?
• something 'trivial' you do when starting your project in e.g. .Profile ?

[HAHR]

@HanOostdijk

e.g. .Profile

Thanks for the answers: I have tried all suggestions: Random Number seeds, setting up completely new environment, making sure libraries are load in the same order. But I still get different results in some calculations, others are the same.

I have for example stored the .Rmd program, load R anew, setup new project, changing working directory. Loaded the Rmd program and run it copying the code chunks to the console. This way I ensure that it is also a new environment in the console. But event then I get different results. It mostly happens with the train( function from the caret package. Is there a bug somewhere? Or maybe I have to make sure that I have installed R studio and R in the correct versions and sequence. Maybe it has to do with my Mac OS version too. I use MacOS High Sierra.

By the way I do not know what you mean with something "trivial" eg. Profile.

Thanks and regards

start a new project in R in a different directory

[HanOostdijk]

Sorry, I made a mistake. I meant the data set .Rprofile .

If you open a project the R statements in that data set (when present) are executed.
When you start RStudio without an open project the default (*) folder is set as working directory and if that contains an .Rprofile data set the R statements it contains are executed.
With 'trivial' I only meant that is so easy to forget that you placed some initialisation code there . That could have an impact of course.

(*) the default folder is the one set with
Tools | Global Options... | General | Default working directory (when not in a project)