val.prob.ci.2 issue

[afenton1985]

I am trying to plot a calibration plot. There are 329443 observations, and I enter the following:

> val.prob.ci.2(Bare$KFRE_Pred, Bare$RRT_in_2YRS, smooth="rcs")

which leads to the following error:

singular information matrix in lrm.fit (rank= 3 ). Offending variable(s):

Warning in value[[3L]](https://github.com/BavoDC/val.prob.ci.2/issues/cond) :
The number of knots led to estimation problems, nk will be set to 4.
singular information matrix in lrm.fit (rank= 3 ). Offending variable(s):

Warning in value[[3L]](https://github.com/BavoDC/val.prob.ci.2/issues/cond) :
Nk 4 also led to estimation problems, nk will be set to 3.
singular information matrix in lrm.fit (rank= 2 ). Offending variable(s):

Error in array(x, c(length(x), 1L), if (!is.null(names(x))) list(names(x), :
'data' must be of a vector type, was 'NULL'
In addition: Warning message:
In val.prob.ci.2(Bare$KFRE_Pred, Bare$RRT_in_2YRS, smooth = "rcs") :
27 observations deleted from logistic calibration due to probs. of 0 or 1

Any suggestions?

Best,
Anthony

[nirgrahamuk]

Here is an example

library(caret)
data(mdrr)
mdrrDescr <- mdrrDescr[, -nearZeroVar(mdrrDescr)]
mdrrDescr <- mdrrDescr[, -findCorrelation(cor(mdrrDescr), .5)]


inTrain <- createDataPartition(mdrrClass)
trainX <- mdrrDescr[inTrain[[1]], ]
trainY <- mdrrClass[inTrain[[1]]]
testX <- mdrrDescr[-inTrain[[1]], ]
testY <- mdrrClass[-inTrain[[1]]]

library(MASS)

ldaFit <- MASS::lda(trainX, trainY)

testProbs <- data.frame(obs = testY,
                        lda = predict(ldaFit, testX)$posterior[,1])

calPlotData <- caret::calibration(obs ~ lda , data = testProbs)
calPlotData

xyplot(calPlotData, auto.key = list(columns = 2))

[afenton1985]

Thank you very much - that's very helpful. One of the advantages of val.prob.ci.2 is the inclusion of 95% confidence intervals - can this be incorporated into your code?

Many thanks,
Anthony

[nirgrahamuk]

according to the documentation:
For the ggplot method, confidence intervals on the estimated proportions (from binom.test) are also shown.

ggplot(calPlotData)+geom_line()