Hi all,

I have a question concerning the val.prob.ci.2 function. I am externally validating a prediction model of mortality. Therefore I'd like to produce a calibration plot between the observed outcome (30DayMortality) and the chance (p) calculated for every patient. Before, I have used the val.prob.ci function in Rstudio, however this resulted in an lower AUC than expected. I got the advice to use val.prob.ci.2 instead, however with this function it did not produce a calibration plot.

Below I show first the val.prob.ci.2 function, and afterwards the val.prob.ci on how I used it before and how I would use the val.prob.ci.2 function.

Hopefully, someone can help me with this.

Thanks in advance.

###############################1. Val.prob.ci.2######################################

val.prob.ci.2 <- function(p, y, logit, group, weights = rep(1, length(y)), normwt = F, pl = T,

smooth = c("loess","rcs",F), CL.smooth="fill",CL.BT=F,

nr.knots=5,logistic.cal = F, xlab = "Predicted probability", ylab =

"Observed proportion", xlim = c(-0.02, 1),ylim = c(-0.15,1), m, g, cuts, emax.lim = c(0, 1),

legendloc = c(0.50 , 0.27), statloc = c(0,.85),dostats=T,roundstats=2,

riskdist = "predicted", cex=0.75,cex.leg = 0.75, mkh = 0.02, connect.group =

F, connect.smooth = T, g.group = 4, evaluate = 100, nmin = 0, d0lab="0", d1lab="1", cex.d01=0.7,

dist.label=0.04, line.bins=-.05, dist.label2=.03, cutoff, las=1, length.seg=1,...)

##############################################2. Val.prob.ci########################

val.prob.ci(p, y=Outcome, riskdist="predicted", g=10, logistic.cal=F, statloc = c(0,.85), dostats=c(12,13,2),

d0lab="Alive", d1lab="Dead", xlim=c(0,1.08),xlab="Probability of 30-day mortality", ylab="Observed 30-day mortality",

cex.d01=1, dist.label=0.03, cex.lab=1.0, cex=1, length.seg=1)

title ("Calibration 30-day mortality")

#############################Val.prob.ci.2. My model##################################

val.prob.ci.2 (p, y, group=TRUE, normwt = F, pl = T,

smooth = F, CL.BT=F,

logistic.cal = F, xlab = "Predicted probability", ylab =

"Observed proportion", xlim = c(-0.02, 1),ylim = c(-0.15,1), g=10, emax.lim = c(0, 1),

legendloc = c(0.50 , 0.27), statloc = c(0,.85),dostats=T,roundstats=2,

riskdist = "predicted", cex=0.75,cex.leg = 0.75, mkh = 0.02, connect.group =

F, connect.smooth = T, g.group = 4, evaluate = 100, nmin = 0, d0lab="0", d1lab="1", cex.d01=0.7,

dist.label=0.04, line.bins=-.05, dist.label2=.03, cutoff, las=1, length.seg=1)