Examples of Non-Linear Ordinal Regression Models?


The only out-of-the-box ordinal regression models I’ve found in R seem to be linear (vglm in vgam and polr in MASS), but I’m looking for something to better incorporate non-linear effects like a random forest or maybe a neural network.

Any suggestions or code examples? All advice is appreciated.


The vgam function can easily include nonlinearity. Also, there is rpartScore for CART models.

Otherwise, you can convert them to unordered factors, fit models, then post-process the probabilities to be monotonic using techniques like the pooled-adjacent violator algorithm (best name ever) or monotonic regression.

Beyond that, you’re stuck with using splines inside a formula method AFAICT


Thanks, Max. A package like rpartScore is exactly what I was looking for.

Now I need to google just about everything you said, because I can understand ~10% of what you wrote there.


Sorry to be too jargony.

Suppose you fit a classification model and treat the classes as unordered. Suppose you have four classes (A though D) and you get class probabilities for a data point that look like:

   A     B     C    D 
0.10  0.35  0.20 0.35

Since they are ordinal, you might expect that the class probabilities should not have an up- and down-pattern and should be monotonically increasing or decreasing. You can post-process these results to fit that constraint.

A simple PAVA algorithm would find where the assumption is violated and change the data values. Here an example using the isotone (pdf) package:

> library(isotone)
> dat <- data.frame(x = 1:4, prob = c(0.10, 0.35, 0.20, 0.35))
> gpava(dat$x, dat$prob)

gpava(z = dat$x, y = dat$prob)

PAVA results:
  Predictor Fitted Values
1         1          0.10
2         2          0.28
3         3          0.28
4         4          0.35

The class probability values for classes B and C were modifed to better fit the assumptions.


Thanks for the reply! And your earlier reply wasn’t too jargony – I’m just not that well-versed in stats talk. Good to know, though – I guess using ordinal regression vs. just multi-class is largely a philosophical choice in most circumstances.


Hey Max,

Do you know if it's possible for the fitted values in gpava to be rounded to more than 2 digits? I didn't see anything about that in the documentation, so I thought maybe you'd know


I'm sure that it stores them to full precision; you might see two digits because of how they are printed (unless you have a counter-example).