SVM with SGD training

Machine Learning and Modeling
1

I'm looking for a package that might have support vector machines with stochastic gradient descent training, like scikitlearn's sgdclassifier. I'm aware of reticulate and the ability to write/run Python with R, but I'm looking for an R implementation, and it doesn't appear to me that caret or e1071 have what I am looking for (but I may be mistaken).

2

I don't know of any. John Platt's SMO algorithm is used solving the SVM quadratic programming objective functions.

3

Thanks, at least now I know I'm not crazy and can stop looking for now :slight_smile:

4

We make no guarantees :stuck_out_tongue_winking_eye:.

1 Like
5

The SGD approach to solving SVMs is fairly new -- it wasn't until somewhat recently that people formulated the hinge loss in a way that allows gradient descent as opposed to solving the quadratic programming problem.

I'm not sure what the actual tradeoffs are between the two methods in terms of computational time / efficiency. I imagine something fancy like ADMM / related is better than SGD. My guess is the advantage of SGD is that you can do early stopping / fit the SVM in a neural net framework using hinge loss. Based on this it's not surprising that there isn't an SGD SVM implementation in R since the advantage isn't entirely clear.

I have a naive Numpy implementation of the SGD approach if you want to translate it into R, but again, I imagine this would be for didactic purposes more than anything.

1 Like