sparklyr 1.0: Apache Arrow, XGBoost, Broom and TFRecords

sparklyr 1.0: Apache Arrow, XGBoost, Broom and TFRecords

Javier Luraschi - 2019-03-15

With much excitement built over the past three years, we are thrilled to share that sparklyr 1.0 is now available on CRAN!

The sparklyr package provides an R interface to Apache Spark. It supports dplyr, MLlib, streaming, extensions and many other features; however, this particular release enables the following new features:

  • Arrow enables faster and larger data transfers between Spark and R.
  • XGBoost enables training gradient boosting models over distributed datasets.
  • Broom converts Spark’s models into tidy formats that you know and love.
  • TFRecords writes TensorFlow records from Spark to support deep learning workflows.


1 Like

This topic was automatically closed after 21 days. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.