sparklyr 1.0: Apache Arrow, XGBoost, Broom and TFRecords

[javierluraschi]

sparklyr 1.0: Apache Arrow, XGBoost, Broom and TFRecords

Javier Luraschi - 2019-03-15

With much excitement built over the past three years, we are thrilled to share that sparklyr 1.0 is now available on CRAN!

The sparklyr package provides an R interface to Apache Spark. It supports dplyr, MLlib, streaming, extensions and many other features; however, this particular release enables the following new features:

• Arrow enables faster and larger data transfers between Spark and R.
• XGBoost enables training gradient boosting models over distributed datasets.
• Broom converts Spark’s models into tidy formats that you know and love.
• TFRecords writes TensorFlow records from Spark to support deep learning workflows.

Read more at sparklyr 1.0: Apache Arrow, XGBoost, Broom and TFRecords - Posit