Ray is a great framework for distributed computing based on C/C++ code, which supports pandas on ray right now and makes pandas distributed computing faster than Sparklyr.
And more importantly, Ray support reinforcement learning on A3C framework better than others.
With the distributed GPU/CPU hybrid condition, playing A3C with Mxnet and Ray is the best choice until now, however, Ray does not officially provide an R interface right now.