One of the data scientist is trying to move data from RDBMS/CSV to DataLake via SparklyRcode, he gets below error while. Can we join two different dataset in sparklyR and execute spark on cluster mode?
Test Case 1 :
Description : Trying to load a small mapping file into spark table. In our analysis we use few
One of the data scientist is trying to move data from RDBMS/CSV to DataLake via SparklyR mapping files which are in excel/csv which are required to be joined with rest of the data.
Result : The copy_to command works with max 15 records, beyond that throws exceptions – Fails