dBwritetable to slow with Odbc Hadoop HIVE

Hello,
I am using the DBI/ODBC package to connect to our Database (Hadoop-HIVE).
To send and write R Data.frame to a sandbox in the Hadoop / HIVE database, I am using dbWriteTable (), but it is very slow.

To send a file of 1.3Gbytes, the transfer time reaches 18 hours.

Our structure is as follows:
Server Linux - R / RStudio Server Pro -> Server Linux - Hadoop / Hive.

Do you have any advice on best practices or some other function to use?

Is it possible that I have an infrastructure problem between the two servers?

Maybe I have to switch from the Hive connection to the Impala connection in the dBwritetable process, for better performance to write to Hadoop?

Thank you!

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.