Hi.
In RStudio Terminal I established ssh connection to my cloud, but with terminalBuffer() I can get only last 1000 rows, however I would like to get the entire hive output into data.frame.
Maybe it is possible to redirect Terminal output to RStudio console and then write it into file or there is direct way ?
There are lots of options, depending on your situation:
- Use Apache Drill's Hive plugin via sergeant or ODBC:
-
Use SparkR
-
Use the hive or RHive packages (though be cautious of their level of maintenance; neither is obviously active)
-
Redirect the output to a file and read that in from R, (least setup, though it requires more steps each time and is more immediately subject to local resource constraints)
...and probably more. Which is the best option depends on what infrastructure is already in place and how often you'll need to do this sort of thing.
4 Likes