Using graylog for centralising logs within the R ecosystem

shiny
logging

#1

Hey folks,

We are trying to centralise the logging infrastructure inside the organisation. The tech stack is mostly NodeJS + Python. We use R for all data processing, reporting and analytics.

The engineering team is planning to use Graylog for some internal tools. Just wanted to understand if people here have some experience in connecting Graylog with the R ecosystem (Shiny applications, scripts, etc)

Python has a grapy package for doing the same. I think this can be used via reticulate. Any other approaches/help/learnings will be appreciated

Thanks


#2

Hello!

I had the same task, I used the REST API of Graylog and then read out the data with elasticsearchr. To display the data in a Shiny app, I made a script which is based on the movie explorer example in R Shiny. Does this help you? I can help out with script fragments if needed, and I am happy to exchange experience with you.


#3

Thanks for the reply @knilda .. Please share your repo, will definitely have a look.


#4

You are very welcome, @apooRv. As I unterstand, graypy package is reporting logs from python to graylog. My script is getting data from graylog to analyse them in R. Is this what you want?


#5

@knilda was more interested in the ingestion of logs in Graylog, any inputs on that front ?


#6

Sorry, I have no experience on that. I think its also possible via REST API.


#7

You might have a look at the logging package, which has some examples of logging to a remote server via REST (I think this is it? Not sure if this is the best example):


#8

@apooRv this is my repo, in case you want to get your logging data from graylog back to R, this might help you https://github.com/knilda/shRlog.git


#9

I've been testing using @atheriel's excellent rsyslog package
(https://cran.r-project.org/web/packages/rsyslog/index.html) to forward via rsyslog. I can confirm that it successfully logs to the main syslog process through rstudio-connect, and then I forward that to a logging box.