How to merge every data files application is reading without user help like in realtime application?

I am using shiny to create a machine learning application. This application will find or detect the fraud cases and shows the fraud data with datatable. I trying to create this as a near real time application and this application reads files that i am creating as a stream using sparklyr "stream_generate()". This application is reading the files as a realtime application ,but I want to merge or keep those all files application reads in one dataframe and use for analysis or visualization.

There will not be user interaction at datainput section or machine learning section. User will only interact with the result of the application .

So , how can i merge or save those input files in single file?

Code:

# Define server logic
# Server will work on the data reading,data passing to model and passing data to ui section for visualization
## calling real time file reading
# source("server/00-datacreator-srv.R", local = TRUE)
data <- read_csv("/home/diwash/projeckt/credit_card_feaud/credit-card-fraud-detector/data/data.csv")

IsThereNewFile <- function() { #  cheap function whose values over time will be tested for equality;
  #  inequality indicates that the underlying value has changed and needs to be
  #  invalidated and re-read using valueFunc
  filenames <- list.files(pattern = "*.csv", full.names = TRUE)
  length(filenames)
}

ReadAllData <- function() { # A function that calculates the underlying value
  filenames <- list.files(pattern = "*.csv", full.names = TRUE)
  read_csv(filenames[length(filenames)])
}



shinyServer(server <- function(input, output, session) {
  values <- reactiveValues()
  values$data <-  read_csv("/home/diwash/projeckt/credit_card_feaud/credit-card-fraud-detector/data/data.csv")
 
  ## read the files in 3 second difference
  sampled_data <- reactivePoll(3000, session, IsThereNewFile, ReadAllData)
  real_data <- reactive({values$data <- rbind(values$data, sampled_data())
  })
  data <- reactive({assign(values$data ,real_data())})

  observe({
    print(values$data)
  }) 
  source("server/01-dashboard-srv.R", local = TRUE)
  source("server/02-alert-srv.R", local = TRUE)
})


This topic was automatically closed 54 days after the last reply. New replies are no longer allowed.