how to store data frame in file

hi there,

i have a data frame, which i construct from json data (fetch from website). now i want to save this data on my pc, (as a file).
when i am trying to save data frame in csv file it give me error.

Error in dimnames(X) <- list(dn[[1L]], unlist(collabs[nc > 0], use.names = FALSE)) :
length of 'dimnames' [2] not equal to array extent

can i save data frame as a file, and then load again, when i needed.
here is my R code

library(curl)
library(jsonlite)
library(dplyr)
h <- new_handle()
handle_setheaders(h,Authorization = "Bearer 60ff659020cbdccfb2b7aacaa03af4f7")
url <- "https://api.deutschebahn.com/stada/v2/stations"
res <- curl_fetch_memory(url,h)
(myjsondata <- jsonlite::prettify(rawToChar(res$content)))
(myjsondata_df <- jsonlite::fromJSON(myjsondata) %>%
as.data.frame())

View(myjsondata_df)
write.csv(myjsondata_df, file = "allStationTODAYNOV.csv")

I am looking forward to hearing from you soon.

many thanks,
rao

Hello,

First of all, a word of warning: you should never provide authentication information in code you share as this might be sensitive information that could lead to issues if it falls into the wrong hands :slight_smile:
I know you did this to help us recreate the issue, but next time try to see if you can generate the same issue without having to share authentication credentials. A very nice start guide is here

To the question at hand now:
JSON is not the same data structure as a table and can have nested lists of data which are not possible in csv. Sometimes you can easily convert if JSON does not have nested data, but in your case it is. Look at the "result.mailingAddress", "result.regionalbereich", ... they are nested data frames within a data frame. So you cannot simply convert this to a regular data frame. You can extract parts or remove them, but flattening everything into one giant table will be impossible or even if possible, will generate a massive table with a lot of repeating values.

Best way is to save the json as a json :slight_smile: Then you can load it back whenever needed.

library(curl)
library(jsonlite)
library(dplyr)

h <- new_handle()
handle_setheaders(h,Authorization = "Bearer 60ff659020cbdccfb2b7aacaa03af4f7")
url <- "https://api.deutschebahn.com/stada/v2/stations"
res <- curl_fetch_memory(url,h)
myjsondata <- rawToChar(res$content)
myjsondata <- fromJSON(prettify(myjsondata))

#Write the json
write_json(myjsondata, "allStationTODAYNOV.json", pretty = T)

#Read the json
test = read_json("allStationTODAYNOV.json", simplifyVector = T)

Notes:

  • pretty = T makes sure the JSON is saved in a pretty more readable multi-line format with indentation instead of just one giant row of text
  • simplifyVector = T will try and simplify JSON files when read in by simplifying nested lists into vectors and data frames

Hope this helps,
PJ

Hi PJ,

Thank you very much. I will be very careful next time regarding authentication key.

one thing I still don't understand, that once I have created, built, constructed as Data Frame, the nested games is over. Now there is no nested in DF. DF have only column header and rows of data.

Nobody know how you created DF. so now I have DF, so I can export this DF in any format. i.e. CSV, SQLIT, XLS, JSON etc...
DF can be exported in any format and voice versa (as per my knowledge).

I am looking forward to hearing from you.

Many thanks,
Rao

the R dataframe, has tabular outer dimensions, but with nesting it supports entries which are themselves dataframes.
simple tabular formats such as csv, do not support this.
Yet, you can save the data.frame - the same as any R object though.
base::saveRDS accomplishes this.

(nested_tbl <- tibble(
  a = c(1, 2),
  b = list(3, tibble(x = 4, y = 5))
))

how would you manually compose a csv so that another program can read it with the same structure ?
remembering that csv is primitive and therefore you can only use commas and line breaks....

a,b
1,2
3, ??????

1 Like

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.