Download data from website into RStudio

rstudio

#1

Hello,

I am using this command to download data from the stated web site into RStusio using cpd <- read.csv(url(“https://pub.data.gov.bc.ca/datasets/176284/BC_Liquor_Store_Product_Price_list.csv?accessType=DOWNLOAD”)) but I get an error message such as wrong / Is this the right command?


#2

Have you tried this?
read.csv(“https://pub.data.gov.bc.ca/datasets/176284/BC_Liquor_Store_Product_Price_list.csv”)


#3

read.csv(“https://pub.data.gov.bc.ca/datasets/176284/BC_Liquor_Store_Product_Price_list.csv”)

I keep getting the same error: unexpected / and unexpected “”


#4

The quotation marks got mangled in @martin.R’s post. Try this instead:

read.csv("https://pub.data.gov.bc.ca/datasets/176284/BC_Liquor_Store_Product_Price_list.csv")

Also, I would suggest trying readr::read_csv instead of read.csv, as it’s faster and less prone to other coding issues:

library(readr)

read_csv("https://pub.data.gov.bc.ca/datasets/176284/BC_Liquor_Store_Product_Price_list.csv")

#5

Thank you! that fixed it.


#6

Hi @giuseppa.cefalu, I would recommend using download.file() to make a local copy of the file, and point read_csv() to that copy. Additionally, having a !file.exists() check will make sure that you don’t download the file every time you run your script.


#7

Could you explain a little bit more? "point read_csv() to that copy?


#8

Sure, here’s an example:

library(readr)
if(!file.exists("BC_Liquor_Store_Product_Price_list.csv")){
  download.file("https://pub.data.gov.bc.ca/datasets/176284/BC_Liquor_Store_Product_Price_list.csv", "BC_Liquor_Store_Product_Price_list.csv")
}

price_list <- read_csv("BC_Liquor_Store_Product_Price_list.csv") # Point to local copy

#9

Thank you for your help.


#10

Thanks Edgar! How about Upload data files to the RStudio server through the Shiny GUI?


#11

This is another question: Could you help me with this?
When I run has_deve() after nstalling devtools I do not get TRUE but an error. I do not know what is causing it.

library(devtools)
has_devel()
“D:/R-34~1.2/bin/x64/R” --no-site-file --no-environ --no-save --no-restore --quiet CMD SHLIB
foo.c

Warning message:
running command ‘make -f “D:/R-34~1.2/etc/x64/Makeconf” -f “D:/R-34~1.2/share/make/winshlib.mk” SHLIB=“foo.dll” WIN=64 TCLBIN=64 OBJECTS=“foo.o”’ had status 127
Error: Command failed (1)


#12

Hi, yes, I think this article can help you with that: http://shiny.rstudio.com/articles/upload.html


#13

Not sure about this one, maybe @jimhester can chime in?


#14

Thanks. I will read the article.


#15

I am trying to install devtools install.packages(“devtools”) and I get the following message:
Warning in install.packages :
package ‘/tmp/Rtmp3ROZiD/downloaded_packages/devtools_1.13.4.tar.gz’ is not available (for R version 3.3.2)
is there an equivalent package for version 3.3.2?


#16

I believe that the package devtools is necessary to create a package in RStudio


#17

This might be the reason why I get
has_devel()
“D:/R-34~1.2/bin/x64/R” --no-site-file --no-environ --no-save --no-restore --quiet CMD SHLIB
foo.c

Warning message:
running command ‘make -f “D:/R-34~1.2/etc/x64/Makeconf” -f “D:/R-34~1.2/share/make/winshlib.mk” SHLIB=“foo.dll” WIN=64 TCLBIN=64 OBJECTS=“foo.o”’ had status 127
Error: Command failed (1)strong text


#18

The problem was solved by running the following command in Linix Ubuntu
sudo apt-get install libcurl4-openssl-dev libssl-dev
After that one can run install.packages(“devtools”) from the RStudio console.


#19

I found another problem. This time I am trying to install roxygen2 -install.packages(“roxygen2”)
and it cannot be done. I get a non-zero status. Do you know what is going on? Bellow is the entire output:
install.packages(“roxygen2”)
Installing package into ‘/home/giuseppa/R/x86_64-pc-linux-gnu-library/3.3’
(as ‘lib’ is unspecified)
also installing the dependency ‘xml2’

trying URL 'https://cran.rstudio.com/src/contrib/xml2_1.1.1.tar.gz
Content type ‘application/x-gzip’ length 244643 bytes (238 KB)

downloaded 238 KB

trying URL 'https://cran.rstudio.com/src/contrib/roxygen2_6.0.1.tar.gz
Content type ‘application/x-gzip’ length 150580 bytes (147 KB)

downloaded 147 KB

  • installing source package ‘xml2’ …
    ** package ‘xml2’ successfully unpacked and MD5 sums checked
    Package libxml-2.0 was not found in the pkg-config search path.
    Perhaps you should add the directory containing libxml-2.0.pc' to the PKG_CONFIG_PATH environment variable No package 'libxml-2.0' found Package libxml-2.0 was not found in the pkg-config search path. Perhaps you should add the directory containinglibxml-2.0.pc’
    to the PKG_CONFIG_PATH environment variable
    No package ‘libxml-2.0’ found
    Using PKG_CFLAGS=
    Using PKG_LIBS=-lxml2
    ------------------------- ANTICONF ERROR ---------------------------
    Configuration failed because libxml-2.0 was not found. Try installing:
  • deb: libxml2-dev (Debian, Ubuntu, etc)
  • rpm: libxml2-devel (Fedora, CentOS, RHEL)
  • csw: libxml2_dev (Solaris)
    If libxml-2.0 is already installed, check that ‘pkg-config’ is in your
    PATH and PKG_CONFIG_PATH contains a libxml-2.0.pc file. If pkg-config
    is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
    R CMD INSTALL --configure-vars=‘INCLUDE_DIR=… LIB_DIR=…’

ERROR: configuration failed for package ‘xml2’

  • removing ‘/home/giuseppa/R/x86_64-pc-linux-gnu-library/3.3/xml2’
    Warning in install.packages :
    installation of package ‘xml2’ had non-zero exit status
    ERROR: dependency ‘xml2’ is not available for package ‘roxygen2’
  • removing ‘/home/giuseppa/R/x86_64-pc-linux-gnu-library/3.3/roxygen2’
    Warning in install.packages :
    installation of package ‘roxygen2’ had non-zero exit status

The downloaded source packages are in
‘/tmp/Rtmp1icqUW/downloaded_packages’

install.packages(“roxygen2_6.0.1.tar.gz”, type=“source”, repos=NULL)
Installing package into ‘/home/giuseppa/R/x86_64-pc-linux-gnu-library/3.3’
(as ‘lib’ is unspecified)
ERROR: dependency ‘xml2’ is not available for package ‘roxygen2’

  • removing ‘/home/giuseppa/R/x86_64-pc-linux-gnu-library/3.3/roxygen2’
    Warning in install.packages :
    installation of package ‘roxygen2_6.0.1.tar.gz’ had non-zero exit status

install.packages(“xml2”)
Installing package into ‘/home/giuseppa/R/x86_64-pc-linux-gnu-library/3.3’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/xml2_1.1.1.tar.gz
Content type ‘application/x-gzip’ length 244643 bytes (238 KB)
==================================================
downloaded 238 KB

  • installing source package ‘xml2’ …
    ** package ‘xml2’ successfully unpacked and MD5 sums checked
    Package libxml-2.0 was not found in the pkg-config search path.
    Perhaps you should add the directory containing libxml-2.0.pc' to the PKG_CONFIG_PATH environment variable No package 'libxml-2.0' found Package libxml-2.0 was not found in the pkg-config search path. Perhaps you should add the directory containinglibxml-2.0.pc’
    to the PKG_CONFIG_PATH environment variable
    No package ‘libxml-2.0’ found
    Using PKG_CFLAGS=
    Using PKG_LIBS=-lxml2
    ------------------------- ANTICONF ERROR ---------------------------
    Configuration failed because libxml-2.0 was not found. Try installing:
  • deb: libxml2-dev (Debian, Ubuntu, etc)
  • rpm: libxml2-devel (Fedora, CentOS, RHEL)
  • csw: libxml2_dev (Solaris)
    If libxml-2.0 is already installed, check that ‘pkg-config’ is in your
    PATH and PKG_CONFIG_PATH contains a libxml-2.0.pc file. If pkg-config
    is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
    R CMD INSTALL --configure-vars=‘INCLUDE_DIR=… LIB_DIR=…’

ERROR: configuration failed for package ‘xml2’

  • removing ‘/home/giuseppa/R/x86_64-pc-linux-gnu-library/3.3/xml2’
    Warning in install.packages :
    installation of package ‘xml2’ had non-zero exit status

The downloaded source packages are in
‘/tmp/Rtmp1icqUW/downloaded_packages’

library(“testthat”)
install.packages(“roxygen2”)
Installing package into ‘/home/giuseppa/R/x86_64-pc-linux-gnu-library/3.3’
(as ‘lib’ is unspecified)
also installing the dependency ‘xml2’

trying URL 'https://cran.rstudio.com/src/contrib/xml2_1.1.1.tar.gz
Content type ‘application/x-gzip’ length 244643 bytes (238 KB)

downloaded 238 KB

trying URL 'https://cran.rstudio.com/src/contrib/roxygen2_6.0.1.tar.gz
Content type ‘application/x-gzip’ length 150580 bytes (147 KB)

downloaded 147 KB

  • installing source package ‘xml2’ …
    ** package ‘xml2’ successfully unpacked and MD5 sums checked
    Package libxml-2.0 was not found in the pkg-config search path.
    Perhaps you should add the directory containing libxml-2.0.pc' to the PKG_CONFIG_PATH environment variable No package 'libxml-2.0' found Package libxml-2.0 was not found in the pkg-config search path. Perhaps you should add the directory containinglibxml-2.0.pc’
    to the PKG_CONFIG_PATH environment variable
    No package ‘libxml-2.0’ found
    Using PKG_CFLAGS=
    Using PKG_LIBS=-lxml2
    ------------------------- ANTICONF ERROR ---------------------------
    Configuration failed because libxml-2.0 was not found. Try installing:
  • deb: libxml2-dev (Debian, Ubuntu, etc)
  • rpm: libxml2-devel (Fedora, CentOS, RHEL)
  • csw: libxml2_dev (Solaris)
    If libxml-2.0 is already installed, check that ‘pkg-config’ is in your
    PATH and PKG_CONFIG_PATH contains a libxml-2.0.pc file. If pkg-config
    is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
    R CMD INSTALL --configure-vars=‘INCLUDE_DIR=… LIB_DIR=…’

ERROR: configuration failed for package ‘xml2’

  • removing ‘/home/giuseppa/R/x86_64-pc-linux-gnu-library/3.3/xml2’
    Warning in install.packages :
    installation of package ‘xml2’ had non-zero exit status
    ERROR: dependency ‘xml2’ is not available for package ‘roxygen2’
  • removing ‘/home/giuseppa/R/x86_64-pc-linux-gnu-library/3.3/roxygen2’
    Warning in install.packages :
    installation of package ‘roxygen2’ had non-zero exit status

The downloaded source packages are in
‘/tmp/Rtmp1icqUW/downloaded_packages’strong text


#20

This problem has been solved too. include deb https://cran.cnr.berkeley.edu//bin/linux/ubuntu zesty/ in the etc/apt/sources.file
run the command -sudo apt-get update
run the command -sudo apt-get install r-base-dev
In the R console
install.packages(“roxygen2”)