Can IoT sensor data from Python program be automatically connected to R?

[marwanzorkani]

I am new to the programing world and i am planning to learn Python for many automation projects that i want to make in the near future. I will be using Python for automation for agriculture purposes, IoT, and to build modules with Quantum Geographic Information System (QGIS). In addition, after I master the use of Python I want to start going to the field of machine learn and artificial intelligence in general.
I have looked into different coding languages like SQL, MySQL, NoSQL, and T-SQL and I do like the feature of each one. However, I am not keen to use any of them because I will need to use multiple software to get the analytical outcome results that I will need since there is no software that combine the four languages.
I will not be merging datasets to the programs that I will make by using Python. For example, I want to build weather station by using five or more open source sensors and transfer their sensor data log to make tables and graphs. The weather station will be connected to the internet and the all the sensor will be running on real time and collecting data every one second. I found ThingSpeak (thingspeak.com) which does the functions of logs data collections from sensors and automatically apply it to tables and customized graphs. I do not want to use a middle application like ThingSpeak to collect and arrange my data because my data will be passing to multiple sources and at some point I will be obligated to subscribe and pay money.

1. I studied statistics for business and have good background about statistics but i know that for fact that statistics is big ocean. Can RStudio be an alternative for (SQL, MySQL, NoSQL, T-SQL)? if yes, what topics in statistics will i need to cover?

2. Can RStudio be used to create database tables?

3. Can RStudio be used instead of ThingSpeak to collect and arrange data from Python and apply them to customized graphs automated on real time?

4. Are there features that RStudio can be used for IoT projects?

5. Do you have recommendation for me or other point of view that can help me?

[technocrat]

Approach from the general to the specific.

IoT is telemetry. Signals travel from one device to another, either one way (turn on/turn off) or two way(turn on/return image). The program handling the communication runs according to a protocol that can be programmed against. Decide whether to give over to proprietary protocols or open.

Protocols can be implemented variously. A lowest common denominator approach has benefits if the universe of devices is known. csv works everywhere. If the universe is unknown, xml acts as a self-documenting data format.

The data store in which to capture depends on volume and frequency. Appending to csv is a standard i/o operation, but is tricky to handle when there are race conditions (two devices trying to write at the same time). That is and large volumes are when SQL is indicated. It is also useful when data validation is required.

Give data in storage, the volume influences the choice of analytic software. R is limited by memory; it has to have the data to be processed in memory at the same time. This is only relevant with very large datasets.

The scope of statistical analysis determines whether R should be considered over Python. Basic descriptive statistics can be done in either. R has an advantage in advanced statistics.

R has more expressive graphing capabilities than Python. If the finish of presentation is important, that argues in favor of R.

Learning two languages at the same time can be difficult if the languages dififer greatly in philosophy. Just learning Russian and Ukrainian (C and Ruby) is easier than learning Basque and Japanese(R and Python). Python is a procedural/imperative language: do this, do that, and then do the other. R is functional, f(x) = y. Having Python as a first language can make it difficult to pick up R; less so, v.v.

The subsequent development of a project created in a scripting language, such as R or Python to scale to large demands, may require compiled language implementation. It is generally easier to migrate Python to C++, say, than R to Go or Haskell.

Specific questions.

1. Undergraduate level business statistics is an R built in. Beyond that depends on the specific questions to be put to particular data.
2. RStudio is a GUI to R; database tables as data frames are native to R and extensions, such as tibble enhance them.
3. R is indifferent to what happened to the data before arriving in a readable format, such as csv or SQL.
4. There are some R projects specifically aimed at IoT, such as the iotables package.
5. Keep in mind that RStudio can be used as a Python IDE

[jlacko]

Your description is a broad one; my understanding (and I can be wrong, in which case do forgive me) is that you are looking for a persistent data storage. If this is the case then you should consider a database.

A database need not be prohibitively expensive - with a little googling you should find dozen walkthroughs for Hadoop installations running on Raspberry pi's - and setting one up yourself is quite fun + it would give you a first hand experience in data engineering. Which is a rather marketable skill to have.