Strava Data - R Views Submission

Category: Other
Repo: https://github.com/duju211/pin_strava


I am a vivid runner and cyclist. Since a couple of years, I’m recording
almost all my activities with some kind of GPS device.

I record my runs with a Garmin device and my bike rides with a Wahoo
device. Both accounts get synchronized with my Strava account. I figured
that it would be nice to directly access my data from my Strava account.

In the following text, I will describe the progress to get the data into
R.

In this analysis, the following packages are used:

library(tarchetypes)
library(conflicted)
library(tidyverse)
library(lubridate)
library(jsonlite)
library(targets)
library(httr)
library(pins)
library(httr)
library(fs)

conflict_prefer("filter", "dplyr")

Data

The whole data pipeline is implemented with the help of the targets
package. Here you can learn more
about the package and its functionalities.

Target Plan

The manifest of the target plan looks like this:

name command pattern cue_mode
my_app define_strava_app() NA thorough
my_endpoint define_strava_endpoint() NA thorough
act_col_types list(moving = col_logical(), velocity_smooth = col_number(), grade_smooth = col_number(), distance = col_number(), altitude = col_number(), heartrate = col_integer(), time = col_integer(), lat = col_number(), lng = col_number(), cadence = col_integer(), watts = col_integer()) NA thorough
my_sig define_strava_sig(my_endpoint, my_app) NA never
df_act_raw read_all_activities(my_sig) NA thorough
df_act pre_process_act(df_act_raw, athlete_id) NA thorough
act_ids pull(distinct(df_act, id)) NA thorough
df_meas read_activity_stream(act_ids, my_sig) map(act_ids) never
df_meas_all bind_rows(df_meas) NA thorough
df_meas_wide meas_wide(df_meas_all) NA thorough
df_meas_pro meas_pro(df_meas_wide) NA thorough
gg_meas vis_meas(df_meas_pro) NA thorough
gg_meas_save save_gg_meas(gg_meas) NA thorough

The most important targets of the plan are described in detail in the
following subsections.

OAuth Dance from R

To get access to your Strava data from R, you have to create a Strava
api. How to do this is documented
here.

The Strava api requires a so called OAuth dance. How this can be done
from within R is described in the following section.

Create an OAuth Strava app:

name command pattern cue_mode
my_app define_strava_app() NA thorough
define_strava_app <- function() {
  oauth_app(
    appname = "r_api",
    key = Sys.getenv("STRAVA_KEY"),
    secret = Sys.getenv("STRAVA_SECRET"))
}

You can find your STRAVA_KEY and STRAVA_SECRET variables under the
Strava api settings after you have created your own personal api. The
name of api is determined during creation. In my case I named it
r_api.

Define an endpoint:

name command pattern cue_mode
my_endpoint define_strava_endpoint() NA thorough
define_strava_endpoint <- function() {
  oauth_endpoint(
    request = NULL,
    authorize = "https://www.strava.com/oauth/authorize",
    access = "https://www.strava.com/oauth/token")
}

The authorize parameter describes the authorization url. And the
access argument is used to exchange the authenticated token.

The final authentication step. Before the user can execute the following
steps, he has to authenticate the api in the web browser.

name command pattern cue_mode
my_sig define_strava_sig(my_endpoint, my_app) NA always
define_strava_sig <- function(endpoint, app) {
  oauth2.0_token(
    endpoint, app,
    scope = "activity:read_all,activity:read,profile:read_all",
    type = NULL, use_oob = FALSE, as_header = FALSE,
    use_basic_auth = FALSE, cache = FALSE)
}

The information in my_sig can now be used to access Strava data. Set
the cue_mode of the target to ‘always’, so that the user has to
authenticate and the following api calls are all executed with an up to
date authorization token.

Activities

We are now authenticated and can directly access Strava data. At first
load an overview table of all available activities. Because the total
number of activities is unknown, use a while loop. Break the execution
of the loop, if there are no more activities to read.

name command pattern cue_mode
df_act_raw read_all_activities(my_sig) NA thorough
read_all_activities <- function(sig) {
  activities_url <- parse_url(
    "https://www.strava.com/api/v3/athlete/activities")

  act_vec <- vector(mode = "list")
  df_act <- tibble::tibble(init = "init")
  i <- 1L

  while (nrow(df_act) != 0) {
    r <- activities_url %>%
      modify_url(
        query = list(
          access_token = sig$credentials$access_token[[1]],
          page = i)) %>%
      GET()

    df_act <- content(r, as = "text") %>%
      fromJSON(flatten = TRUE) %>%
      as_tibble()
    if (nrow(df_act) != 0)
      act_vec[[i]] <- df_act
    i <- i + 1L
  }

  df_activities <- act_vec %>%
    bind_rows() %>%
    mutate(start_date = ymd_hms(start_date))
}

The resulting data frame consists of one row per activity:

## # A tibble: 592 x 60
##    resource_state name  distance moving_time elapsed_time total_elevation~ type 
##             <int> <chr>    <dbl>       <int>        <int>            <dbl> <chr>
##  1              2 "Wee~  55641          9037        10610             923. Ride 
##  2              2 "Bre~  43892.         5594         5721             606  Ride 
##  3              2 "Rad~  18244.         3454        35536             300. Ride 
##  4              2 "Sta~  28529.         4214        11105             370  Ride 
##  5              2 "Abe~     26.5           3            3               0  Ride 
##  6              2 "Men~  32175.         9077        16895             996  Ride 
##  7              2 "Boz~  29377.         5411        12836             310  Ride 
##  8              2 "Mit~   8328          1173         1259              13  Ride 
##  9              2 "Fah~  36886.         5578        11141             468  Ride 
## 10              2 "Mor~   6552.         2693         2823             104. Run  
## # ... with 582 more rows, and 53 more variables: workout_type <int>, id <dbl>,
## #   external_id <chr>, upload_id <dbl>, start_date <dttm>,
## #   start_date_local <chr>, timezone <chr>, utc_offset <dbl>,
## #   start_latlng <list>, end_latlng <list>, location_city <lgl>,
## #   location_state <lgl>, location_country <chr>, start_latitude <dbl>,
## #   start_longitude <dbl>, achievement_count <int>, kudos_count <int>,
## #   comment_count <int>, athlete_count <int>, photo_count <int>, ...

Preprocess activities. Make sure that all id columns are represented as
characters and improve the column names:

name command pattern cue_mode
df_act pre_process_act(df_act_raw, athlete_id) NA thorough
pre_process_act <- function(df_act_raw, athlete_id) {
  df_act <- df_act_raw %>%
    mutate(
      across(contains("id"), as.character),
      `athlete.id` = athlete_id)
}

Extract all ids of the activities:

name command pattern cue_mode
act_ids pull(distinct(df_act, id)) NA thorough

Measurements

Read the ‘stream’ data from Strava. A ‘stream’ is a nested list (json
format) with all available measurements of the corresponding activity.

To get all available variables and turn the result into a data frame,
define a helper function. This function takes an id of an activity and
an authentication token, which we have created earlier.

name command pattern cue_mode
df_meas read_activity_stream(act_ids, my_sig) map(act_ids) never
read_activity_stream <- function(id, sig) {
  act_url <- parse_url(stringr::str_glue(
    "https://www.strava.com/api/v3/activities/{id}/streams"))
  access_token <- sig$credentials$access_token[[1]]

  r <- modify_url(
    act_url,
    query = list(
      access_token = access_token,
      keys = str_glue(
        "distance,time,latlng,altitude,velocity_smooth,heartrate,cadence,watts,
        temp,moving,grade_smooth"))) %>%
    GET()

  stop_for_status(r)

  fromJSON(content(r, as = "text"), flatten = TRUE) %>%
    as_tibble() %>%
    mutate(id = id)
}

The target is defined with dynamic branching which maps over all
activity ids. Define the cue mode as never to make sure, that every
target runs exactly once.

Bind the single targets into one data frame:

name command pattern cue_mode
df_meas_all bind_rows(df_meas) NA thorough

The data now is represented by one row per measurement series:

## # A tibble: 4,724 x 6
##    type            data              series_type original_size resolution id    
##    <chr>           <list>            <chr>               <int> <chr>      <chr> 
##  1 moving          <lgl [9,045]>     distance             9045 high       59753~
##  2 latlng          <dbl [9,045 x 2]> distance             9045 high       59753~
##  3 velocity_smooth <dbl [9,045]>     distance             9045 high       59753~
##  4 grade_smooth    <dbl [9,045]>     distance             9045 high       59753~
##  5 distance        <dbl [9,045]>     distance             9045 high       59753~
##  6 altitude        <dbl [9,045]>     distance             9045 high       59753~
##  7 time            <int [9,045]>     distance             9045 high       59753~
##  8 moving          <lgl [5,607]>     distance             5607 high       59472~
##  9 latlng          <dbl [5,607 x 2]> distance             5607 high       59472~
## 10 velocity_smooth <dbl [5,607]>     distance             5607 high       59472~
## # ... with 4,714 more rows

Turn the data into a wide format so that every activity is one row
again:

name command pattern cue_mode
df_meas_wide meas_wide(df_meas_all) NA thorough
meas_wide <- function(df_meas) {
  pivot_wider(df_meas, names_from = type, values_from = data)
}

## # A tibble: 592 x 14
##    series_type original_size resolution id         moving        latlng velocity_smooth
##    <chr>               <int> <chr>      <chr>      <list>        <list> <list>         
##  1 distance             9045 high       5975328478 <lgl [9,045]> <dbl ~ <dbl [9,045]>  
##  2 distance             5607 high       5947271836 <lgl [5,607]> <dbl ~ <dbl [5,607]>  
##  3 distance             3460 high       5944515311 <lgl [3,460]> <dbl ~ <dbl [3,460]>  
##  4 distance             4234 high       5936333308 <lgl [4,234]> <dbl ~ <dbl [4,234]>  
##  5 distance                4 high       5936332751 <lgl [4]>     <dbl ~ <dbl [4]>      
##  6 distance             9151 high       5937299994 <lgl [9,151]> <dbl ~ <dbl [9,151]>  
##  7 distance             5551 high       5882522249 <lgl [5,551]> <dbl ~ <dbl [5,551]>  
##  8 distance             1211 high       5882541097 <lgl [1,211]> <dbl ~ <dbl [1,211]>  
##  9 distance             2186 high       5852593500 <lgl [2,186]> <dbl ~ <dbl [2,186]>  
## 10 distance             2733 high       5843110297 <lgl [2,733]> <dbl ~ <dbl [2,733]>  
## # ... with 582 more rows, and 7 more variables: grade_smooth <list>,
## #   distance <list>, altitude <list>, time <list>, heartrate <list>,
## #   cadence <list>, watts <list>

Preprocess and unnest the data. The column latlng needs special
attention, because it contains latitude and longitude information.
Separate the two measurements before unnesting all list columns.

name command pattern cue_mode
df_meas_pro meas_pro(df_meas_wide) NA thorough
meas_pro <- function(df_meas_wide) {
  df_meas_wide %>%
    mutate(
      lat = map_if(
        .x = latlng, .p = ~ !is.null(.x), .f = ~ .x[, 1]),
      lng = map_if(
        .x = latlng, .p = ~ !is.null(.x), .f = ~ .x[, 2])) %>%
    select(-c(latlng, original_size, resolution)) %>%
    unnest(where(is_list))
}

After this step every row is one point in time and every column is (if
present) a measurement at this point in time.

## # A tibble: 2,111,660 x 13
##    series_type id    moving velocity_smooth grade_smooth distance altitude  time
##    <chr>       <chr> <lgl>            <dbl>        <dbl>    <dbl>    <dbl> <int>
##  1 distance    5975~ FALSE              0            1.5      0       570.     0
##  2 distance    5975~ TRUE               0            2        6.7     570.     1
##  3 distance    5975~ TRUE               0            2.2     13.5     571.     2
##  4 distance    5975~ TRUE               0            2.9     20.5     571.     3
##  5 distance    5975~ TRUE               6.9          2.1     27.6     571      4
##  6 distance    5975~ TRUE               6.9          1.4     34.7     571.     5
##  7 distance    5975~ TRUE               7            1.4     41.5     571.     6
##  8 distance    5975~ TRUE               7            0.7     48.4     571.     7
##  9 distance    5975~ TRUE               7            0.7     55.3     571.     8
## 10 distance    5975~ TRUE               6.9          1.5     62.2     571.     9
## # ... with 2,111,650 more rows, and 5 more variables: heartrate <int>,
## #   cadence <int>, watts <int>, lat <dbl>, lng <dbl>

Visualisation

Visualize the final data by displaying the geospatial information in the
data. Every facet is one activity. Keep the rest of the plot as minimal
as possible.

name command pattern cue_mode
gg_meas vis_meas(df_meas_pro) NA thorough
vis_meas <- function(df_meas_pro) {
  df_meas_pro %>%
    filter(!is.na(lat)) %>%
    ggplot(aes(x = lng, y = lat)) +
    geom_path() +
    facet_wrap(~ id, scales = "free") +
    theme(
      axis.line = element_blank(),
      axis.text.x = element_blank(),
      axis.text.y = element_blank(),
      axis.ticks = element_blank(),
      axis.title.x = element_blank(),
      axis.title.y = element_blank(),
      legend.position = "bottom",
      panel.background = element_blank(),
      panel.border = element_blank(),
      panel.grid.major = element_blank(),
      panel.grid.minor = element_blank(),
      plot.background = element_blank(),
      strip.text = element_blank())
}


This is a submission to the R Views Call for Documentation. For more information see rviews.rstudio.com.

1 Like