High-Frequency Data

Spoiler, it's a LOT of data

The following section will discuss our High-Frequency Data model and help you understand the limitations, processes, and methods for accessing High-Frequency Data.

What is High-Frequency Data

In the scope of this conversation, we're referring to events captured at a rate of at least one reading per minute or higher. Intraday data is used for activity and heart rate data. CGM is used for blood glucose (CGM) data.

High-Frequency Terminology

As you learn more about High-Frequency data, you may also hear people describe this data differently. Some data sources refer to metrics calculated at this frequency as intraday, and others call it epoch or time series data.

Additionally, Validic supports activity high-frequency data in the intraday resource and glucose high-frequency data (CGM) in the cgm resource.

Supported Data Sources

At this time, there are limited data sources that expose High-Frequency Data. Below, we've outlined the requirements to access, the frequency it's delivered, and the consistency of the data.

📘

Enabling your Validic account to access High-Frequency Data

Before your account can request High-Frequency Data, you must contact your Validic representative or contact our client services at [email protected].

Data SourceInform ResourceSupported MetricsFrequencyLimitations
Fitbitintradaysteps
heart_rate
elevation
distance
avg_spo2
Per Minute
(IE count of steps accumulated over one minute)
3rd-party developers who want access to retrieve other Fitbit users’ Intraday data through the “Client” or “Server” application type is granted on a case-by-case basis. Applications must demonstrate necessity to create a great user experience. Fitbit is very supportive of non-profit research and personal projects. Commercial applications require thorough review and are subject to additional requirements. Only select applications are granted access and Fitbit reserves the right to limit this access. https://dev.fitbit.com/build/reference/web-api/intraday-requests/
Apple HealthintradaystepsPer Minute
(IE count of steps accumulated over one minute)
Requires customers use the Validic Mobile Library to access data. Steps are calculated from the Apple Watch only. Steps are an average over the reported period.
Garminintradaysteps
distance
15 minutes
(IE: count of steps accumulated over 15 minutes)
No restrictions.
Dexcomcgmblood_glucose5 Minutes, reported Hourly
(IE One event, every five minutes)
Dexcom requires pre-approval and must review your application before you're granted full access to their data. You can learn more here. When you're ready to get started, contact [email protected] and we'll help you navigate the application process. This is currently only supported in the US.

"Data from the Dexcom API is available with a one-hour delay. This delay is enforced on the data upload time, not on the timestamps of individual data points. The G5® and G6® Mobile applications upload once per hour, so the data will (on average) be 1.5 hours delayed. On the other hand, data uploaded directly from a receiver over USB (via the Dexcom CLARITY® uploader) is available immediately because it is viewed as an active, rather than passive, upload."

Data Accessibility Notes

In all circumstances, the frequency and consistency of data delivery are determined by the end-user and how often they submit data to the cloud. In most cases, high-frequency data generally appears on predictable intervals of 5, 10, 15 Minutes, up to 1 hour. However, this may not always be the case.

Additionally, when working with this data, consider a user may only sync every few days. We suggest you replicate these conditions when testing to ensure your systems can scale appropriately.

Code Samples

Spoiler, it's a LOT of data. Below you'll find examples to help set your expectations. While the identities have been changed to protect the innocent, these represent actual data collected from Validic employees.

Intraday

Below are samples of activity High-Frequency Data, which is reported as intraday.

event:data
data:{ 
   "checksum":"9a442c33c5415f3757421b1c107e7a0d",
   "created_at":"2019-10-13T14:59:50.094Z",
   "deleted_at":null,
   "end_time":"2019-10-13T14:45:00Z",
   "granularity":{ 
      "interval":15,
      "unit":"min"
   },
   "id":"f814a36d2SAMPLE33740e5481555c",
   "log_id":"30b5984SAMPLEd6ff4cf75f17e",
   "metrics":[ 
      { 
         "data_points":[ 
            { 
               "time":"2019-10-13T04:30:00.000Z",
               "value":28
            },
            { 
               "time":"2019-10-13T05:15:00.000Z",
               "value":52
            },
            { 
               "time":"2019-10-13T08:15:00.000Z",
               "value":17
            },
            { 
               "time":"2019-10-13T13:45:00.000Z",
               "value":7
            },
            { 
               "time":"2019-10-13T14:45:00.000Z",
               "value":41
            }
         ],
         "origin":"device",
         "type":"steps",
         "unit":"count"
      },
      { 
         "data_points":[ 
            { 
               "time":"2019-10-13T04:00:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T04:15:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T04:30:00.000Z",
               "value":22.68
            },
            { 
               "time":"2019-10-13T04:45:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T05:00:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T05:15:00.000Z",
               "value":42.11
            },
            { 
               "time":"2019-10-13T05:30:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T05:45:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T06:00:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T06:15:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T06:30:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T06:45:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T07:00:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T07:15:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T07:30:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T07:45:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T08:00:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T08:15:00.000Z",
               "value":13.77
            },
            { 
               "time":"2019-10-13T08:30:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T08:45:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T09:00:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T09:15:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T09:30:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T09:45:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T10:00:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T10:15:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T10:30:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T10:45:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T11:00:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T11:15:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T11:30:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T11:45:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T12:00:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T12:15:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T12:30:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T12:45:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T13:00:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T13:15:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T13:30:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T13:45:00.000Z",
               "value":5.67
            },
            { 
               "time":"2019-10-13T14:00:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T14:15:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T14:30:00.000Z",
               "value":0
            },
            { 
               "time":"2019-10-13T14:45:00.000Z",
               "value":33.22
            }
         ],
         "origin":"device",
         "type":"distance",
         "unit":"m"
      }
   ],
   "offset_origin":"source",
   "source":{ 
      "device":null,
      "type":"garmin"
   },
   "start_time":"2019-10-13T04:00:00Z",
   "tags":[ 

   ],
   "type":"intraday",
   "user":{ 
      "organization_id":"59ba7cSAMPL001c29216",
      "uid":"garmin-intraday",
      "user_id":"5c65a7bSAMPLE700019df1a5"
   },
   "utc_offset":-14400,
   "version":"1.0"
}
event:data
data:{ 
   "checksum":"0b59aa330b4b7fc632fb018d7c1f11a1",
   "created_at":"2019-09-24T16:49:01.475Z",
   "deleted_at":null,
   "end_time":"2019-09-24T03:59:00Z",
   "granularity":{ 
      "interval":1,
      "unit":"min"
   },
   "id":"bcd0f59daSAMPLE08e2e685ff",
   "log_id":"afbbfSAMPLE84b6bc9f3295",
   "metrics":[ 
      { 
         "data_points":[ 
            { 
               "time":"2019-09-23T20:14:00Z",
               "value":4
            },
            { 
               "time":"2019-09-23T20:15:00Z",
               "value":7
            },
            { 
               "time":"2019-09-23T20:16:00Z",
               "value":30
            },
            { 
               "time":"2019-09-23T20:17:00Z",
               "value":80
            },
            { 
               "time":"2019-09-23T20:18:00Z",
               "value":102
            },
            { 
               "time":"2019-09-23T20:19:00Z",
               "value":110
            },
            { 
               "time":"2019-09-23T20:20:00Z",
               "value":110
            },
            { 
               "time":"2019-09-23T20:21:00Z",
               "value":112
            },
            { 
               "time":"2019-09-23T20:22:00Z",
               "value":108
            },
            { 
               "time":"2019-09-23T20:23:00Z",
               "value":110
            },
            { 
               "time":"2019-09-23T20:24:00Z",
               "value":110
            },
            { 
               "time":"2019-09-23T20:25:00Z",
               "value":27
            },
            { 
               "time":"2019-09-23T20:40:00Z",
               "value":7
            },
            { 
               "time":"2019-09-23T20:41:00Z",
               "value":16
            },
            { 
               "time":"2019-09-23T20:42:00Z",
               "value":34
            },
            { 
               "time":"2019-09-23T20:44:00Z",
               "value":31
            },
            { 
               "time":"2019-09-23T20:45:00Z",
               "value":26
            },
            { 
               "time":"2019-09-23T20:46:00Z",
               "value":104
            },
            { 
               "time":"2019-09-23T20:47:00Z",
               "value":42
            },
            { 
               "time":"2019-09-23T20:48:00Z",
               "value":9
            },
            { 
               "time":"2019-09-23T20:49:00Z",
               "value":18
            },
            { 
               "time":"2019-09-23T20:57:00Z",
               "value":14
            },
            { 
               "time":"2019-09-23T20:58:00Z",
               "value":18
            }
         ],
         "origin":"device",
         "type":"steps",
         "unit":"count"
      }
   ],
   "offset_origin":"profile",
   "source":{ 
      "device":null,
      "type":"fitbit"
   },
   "start_time":"2019-09-23T13:54:00Z",
   "tags":[ 
      { 
         "name":"range_start",
         "value":"2019-09-24T09:54:00+00:00"
      },
      { 
         "name":"range_end",
         "value":"2019-09-23T23:59:59+00:00"
      }
   ],
   "type":"intraday",
   "user":{ 
      "organization_id":"59ba7cdafSAMPLEc29216",
      "uid":"fitbit-intraday",
      "user_id":"5c1003aSAMPLE012a6fc1"
   },
   "utc_offset":-14400,
   "version":"1.0"
}
event:data
data:{ 
   "checksum":"0c1e07c59b1e2b03bf0cffdf633e8e5e",
   "created_at":"2019-10-14T12:29:55.743Z",
   "deleted_at":null,
   "end_time":"2019-10-14T12:28:00Z",
   "granularity":{ 
      "interval":1,
      "unit":"min"
   },
   "id":"9ae640f4219SAMPLEc7d912df027c",
   "log_id":"25bb6e041SAMPLE9ee53000a14a6",
   "metrics":[ 
      { 
         "data_points":[ 
            { 
               "time":"2019-10-14T11:28:00Z",
               "value":86
            },
            { 
               "time":"2019-10-14T11:29:00Z",
               "value":82
            },
            { 
               "time":"2019-10-14T11:30:00Z",
               "value":74
            },
            { 
               "time":"2019-10-14T11:31:00Z",
               "value":72
            },
            { 
               "time":"2019-10-14T11:32:00Z",
               "value":72
            },
            { 
               "time":"2019-10-14T11:33:00Z",
               "value":70
            },
            { 
               "time":"2019-10-14T11:34:00Z",
               "value":70
            },
            { 
               "time":"2019-10-14T11:35:00Z",
               "value":71
            },
            { 
               "time":"2019-10-14T11:36:00Z",
               "value":71
            },
            { 
               "time":"2019-10-14T11:37:00Z",
               "value":71
            },
            { 
               "time":"2019-10-14T11:38:00Z",
               "value":71
            },
            { 
               "time":"2019-10-14T11:39:00Z",
               "value":70
            },
            { 
               "time":"2019-10-14T11:40:00Z",
               "value":69
            },
            { 
               "time":"2019-10-14T11:41:00Z",
               "value":70
            },
            { 
               "time":"2019-10-14T11:42:00Z",
               "value":72
            },
            { 
               "time":"2019-10-14T11:43:00Z",
               "value":75
            },
            { 
               "time":"2019-10-14T11:44:00Z",
               "value":69
            },
            { 
               "time":"2019-10-14T11:45:00Z",
               "value":70
            },
            { 
               "time":"2019-10-14T11:46:00Z",
               "value":69
            },
            { 
               "time":"2019-10-14T11:47:00Z",
               "value":68
            },
            { 
               "time":"2019-10-14T11:48:00Z",
               "value":69
            },
            { 
               "time":"2019-10-14T11:49:00Z",
               "value":70
            },
            { 
               "time":"2019-10-14T11:50:00Z",
               "value":68
            },
            { 
               "time":"2019-10-14T11:51:00Z",
               "value":70
            },
            { 
               "time":"2019-10-14T11:52:00Z",
               "value":70
            },
            { 
               "time":"2019-10-14T11:53:00Z",
               "value":69
            },
            { 
               "time":"2019-10-14T11:54:00Z",
               "value":67
            },
            { 
               "time":"2019-10-14T11:55:00Z",
               "value":67
            },
            { 
               "time":"2019-10-14T11:56:00Z",
               "value":67
            },
            { 
               "time":"2019-10-14T11:57:00Z",
               "value":67
            },
            { 
               "time":"2019-10-14T11:58:00Z",
               "value":67
            },
            { 
               "time":"2019-10-14T11:59:00Z",
               "value":67
            },
            { 
               "time":"2019-10-14T12:00:00Z",
               "value":65
            },
            { 
               "time":"2019-10-14T12:01:00Z",
               "value":66
            },
            { 
               "time":"2019-10-14T12:02:00Z",
               "value":65
            },
            { 
               "time":"2019-10-14T12:03:00Z",
               "value":64
            },
            { 
               "time":"2019-10-14T12:04:00Z",
               "value":66
            },
            { 
               "time":"2019-10-14T12:05:00Z",
               "value":65
            },
            { 
               "time":"2019-10-14T12:06:00Z",
               "value":66
            },
            { 
               "time":"2019-10-14T12:07:00Z",
               "value":66
            },
            { 
               "time":"2019-10-14T12:08:00Z",
               "value":66
            },
            { 
               "time":"2019-10-14T12:09:00Z",
               "value":66
            },
            { 
               "time":"2019-10-14T12:10:00Z",
               "value":66
            },
            { 
               "time":"2019-10-14T12:11:00Z",
               "value":65
            },
            { 
               "time":"2019-10-14T12:12:00Z",
               "value":64
            },
            { 
               "time":"2019-10-14T12:13:00Z",
               "value":65
            },
            { 
               "time":"2019-10-14T12:14:00Z",
               "value":64
            },
            { 
               "time":"2019-10-14T12:15:00Z",
               "value":65
            },
            { 
               "time":"2019-10-14T12:16:00Z",
               "value":66
            },
            { 
               "time":"2019-10-14T12:17:00Z",
               "value":66
            },
            { 
               "time":"2019-10-14T12:18:00Z",
               "value":64
            },
            { 
               "time":"2019-10-14T12:19:00Z",
               "value":65
            },
            { 
               "time":"2019-10-14T12:20:00Z",
               "value":67
            },
            { 
               "time":"2019-10-14T12:21:00Z",
               "value":67
            },
            { 
               "time":"2019-10-14T12:22:00Z",
               "value":64
            },
            { 
               "time":"2019-10-14T12:23:00Z",
               "value":63
            },
            { 
               "time":"2019-10-14T12:24:00Z",
               "value":63
            },
            { 
               "time":"2019-10-14T12:25:00Z",
               "value":64
            },
            { 
               "time":"2019-10-14T12:26:00Z",
               "value":66
            },
            { 
               "time":"2019-10-14T12:27:00Z",
               "value":61
            },
            { 
               "time":"2019-10-14T12:28:00Z",
               "value":62
            }
         ],
         "origin":"device",
         "type":"avg_heart_rate",
         "unit":"bpm"
      }
   ],
   "offset_origin":"profile",
   "source":{ 
      "device":null,
      "type":"fitbit"
   },
   "start_time":"2019-10-14T11:28:00Z",
   "tags":[ 
      { 
         "name":"range_start",
         "value":"2019-10-14T00:00:00Z"
      },
      { 
         "name":"range_end",
         "value":"2019-10-14T23:59:59Z"
      }
   ],
   "type":"intraday",
   "user":{ 
      "organization_id":"59ba7cdaSAMPLEc29216",
      "uid":"fitbit-intraday",
      "user_id":"5c1003SAMPLE00012a6fc1"
   },
   "utc_offset":-14400,
   "version":"1.0"
}

CGM

Below are samples of CGM High-Frequency Data, which is reported as cgm.

event:data
data: {
  "type": "cgm",
  "checksum": "51502ff934e3b47b3df8284494f814b8",
  "created_at": "2024-10-25T18:23:33.220Z",
  "deleted_at": null,
  "end_time": "2024-10-24T23:59:59Z",
  "id": "7db302193cc6b9b80405f189be67b9c8",
  "log_id": "2024-10-24",
  "metrics": [
    {
      "origin": "device",
      "type": "blood_glucose",
      "unit": "mg/dL",
      "reading_log": [
      ...,
        {
          "time": "2024-10-24T01:59:33Z",
          "value": 315,
          "display_time": "2024-10-23T21:59:33-04:00",
          "scheduled": true,
          "device_id": "69bbc1b66cfe49597c5eff1376ac2cd1cad5dae010a997b0a2879ffad39b07d9"
        },
        {
          "time": "2024-10-24T02:04:32Z",
          "value": 308,
          "display_time": "2024-10-23T22:04:32-04:00",
          "scheduled": true,
          "device_id": "69bbc1b66cfe49597c5eff1376ac2cd1cad5dae010a997b0a2879ffad39b07d9"
        },
        {
          "time": "2024-10-24T02:09:32Z",
          "value": 284,
          "display_time": "2024-10-23T22:09:32-04:00",
          "scheduled": true,
          "device_id": "69bbc1b66cfe49597c5eff1376ac2cd1cad5dae010a997b0a2879ffad39b07d9"
        },
      ...  
      ]
    }
  ],
  "source": {
    "type": "dexcom",
    "devices": {
      "69bbc1b66cfe49597c5eff1376ac2cd1cad5dae010a997b0a2879ffad39b07d9": {
        "id": "69bbc1b66cfe49597c5eff1376ac2cd1cad5dae010a997b0a2879ffad39b07d9",
        "manufacturer": "dexcom",
        "model": "g6"
      }
    }
  },
  "start_time": "2024-10-24T00:00:00Z",
  "tags": [],
  "user": {
    "organization_id": "6584a3e40f32d5000fc254a7",
    "uid": "7fd835687f1f41ed8b125ca0ebfeaeb8",
    "user_id": "66be73022d30800011ae066e"
  },
  "user_notes": [],
  "utc_offset": -18000,
  "offset_origin": "user_defined",
  "version": "1.0"
}