High-Frequency Data
Spoiler, it's a LOT of data
The following section will discuss our High-Frequency Data model and help you understand the limitations, processes, and methods for accessing High-Frequency Data.
What is High-Frequency Data
In the scope of this conversation, we're referring to events captured at a rate of at least one reading per minute or higher. Intraday data is used for activity and heart rate data. CGM is used for blood glucose (CGM) data.
High-Frequency Terminology
As you learn more about High-Frequency data, you may also hear people describe this data differently. Some data sources refer to metrics calculated at this frequency as intraday, and others call it epoch or time series data.
Additionally, Validic supports activity high-frequency data in the intraday resource and glucose high-frequency data (CGM) in the cgm resource.
Supported Data Sources
At this time, there are limited data sources that expose High-Frequency Data. Below, we've outlined the requirements to access, the frequency it's delivered, and the consistency of the data.
Enabling your Validic account to access High-Frequency Data
Before your account can request High-Frequency Data, you must contact your Validic representative or contact our client services at [email protected].
| Data Source | Inform Resource | Supported Metrics | Frequency | Limitations |
|---|---|---|---|---|
| Fitbit | intraday | steps heart_rate elevation distance avg_spo2 | Per Minute (IE count of steps accumulated over one minute) | 3rd-party developers who want access to retrieve other Fitbit users’ Intraday data through the “Client” or “Server” application type is granted on a case-by-case basis. Applications must demonstrate necessity to create a great user experience. Fitbit is very supportive of non-profit research and personal projects. Commercial applications require thorough review and are subject to additional requirements. Only select applications are granted access and Fitbit reserves the right to limit this access. https://dev.fitbit.com/build/reference/web-api/intraday-requests/ |
| Apple Health | intraday | steps | Per Minute (IE count of steps accumulated over one minute) | Requires customers use the Validic Mobile Library to access data. Steps are calculated from the Apple Watch only. Steps are an average over the reported period. |
| Garmin | intraday | steps distance | 15 minutes (IE: count of steps accumulated over 15 minutes) | No restrictions. |
| Dexcom | cgm | blood_glucose | 5 Minutes, reported Hourly (IE One event, every five minutes) | Dexcom requires pre-approval and must review your application before you're granted full access to their data. You can learn more here. When you're ready to get started, contact [email protected] and we'll help you navigate the application process. This is currently only supported in the US and EU. "Data from the Dexcom API is available with a one-hour delay. This delay is enforced on the data upload time, not on the timestamps of individual data points. The G5® and G6® Mobile applications upload once per hour, so the data will (on average) be 1.5 hours delayed. On the other hand, data uploaded directly from a receiver over USB (via the Dexcom CLARITY® uploader) is available immediately because it is viewed as an active, rather than passive, upload." |
| Abbott | cgm | blood_glucose | Varies by sensor, reported Hourly | Abbott requires pre-approval and must review your application before you're granted full access to their data. When you're ready to get started, contact [email protected] and we'll help you navigate the application process. This is currently only supported in the US. Data from the Abbott API is available with a one-hour delay, so the data will (on average) be 1.5 hours delayed. |
Data Accessibility Notes
In all circumstances, the frequency and consistency of data delivery are determined by the end-user and how often they submit data to the cloud. In most cases, high-frequency data generally appears on predictable intervals of 5, 10, 15 Minutes, up to 1 hour. However, this may not always be the case.
Additionally, when working with this data, consider a user may only sync every few days. We suggest you replicate these conditions when testing to ensure your systems can scale appropriately.
Code Samples
Spoiler, it's a LOT of data. Below you'll find examples to help set your expectations. While the identities have been changed to protect the innocent, these represent actual data collected from Validic employees.
Intraday
Below are samples of activity High-Frequency Data, which is reported as intraday.
event:data
data:{
"checksum":"9a442c33c5415f3757421b1c107e7a0d",
"created_at":"2019-10-13T14:59:50.094Z",
"deleted_at":null,
"end_time":"2019-10-13T14:45:00Z",
"granularity":{
"interval":15,
"unit":"min"
},
"id":"f814a36d2SAMPLE33740e5481555c",
"log_id":"30b5984SAMPLEd6ff4cf75f17e",
"metrics":[
{
"data_points":[
{
"time":"2019-10-13T04:30:00.000Z",
"value":28
},
{
"time":"2019-10-13T05:15:00.000Z",
"value":52
},
{
"time":"2019-10-13T08:15:00.000Z",
"value":17
},
{
"time":"2019-10-13T13:45:00.000Z",
"value":7
},
{
"time":"2019-10-13T14:45:00.000Z",
"value":41
}
],
"origin":"device",
"type":"steps",
"unit":"count"
},
{
"data_points":[
{
"time":"2019-10-13T04:00:00.000Z",
"value":0
},
{
"time":"2019-10-13T04:15:00.000Z",
"value":0
},
{
"time":"2019-10-13T04:30:00.000Z",
"value":22.68
},
{
"time":"2019-10-13T04:45:00.000Z",
"value":0
},
{
"time":"2019-10-13T05:00:00.000Z",
"value":0
},
{
"time":"2019-10-13T05:15:00.000Z",
"value":42.11
},
{
"time":"2019-10-13T05:30:00.000Z",
"value":0
},
{
"time":"2019-10-13T05:45:00.000Z",
"value":0
},
{
"time":"2019-10-13T06:00:00.000Z",
"value":0
},
{
"time":"2019-10-13T06:15:00.000Z",
"value":0
},
{
"time":"2019-10-13T06:30:00.000Z",
"value":0
},
{
"time":"2019-10-13T06:45:00.000Z",
"value":0
},
{
"time":"2019-10-13T07:00:00.000Z",
"value":0
},
{
"time":"2019-10-13T07:15:00.000Z",
"value":0
},
{
"time":"2019-10-13T07:30:00.000Z",
"value":0
},
{
"time":"2019-10-13T07:45:00.000Z",
"value":0
},
{
"time":"2019-10-13T08:00:00.000Z",
"value":0
},
{
"time":"2019-10-13T08:15:00.000Z",
"value":13.77
},
{
"time":"2019-10-13T08:30:00.000Z",
"value":0
},
{
"time":"2019-10-13T08:45:00.000Z",
"value":0
},
{
"time":"2019-10-13T09:00:00.000Z",
"value":0
},
{
"time":"2019-10-13T09:15:00.000Z",
"value":0
},
{
"time":"2019-10-13T09:30:00.000Z",
"value":0
},
{
"time":"2019-10-13T09:45:00.000Z",
"value":0
},
{
"time":"2019-10-13T10:00:00.000Z",
"value":0
},
{
"time":"2019-10-13T10:15:00.000Z",
"value":0
},
{
"time":"2019-10-13T10:30:00.000Z",
"value":0
},
{
"time":"2019-10-13T10:45:00.000Z",
"value":0
},
{
"time":"2019-10-13T11:00:00.000Z",
"value":0
},
{
"time":"2019-10-13T11:15:00.000Z",
"value":0
},
{
"time":"2019-10-13T11:30:00.000Z",
"value":0
},
{
"time":"2019-10-13T11:45:00.000Z",
"value":0
},
{
"time":"2019-10-13T12:00:00.000Z",
"value":0
},
{
"time":"2019-10-13T12:15:00.000Z",
"value":0
},
{
"time":"2019-10-13T12:30:00.000Z",
"value":0
},
{
"time":"2019-10-13T12:45:00.000Z",
"value":0
},
{
"time":"2019-10-13T13:00:00.000Z",
"value":0
},
{
"time":"2019-10-13T13:15:00.000Z",
"value":0
},
{
"time":"2019-10-13T13:30:00.000Z",
"value":0
},
{
"time":"2019-10-13T13:45:00.000Z",
"value":5.67
},
{
"time":"2019-10-13T14:00:00.000Z",
"value":0
},
{
"time":"2019-10-13T14:15:00.000Z",
"value":0
},
{
"time":"2019-10-13T14:30:00.000Z",
"value":0
},
{
"time":"2019-10-13T14:45:00.000Z",
"value":33.22
}
],
"origin":"device",
"type":"distance",
"unit":"m"
}
],
"offset_origin":"source",
"source":{
"device":null,
"type":"garmin"
},
"start_time":"2019-10-13T04:00:00Z",
"tags":[
],
"type":"intraday",
"user":{
"organization_id":"59ba7cSAMPL001c29216",
"uid":"garmin-intraday",
"user_id":"5c65a7bSAMPLE700019df1a5"
},
"utc_offset":-14400,
"version":"1.0"
}
event:data
data:{
"checksum":"0b59aa330b4b7fc632fb018d7c1f11a1",
"created_at":"2019-09-24T16:49:01.475Z",
"deleted_at":null,
"end_time":"2019-09-24T03:59:00Z",
"granularity":{
"interval":1,
"unit":"min"
},
"id":"bcd0f59daSAMPLE08e2e685ff",
"log_id":"afbbfSAMPLE84b6bc9f3295",
"metrics":[
{
"data_points":[
{
"time":"2019-09-23T20:14:00Z",
"value":4
},
{
"time":"2019-09-23T20:15:00Z",
"value":7
},
{
"time":"2019-09-23T20:16:00Z",
"value":30
},
{
"time":"2019-09-23T20:17:00Z",
"value":80
},
{
"time":"2019-09-23T20:18:00Z",
"value":102
},
{
"time":"2019-09-23T20:19:00Z",
"value":110
},
{
"time":"2019-09-23T20:20:00Z",
"value":110
},
{
"time":"2019-09-23T20:21:00Z",
"value":112
},
{
"time":"2019-09-23T20:22:00Z",
"value":108
},
{
"time":"2019-09-23T20:23:00Z",
"value":110
},
{
"time":"2019-09-23T20:24:00Z",
"value":110
},
{
"time":"2019-09-23T20:25:00Z",
"value":27
},
{
"time":"2019-09-23T20:40:00Z",
"value":7
},
{
"time":"2019-09-23T20:41:00Z",
"value":16
},
{
"time":"2019-09-23T20:42:00Z",
"value":34
},
{
"time":"2019-09-23T20:44:00Z",
"value":31
},
{
"time":"2019-09-23T20:45:00Z",
"value":26
},
{
"time":"2019-09-23T20:46:00Z",
"value":104
},
{
"time":"2019-09-23T20:47:00Z",
"value":42
},
{
"time":"2019-09-23T20:48:00Z",
"value":9
},
{
"time":"2019-09-23T20:49:00Z",
"value":18
},
{
"time":"2019-09-23T20:57:00Z",
"value":14
},
{
"time":"2019-09-23T20:58:00Z",
"value":18
}
],
"origin":"device",
"type":"steps",
"unit":"count"
}
],
"offset_origin":"profile",
"source":{
"device":null,
"type":"fitbit"
},
"start_time":"2019-09-23T13:54:00Z",
"tags":[
{
"name":"range_start",
"value":"2019-09-24T09:54:00+00:00"
},
{
"name":"range_end",
"value":"2019-09-23T23:59:59+00:00"
}
],
"type":"intraday",
"user":{
"organization_id":"59ba7cdafSAMPLEc29216",
"uid":"fitbit-intraday",
"user_id":"5c1003aSAMPLE012a6fc1"
},
"utc_offset":-14400,
"version":"1.0"
}
event:data
data:{
"checksum":"0c1e07c59b1e2b03bf0cffdf633e8e5e",
"created_at":"2019-10-14T12:29:55.743Z",
"deleted_at":null,
"end_time":"2019-10-14T12:28:00Z",
"granularity":{
"interval":1,
"unit":"min"
},
"id":"9ae640f4219SAMPLEc7d912df027c",
"log_id":"25bb6e041SAMPLE9ee53000a14a6",
"metrics":[
{
"data_points":[
{
"time":"2019-10-14T11:28:00Z",
"value":86
},
{
"time":"2019-10-14T11:29:00Z",
"value":82
},
{
"time":"2019-10-14T11:30:00Z",
"value":74
},
{
"time":"2019-10-14T11:31:00Z",
"value":72
},
{
"time":"2019-10-14T11:32:00Z",
"value":72
},
{
"time":"2019-10-14T11:33:00Z",
"value":70
},
{
"time":"2019-10-14T11:34:00Z",
"value":70
},
{
"time":"2019-10-14T11:35:00Z",
"value":71
},
{
"time":"2019-10-14T11:36:00Z",
"value":71
},
{
"time":"2019-10-14T11:37:00Z",
"value":71
},
{
"time":"2019-10-14T11:38:00Z",
"value":71
},
{
"time":"2019-10-14T11:39:00Z",
"value":70
},
{
"time":"2019-10-14T11:40:00Z",
"value":69
},
{
"time":"2019-10-14T11:41:00Z",
"value":70
},
{
"time":"2019-10-14T11:42:00Z",
"value":72
},
{
"time":"2019-10-14T11:43:00Z",
"value":75
},
{
"time":"2019-10-14T11:44:00Z",
"value":69
},
{
"time":"2019-10-14T11:45:00Z",
"value":70
},
{
"time":"2019-10-14T11:46:00Z",
"value":69
},
{
"time":"2019-10-14T11:47:00Z",
"value":68
},
{
"time":"2019-10-14T11:48:00Z",
"value":69
},
{
"time":"2019-10-14T11:49:00Z",
"value":70
},
{
"time":"2019-10-14T11:50:00Z",
"value":68
},
{
"time":"2019-10-14T11:51:00Z",
"value":70
},
{
"time":"2019-10-14T11:52:00Z",
"value":70
},
{
"time":"2019-10-14T11:53:00Z",
"value":69
},
{
"time":"2019-10-14T11:54:00Z",
"value":67
},
{
"time":"2019-10-14T11:55:00Z",
"value":67
},
{
"time":"2019-10-14T11:56:00Z",
"value":67
},
{
"time":"2019-10-14T11:57:00Z",
"value":67
},
{
"time":"2019-10-14T11:58:00Z",
"value":67
},
{
"time":"2019-10-14T11:59:00Z",
"value":67
},
{
"time":"2019-10-14T12:00:00Z",
"value":65
},
{
"time":"2019-10-14T12:01:00Z",
"value":66
},
{
"time":"2019-10-14T12:02:00Z",
"value":65
},
{
"time":"2019-10-14T12:03:00Z",
"value":64
},
{
"time":"2019-10-14T12:04:00Z",
"value":66
},
{
"time":"2019-10-14T12:05:00Z",
"value":65
},
{
"time":"2019-10-14T12:06:00Z",
"value":66
},
{
"time":"2019-10-14T12:07:00Z",
"value":66
},
{
"time":"2019-10-14T12:08:00Z",
"value":66
},
{
"time":"2019-10-14T12:09:00Z",
"value":66
},
{
"time":"2019-10-14T12:10:00Z",
"value":66
},
{
"time":"2019-10-14T12:11:00Z",
"value":65
},
{
"time":"2019-10-14T12:12:00Z",
"value":64
},
{
"time":"2019-10-14T12:13:00Z",
"value":65
},
{
"time":"2019-10-14T12:14:00Z",
"value":64
},
{
"time":"2019-10-14T12:15:00Z",
"value":65
},
{
"time":"2019-10-14T12:16:00Z",
"value":66
},
{
"time":"2019-10-14T12:17:00Z",
"value":66
},
{
"time":"2019-10-14T12:18:00Z",
"value":64
},
{
"time":"2019-10-14T12:19:00Z",
"value":65
},
{
"time":"2019-10-14T12:20:00Z",
"value":67
},
{
"time":"2019-10-14T12:21:00Z",
"value":67
},
{
"time":"2019-10-14T12:22:00Z",
"value":64
},
{
"time":"2019-10-14T12:23:00Z",
"value":63
},
{
"time":"2019-10-14T12:24:00Z",
"value":63
},
{
"time":"2019-10-14T12:25:00Z",
"value":64
},
{
"time":"2019-10-14T12:26:00Z",
"value":66
},
{
"time":"2019-10-14T12:27:00Z",
"value":61
},
{
"time":"2019-10-14T12:28:00Z",
"value":62
}
],
"origin":"device",
"type":"avg_heart_rate",
"unit":"bpm"
}
],
"offset_origin":"profile",
"source":{
"device":null,
"type":"fitbit"
},
"start_time":"2019-10-14T11:28:00Z",
"tags":[
{
"name":"range_start",
"value":"2019-10-14T00:00:00Z"
},
{
"name":"range_end",
"value":"2019-10-14T23:59:59Z"
}
],
"type":"intraday",
"user":{
"organization_id":"59ba7cdaSAMPLEc29216",
"uid":"fitbit-intraday",
"user_id":"5c1003SAMPLE00012a6fc1"
},
"utc_offset":-14400,
"version":"1.0"
}
CGM
Below are samples of CGM High-Frequency Data, which is reported as cgm.
event:data
data: {
"type": "cgm",
"checksum": "51502ff934e3b47b3df8284494f814b8",
"created_at": "2024-10-25T18:23:33.220Z",
"deleted_at": null,
"end_time": "2024-10-24T23:59:59Z",
"id": "7db302193cc6b9b80405f189be67b9c8",
"log_id": "2024-10-24",
"metrics": [
{
"origin": "device",
"type": "blood_glucose",
"unit": "mg/dL",
"reading_log": [
...,
{
"time": "2024-10-24T01:59:33Z",
"value": 315,
"display_time": "2024-10-23T21:59:33-04:00",
"scheduled": true,
"device_id": "69bbc1b66cfe49597c5eff1376ac2cd1cad5dae010a997b0a2879ffad39b07d9"
},
{
"time": "2024-10-24T02:04:32Z",
"value": 308,
"display_time": "2024-10-23T22:04:32-04:00",
"scheduled": true,
"device_id": "69bbc1b66cfe49597c5eff1376ac2cd1cad5dae010a997b0a2879ffad39b07d9"
},
{
"time": "2024-10-24T02:09:32Z",
"value": 284,
"display_time": "2024-10-23T22:09:32-04:00",
"scheduled": true,
"device_id": "69bbc1b66cfe49597c5eff1376ac2cd1cad5dae010a997b0a2879ffad39b07d9"
},
...
]
}
],
"source": {
"type": "dexcom",
"devices": {
"69bbc1b66cfe49597c5eff1376ac2cd1cad5dae010a997b0a2879ffad39b07d9": {
"id": "69bbc1b66cfe49597c5eff1376ac2cd1cad5dae010a997b0a2879ffad39b07d9",
"manufacturer": "dexcom",
"model": "g6"
}
}
},
"start_time": "2024-10-24T00:00:00Z",
"tags": [],
"user": {
"organization_id": "6584a3e40f32d5000fc254a7",
"uid": "7fd835687f1f41ed8b125ca0ebfeaeb8",
"user_id": "66be73022d30800011ae066e"
},
"user_notes": [],
"utc_offset": -18000,
"offset_origin": "user_defined",
"version": "1.0"
}
event:data
data: {
"type": "cgm",
"checksum": "51502ff934e3b47b3df8284494f814b8",
"created_at": "2024-10-12T18:23:33.220Z",
"deleted_at": null,
"end_time": "2024-10-11T23:59:59Z",
"id": "7db302193cc6b9b80405f189be67b9c8",
"log_id": "2024-10-11",
"metrics": [
{
"origin": "device",
"type": "blood_glucose",
"unit": "mg/dL",
"reading_log": [
...,
{
"time": "2024-10-12T01:13:47Z",
"value": 90,
"display_time": "2024-10-11T21:13:47-04:00",
"scheduled": true,
"device_id": "ed7c4e65-3f27-4de2-8f55-fa0c70d33939",
"new_sensor": false
},
{
"time": "2024-10-11T21:18:46",
"value": 82,
"display_time": "2024-10-11T21:18:46-04:00",
"scheduled": true,
"device_id": "ed7c4e65-3f27-4de2-8f55-fa0c70d33939",
"new_sensor": false
},
{
"time": "2024-10-11T21:23:46",
"value": 91,
"display_time": "2024-10-11T21:23:46-04:00",
"scheduled": true,
"device_id": "ed7c4e65-3f27-4de2-8f55-fa0c70d33939",
"new_sensor": false
},
...
]
}
],
"source": {
"type": "abbott",
"devices": {
"ed7c4e65-3f27-4de2-8f55-fa0c70d33939": {
"id": "ed7c4e65-3f27-4de2-8f55-fa0c70d33939",
"manufacturer": "abbott",
"model": "freeStyleLibre3"
}
}
},
"start_time": "2024-10-11T00:00:00Z",
"tags": [],
"user": {
"organization_id": "6584a3e40f32d5000fc254a7",
"uid": "7fd835687f1f41ed8b125ca0ebfeaeb8",
"user_id": "66be73022d30800011ae066e"
},
"user_notes": [],
"utc_offset": -18000,
"offset_origin": "user_defined",
"version": "1.0"
}
Updated about 1 month ago