Sleep Infographic Using Raspberry Pi and Fitbit API

Infographics.  They're everywhere.  Here's one I did (it's my first one so be kind!):

So why did I do this?  In a previous post I spotted how the Fitbit API contains lots of delicious Geek data for me to analyse.  Hence I wanted to extract this data, analyse it and present it.  My inclination is to present  detailed charts and tables of data but I decided to present it as an infographic as they seem to be everywhere these days.

In terms of my Fitbit Charge HR and sleep measurement, I think it's reasonably accurate.  If I've had a good night sleep then this is generally registered.  Similar for a bad nights sleep.  When I get up in the night  to answer nature's call it's always logged.  Needs more analysis to be sure though...

So the process was:

• Write a Python script on my Raspberry Pi to access the Fitbit API.
• Have the script output data in a raw format.
• Use Excel to process and present the data.  

Full code is at the bottom but highlights from it are summarised below.

Accessing the API

Uses methods described in this previous post.  Snippet:

#Get a client

authd_client = fitbit.Fitbit(CLIENT_KEY, CLIENT_SECRET, resource_owner_key=USER_KEY, resource_owner_secret=USER_SECRET)

Calculating How Many Days to Process For

The Fitbit API takes a single date as a parameter.  I wanted to process for every day that I'd had the Fitbit.  Hence I wrote a function called CountTheDays() to count how many days had passed between the day the script is run and the day I got my Fitbit.

Looping through Each Day

Having the number of days to process for, I could then create a for loop to look at each day in turn.  

This snippet:

for i in range(DayCount,-1,-1):

  #Get the date to process

  DateForAPI = ComputeADate(i)

Means you loop with the loop var i decrementing each time.  It starts off back at the first day I got the Fitbit and finishes on today's date.

The code then gets an actual date using ComputeADate(i) which means it computes a date that is i days different from today.

Getting the API Data

This method gets the API data for you:

fitbit_sleep = authd_client._COLLECTION_RESOURCE('sleep',DateForAPI)

...and the resulting JSON structure starts like this:

{u'sleep': [{u'logId': 778793259, u'isMainSleep': True, u'minutesToFallAsleep': 0, u'awakeningsCount': 26, u'minutesAwake': 45, u'timeInBed': 483, u'minutesAsleep': 417, u'awakeDuration': 11, u'efficiency': 90, u'startTime': u'2015-03-16T22:18:00.000', u'restlessCount': 29, u'duration': 28980000, u'restlessDuration': 52, u'minuteData': 

....has a bunch of "per minute" records like this....

{u'value': u'1', u'dateTime': u'22:38:00'}, {u'value': u'1', u'dateTime': u'22:39:00'}, {u'value': u'1', u'dateTime': u'22:40:00'}

...then finishes with more summary data like this.

u'awakeCount': 3, u'minutesAfterWakeup': 1}], u'summary': {u'totalTimeInBed': 483, u'totalMinutesAsleep': 417, u'totalSleepRecords': 1}}

Extracting Elements of the JSON

Using my patented trial and error method I managed to work out you pick out different parts of the JSON like this:

#Get the total minutes in bed value.  This will control our loop that gets the sleep

MinsInBed = fitbit_sleep['sleep'][0]['timeInBed']

#Get the total sleep value

MinsAsleep = fitbit_sleep['sleep'][0]['minutesAsleep']

...and that the [timeInBed] field could be used to determine how many per minute records there are.  Hence you can loop, logging each of the per minute records:

#Loop through the lot

  for i in range(0,MinsInBed):

    SleepVal = fitbit_sleep['sleep'][0]['minuteData'][i]['value']

    TimeVal = fitbit_sleep['sleep'][0]['minuteData'][i]['dateTime']

    MyFile.write(DateForAPI + ',' + TimeVal + ',' + SleepVal + '\r\n')

    

If this is the first loop iteration, grab the time which is when Fitbit thought I went to bed:

    if (i == 0):

      FirstTimeInBed = TimeVal

Logging to File

In the loop above I log the per minute data to file.  I also log summary data to file like this:

#Write a log of summary data

  SummaryFile.write(DateForAPI + ',' + str(MinsInBed) + ',' + str(MinsAsleep) + ',' + FirstTimeInBed + '\r\n')

One of these logs is written per day.

Analysing the Summary Data

I'll leave analysis of the per minute data until later, (can't think what to do with it now).  The summary data looks like this:

2015-01-27,478,434,22:21:00

2015-01-28,447,420,22:50:00

2015-01-29,491,446,22:11:00

2015-01-30,414,359,23:29:00

The format is:

<Date>,<Minutes in bed>,<Minutes asleep>,<Bed Time>

So for the infographic I used Excel to:

1-Add up all the time asleep values to give the total time asleep (expressed in hours).

2-Worked out the average time asleep each night (again expressed in hours).

3-Used a pivot table to examine the data on a per day of week basis.

So the Saturday to Sunday sleep is my best at a mean of 7.6 hours.  Sunday to Monday sleep is the worst at 5.8 hours (this is because I do a swim training session which finishes late on a Sunday night so I go to bed later and am also a bit wired so it takes me longer to get to sleep). 

4-Worked out an "efficiency" figure by calculating what proportion of time in bed I'm actually asleep for.

5-Looked at the frequency distribution of Fitbit logged bed time:

The mode value of 22:12 went on the infographic.  Latest bed time is still before midnight.  Rock 'n' roll baby!

Full Code

Secret stuff changed of course.  Look at my previous posts to see how to authenticate etc.

#V1 - Minute by minute data

#V2 - Summary data added

import fitbit

from datetime import datetime, timedelta

#Constants

CLIENT_KEY = 'uptownfunkyouup'

CLIENT_SECRET = 'livinlifeinthecity'

USER_KEY = 'iliketomoveitmoveit'

#USER_KEY = 'iliketomoveitmove'

USER_SECRET = 'iliketohuhmoveit'

#The first date I used Fitbit

FirstFitbitDate = '2015-01-27'

#Determine how many days to process for.  First day I ever logged was 2015-01-27

def CountTheDays():

  #See how many days there's been between today and my first Fitbit date.

  now = datetime.now()                                         #Todays date

  FirstDate = datetime.strptime(FirstFitbitDate,"%Y-%m-%d")    #First Fitbit dat

e as a Python date object

  #Calculate difference between the two and return it

  return abs((now - FirstDate).days)

#Produce a date in yyyy-mm-dd format that is n days before today's date (where n is a passed parameter)

def ComputeADate(DaysDiff):

  #Get today's date

  now = datetime.now()

  #Compute the difference betwen now and the day difference paremeter passed

  DateResult = now - timedelta(days=DaysDiff)

  return DateResult.strftime("%Y-%m-%d")

#Get a client

authd_client = fitbit.Fitbit(CLIENT_KEY, CLIENT_SECRET, resource_owner_key=USER_KEY, resource_owner_secret=USER_SECRET)

#Find out how many days to compute for

DayCount = CountTheDays()

#Open a file to write the output - minute by minute and summary

FileToWrite = '/home/pi/sleep/' + 'minuteByMinute_' + datetime.now().strftime("%Y-%m-%d") + '.csv'

MyFile = open(FileToWrite,'w')

SummaryFileToWrite = '/home/pi/sleep/' + 'summary_' + datetime.now().strftime("%Y-%m-%d") + '.csv'

SummaryFile = open(SummaryFileToWrite,'w')

#Process each one of these days stepping back in the for loop and thus stepping up in time

for i in range(DayCount,-1,-1):

  #Get the date to process

  DateForAPI = ComputeADate(i)

  #Tell the user what is happening

  print 'Processing this date: ' + DateForAPI

  #Get sleep

  fitbit_sleep = authd_client._COLLECTION_RESOURCE('sleep',DateForAPI)

  #Get the total minutes in bed value.  This will control our loop that gets the sleep

  MinsInBed = fitbit_sleep['sleep'][0]['timeInBed']

  #Get the total sleep value

  MinsAsleep = fitbit_sleep['sleep'][0]['minutesAsleep']

  #Loop through the lot

  for i in range(0,MinsInBed):

    SleepVal = fitbit_sleep['sleep'][0]['minuteData'][i]['value']

    TimeVal = fitbit_sleep['sleep'][0]['minuteData'][i]['dateTime']

    MyFile.write(DateForAPI + ',' + TimeVal + ',' + SleepVal + '\r\n')

    #If this is the first itteration of the loop grab the time which says when I got to bed

    if (i == 0):

      FirstTimeInBed = TimeVal

  #Write a log of summary data

  SummaryFile.write(DateForAPI + ',' + str(MinsInBed) + ',' + str(MinsAsleep) + ',' + FirstTimeInBed + '\r\n')

#We're now at the end of theloops.  Close the file

MyFile.close()

SummaryFile.close()