More on the LightwaveRF UDP Interface and COSM

Previously I've blogged on the LightwaveRF UDP interface and logging measurements to COSM.  I've tinkered with the scripts to make them more reliable so I thought I would share a full code listing with some notes as to how to do it yourself.

Before I get into the code, here's a chart from COSM showing recent electricity costs for my home:

This shows how Christmas Eve has been the most expensive day this week, (actually the second most expensive since I started logging).  This was down to us doing stacks of cooking that day in preparation for the Christmas festivities.

The method consists of two Python scripts running on an Android handset.  One to send the command to the LightwaveRF “WiFi Link”, one to receive the response and post it to COSM.

The code listing for the sender script is shown below, (I'll then go on to describe the longer receiver script).  The only thing you'll need to change is the constant "UDP_IP" to represent the IP address that your WiFi Link is on.  I have a feature on my ADSL router to always assign this address to the WiFi link.

The script then stays in and endless loop, sending the command, pausing 60 seconds then starting again.  Change time.sleep(60) if you want it to send more / less frequently.

When you run this script, you’ll have to press a button on the WiFi Link to authorise the handset to send commands to it.  After this you’ll never have to do this again.  The try: and except: constructs allow for error capture and handling.  Until I put these in the script was quite unreliable but with them the script has been running non-stop for 2 months.

import socket
import time

UDP_IP = '192.168.0.2'
UDP_PORT = 9760
INET_ADDR = (UDP_IP,UDP_PORT)

MESSAGE = "123,@?\0"

print "UDP target IP:", UDP_IP 
print "UDP target port:", UDP_PORT 
print "message:", MESSAGE

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

#sock.sendto(MESSAGE, (UDP_IP, UDP_PORT))

while True:
  try:
    sock.sendto(MESSAGE, INET_ADDR)
    print "done sending udp"
    time.sleep(60)
  except Exception, err:
    #Write a log with the error 

print "Got us an exception: " + str(err)   


The receiving script is a lot longer and is shown below.You'll have to set up your COSM account
accordingly and edit a number of number of things to get it working.

Here are the things to edit and set up:

1)The IP address assigned to the Android handset, (UDP_IP).  Again I use my ADSL router feature to make sure this IP address is always assigned.

2)Change the COSM parameters:

• TimeZoneOffset - This is used to manage timezones and daylight saving time.  I had this as "+01:00" during UK daylight saving time and changed this to "+00:00" when we moved back to Greenwich Mean Time.  Change this to reflect the timezone you're in, (COSM uses UTC).
• MyFeedID - This is the feed ID created in COSM
• MyWattsNowAPIKey - The API key set up for this feed and the associated datastreams. Make sure you have full priviledges for the key.
• The six datastream names.  Four are for the distinct LightwaveRF measurements.  The cost ones are derived from the Wh values and my personal energy costs.

3)The UnitCost and DailyCost parameters represent my electricity costs which are calculate as DailyCost + (UnitCost * KWh) in the CalculateCosts sub-routine.

4)Naughty, naughty - I've embedded what could be a constant quite low down in the code with this line "chdir('/mnt/sdcard/webserver/energy_measurements')".   Change the directory to one you've got set up on your handset, (or delete / comment the line out and the script will write to the /scripts directory).  In this directory I write two log files:

• energy_measurements.csv - This is a local copy of all the measurements I take.
• energy_measurements_log_file.txt - At various points in the code I call a sub-routine called WriteDebugLog which simply write a line of text to a file associated with key points of code execution.  I used this when I set the script up to give me information as to where the code was getting "stuck".

The main body of code then just stays in a continuous loop, waits for a UDP segment to be received, parses it and writes the measurements to COSM.

To run the code simply create 2 .py files with each of the scripts listed, copy them to the /sl4a/scripts directory on the handset and run them.  If it works like mine it will be very reliable!

#v1=Just Watts measurement. V2=Added other4 values. V3=Added GBP values. V4=Added errorlogging

#Import statements
import socket
import datetime
from os import chdir 
import httplib
import sys

#Some constants for this, the server (192.168.0.3)
UDP_IP = "192.168.0.3"
UDP_PORT = 9761            #Responses always sent to this port

#These are constants related to the COSM feed
TimeZoneOffset = "+00:00"
MyFeedID = ""
MyWattsNowAPIKey = ""
MyWattsNowDataStream = "WattsNow"
MyMaxWattsDataStream = "WattsNow_Max"
MyCumulativeWattsDataStream = "WattsNow_Cumulative"
MyYesterdayTotalDataStream = "WattsNow_TotalYesterday"
MyCumulativeCostDataStream = "WattsNow_CostToday"
MyCostYesterdayDataStream = "WattsNow_CostYesterday"

#Constants related to costs
UnitCost =13.42
DailyCost = 16.45

#This is a Python function that writes a log file.  Used for debugging purposes
def WriteDebugLog(StrToLog):
  #Form a date and time for this
  #Get the date and time
  DateToday = datetime.date.today()
  TimeNow = datetime.datetime.now()
    
  #Form the string we will write to screen and local file
  LogFileString = str(DateToday) + "," + str(TimeNow) + "," + StrToLog  


  #And log to file.  "a" means append if necessary
  logfile = open("energy_measurements_log_file.txt", "a")
  logfile.write(LogFileString + "\n")
  logfile.close()  

  return

#This is a Python function to log to COSM
def SendToCOSM(ValToSend,KeyToUse,FeedToUse,DataStreamToUse):
  #Use this try statement to capture errors  
  try:
    #Write to our debug log file
    WriteDebugLog("Start of write to COSM Function. " + DataStreamToUse)  

    #First form the string to send.  Here be an example '2012-09-30T22:00:00.676045+01:00,66'
    #So we need some date geekery for this  
    #Get a variable to hold the date
    today = datetime.datetime.now()

    #Create an overall string with the story so far
    MyDateTimeString = today.strftime("%Y-%m-%d") + "T"

    #Now for the time bit - First the format string
    FormattedTime = today.strftime("%H:%M:%S")    #Get the formatted time

    #Now form the full monty string
    MyDateTimeString = MyDateTimeString + FormattedTime + TimeZoneOffset + "," + ValToSend
  
    #And get it's length
    MyStrLen = str(len(MyDateTimeString))

    #Print what we got so far
    print 'FullString:', MyDateTimeString
  
    #Now do the HTTP magic - Connect to the server
    h = httplib.HTTP('api.cosm.com')
  
    # build url we want to request
    FullURL = 'http://api.cosm.com/v2/feeds/'+ FeedToUse + '/datastreams/' + DataStreamToUse + '/datapoints.csv'

    #Print the URI string we will use
    print "Full URL: " + FullURL
  
    # POST our data.  
    h.putrequest('POST',FullURL)    
 
    # setup the user agent
    h.putheader('X-ApiKey',KeyToUse)
    h.putheader('Content-Length',MyStrLen)   

    # we're done with the headers....
    h.endheaders()
  
    #Send the data
    h.send(MyDateTimeString)

    #Get the response from the request
    returncode, returnmsg,headers = h.getreply()
   
    #display whatever the results are....
    f = h.getfile()
    MyData = f.read()
    print f.read()
  
    #Write to our debug log file
    WriteDebugLog("End of write to COSM Function")
    
    #Now just return
    return 
  #Catch an exception
  except Exception, err:
    #Write a log with the error
    print "Got us an exception: " + str(err)
    #WriteDebugLog("Caught this error in log to COSM function: " + str(err)     
           
#This function calculates the cost in pounds for the electricity used.
#The formula is ((WattHours/ 1000) * (UnitCost / 100)) + (DailyCharge / 100)
def CalculateCosts(InWattHours):
  #WattHours comes in as a string so need to turn to a number
  
  #do the calculation
  CostInPoundsFloat = ((float(InWattHours) / 1000) * (UnitCost / 100)) + (DailyCost / 100)

  #Round it to 2 decimal places
  CostInPoundsFloat = round(CostInPoundsFloat,2)
  
  #return a string
  return str(CostInPoundsFloat)

########################################
#Now we start the main part of the code
########################################


#Change directory that we will write to
chdir('/mnt/sdcard/webserver/energy_measurements') 

#Tell the user we've started
print "UDP server started.  Waiting for response...."

#Bind a socket 
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP
sock.bind((UDP_IP, UDP_PORT))

#Now just loop until you receive a response
while True:
    #Read data from the buffer
    data, addr = sock.recvfrom(1024) #buffer size is 1024               
    
    #Write to our debug log file
    WriteDebugLog("What we read from the buffer: " + data)
    
    #Get rid of the initial part of the result string as this
    #Is just a static command portion.  First get the length
    DataLength = len(data) - 1
    #Now extract everything from character 7 to the end           
    MeasurementCSV = data[7:DataLength]

    #Write to our debug log file
    WriteDebugLog("Just the measurements after removing the command prefix: " + MeasurementCSV)

    #Get the date and time
    today = datetime.date.today()
    TheTime = datetime.datetime.now()
    
    #Form the string we will write to screen and local file
    OutString = str(today) + "," + str(TheTime) + "," + MeasurementCSV
    
    #Print the result...
    print OutString
    
    #Write to our debug log file
    WriteDebugLog("The string that we will log to the log file: " + OutString)
     
    #And log to file.  "a" means append if necessary
    logfile = open("energy_measurements.csv", "a")
    logfile.write(OutString)
    logfile.close()
     
    #Write to our debug log file
    WriteDebugLog("Have just written the log file CSV") 

    #Split the string and assign to variables  
    SplitMeasurement = MeasurementCSV.split(',')    
    WattsNow = SplitMeasurement[0]            #The power value for now (Watts)
    MaxWatts = SplitMeasurement[1]            #The max power today (Watts)
    CumToday = SplitMeasurement[2]            #Cumulative today (Watt Hours)
    TotalYesterday = SplitMeasurement[3]      #Total yesterday (Watt Hours)

    #Write to our debug log file
    WriteDebugLog("Have just split the string in 4") 

    #Print the output 
    print "Watts Now [W]:" + WattsNow
    print "Max Watts Today [W]:" + MaxWatts
    print "Cumulative Today [Wh]:" + CumToday
    print "Total Yesterday [Wh]:" + TotalYesterday    

    #Write to our debug log file
    WriteDebugLog("Have just printed the measurements to screen") 

    #Log to COSM dude!!! First check it's not 0 as that looks rubbish!
    if WattsNow == "0":
      print "Not sending as it's 0 Watts"
      
      #Write to our debug log file
      WriteDebugLog("Saw that the Watts measurement was 0 so didn't log to COSM") 
    else:
      SendToCOSM(WattsNow,MyWattsNowAPIKey,MyFeedID,MyWattsNowDataStream)
      SendToCOSM(MaxWatts,MyWattsNowAPIKey,MyFeedID,MyMaxWattsDataStream)
      SendToCOSM(CumToday,MyWattsNowAPIKey,MyFeedID,MyCumulativeWattsDataStream)
      SendToCOSM(TotalYesterday,MyWattsNowAPIKey,MyFeedID,MyYesterdayTotalDataStream)

      #Write to our debug log file
      WriteDebugLog("Have just sent the 4 measurements to COSM.  Now calculate costs.") 

      #Now calculate the costs
      CumulativeCost = CalculateCosts(CumToday)
      TotalYesterdayCost = CalculateCosts(TotalYesterday) 
      
      print "Cumulative Cost GBP" + CumulativeCost
      print "TotalCost GBP" + TotalYesterdayCost
      
      #Write to our debug log file
      WriteDebugLog("Have calculated costs. Was cumulative GBP" + CumulativeCost + "and yesterday GBP" + TotalYesterdayCost + ". Now send to COSM") 

      #Send them to COSM
      SendToCOSM(CumulativeCost,MyWattsNowAPIKey,MyFeedID,MyCumulativeCostDataStream)
      SendToCOSM(TotalYesterdayCost,MyWattsNowAPIKey,MyFeedID,MyCostYesterdayDataStream)

      #Write to our debug log file
      WriteDebugLog("Sent costs to COSM.")