This blog has moved! All new posts will be on charlesmartin.com.au and all the current posts have migrated there too, so please leave comments there instead.

19 July 2009

Networked Arduino Heartbeat sensor + SuperCollider

I made a simple heartbeat sensor using an Arduino which sends OSC signals at each heartbeat over a network. I'm using the heartbeat sensor as an awesome prop in my show Vital LMTD which is on at the Street Theatre in Canberra!


There's a new video here.
And an article on Makezine here!
I got the idea from Recotana's Heartbeat Midi Controller. Recotana also wrote the OSC library for Arduino which enabled this project.

Networked Arduino Heartbeat Sensor Code (Requires Recotana's OSC Library):
// cpm_heartbeatEthernet
// Version 1.0 October 2009.
// Copyright Charles Martin (http://www.charlesmartin.com.au).
// Uses recotana's OSCClass (http://www.recotana.com)

// Detect heartbeat using a light reading through skin
// On each beat, send an OSC message of the instantaneous
// heartrate.

#include "Ethernet.h"
#include "OSCClass.h"

// Pins
const int ledPin = 13;
const int sensePin = 0;

// LED blink variables
int ledState = LOW;
long ledOnMillis = 0;
long ledOnInterval = 50;

// Hearbeat detect variables
int newHeartReading = 0;
int lastHeartReading = 0;
int Delta = 0;
int recentReadings[8] = {0,0,0,0,0,0,0,0};
int historySize = 8;
int recentTotal = 0;
int readingsIndex = 0;
boolean highChange = false;
int totalThreshold = 2;

// Heartbeat Timing
long lastHeartbeatTime = 0;
long debounceDelay = 150;
int currentHeartrate = 0;

// Ethernet and OSC information
byte serverMac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte serverIp[] = { 192, 168, 0, 99 };
int serverPort = 10000;
// byte gateway[] = { 192, 168, 0, 1 };
// byte subnet[] = { 255, 255, 255, 0 };
byte destIp[] = {192,168,0,255};
int destPort = 3333;
char *topAddress="/heartbeat";
OSCMessage recMes;
OSCMessage sendMes;
OSCClass osc(&recMes);

void setup() {
Ethernet.begin(serverMac ,serverIp);
osc.begin(serverPort);
osc.flush();
sendMes.setIp( destIp );
sendMes.setPort( destPort );
sendMes.setTopAddress(topAddress);
// initialize the serial communication:
Serial.begin(9600);
// initialize the digital pin as an output:
pinMode(ledPin, OUTPUT);
}

void loop() {
// Turn off LED
digitalWrite(ledPin, LOW);

// Read analogue pin.
newHeartReading = analogRead(sensePin);
//Serial.println(newHeartReading);
//Calculate Delta
Delta = newHeartReading - lastHeartReading;
lastHeartReading = newHeartReading;

// Find new recent total
recentTotal = recentTotal - recentReadings[readingsIndex] + Delta;
// replace indexed recent value
recentReadings[readingsIndex] = Delta;
// increment index
readingsIndex = (readingsIndex + 1) % historySize;

//Debug
//Serial.println(recentTotal);

// Decide whether to start an LED Blink.
if (recentTotal >= totalThreshold) {
// Possible heartbeart, check time
if (millis() - lastHeartbeatTime >= debounceDelay) {
// Heartbeat
digitalWrite(ledPin, HIGH);
currentHeartrate = 60000 / (millis() - lastHeartbeatTime);
lastHeartbeatTime = millis();
// Print Results
//Serial.println("Beat");
if (currentHeartrate <= 200) { Serial.println(currentHeartrate); // Send a serial message sendMes.setArgs("i" , &currentHeartrate); // Setup an OSC message osc.sendOsc( &sendMes ); // Send the heartbeat OSC message } } } delay(10); }


Here's some older information about the process of making this sensor!

I wanted it to communicate with the computer and send data to SuperCollider. As a proof of concept, I've connected the Arduino to SuperCollider via a Processing script that translates serial data from the Arduino into OSC messages. Here's a video demonstration:




Arduino and the sensor circuit. My phone camera can see in infrared.

The sensor uses two simple components, an IR LED and an IR phototransistor. Both components are powered by the Arduino's 5V output and one analogue input reads the voltage across the phototransistor.
  1. IR LED (Jaycar ZD1945)
  2. IR Phototransistor (Jaycar ZD1950)
  3. 10KOhm resistor
  4. 220Ohm resistor



The simple circuit is the same as for an IR range sensor, commonly used in robot projects. The easiest way to start looking at data form an Arduino's analogue input is to follow the Arduino Graph tutorial.

The idea is that when your heart beats you have a quick rush of blood into tiny blood vessels close to your skin which makes it less transparent. This effect is easiest to observe on your finger tips or earlobe. So the IR emitter and phototransistor are placed next to each other (not much light goes through the side of the emitter!) and I put my finger on top. Light from the IR emitter illuminates my skin and is reflected into the phototransistor.

The phototransistor is connected to the Arduino in a similar way to a potentiometer. One lead is connected to +5V and the other to ground. The +5V lead is also connected to an analogue input on the Arduino. When the phototransistor receives more IR light it becomes more resistive and a lower voltage is detected by the analogue input.



The circuit all soldered together and held together with double sided tape. It was then wrapped up in electrical tape to protect it and shield the phototransistor from other light sources.


Graph of the sensor output! Each little bump is a heartbeat!

Similar projects around the internet have used an amplifier to boost the signal from the phototransistor. I found that the data was clear enough for the Arduino to track heartbeats accurately. My Arduino program follows the (average) rate of change of the phototransistor voltage and uses this to judge whether a heartbeat is occuring or not.

Arduino code:

// Pins
const int ledPin = 13;
const int sensePin = 0;

// LED blink variables
int ledState = LOW;
long ledOnMillis = 0;
long ledOnInterval = 50;

// Hearbeat detect variables
int newHeartReading = 0;
int lastHeartReading = 0;
int Delta = 0;
int recentReadings[8] = {0,0,0,0,0,0,0,0};
int historySize = 8;
int recentTotal = 0;
int readingsIndex = 0;
boolean highChange = false;
int totalThreshold = 2;

// Heartbeat Timing
long lastHeartbeatTime = 0;
long debounceDelay = 150;
int currentHeartrate = 0;

void setup() {
// initialize the serial communication:
Serial.begin(9600);
// initialize the digital pin as an output:
pinMode(ledPin, OUTPUT);
}

void loop() {
// Turn off LED
digitalWrite(ledPin, LOW);

// Read analogue pin.
newHeartReading = analogRead(sensePin);
//Serial.println(newHeartReading);
//Calculate Delta
Delta = newHeartReading - lastHeartReading;
lastHeartReading = newHeartReading;

// Find new recent total
recentTotal = recentTotal - recentReadings[readingsIndex] + Delta;
// replace indexed recent value
recentReadings[readingsIndex] = Delta;
// increment index
readingsIndex = (readingsIndex + 1) % historySize;

//Debug
//Serial.println(recentTotal);

// Decide whether to start an LED Blink.
if (recentTotal >= totalThreshold) {
// Possible heartbeart, check time
if (millis() - lastHeartbeatTime >= debounceDelay) {
// Heartbeat
digitalWrite(ledPin, HIGH);
currentHeartrate = 60000 / (millis() - lastHeartbeatTime);
lastHeartbeatTime = millis();
// Print Results
//Serial.println("Beat");
if (currentHeartrate <= 200) { Serial.println(currentHeartrate); } } } delay(10); }



Processing code:

// Based on examples from Arduino's Graphing Tutorial and OscP5 documentation
import processing.serial.*;
Serial myPort; // The serial port
int xPos = 1; // horizontal position of the graph
import oscP5.*;
import netP5.*;
OscP5 oscP5;
NetAddress myRemoteLocation;

void setup () {
// set the window size:
size(640, 480);
frameRate(25);
// Start OscP5
oscP5 = new OscP5(this,12000);

// List availabl serial ports.
println(Serial.list());

// Setup which serial port to use.
// This line might change for different computers.
myPort = new Serial(this, Serial.list()[0], 9600);

myPort.bufferUntil('\n');
// Configure NetAddress to send OSC messages to
myRemoteLocation = new NetAddress("127.0.0.1",57120);
// set inital background:
background(0);
}

void draw () {
}

void serialEvent (Serial myPort) {
// read the string from the serial port.
String inString = myPort.readStringUntil('\n');

if (inString != null) {
// trim off any whitespace:
inString = trim(inString);
// convert to an int
println(inString);
int currentHeartrate = int(inString);

if (currentHeartrate > 0) {
// Construct and send OSC message of the current heartrate
OscMessage myMessage = new OscMessage("/heartbeat");
myMessage.add(currentHeartrate);
oscP5.send(myMessage, myRemoteLocation);

// draw the Heartrate BPM Graph.
float heartrateHeight = map(currentHeartrate, 0, 200, 0, height);
stroke(127,34,255);
line(xPos, height, xPos, height - heartrateHeight);
// at the edge of the screen, go back to the beginning:
if (xPos >= width) {
xPos = 0;
background(0);
} else {
// increment the horizontal position:
xPos++;
}
}
}
}

/* incoming osc message are forwarded to the oscEvent method. */
void oscEvent(OscMessage theOscMessage) {
/* print the address pattern and the typetag of the received OscMessage */
print("### received an osc message.");
print(" addrpattern: "+theOscMessage.addrPattern());
println(" typetag: "+theOscMessage.typetag());
}


SuperCollider Code:

// create the OSCresponder
// Beeps each time it receives a heartbeat OSC message.
(
n = NetAddr.new("127.0.0.1", nil);
o = OSCresponder.new(n, "/heartbeat", {
arg time, resp, msg;
msg.postln;
{ EnvGen.kr(Env.perc, 1.0, doneAction: 2) * SinOsc.ar([440,440], 0, 0.1) }.play;
} ).add;
)

o.remove; // remove the OSCresponder.


24 comments:

Rohan said...

This is super cool. I gather it is working in the same way as the things they put on your finger in hospital? Will be great when you have combined it with midi!

casper said...

Hi! Great work, thanks for sharing this. This might be a stupid question, but just to make sure: this system is capable of outputting the registered heart rate as a signal, e.g. to have a LED blinking at the measured heart rate, without the use of Processing, right? The Arduino is calculating the current heart rate itself, and is not just passing unprocessed data to the PC?

I would like to use this technique in a standalone installation, without using a computer, just an Arduino. Thank you in advance!

Charles Martin said...

Hey Casper,

Yes the Arduino is doing all the work. It sends a message containing the current heartrate (pretty approximate...) on every detected heartbeat.

This message could be send over serial to Processing or another program as in the older example, or as an OSC message over a network using the Arduino ethernet shield.

I'm currently using this gadget in a show in Canberra, using it as a standalone device running on a battery and sending the messages over OSC to my computer for an audio response and my friend Benjamin Forster's computer for a video response. It works really well!

casper said...

Hi Charles,

How cool you can use the device for live performance! I gave your hardware setup a try but I never got really usable results - although I bought the exact same parts you used. The values returned by the IR photoresistor didn't seem to vary in a way that made any sense. Displaying the values through Processing showed no rhythm whatsoever, and the change in registered values was very small.
I tried the same setup with various resistors, LED's and photoresistors, but up to no avail. I'm still wondering what I did wrong.

Charles Martin said...

I'm trying to think of why it wouldn't work for you casper...

Maybe you could try *just* the phototransistor and not the IR LED and then run the Arduino Graph Tutorial to look at the data in Processing, then you'd be able to tell whether the phototransistor is work by covering it and shining light etc.

Another idea... it might be that there is too much light leaking from the IR LED or something?

When I first tried this, the heartbeat signal from the phototransistor was really weak, varying only by about 10 points in each heartbeat, sometimes less. I could see it better by limiting y axis of the graph in processing, maybe instead of being 1024 just limit it to 100 around the level that is usually reported when you hold it to your finger. That way you can see the variation.

A lot of people use an opamp to boost the signal of the phototransistor before sending it to the Arduino, but I found that even with a low resolution measurement I could detect it pretty accurately with the Arduino program in the post so I didn't use any extra hardware.

There is only one threshold variable in my program that limits the sensitivity: "totalThreshold" so you could experiment changing that.

The program records the difference between each reading from the arduino analogue input, then it uses the sum of the last 8 differences as a basis for reporting a heartbeat. If the sum is greater than totalThreshold then a heartbeat is triggered.

I found that having totalThreshold really low... like 2... had the best results.

geniemaster said...

Hi I have a question in regard your heartbeat sensor. I'm making an interactive art and I want to use MaxMSP to detect the heartbeats with the IR emitter and detector. I'm new to arduino and was wondering how to read your codes and translate that to MaxMSP.

casper said...

Hi geniemaster,

Charles' hardware setup in combination with his Arduino code will make the Arduino write values to the serial port. In the example above, Processing is reading these values and visualizes them as a graph. Max/MSP can also read the signals sent to the serial port. You can read more about that here.

geniemaster said...

Thanks Casper,
Is it possible just do it straight from MaxMSP without making another patch to read from the arduino code?
I'm very new to arduino and we only did MaxMSP in class and have not touch Arduino itself.

casper said...

Hey geniemaster,

The Arduino is a micro controller that needs to be programmed in order to do what you want it to do, for instance reading out a sensor value and writing that value to the serial port. In Max, you can access this data by reading the serial port. You can implement this functionality in your existing patch.

The Arduino is very accessible and easy to use, and to get started you can find many tutorials online. I can recommend the book Getting Started with Arduino, a beginners guide written by one of the Arduino founders, Massimo Banzi.

Have fun tinkering!

geniemaster said...

Sorry got another general question. So the idea of IR sensor is to detect every blood rush (=heart beat) at the finger tip. Does that mean the IR emitter and detector are on all the time, and not blinking, in order to get an array of constant integer inputs determined by the detector? And a threshold number would determine a heart beat in my MAX/MSP?

geniemaster said...

I tried the arduino code provided by Charles, but I kept getting this message when I upload to the board.

avrdude: stk500_recv(): programmer is not responding

Any idea what that mean?

casper said...

geniemaster, did you select the right board in the Tools menu? Make sure that you select the micro controller chip that matches the one on your Arduino, e.g. AtMega 328.

geniemaster said...

yes I did.
I set the board Arduino Duemilanove/ Nano with AtMega 328, and serial port to tty.usbserial

casper said...

geniemaster,

Since your issue is not specifically related to Charles Martin's setup, you may want to look for a solution to this problem on more general Arduino websites/forums/blogs. If you do a Google search on your error message you can find a lot of information and possible solutions.

geniemaster said...

Apparently there is something wrong with my arduino board, so I switch to a new one. And it seems to work fine now uploading the arduino code. Now I just need to figure out how to read from code updated arduino using Max MSP. I'm juggling between using firmata and Max MSP (since that's what my professor wants me to do, but I'm open to any easier way.)

geniemaster said...

well...I think I got it to work...but still got a question about the infrared reading. I basically uploaded the heart beat arduino code to my arduino and polling data using MaxMSP. I got an array of numbers (which is good I think) that kind of act according to the pulse reading, but it's reading the serial inputs awfully slow. It seems like it prints/bang only when the previous number changed.
I guess what I'm saying is if 10,10,10,50,10,10 (over a course of 5 sec for example) is the array of numbers, it displays 10,50,10 (5 sec duration).
Maybe I need to tweet MaxMSP so it can continue to read the serial input and bang/print without gaps.

I used the Max code at the bottom of this page.
http://images.google.com/imgres?imgurl=http://1.bp.blogspot.com/_KBlvp5i4Mkk/SmMYCqG594I/AAAAAAAABl0/W5RWTIvFqMM/s320/heartbeatSensorTestCircuit.jpg&imgrefurl=http://cmpercussion.blogspot.com/2009/07/heartbeat-sensor.html&usg=__cBLJ9KVjeZwoMnemyro4bg4PsNM=&h=240&w=320&sz=21&hl=en&start=8&um=1&itbs=1&tbnid=Yb00tbODCv3tdM:&tbnh=89&tbnw=118&prev=/images%3Fq%3Dvital%2Bsensors,%2Bheart%2Bbeat%26hl%3Den%26client%3Dfirefox-a%26rls%3Dorg.mozilla:en-US:official%26sa%3DN%26um%3D1

Charles Martin said...

I'm sure it would work fine with any kind of arduino or arduino-compatible board.

As for sending data wirelessly... anything is possible. Checkout the Xbee system for a simple wireless serial connection, I use those boards in performances now.

JMK-Marvel said...

What libraries did you used. I am getting an error that say no library for oscPS and netPS

JMK-Marvel said...

I've tried your code using a Red LED and a photoresistor and got the same results.

Charles Martin said...

@JMK-Marvel - Great! I'm glad you got it to work. Yes, as you said, the Processing code needs the oscP5 library. (netP5 is part of oscP5)

ray said...

hi charles, i've got the same problem as casper. The IR detector's output doesn't seem to vary with a rhythm. setting the threshold to 2 or 3 picked up seemingly rhythmic pulses from ambient light though. any fixes for that?

PS Casper. Did you overcome that problem?

Christiaan said...

Hey Charles,

I was wondering have you tried it on a different body part, for example your wrist/arm? And did it work? Thanks in advance.

Smith Powell said...

Thanks for giving such an impressive post on XBee module. Xbee quick start
is a good source for learning XBee.

Nelson Graham said...

Hey I am trying to use your design to help my son with a school project and I cant seem to get any real sensitivity out of the receiver. I have the same resistors and Ir pair as you do but my readings are +-1, when i look at my raw readings. I have tried playing around with both the resistors to increase the IR emitter output or thresholds with the receiver.. no joy. I just end up with moving the reference level but no sensitivity to the pulse. If I remove the analog lead, I see the reading go to 0 so I know I am reading. Looking at the IR emitter with a camera I see that it is on...Help ! ..Please