Firmata

Problem

Often times, working with embedded hardware, we want to have the low level bit-flipping access of embedded programming but keep the high level programming ability of languages like Python or Ruby. Let’s say we want to control the position of a stepper motor connected through a motor driver to an Arduino, but we don’t want a “closed” control system with just the Arduino–we want to control the position from a Python script on a computer (maybe connected to a web service, etc.).

Solution

I’ve found my best solution to be to communicate between the computer and the Arduino using the Firmata protocol, a serial communication protocol specification for the Arduino that has libraries for C, C++, Python, Ruby, and some other languages too. This lets us write some commands and routines, even passing data back and forth, facilitating communication between the embedded system and the user’s computer.

Step-by-Step

Arduino sketch

Let’s start with flashing a sketch to the Arduino that contains some custom commands. The Firmata library should exist in the most recent version of the Arduino IDE. You can open the “StandardFirmata” example sketch, and save it as a copy in your project folder so that you can edit it (example sketches themselves can’t be edited). This sketch alone provides a large variety of commands that you can send over serial, like flipping pins and sending PWM. However, for my needs I also wanted to extend these abilities. To start, add a custom command code that isn’t already in use. I found that the 0x30-0x3F range should be free. Let’s say you want to call commands “MOVE_STEPPER” and “REPORT_SENSOR”: Add this to the beginning of the sketch:

#define MOVE_STEPPER  0x34
#define REPORT_SENSOR 0x30
#define SENSOR_RESPONSE 0x31
#define READY 0x35

Later in the sketch you will find a function called void sysexCallback(byte command, byte argc, byte *argv). The switch case in this function accepts the command code for incoming “sysex” commands, which you can think of as custom commands. For the sketch to accept a command with the MOVE_STEPPER code, you need to add a case in the switch-case for that code, as follows:

case MOVE_STEPPER:
  int steps;
  steps = argv[0] | (argv[1] << 7); //compose 7-bit bytes into steps
  bool dir;
  dir = (bool)(argv[2] & 1); //compose 7-bit byte into direction bool
  int time_micros;
  time_micros = argv[3] | (argv[4] << 7); //compose 7-bit byte into delay int
  digitalWrite(DIR_PIN, dir); //set direction to what was received
  for (int i = 0; i < steps; i++) { //move axis
    //TODO: add check for collision
    digitalWrite(STEP_PIN, HIGH);
    delayMicroseconds(time_micros); //add proper delay from argv
    digitalWrite(STEP_PIN, LOW);
    delayMicroseconds(time_micros);
  }
  Firmata.write(START_SYSEX);
  Firmata.write(READY); //fire off the READY response
  Firmata.write(END_SYSEX);
break;

Note that this snippet requires that you have STEP_PIN and DIR_PIN set before hand, as well as having already sent the pinmode for both. I also have a READY command defined (can be 32-bit number) that is a placeholder (no data is sent) that the Arduino sends back to the computer to tell it that it is done with the operation. If you did want to send data back, if you for example made a command to retrieve data from an I2C or SPI sensor, you can use the following pattern into another case (case REPORT_SENSOR):

Firmata.write(START_SYSEX);
Firmata.write(SENSOR_RESPONSE_CODE); //send response code (defined earlier)
Firmata.write(sensordata & 0x7F); //report data
Firmata.write((sensordata >> 7) & 0x7F);
Firmata.write(END_SYSEX);

Python script

For simple projects we can get away without extending the Arduino class provided by the pyFirmata library, but in this case we want to add some functionality in the form of extra sysex commands. Let’s start with a customarduino.py file to extend this class. Here is an example of what that file might look like in this case:

from pyfirmata import Arduino, util
import time
from utils import force_except


# sysex callback code definitions
REPORT_SENSOR = 0x30
SENSOR_RESPONSE = 0x31
MOVE_AXIS = 0x34
READY = 0x35


# this class extends the PyFirmata "Arduino" class with custom "SYSEX" commands
class CustomArduino(Arduino):
    # required otherwise you can't prove the variables exist
    sensor_data = None
    ready = False

    def __init__(self, comm):
        super().__init__(comm)
        # some custom handlers for responses that we need to watch
        self.add_cmd_handler(SENSOR_RESPONSE, self._handle_sensor_response)
        self.add_cmd_handler(READY, self._handle_ready)

    def _handle_sensor_response(self, *data):
        # *data takes in the args as an array

        # decompose the "7-bit byte" packets
        self.sensor_data = data[0] | (data[1] << 7) | (data[2] << 14)

    # updates the "ready" variable, representing if the current operation
    # has finished on the Arduino
    # currently not really used, in lieu of a dumb timeout
    def _handle_ready(self, *data):
        # print("ready received", flush=True)
        self.ready = True

    # returns the state of the ready variable, and UPDATES it back to
    # false if it if was ready (can't just read it)
    def _check_ready(self):
        if self.ready is True:
            self.ready = False
            return True
        else:
            return False

    # returns the value of the sensor
    def read_sensor(self):
        # requests the sensor data from the Arduino
        self.send_sysex(REPORT_SENSOR, [])
        # sets a timeout, probably would just block the entire program
        t_end = time.time() + 5
        # waits until data has come back
        while ((time.time() < t_end) and self.bytes_available() == 0):
            pass
        # once data has come back, the self.sensor_data variable has already
        # been updated by the _handle_sensor_response callback
        return self.sensor_data  # 0 is analog cap, 1 is laser

    # requests that the Arduino moves the axis by step, direction, and delay
    def move_axis(self, steps, direction, micros):
        # decompose the int step and delay args into 7-bit bytes that can
        # be sent to the Arduino (firmata only accepts that size packet)
        stepsA = steps & 0x7F  # TODO add size checking
        stepsB = steps >> 7
        microsA = micros & 0x7F
        microsB = micros >> 7
        # send the request to the arduino, with data payload
        self.send_sysex(MOVE_STEPPER,
                        [stepsA, stepsB, direction, microsA, microsB])
        # set timeout
        t_end = time.time() + 5
        # wait until the READY has come back, but we don't need to read
        # anything  because the READY command has no payload
        while ((time.time() < t_end) and self.bytes_available() == 0):
            pass
        #if not self._check_ready():
        #    raise Exception("Ready msg not received")
        return

Now we can instantiate this extended Arduino class in our main script to leverage our added commands. You need the following imports:

from customarduino import CustomArduino
from pyfirmata import util

Now you can create a CustomArduino object with the correct COM port:

board = CustomArduino('/dev/tty.usbmodem1411')

The library requires that you start the board “iterator” before making any communications:

it = util.Iterator(board)
it.start()

You can still use pyFirmata’s existing features, like manually flipping pins:

pin = board.get_pin('d:11:i')
pin.enable_reporting()
print(pin.read())

This is important since I’ve found you need to block your script until the serial init procedure is done. You can do this as follows, with any pin:

# block the script until the Arduino has initialized and
# actually returns a value (not None)
while(cap_pin.read() is None):
    pass  # do nothing

You can also use the commands that we just made:

board.move_axis(500, 1, 500)
print(board.read_sensor())