Deafblind Robodrummer has no eyes and no ears, all he got is his sense of touch and rhythm.
Cluelessly he scans the environment with pegs in his hands, that he uses as drumsticks, looking for objects to drum on.
When he feels the presence of drum-able objects close to him, he plays a beat.
It's those moments that bring a glimpse of joy to his Land of Silence and Darkness.
Deafblind Robodrummer (DBRD) is equipped with a torso, to which two arms and a head are attached. The arms can turn on the shoulders and the wrist is able to move vertically. This allows DBRD to turn towards a drum and beat it. As there are no further sensors or input devices, the robot is dependant on sensing his environments through the servo motors.
After initialisation, DBRD begins to look for drum-able objects, by moving his arms as far out as possible and lowering the sticks by moving the wrists. If he feels the resistance of an object, he remembers the position of the wrist at which it was felt, moves the wrist upward and turns the shoulder slightly, continuing the probing procedure. He continues until the two arms meet in the middle and the entire environment has been scanned.
If the scanning procedure was successful, DBRD now has a list of positions at which an object has been felt, however he does not know yet where an object begins or ends and how many there are around him. Therefore he marks the first occurrence of a measurement with a "b" (begin) and all the subsequent measurements with "i" (inside) until the measurement returned false (no drum encountered). Positions where no object was detected are marked with "o" (outside).
The measurements tagged with "b" are combined with their neighbouring "i" and averaged to find the center of the objects, at which position DBRD imagines to be a drum. Below is a table illustrating the process for some sample measurements with position shifts of 0.5.
| Positions | -2.5 | -2 | -1.5 | -1 | -0.5 | 0 | 0.5 | 1 |
|---|---|---|---|---|---|---|---|---|
| Measurements | 2.1312 | 2.2132 | false | 1.831 | 1.7932 | 1.8123 | false | false |
| Interpretation | b | i | o | b | i | i | o | o |
| Construction | Drum 1 |   | Drum 2 | |||||
When all measurements have been analysed, DBRD has internalised the positions of the possible drums for both left and right arms. He now can use as much objects as are around him to play his beats.
DBRD continues playing for a predefined amount of time. Originally, the idea was to make him go on for as long as there are objects to drum on (objects might move away during drumming or be removed maliciously). However, when hitting a drum, thereby executing a fast down- and upwards motion with the wrist, the forces acting on the wrist are as high when an object has been hit as they are when the object is missed. This is because in both cases the speed of the wrist drops to zero instantaneously. DBRD therefore is unable to tell if a hit was successful or not.
The lack of sensitivity in the servo motors was a general problem in the process. As visible in the video, DBRD has to execute a lot of force, almost lifting himself up in order to detect an object. A threshold defines at which point the torque applied to the motor represents an object. If it is chosen too high, the robot ignores the object and might fall, if it is too low, too many false positives are registered due to normal fluctuations in torque when the motors are moving.
The speed and step size at which DBRD explores his environment have been chosen in order to minimise false positives, while at the same time keeping the duration of the scanning procedure to a bearable level. With the current setting, small objects are missed and objects too close to each other would not be recognised separately. Reducing the size of the steps the robot arms move horizontally would make it possible to also detect smaller objects. A faster vertical movement however proved unsuccessful, as the forces on the motor in normal motion become too strong.
When developing the software, some minor issues have come up, mainly due to conflicts arising from the parallel exploration of the environment with both arms. This was solved by encapsulating the scanning process in separate classes and instantiating them for both arms individually.
An interesting continuation of the project, would be to add some mobility to the robot, make him able to move around and gather objects to use as a drum set. At the moment DBRD relies on suitable objects being placed in his surrounding. He is also not able to distinguish rigid objects, that make a sound, from flexible ones, that are less interesting to drum on. It might be possible to improve the algorithm for detection to make this differentiation. However, it could be necessary to add a force resistant sensor to the hands of the robot, but this would be outside the interest of this project, which looks at the possibility of using only the servos for robot localisation and orientation.
The main code is in the file DrummerBoy.u, which also creates an instance of the main class. The robot automatically resets and after calling the function start() begins the scanning procedure. The class DrummerBoy creates two instances of the class DrumSet, one for each arm. All the functions for scanning and detecting objects are within the class DrumSet. For each detected object, an instance of the class Drum is called. The class Drum is used to store the horizontal and vertical position of the drums. Below is a diagram illustrating the classes and their processes.
load("dynamixel.u");
var Global.d = Dynamixel.new;
class DrummerBoy {
load("drumset.u");
/* Initialisation */
function init() {
var this.hitDistance = 0.5; // Force with which the drums will be hit
var this.hitSpeed = 0.1; // Delay after drum has been hit
var this.duration = 20; // Number of beats to play
var this.drumTag = Tag.new;
this.drumTag.freeze;
// Initialise motors
var this.head = Dynamixel.Device.new(d,4,Dynamixel.DeviceDesc.AX12);
var this.rightWrist = Dynamixel.Device.new(d,1,Dynamixel.DeviceDesc.AX12);
var this.rightShoulder = Dynamixel.Device.new(d,2,Dynamixel.DeviceDesc.AX12);
var this.leftWrist = Dynamixel.Device.new(d,11,Dynamixel.DeviceDesc.AX12);
var this.leftShoulder = Dynamixel.Device.new(d,12,Dynamixel.DeviceDesc.AX12);
var this.torso = Dynamixel.Device.new(d,8,Dynamixel.DeviceDesc.AX12);
var this.kick = Dynamixel.Device.new(d,10,Dynamixel.DeviceDesc.AX12);
// Initialise center and extreme positions of robot limbs
this.initPositions();
// Reset positions
this.setSpeed(1);
this.reset();
sleep(1s);
this.setSpeed(0);
};
function initPositions() {
// Set min and max positions as well as center positions for all joints
var this.head.initPos = 0;
var this.head.maxLeft = 1.5;
var this.head.maxRight = -1.5;
var this.rightWrist.initPos = 0;
var this.rightWrist.maxUp = -1.5;
var this.rightWrist.maxDown = 1.5;
var this.rightShoulder.initPos = 1.5;
var this.rightShoulder.maxRight = 0;
var this.rightShoulder.maxLeft = 2.5;
var this.leftWrist.initPos = 0;
var this.leftWrist.maxUp = 1.5;
var this.leftWrist.maxDown = -1.5;
var this.leftShoulder.initPos = -1.5;
var this.leftShoulder.maxRight = -2.5;
var this.leftShoulder.maxLeft = 0;
var this.torso.initPos = 0;
var this.torso.maxRight = -1.5;
var this.torso.maxLeft = 1.5;
var this.kick.initPos = 0;
var this.kick.maxUp = -1.5;
var this.kick.maxDown = 1;
};
/* Main behaviour */
function start() {
reset();
this.drumTag.freeze;
// Create drumset object for left and right arm
var this.leftDrumSet = DrumSet.new(this.leftShoulder,this.leftWrist,"left"),
var this.rightDrumSet = DrumSet.new(this.rightShoulder,this.rightWrist,"right");
echo("Found "+this.rightDrumSet.drums.size()+" drum(s) on the right");
echo("Found "+this.leftDrumSet.drums.size()+" drum(s) on the left");
// Start drumming if drums have been found
if(this.rightDrumSet.drums.size()>0 || this.leftDrumSet.drums.size()>0) {
this.drumTag.unfreeze;
this.drumTag :while (1 == 1){
var countdown = this.duration;
every(2s) {
this.drum();
countdown--;
if(countdown<=0) {
this.drumTag.freeze;
break;
}
};
};
};
};
function drum() {
var i;
var drumToHit;
if(this.rightDrumSet.drums.size()>0) {
// Pick a random drum and hit it
i = random(this.rightDrumSet.drums.size());
echo("Hitting drum "+i+" on the right");
drumToHit = this.rightDrumSet.drums[i];
// Lift wrist up to initial position
this.rightWrist.targetPos = this.rightWrist.initPos;
sleep(0.2s);
// Turn shoulder to position of the drum
this.rightShoulder.targetPos = drumToHit.getShoulderPos();
sleep(0.2s);
// Place wrist above drum
this.rightWrist.targetPos = drumToHit.getWristPos()-this.hitDistance;
sleep(0.2s);
// Hit it
this.hitRight();
};
if(this.leftDrumSet.drums.size()>0) {
i = random(this.leftDrumSet.drums.size());
echo("Hitting drum "+i+" on the left");
drumToHit = this.leftDrumSet.drums[i];
this.leftWrist.targetPos = this.leftWrist.initPos;
sleep(0.2s);
this.leftShoulder.targetPos = drumToHit.getShoulderPos();
sleep(0.2s);
this.leftWrist.targetPos = drumToHit.getWristPos()+this.hitDistance;
sleep(0.2s);
this.hitLeft();
};
};
function stop() {
this.drumTag.freeze;
};
/* Commands */
function hitLeft() {
// Execute hit motion with left wrist
var curPos = this.leftWrist.position;
this.leftWrist.targetPos = curPos-this.hitDistance;
sleep(this.hitSpeed);
this.leftWrist.targetPos = curPos;
};
function hitRight() {
// Execute hit motion with right wrist
var curPos = this.rightWrist.position;
this.rightWrist.targetPos = curPos+this.hitDistance;
sleep(this.hitSpeed);
this.rightWrist.targetPos = curPos;
};
function hitKick() {
// Execute hit motion with kick
var curPos = this.kick.position;
this.kick.targetPos = curPos+this.hitDistance;
sleep(this.hitSpeed);
this.kick.targetPos = curPos;
};
/* Utilities */
function pos2Deg(var position, var left, var right) {
// Convert motor position to degrees
return (position-left)/(right-left)*(-90-90)+90;
};
function deg2Pos(var degree, var maxLeft, var maxRight, var leftDeg, var rightDeg) {
// Convert degrees to motor position
return (degree-rightDeg)/(leftDeg-rightDeg)*(maxRight-maxLeft)+maxLeft;
};
function reset() {
// Initialise all limbs
var delay = 0.1;
this.rightShoulder.targetPos = 1.5;
sleep(delay);
this.rightWrist.targetPos = 0;
sleep(delay);
this.leftWrist.targetPos = 0;
sleep(delay);
this.leftShoulder.targetPos = -1.5;
sleep(delay);
this.torso.targetPos = 0;
sleep(delay);
this.kick.targetPos = 0;
sleep(delay);
this.head.targetPos = 0;
};
function setSpeed(var speed) {
this.head.targetSpeed = speed;
this.leftShoulder.targetSpeed = speed;
this.leftWrist.targetSpeed = speed;
this.rightShoulder.targetSpeed = speed;
this.rightWrist.targetSpeed = speed;
this.torso.targetSpeed = speed;
this.kick.targetSpeed = speed;
};
};
var drummer = DrummerBoy.new;
class DrumSet {
load("drum.u");
function init(shoulder,wrist,side) {
var this.torqueThreshold = 80; // Sensitivity for drum detection
// Initialise drum scan
var this.drums = [];
var drumSet = scan(shoulder,wrist,side);
// Setup drum set
if(drumSet.size()) {
for(var i=0;i<drumSet.size();i++) {
this.drums += [Drum.new(drumSet[i][0],drumSet[i][1])]
};
};
};
function scan(shoulder,wrist,side) {
var step = 0.25; // Shoulder scan intervals
var gap = 0.3; // Buffer space
var init;
var goal;
var positions = [];
var detects = [];
if(side=="left") {
// Start from left position, move to right
init = shoulder.maxLeft;
goal = shoulder.maxRight+gap;
// Inverse step size for left shoulder
step = -1*step;
// Scan step by step from left to right
for(var i = init;i>goal;i+=step) {
shoulder.targetPos = i;
positions += [i];
// Detect drum at current position
detects += [detect(wrist,side)];
sleep(0.5s);
};
} else {
init = shoulder.maxRight;
goal = shoulder.maxLeft-gap;
// Scan step by step from right to left
for(var i = init;i<goal;i+=step) {
shoulder.targetPos = i;
positions += [i];
// Detect drum at current position
detects += [detect(wrist,side)];
sleep(0.5s);
};
};
// Return the evaluated scans
return evaluateScans(positions,detects);
};
function detect(wrist,side) {
var step = 0.1; // Steps to lower the wrist
var ignore = 5; // The first few steps are ignored
var gap = 1; // Leave some space before reaching maximum position
// Don't change these variables
var detected = false;
var ok = true;
var position = 0;
var torque = 0;
// Pick limb
if(side=="left") {
// If we deal with the left wrist, we have to inverse the direction
step = -1*step;
};
for(var i=0;i<60;i= i+1) {
// Ignore first steps
if(ignore>0) {
ignore--;
};
torque = wrist.torque%1024; // prevent wraparound
// If torque exceeds thresholds, drum has been detected
if(torque>this.torqueThreshold && !ignore) {
detected = true;
position = wrist.position;
break;
};
// If position is going to exceed maximum position, exit loop
if(side=="left" && wrist.position+(step*gap)<wrist.maxDown) {
ok = false;
} else if (side!="left" && wrist.position+(step*gap)>wrist.maxDown) {
ok = false;
};
if(ok) {
wrist.targetPos = wrist.position+step;
} else {
break;
};
sleep(0.3s);
};
// reset position
wrist.targetPos = wrist.initPos;
// return position if drum has been detected
if(detected) {
return position;
} else {
return false;
}
};
function evaluateScans(positions,detects) {
// Array sizes of positions and detects have to match
if(positions.size()!=detects.size()) {
echo("Incompatible list sizes");
return false;
};
var out = true;
var inside = false;
var drums = [];
var drumSet = [];
var p;
for(var i=0;i<positions.size();i++) {
// Mark the beginning, inside and outside positions of the drum
if(detects[i] && out) {
drums += ["b"];
out = false;
inside = true;
} else if (detects[i] && inside) {
drums +=["i"];
} else if (!detects[i]) {
drums +=["o"];
out = true;
inside = false;
};
};
// Construct the drum position from the measurements
for(var i=0;i<drums.size();i++) {
p = 0;
var j;
var k;
// If the beginning of a drum has been detected
if(drums[i]=="b") {
if(i<drums.size()-1) {
// Look for the end of it
for(j=i+1;j<drums.size();j++) {
if(drums[j]=="o") {
break;
}
};
j--;
// Sum up the positions that comprise the drum
for(k=i;k<=j;k++) {
p+=positions[k];
};
} else {
j = i;
};
// And take the average of all the positions
// to get the center of the drum
p = p/(j-i+1);
// Add the shoulder and wrist position to the drum set
drumSet += [[p,detects[i]]];
}
};
return drumSet;
};
}
class Drum {
function init(x,y) {
var this.x = x;
var this.y = y;
};
function getShoulderPos() {
return this.x;
};
function getWristPos() {
return this.y;
};
};
BDRD is by far not the first percussionist robot, some of the ones around even do somersaults.
From the german robocross machine comes StickyBoy. The
four-armed two-legged metal robo drummer, runs on compressed air and can be programmed via MIDI.
A particularly witty one is the Yellow Drum Machine by Frits Lyneborg. With its belt tracks it looks more like a digger than a robot, but they enable it to move around and look for objects to drum on. If it finds interesting sounding objects, it records the sound and composes a rhythm with the samples.
In contrast to BDRD however, the Yellow Drum Machine uses ultrasonic sensors to locate objects. Most current robots use ultrasound, cameras or laser to orient themselves, at the moment there are a few who rely solely on touch. A recent paper titled Biomimetic vibrissal sensing for robots examines the use of whiskers as an additional sense for robot orientation. At the time of writing however, BDRD does not have many deafblind colleagues.
This project was realised as part of the course Creative Robotics at Goldsmiths, lead by Patrick Tresset. The robot was built in collaboration with Thibault Lelievre, Thomas Tanay and Dean Spiridonov.