Rock and roll

Rensselaer Polytechnic Institute student Zane Van Dusen has created a computer based musical instrument that allows all people, regardless of physical mobility, the opportunity to make music.

Van Dusen, a senior majoring in electronic media, arts, and communication (EMAC) and computer science, worked with an interdisciplinary group of students led by Pauline Oliveros, a musician and professor of the arts at Rensselaer.

The team designed and implemented a computer interface that tracks the movement of a user’s head to allow them to produce electronic sounds and compose music on a virtual keyboard in both solo and ensemble settings.

The device provides a much-needed outlet for creative expression for people with extremely limited mobility, particularly individuals with severe cerebral palsy (CP) — a neurological disorder that permanently affects body movement and muscle coordination and has the capacity to render people unable to speak or move. It also has therapeutic benefits, according to Van Dusen.

The user positions him/herself in front of the camera and chooses a point to track (usually the tip of the nose).

When the user moves, the software determines the new X and Y coordinates of the point, and uses the information to play a note or trigger a sound.

‘We recently tested the adaptive use musical instrument in a clinic and noticed that many of the children were more focused on their movements because they were motivated by the sounds they were creating,’ he said. ‘One child played the instrument for almost an hour, even though it took a lot of effort for him to keep his head up that long.’

Beyond musical communication, Van Dusen sees potential for the device to allow users to create verbal exchanges: ‘The interface could be adapted to create speech software, allowing those who suffer from CP to form full sentences, rather than just yes or no responses.’

Following his graduation, Van Dusen plans to continue working with Oliveros through the summer to perfect the prototype instrument.