Technology: Behind the Air Guitar

This work is licensed under a Creative Commons License.

Technology: Behind the Air Guitar

Source: http://airguitar.tml.hut.fi/tech.html

The insides of the Virtual Air Guitar

The Virtual Air Guitar consists of several pieces of software. The user's actions are read by the input device, such as a webcam, and passed through gesture recognition. A musical intelligence module interprets these gestures and sends commands to the sound model, which produces the final sound. The mapping of the user's gestures into sound model controls is very complex, making the sound model itself totally invisible - the user does not need to know anything about it to play.

Input

The web camera was chosen as input device for two reasons. It allows the user to play just as they would do with a real air guitar, without having to learn to use any additional devices, without any cables or buttons. And hidden inside a device rack, the interface has no breakable parts at all, and is ideal for use at a science centre, where experimenters of all ages subject the exhibits to considerable stress. In addition to the webcam, there are also two foot pedals - one for starting play, and one for changing between play modes.

The input software sees everything orange in a picture - hence the orange gloves - and assumes that the left-most orange blob is the left hand and the other the right hand. The gesture recognition module tracks the movement of the blobs, detecting when the right hand passes over the imaginary guitar centreline, and checking the relative distance between the hands to make out a fret position on the neck. The guitar moves with the perfomer, allowing for small and large guitars for small and large people, or even playing near your feet or behind your neck (as long as the gloves are visible). The gesture recognition module then sends this information onward to the musical intelligence module.

Musical intelligence

The sound model has been around for a long time, it hasn't really been used to produce actual music, because it's difficult to control. In other words, the mapping of user control to sound model parameters is not intuitive. The sound may be physically correct, but playing a guitar is more than just plucking one string and letting it ring - it's the playing style that matters.

The guitar meta language and the musical intelligence module make the sound model invisible, allowing it to be controlled with musical concepts instead of delay line parameters. The meta language is a collection of guitar playing styles, all the things you can do with a guitar, expressed as musical concepts such as plucking, fretting or bending strings.

The musical intelligence module also knows a certain musical context, such as a hard rock guitar solo on the E minor pentatonic scale. The user's gestures are first interpreted into musical concepts within this context, then converted into control parameters for the sound model. The musical intelligence thus contains both the rules by which gestures are converted into the meta language, and the implementations of the meta language for a certain sound model.

Sound synthesis

The actual sound of the Virtual Air Guitar is produced by a physical sound model of a plucked string - or more precisely, six models, one for each string. The model is an extended version of Karplus-Strong synthesis, on which many sound models of guitars and other plucked-string instruments are based on. Each of the six stings has been manually calibrated to match the sound of a hand-made copy of a Fender Stratocaster guitar.

The digital waveguide model starts with a sampled excitation signal, which is essentially the sound of plucking a string without the string vibration, although it can be as simple as a square impulse, too. This signal is processed by a timbre control filter, then modified by a comb filter that represents the position along the string at which it is plucked. This signal is then fed into a feedback loop, the delay line. The length of the delay line defines the frequency at which the string vibrates, thus controlling pitch. At each loop, the sound is damped by two filters, one for all frequencies and one for high frequencies. The main parameters for the sound model are thus the sharpness of the pluck, pluck position, length of delay line, and two damping parameters.

At this point, the generated sound is what the pickup of an electric guitar would hear. It still needs to be amplified, and for that purpose, a physical model of a tube amplifier is used. Tube amplifiers are based on valves in which heated air conducts the sound, and are still used by the majority of guitarists. When driven hard, the valves add harmonic distortions to the sound which are pleasing to the human ear. After this, the sound is directed to a physical model of a guitar speaker cabinet, and finally, some common guitar effects are added to the sound, such as equalization, reverb and delay.

Equipment

The webcam Air Guitar runs on a regular desktop computer, and all hardware can be bought off-the-shelf from a computer store. The current version at Heureka runs on a Linux computer, and all software used is free or open source. A Windows version is under development.

Top