Published on
Scientists in Italy have demonstrated that robotics can actually sharpen musical timing and coordination between performers.
In the study, which has been the Science Robotics journal_,_ professional violinists were fitted with exoskeletons on their bow-playing arms.
The robots delivered subtle haptic feedback, helping the musicians synchronise their movements.
Francesco Di Tommaso, one of the robotics researchers at Università Campus Bio-Medico, explained the approach: “To answer our scientific question, we designed an experiment in which we tested different sensory feedback conditions. So, most notably, we had one condition where they could hear each other and see each other, which is the traditional one, and one condition where we prevented vision and we introduced haptics, so this (was) feedback from the devices.”
He added: “In this way, we were able to demonstrate that replacing vision with haptics actually enhances coordination both in kinematics and in musical alignment so they were able to perform better both in terms of their movements and in terms of musical output.”
How the study was carried out
The violinists were tested under four conditions: hearing but not seeing each other; both hearing and seeing; vision blocked but the exoskeleton active; and finally, full sensory feedback combined with the exoskeleton.
Infrared cameras and sensors recorded the angles of their arms, shoulder positions, and the force applied to the bows.
“The exoskeletons are specifically designed for the upper limbs and they assist the movements of the shoulder and the elbow, specifically the flexion extension of the elbow and the internal and external rotation of the shoulder,” Di Tommaso said.
“They were designed to physically couple two individuals performing a movement, which means that they have sort of a virtual communication between them. So, the movements that are recorded by one exoskeleton are transmitted to the other one.”
When the musicians’ movements differed, the exoskeletons delivered bidirectional forces to synchronise them. While some participants reported discomfort, the system ultimately improved their coordination.
“So at the end of the experiments, we asked participants what they thought about the forces that they were feeling. They were unaware of the origin of those forces and most of them did not realize that they were actually coming from their partner and some of them actually reported that they felt discomfort in perceiving those forces. But the interesting part is that actually those forces helped them to coordinate better,” Di Tommaso added.
Beyond the music
The study was not designed for clinical use, but Professor Domenico Formica, a bioengineer involved in the research, suggested the technology could have broader applications.
“In theory the same concept may be applied to several fields and one may be, for example, motor rehabilitation. So in motor rehabilitation mediated by robots, we usually have patients that interact with the robot to improve their recovery. Using the same technology, we may have, for example, a therapist interacting with the patient and by this bilateral exchange of forces, this may improve the recovery during the therapy. Or we can also consider having two patients that work together to be challenged in the process of their recovery,” Formica said.