Metis a Tangible User Interface created by NUME Group and presented at the last Festival of Science in Genova – Workshop “Orizzonti Sonori”. The Metis has been realised with the purpose to offer a new interactive educational tool to the community.
After 3 years from its presentation, under supervision of Michelangelo Lupone, Balandino decided to address his thesis to the development of a graphical interface and a audio processing toolkit designed for Metis, in order to bring on stage the live electronics performance Il sogno nel sogno.
The long work conducted by Balandino, can be summarised in three steps: understanding the Metis, development of the dedicated GUI and audio toolkit to perform Il Sogno nel Sogno and at the end the performance itself.
Understanding the Metis.
The hardware of the METIS is composed by a metal table with a plexiglas square as top, which is the tangible surface. Under that, is allocated a infra-red camera, which job is to send the information concerning the physical interaction with the Metis to a computer. This last one, after achieved all data, will elaborate the GUI which is projected to the plexiglas by a projector. In addition, a mirror has been added in order to project the GUI over the whole tangible surface. From the software point of view the Metis is builded upon three layers. The first layer is formed by reacTIVision, a computer vision framework for object tracking and basic multi-touch, which is responsible about the calibration and real time video tracking process. Furthermore, using the TUIO protocol, reacTIVision is able to send all interactions data to third applications.
Secondly, a Java application developed using Processing generates the GUI, which able the user to interact with the Metis. When an interaction is detected, the GUI is subjected to real time graphical elaborations.
In addition, the Java application developed to generate the GUI is also a data bridge btween reacTIVision and Max/MSP using the OSC Protocol.
Finally, a Max/MSP patch is responsible of the audio signal processing, driven trough interaction with the Metis.
Il sogno nel sogno
The performance of Il sogno nel sogno is a performance inspired from the movie Inception by C.Nolan. As in the movie, the music describes the three dreams developed in three layers, which are chronologically expressed. Each dream has a protagonist, the piano, the cello and the bass saxophone; which have been elaborated using three different signal processing techniques. All dreams have been describe by graphical elements as well. In fact, it is possible to notice a change of background every time that a new dream is introduced.