During the development and the operation of the HapticSOUND project, the safety and the integrity of the museum’s collections are guaranteed as non-interventive techniques will be used. HapticSOUND is based on the aspects of presenting information, providing opportunities for interaction and offering entertainment. Specifically, the HapticSOUND project involves the development of a system, which will demonstrate information on traditional Cretan instruments; it will provide to the visitors of the Museum the opportunities to play notes, chords or melodies to familiarize with the sounds instruments produce; it will aid visitors to appreciate the manner the instruments are built by constructing the instrument. The visitors will study the instruments, they will comprehend their functional models and they will experiment with the instruments by scaling their experiences from playing notes to playing songs. Moreover, the visitors will study the structural model of the musical instrument by assembling the instrument from its structural units.

Through a gesture recognition system, the visitors will be able to use the HapticSOUND system to:

1) Get information on the traditional Cretan instruments through:

a) the demonstration of texts, videos, photographs and 3D models. The demonstration will include the structural analysis of the instruments and the appropriate documentation.

b) playing of music samples in a special sound device that will reproduce sounds and vibrations in such a way that people with disabilities and hearing problems will be able to experience the transmissions of sounds.

2) Interact with a replica of the musical instrument (product of 3D printing) to:

a) explore haptically (with the use of appropriate sensor devices) and acoustically the capabilities of the instrument by playing notes under the guidance of the HapticSOUND system and by receiving visual and audio feedback in real-time (with the help of an avatar).

b) learn the instrument, in the context of an open music lab, through a process of imitating authentic performances.

3) Enjoy by playing a serious game, where visitors will try to virtually assemble a traditional Cretan instrument from a set of 3D objects.

The HapticSOUND project is an innovative approach, which utilizes novel sensorimotor learning and machine learning techniques in the context of playing sounds based on gesture recognition and fingerprinting at real time. Moreover, the utilization of bone conduction techniques is investigated, as well as the development of serious games based on serious games development models and modern learning theories. The research which will be carried out and the results of its application are expected to significantly promote the production and transmission of knowledge.