Helmholtz lecture by Monica Gori from the Italian Institute of Technology in Genua: ‘Interaction between senses during the child development and the creation of new rehabilitation devices’

Abstract

During the first years of life, sensory modalities communicate with each other. Since 2004 we have studied how the haptic, visual, and auditory modalities interact and are integrated during development. We have observed that specific sensory modalities are crucial for developing particular skills, and the absence of one sensory input affects the development of different modalities. For example, in young children, the haptic modality is essential to perceive size, and the lack of haptic skills (in motor impairment) affects the ability to understand the visual size of objects. Similarly, the visual information is crucial to perceive orientation, and the lack of vision (in blindness) influences haptic orientation perception and verticality perception.

These results suggest a strong interaction between sensory systems during the early period of life for perceptual development. Starting from these premises, we have developed new multisensory technology based on multisensory feedback. The goal is to improve children’s perceptual, motor, social, and learning skills when these are impaired. During the presentation, I’ll present the results obtained with ABBI and TechArm. These are two wearable devices that provide audio, visual, and tactile feedbacks associated with body movement to improve haptic manipulation and social interaction. I will also show the results obtained with the WeDraw platforms, a set of applications based on audio, tactile, and visual feedback designed to improve learning skills at the elementary school level. The results are discussed, considering applications for rehabilitation settings in hospitals.