ESR11 - Active Touch and Behaviour

Implementation of spiking tactile neural encoding on a humanoid robot, development of decoding strategies for perception and behavior generation.

Overview

Objectives

Implementation of spiking tactile neural encoding on a humanoid robot, development of decoding strategies for perception and behavior generation. The robot will be a testbed to reproduce biological touch experiments (in collaboration with SISSA) of the perception of multiple stimulus properties arising from active object exploration, including light pressure, vibration, texture, lateral motion, and stretch. The task will be active exploration and modelling of an object using tactile and visual information. Visual information will be used to form an initial guess of the object shape, tactile information will be used to refine this information and complement it using features that characterize the local curvature of the object and areas that are not visible (due to occlusion). This project will use the algorithms developed within research themes 5 and 6 to classify local features from the object and machine learning models (e.g. Gaussian Processes) to model the object surface and provide an accurate shape. In the final part of the project, we will validate the surface reconstruction method in the context of object grasping, using state-of-the-art techniques that rely on object models. For comparison evaluation, we will use a dataset of objects for which accurate models are available (the Yale-CMU-Berkeley Object Data set, a dataset of object manipulation benchmarking).

Expected Results

Active exploration strategy based on tactile feedback for object modelling, experimental validation and benchmarking in a grasping scenario.

Secondments

SISSA

definition of tasks and protocols for psychophysics
USFD

learn prediction and action selection
PAL

test on different robotic platform (generalisation)

Supervisors

C. Bartolozzi
L. Natale
M. Diamond
S. Panzeri
T. Prescott
S. Terreri

Simon Frederik Müller-Cleve

Since my childhood, I am fascinated by the complexity and functionality of the human brain. While I was more on the technical side during my bachelor, I come closer to biology in my master. Here I came in contact with neuromorphic engineering, as a step from the technical side towards biology. To implement biological models and reduce them to the essentials, knowledge about about functionalities in biology can be won and new technical applications developed. It is very exciting for me to work as close to biology as never befor!

Neutouch for me

I chose NeuTouch, because I always wanted to be part of a great research community and get to know many engaged people. To develope solutions in a team, as an ongoing research project is a great chance to built deeper networks, friendships and learn new cool things, daily! The topic "active touch and behavior" is very exciting for me and one step further than my MCs. To work with event-based sensors is one step further towards applications in human prothetics.

Info

Research Topics

Robotics, Event-based computation
Institution

Istituto Italiano di Tecnologia (IIT)
Background

B.Sc. Mechatronic (University of Applied Science Bielefeld, 2014-2018)
M.Sc. BioMechatronic (UAS Bielefeld and University of Bielefeld, 2018-2020)

Thesis

Active Touch and Behaviour. Groningen University, 2025.

Abstract

Humanoid robots have garnered significant attention for their adaptability to dynamic tasks. Despite advancements in vision-based sensing and deep learning, tactile sensing -- crucial for safe human-robot interaction and dexterous manipulation -- remains underutilized. This thesis addresses the challenges of integrating touch, a fundamental human sensory modality, into robotic systems, focusing on bio-inspired approaches and neuromorphic computation.

The human tactile system, characterized by mechanoreceptors, neural pathways, and proprioceptive integration, provides a blueprint for designing tactile sensors. Drawing on these principles, recent developments in tactile sensing technologies are reviewed, from low-resolution designs to high-resolution sensors enabled by advanced manufacturing. This work highlights the advantages of event-driven processing for efficient tactile encoding, which mimics biological systems by transmitting data only upon input changes. Using these methods, a novel tactile Braille dataset was created, featuring the iCub fingertip sliding over 3D-printed Braille letters. Encoding strategies inspired by Fast Adaptive (FA) mechanoreceptors reduced data by up to 122 times. Neuromorphic hardware (Intel Loihi) demonstrated significant power savings during letter classification, achieving efficiency gains over traditional GPU-based methods, though at a cost to accuracy.

To further advance encoding methods, the WiN-GUI was developed, allowing researchers to visualize and optimize spike-pattern encoding in real-time. This tool integrates temporal, spatial, and spatio-temporal datasets to evaluate neuron behavior and classification performance, with applications extending beyond tactile sensing. Complementing this, human haptic object exploration was analyzed to uncover kinematic synergies and Exploratory Procedures (EPs). Using a multimodal dataset of blindfolded participants solving tactile discrimination tasks, this study identified recurring movement patterns and dimensionality reduction strategies that could inform robotic hand control.

By bridging insights from biology, neuromorphic engineering, and robotics, this work offers scalable solutions for touch-sensitive robots designed for dynamic and human-centric environments, paving the way for safer and more capable systems.

Publications

Müller-Cleve, S. F., Fra, V., Khacef, L., Pequeño-Zurro, A., Klepatsch, D., Forno, E., ... & Bartolozzi, C. (2022). Braille letter reading: A benchmark for spatio-temporal pattern recognition on neuromorphic hardware. Frontiers in Neuroscience, 16, 951164.

Di Domenico, D., Forsiuk, I., Müller-Cleve, S., Tanzarella, S., Garro, F., Marinelli, A., ... & Semprini, M. (2025). Reach&Grasp: a multimodal dataset of the whole upper-limb during simple and complex movements. Scientific Data, 12(1), 233.

Müller-Cleve, S. F., Quintana, F. M., Fra, V., Galindo, P. L., Perez-Peña, F., Urgese, G., & Bartolozzi, C. (2024). WiN-GUI: A graphical tool for neuron-based encoding. SoftwareX, 27, 101759.

Fra, V., Müller-Cleve, S. F., Urgese, G., & Bartolozzi, C. (2025). WiN-GUI Version 2: A graphical tool for neuron-based encoding. SoftwareX, 29, 102031.

Top