Application Areas

Autonomous grasping and manipulation with robotic hands predominantly builds on vision and feed-forward approaches: A camera acquires a scene once before taking an action, the desired object is located and finally grasped or manipulated with no or only limited feedback. This is, on the one hand, due to the vision problem not yet fully solved when it comes to heavy occlusions and on the other hand due to the lack of tactile sensing and processing methods providing alternative feedback channels. Previous work has proposed to use approximate models hat assume symmetric shape to plan grasping . Recently there was some progress utilizing Deep Neural Networks and end-to-end-learning, providing methods to learn visuo-motor control policies for grasping and complex manipulation tasks . The advent and miniaturization of tactile sensors also stimulated the development of corresponding tactile processing methods. Using tactile feedback, researchers have investigated exploration strategies for object modelling , and refine a model obtained with visual information . A body of literature addresses the problem of slip detection (estimating friction cones or utilizing neural networks to classify raw sensor data or vibrations ) and judging grasp quality based on tactile and proprioceptive data . Drawing on previous work on tactile surface exploration , NeuTouch will study algorithms to actively explore objects and refine visual models by integrating information derived from touch encoding local features, especially from object areas that are not visible. This information will be used to model objects and plan object grasping or drive manipulation. Learning the characteristic interaction force profile through initial demonstration and subsequent reinforcement learning will finally achieve a robust skill level for dexterous manipulation and operation e.g. of switches. The developed methods should be generic and allow for simple transfer of acquired skills to new, previously unseen objects. This work will be in synergy with the activities investigating spike-based, bio-inspired algorithms for encoding and recognition of tactile features in the context of explorative behaviour and active touch. We will leverage on the understanding of neural encoding of tactile features during exploratory phases, to implement effective strategies in robots. A predictive behaviour is needed to support robots that interact with objects and humans in unconstrained scenarios and need to adaptively and accurately integrate sensory information. A family of Bayesian, latent variable models referred to as “Deep Gaussian Process” (DGP) offers a probabilistic framework for modelling complex data in (un/semi)-supervised mode. They mimic the compression, chunking and consolidation, and recall functions of mammalian episodic memory and have been validated within three core aspects of robot perception: face recognition, arm/hand action recognition and passive touch gesture recognition using the iCub humanoid robot as part of the FP7 WYSIWYD project . NeuTouch will extend this approach to active touch for the control of tactile object exploration and manipulation. The principled handling of uncertainty in the perception and prediction stages, through the Bayesian probabilistic formulation, constitutes a promising path to the development of robust systems for assistive and industrial robotics. Eventually, the work proposed by NeuTouch extends and integrates the results of eMorph (neuromorphic perception in robotics) and development and use of tactile sensing in robots (RoboSkin, TacMAN).