Technologies for Touch
Significant progress in the development of artificial skin has been achieved in recent years, with specific focus on robotics, however, the research focused on materials and transduction properties, delivering a plethora of different technologies, most of which have too many trade-offs to be integrated on robots or prosthetic devices. So far, few prototypes have been successfully applied, leading to the design of fingertips or large skin. In these sensors, the physical signal from the front-end material is continuously sampled and stored, without a focus on strategies for efficient, application-specific encoding. In this sense, the approach of NeuTouch in completely original, as it will select a few candidate technologies and work at the encoding level, with embedded neuromorphic circuits. Specifically, we will target capacitive and piezoelectric sensors integrated with traditional silicon technology, graphene integrated with photovoltaic cells, and silicon nano-wires for neuromorphic processing. The use of neuromorphic, event-driven spike coding is especially suited for data compression, as spikes will be sent only when and where there is contact. Their use will improve latency, as spikes are sent asynchronously, as soon as contact is detected. Employing a task-specific encoding will maximize the transfer of information, effectively developing a system where hardware and software are co-designed to improve the function of the system as a whole.
Technologies for Touch
Touch for Robotics
Autonomous grasping and manipulation with robotic hands predominantly builds on vision and feed-forward approaches: A camera acquires a scene once before taking an action, the desired object is located and finally grasped or manipulated with no or only limited feedback. This is, on the one hand, due to the vision problem not yet fully solved when it comes to heavy occlusions and on the other hand due to the lack of tactile sensing and processing methods providing alternative feedback channels. Previous work has proposed to use approximate models hat assume symmetric shape to plan grasping . Recently there was some progress utilizing Deep Neural Networks and end-to-end-learning, providing methods to learn visuo-motor control policies for grasping and complex manipulation tasks . The advent and miniaturization of tactile sensors also stimulated the development of corresponding tactile processing methods. Using tactile feedback, researchers have investigated exploration strategies for object modelling , and refine a model obtained with visual information . A body of literature addresses the problem of slip detection (estimating friction cones or utilizing neural networks to classify raw sensor data or vibrations ) and judging grasp quality based on tactile and proprioceptive data . Drawing on previous work on tactile surface exploration , NeuTouch will study algorithms to actively explore objects and refine visual models by integrating information derived from touch encoding local features, especially from object areas that are not visible. This information will be used to model objects and plan object grasping or drive manipulation. Learning the characteristic interaction force profile through initial demonstration and subsequent reinforcement learning will finally achieve a robust skill level for dexterous manipulation and operation e.g. of switches. The developed methods should be generic and allow for simple transfer of acquired skills to new, previously unseen objects. This work will be in synergy with the activities investigating spike-based, bio-inspired algorithms for encoding and recognition of tactile features in the context of explorative behaviour and active touch. We will leverage on the understanding of neural encoding of tactile features during exploratory phases, to implement effective strategies in robots. A predictive behaviour is needed to support robots that interact with objects and humans in unconstrained scenarios and need to adaptively and accurately integrate sensory information. A family of Bayesian, latent variable models referred to as “Deep Gaussian Process” (DGP) offers a probabilistic framework for modelling complex data in (un/semi)-supervised mode. They mimic the compression, chunking and consolidation, and recall functions of mammalian episodic memory and have been validated within three core aspects of robot perception: face recognition, arm/hand action recognition and passive touch gesture recognition using the iCub humanoid robot as part of the FP7 WYSIWYD project . NeuTouch will extend this approach to active touch for the control of tactile object exploration and manipulation. The principled handling of uncertainty in the perception and prediction stages, through the Bayesian probabilistic formulation, constitutes a promising path to the development of robust systems for assistive and industrial robotics. Eventually, the work proposed by NeuTouch extends and integrates the results of eMorph (neuromorphic perception in robotics) and development and use of tactile sensing in robots (RoboSkin, TacMAN).
Touch for Robotics
Touch for Prosthetics
The need/desire for functional replacement of a missing upper limb is an ancient one: historically humans have replaced a missing limb with a prosthesis for cosmetic, vocational, or personal autonomy reasons. The hand is a powerful tool and its loss causes severe physical and often mental debilitation. Together with the obvious inability to grasp and manipulate objects, an amputee loses the capability to sense and explore the surrounding world as well as the ability to use gestures to support speech and express emotions; moreover she/he may develop psychological problems and encumbrance due to physical differences compared with non-disabled people. Significant advances have been achieved in the recent past, such as the development of dexterous artificial hands capable of complex motor tasks and of decoding control methods that restore more natural motor control . Implanted neural interfaces surgically inserted into the peripheral nerves of the residuum have shown remarkable potential as tools for restoring the sensory information flow between the hand prosthesis and the nervous system. Initial very promising demonstrations have shown that tactile information can be restored , opening new and exciting possibilities. Intraneural electrodes, in particular, have been used to restore several features of the sense of touch including sophisticated ones such as texture discrimination , object compliance and shape recognition . Notwithstanding these very promising results, and recent/ongoing research networks (e.g., EU projects: INTER, GRIP, CYBERHAND, NEUROBOTICS, SMARTHAND, NANOBIOTACT, TIME, NEBIAS, or the HAPTIX DARPA initiative) have contributed to the progress of neuro-controlled hand prostheses, important efforts are still necessary to make these solutions more effective (i.e., able to provide more natural and rich sensations). In particular, this proposal is going to address two important open issues, i.e., the development of more biocompatible neural interfaces and the identification of more natural encoding (stimulation) strategies.
Touch for Prosthetics