Human and Robot Hands, Human and Robot Touch: Sensorimotor Synergies to Bridge the Gap Between Neuroscience and Robotics

Full-Day Workshop (Maestro A, 9:00-17:00)

The human hand is our preeminent and most versatile tool to explore and modify the external environment. It represents both the cognitive organ of the sense of touch and the most important end effector in object manipulation and grasping. Our brain can cope efficiently with high degree of complexity of the hand, which arises from the large amount of actuators and sensors. This allows us to perform a large number of daily life tasks, from the simple ones, such as determining the ripeness of a fruit or drive a car, to the more complex ones, as for example performing surgical procedures, playing an instrument or painting. Not surprisingly, an intensive research effort has been devoted to (i) understand the neuroanatomical and physiological mechanisms underpinning the sensorimotor control of human hands and (ii) to attempt to reproduce such mechanisms in artificial robotic systems.

This workshop focuses on the issues that human and robotic hands faces when controlling the large number of joints, actuators and sensors to perform motor tasks such as object exploration, manipulation and grasping.

We present an integrated approach to explore the control of the hand based on sensorimotor synergies that can be applied in both neuroscience and robotics. Hand synergies are based on goal-directed, combined muscle and kinematic activation leading to a reduction of the dimensionality of the motor and sensory space, and provide a highly effective solution for the fast and simplified design of artificial systems.

We will present results on the theoretical and experimental foundations to describe the synergistic organization of the human hand. At the same time, we will show how to exploit the framework of hand synergies to better control and design robotic hands and haptic/sensing systems/ tools, using a reduced number of control inputs/sensors, with the goal of pushing their effectiveness close to the natural one.

The workshop is motivated by the contributed book entitled Human and Robot Hands edited by Matteo Bianchi and Alessandro Moscatelli. A (pre-)print copy of the book will be visible and participants will be able to (pre-)order it. Speakers will be either authors of the book and respondent to a call for internationally recognized scientists working on the study of human and artificial hands. Talks will be overview or research and, in the case of authors, could be excerpts from the chapter contributions.


The workshop is open to any student, researcher as well as developer and end user interested in the study of sensory-motor aspects of human hands and in the design of artificial hands and haptic sensing systems. Multidisciplinary aspects will be covered, ranging from neuroscience and psychology to computer science, mechatronics and robotics.

The goal is to encourage discussion between neuroscience experts and technology developers, to elaborate suitable tools and guidelines for the construction of more efficient artificial systems and to endow perceptual research of more advanced instruments for the investigation of human behaviour.

Tentative Program

9:00 General Introduction
  Part I: Neuroscience
9:15 ​Dexterous Manipulation: from High-level Representation to Low-level Coordination of Digit Force and Position
Qiushi Fu and Marco Santello   Arizona State University
9:45 Digit Position and Force Synergies During Unconstrained Grasping
Abdeldjallil Naceri   University of Bielefeld
10:15 Break
10:30 Sensorimotor Synergies: Fusion of Cutaneous Touch and Proprioception in the Perceived Hand Kinematics
Alessandro Moscatelli   University of Bielefeld and University of Rome Tor Vergata
11:00 Cross-Sensory Information Processing as an Instance of Dimensionality Reduction in Perception
Jessica Hartcher-O'Brien, Alexander Terekhov and Vincent Hayward   UPMC
11:30 Synergies in the Brain: Measuring Brain Activity for Complex Finalized Hand Movements
Andrea Leo and Emiliano Ricciardi   University of Pisa
  Part II: Robotics, Models and Sensing Tools
14:00 Sensorimotor Synergies for the Design of Haptic, Sensing and Robotic Systems
Matteo Bianchi   Istituto Italiano di Tecnologia and University of Pisa
14:30 A Myoelectric Prosthetic Prototype Based on Novel Robotic Hand and First Grasp Synergy: Patient Outcomes
Kristin D. Zhao   Mayo Clinic
15:00 Single Grasp Object Identification with Adaptive Hands and Tactile Sensing
Minas Liarokapis   Yale University
15:30 Break
15:45 Kineto-Static Analysis and Control of Synergistic Underactuated Robotic Hands
Edoardo Farnioli   Istituto Italiano di Tecnologia
16:15 How to Map Human Hand Synergies onto Robotic Hands Using the Syngrasp Matlab Toolbox
Gionata Salvietti and Domenico Prattichizzo   University of Siena


Matteo Bianchi, Eng Ph.D.   Istituto Italiano di Tecnologia and University of Pisa

Matteo Bianchi received the B.S degree and the M.S cum laude in Biomedical Engineering from University of Pisa, Italy, in 2004 and 2007, respectively and the PhD in Automatics, Robotics and Bioengineering from University of Pisa in 2012. Currently, he is working as a Post-Doctoral researcher at the Advanced Robotics Department of Istituto Italiano di Tecnologia, as a collaborator of Centro di Ricerca "E. Piaggio" of University of Pisa and Clinic Research Affiliate at Mayo Clinic (Rochester, US). From January to June 2011, he worked as visiting student at the Laboratory for Computational Sensing and Robotics at Johns Hopkins University, Baltimore, MD. His research interests include haptic interface design, with applications in medical robotics and assistive/affective human-robot interaction; human and robotic hands: optimal sensing and control; psychophysics and mathematical modelling of the sense of touch and human manipulation. He is an author of contributions to international conferences and journals. In 2010 he was finalist for the Best Student Paper and Best Paper Award at the IEEE Haptics Symposium; in 2012 he was awarded with the JCTF novel technology paper award at the IEEE/RSJ IROS Conference and in 2014 with the best paper award (poster category) at the Eurohaptics Conference.

Alessandro Moscatelli, M.D., Ph. D.   University of Bielefeld and University of Rome "Tor Vergata"

Alessandro Moscatelli received the Medical Doctor degree in July 2006 from the University of Rome "Tor Vergata", with a grade of 110/110 cum laude. In 2008 he received the Master degree in Biostatistics from the University of Rome "Tor Vergata" with a grade of 110/110 cum laude. In 2011 was awarded with the Ph.D. in Neuroscience (supervisor: Prof. F. Lacquaniti). From October 2011 to October 2015 he worked as post doc at the Department of Cognitive Neuroscience and Cognitive Interaction Technology - Center of Excellence of the Bielefeld University (head of the department: Prof. Marc O. Ernst). He collaborated to the European Project "The Hand Embodied" and "WearHap". His current research interests are the behavioral and neural correlates of haptic perception. His recent studies focused on the representation of space from tactile motion cues, results of these studies showed the capacity of the tactile system to integrate the velocity-signal over time. He published several studies on the integration of proprioception and touch in the perception of object motion. Moscatelli developed an R package for the estimate of psychophysical parameters in mixed model framework. He was awarded in 2014 with the best paper award (poster category) at the Eurohaptics Conference.