Publications

by Keyword: Learning


By year:[ 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010 | 2009 | 2008 | 2007 | 2006 | 2005 ]

Sánchez-Fibla, M., Forestier, S., Moulin-Frier, C., Puigbò, J. Y., Verschure, P., (2019). From motor to visually guided bimanual affordance learning Adaptive Behavior Article first published online

The mechanisms of how the brain orchestrates multi-limb joint action have yet to be elucidated and few computational sensorimotor (SM) learning approaches have dealt with the problem of acquiring bimanual affordances. We propose a series of bidirectional (forward/inverse) SM maps and its associated learning processes that generalize from uni- to bimanual interaction (and affordances) naturally, reinforcing the motor equivalence property. The SM maps range from a SM nature to a solely sensory one: full body control, delta SM control (through small action changes), delta sensory co-variation (how body-related perceptual cues covariate with object-related ones). We make several contributions on how these SM maps are learned: (1) Context and Behavior-Based Babbling: generalizing goal babbling to the interleaving of absolute and local goals including guidance of reflexive behaviors; (2) Event-Based Learning: learning steps are driven by visual, haptic events; and (3) Affordance Gradients: the vectorial field gradients in which an object can be manipulated. Our modeling of bimanual affordances is in line with current robotic research in forward visuomotor mappings and visual servoing, enforces the motor equivalence property, and is also consistent with neurophysiological findings like the multiplicative encoding scheme.

Keywords: Affordances, Bimanual affordances, Goal babbling, Interlimb coordination, Motor equivalence, Sensorimotor learning


Fonollosa, Jordi, Solórzano, Ana, Marco, Santiago, (2018). Chemical sensor systems and associated algorithms for fire detection: A review Sensors 18, (2), 553

Indoor fire detection using gas chemical sensing has been a subject of investigation since the early nineties. This approach leverages the fact that, for certain types of fire, chemical volatiles appear before smoke particles do. Hence, systems based on chemical sensing can provide faster fire alarm responses than conventional smoke-based fire detectors. Moreover, since it is known that most casualties in fires are produced from toxic emissions rather than actual burns, gas-based fire detection could provide an additional level of safety to building occupants. In this line, since the 2000s, electrochemical cells for carbon monoxide sensing have been incorporated into fire detectors. Even systems relying exclusively on gas sensors have been explored as fire detectors. However, gas sensors respond to a large variety of volatiles beyond combustion products. As a result, chemical-based fire detectors require multivariate data processing techniques to ensure high sensitivity to fires and false alarm immunity. In this paper, we the survey toxic emissions produced in fires and defined standards for fire detection systems. We also review the state of the art of chemical sensor systems for fire detection and the associated signal and data processing algorithms. We also examine the experimental protocols used for the validation of the different approaches, as the complexity of the test measurements also impacts on reported sensitivity and specificity measures. All in all, further research and extensive test under different fire and nuisance scenarios are still required before gas-based fire detectors penetrate largely into the market. Nevertheless, the use of dynamic features and multivariate models that exploit sensor correlations seems imperative

Keywords: Fire detection, Gas sensor, Pattern recognition, Sensor fusion, Machine learning, Toxicants, Carbon monoxide, Hydrogen cyanide, Standard test fires, Transducers, Smoke


Puigbò, J. Y., Arsiwalla, X. D., Verschure, P., (2018). Challenges of machine learning for living machines Biomimetic and Biohybrid Systems 7th International Conference, Living Machines 2018 (Lecture Notes in Computer Science) , Springer International Publishing (Paris, France) 10928, 382-386

Machine Learning algorithms (and in particular Reinforcement Learning (RL)) have proved very successful in recent years. These have managed to achieve super-human performance in many different tasks, from video-games to board-games and complex cognitive tasks such as path-planning or Theory of Mind (ToM) on artificial agents. Nonetheless, this super-human performance is also super-artificial. Despite some metrics are better than what a human can achieve (i.e. cumulative reward), in less common metrics (i.e. time to learning asymptote) the performance is significantly worse. Moreover, the means by which those are achieved fail to extend our understanding of the human or mammal brain. Moreover, most approaches used are based on black-box optimization, making any comparison beyond performance (e.g. at the architectural level) difficult. In this position paper, we review the origins of reinforcement learning and propose its extension with models of learning derived from fear and avoidance behaviors. We argue that avoidance-based mechanisms are required when training on embodied, situated systems to ensure fast and safe convergence and potentially overcome some of the current limitations of the RL paradigm.

Keywords: Avoidance, Neural networks, Reinforcement learning


Herreros, I., (2018). Learning and control Living machines: A handbook of research in biomimetics and biohybrid systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 239-255

This chapter discusses basic concepts from control theory and machine learning to facilitate a formal understanding of animal learning and motor control. It first distinguishes between feedback and feed-forward control strategies, and later introduces the classification of machine learning applications into supervised, unsupervised, and reinforcement learning problems. Next, it links these concepts with their counterparts in the domain of the psychology of animal learning, highlighting the analogies between supervised learning and classical conditioning, reinforcement learning and operant conditioning, and between unsupervised and perceptual learning. Additionally, it interprets innate and acquired actions from the standpoint of feedback vs anticipatory and adaptive control. Finally, it argues how this framework of translating knowledge between formal and biological disciplines can serve us to not only structure and advance our understanding of brain function but also enrich engineering solutions at the level of robot learning and control with insights coming from biology.

Keywords: Feedback control, Feed-forward control, Supervised learning, Unsupervised learning, Reinforcement, Learning, Classical conditioning, Operant conditioning, Reflex, Anticipatory reflex


Freire, I. T., Arsiwalla, X. D., Puigbò, J. Y., Verschure, P., (2018). Limits of multi-agent predictive models in the formation of social conventions Frontiers in Artificial Intelligence and Applications (ed. Falomir, Z., Gibert, K., Plaza, E.), IOS Press (Amsterdam, The Netherlands) Volume 308: Artificial Intelligence Research and Development, 297-301

A major challenge in cognitive science and AI is to understand how intelligent agents might be able to predict mental states of other agents during complex social interactions. What are the computational principles of such a Theory of Mind (ToM)? In previous work, we have investigated hypotheses of how the human brain might realize a ToM of other agents in a multi-agent social scenario. In particular, we have proposed control-based cognitive architectures to predict the model of other agents in a game-theoretic task (Battle of the Exes). Our multi-layer architecture implements top-down predictions from adaptive to reactive layers of control and bottom-up error feedback from reactive to adaptive layers. We tested cooperative and competitive strategies among different multi-agent models, demonstrating that while pure RL leads to reasonable efficiency and fairness in social interactions, there are other architectures that can perform better in specific circumstances. However, we found that even the best predictive models fall short of human data in terms of stability of social convention formation. In order to explain this gap between humans and predictive AI agents, in this work we propose introducing the notion of trust in the form of mutual agreements between agents that might enhance stability in the formation of conventions such as turn-taking.

Keywords: Cognitive Architectures, Game Theory, Multi-Agent Models, Reinforcement Learning, Theory of Mind


Vouloutsi, Vasiliki, Halloy, José, Mura, Anna, Mangan, Michael, Lepora, Nathan, Prescott, T. J., Verschure, P., (2018). Biomimetic and Biohybrid Systems 7th International Conference, Living Machines 2018, Paris, France, July 17–20, 2018, Proceedings , Springer International Publishing (Lausanne, Switzerland) 10928, 1-551

This book constitutes the proceedings of the 7th International Conference on Biomimetic and Biohybrid Systems, Living Machines 2018, held in Paris, France, in July 2018. The 40 full and 18 short papers presented in this volume were carefully reviewed and selected from 60 submissions. The theme of the conference targeted at the intersection of research on novel life-like technologies inspired by the scientific investigation of biological systems, biomimetics, and research that seeks to interface biological and artificial systems to create biohybrid systems.

Keywords: Artificial neural network, Bio-actuators, Bio-robotics, Biohybrid systems, Biomimetics, Bipedal robots, Earthoworm-like robots, Robotics, Decision-making, Tactile sensing, Soft robots, Locomotion, Insects, Sensors, Actuators, Robots, Artificial intelligence, Neural networks, Motion planning, Learning algorithms


Maffei, Giovanni, Herreros, Ivan, Sanchez-Fibla, Marti, Friston, Karl J., Verschure, Paul F. M. J., (2017). The perceptual shaping of anticipatory actions Proceedings of the Royal Society B , 284, (1869)

Humans display anticipatory motor responses to minimize the adverse effects of predictable perturbations. A widely accepted explanation for this behavior relies on the notion of an inverse model that, learning from motor errors, anticipates corrective responses. Here, we propose and validate the alternative hypothesis that anticipatory control can be realized through a cascade of purely sensory predictions that drive the motor system, reflecting the causal sequence of the perceptual events preceding the error. We compare both hypotheses in a simulated anticipatory postural adjustment task. We observe that adaptation in the sensory domain, but not in the motor one, supports the robust and generalizable anticipatory control characteristic of biological systems. Our proposal unites the neurobiology of the cerebellum with the theory of active inference and provides a concrete implementation of its core tenets with great relevance both to our understanding of biological control systems and, possibly, to their emulation in complex artefacts.

Keywords: Active inference, Cerebellum, Computational model, Motor control, Perceptual learning


Aviles, A. I., Alsaleh, S., Montseny, E., Sobrevilla, P., Casals, A., (2016). A Deep-Neuro-Fuzzy approach for estimating the interaction forces in Robotic surgery FUZZ-IEEE IEEE International Conference on Fuzzy Systems , IEEE (Vancouver, Canada ) , 1113-1119

Fuzzy theory was motivated by the need to create human-like solutions that allow representing vagueness and uncertainty that exist in the real-world. These capabilities have been recently further enhanced by deep learning since it allows converting complex relation between data into knowledge. In this paper, we present a novel Deep-Neuro-Fuzzy strategy for unsupervised estimation of the interaction forces in Robotic Assisted Minimally Invasive scenarios. In our approach, the capability of Neuro-Fuzzy systems for handling visual uncertainty, as well as the inherent imprecision of real physical problems, is reinforced by the advantages provided by Deep Learning methods. Experiments conducted in a realistic setting have demonstrated the superior performance of the proposed approach over existing alternatives. More precisely, our method increased the accuracy of the force estimation and compared favorably to existing state of the art approaches, offering a percentage of improvement that ranges from about 35% to 85%.

Keywords: Estimation, Force, Machine learning, Robots, Three-dimensional displays, Uncertainty, Visualization


Marbán, Arturo, Casals, Alicia, Fernández, Josep, Amat, Josep, (2014). Haptic feedback in surgical robotics: Still a challenge Advances in Intelligent Systems and Computing ROBOT2013: First Iberian Robotics Conference (ed. Armada, Manuel A., Sanfeliu, Alberto, Ferre, Manuel), Springer International Publishing 252, 245-253

Endowing current surgical robotic systems with haptic feedback to perform minimally invasive surgery (MIS), such as laparoscopy, is still a challenge. Haptic is a feature lost in surgical teleoperated systems limiting surgeons capabilities and ability. The availability of haptics would provide important advantages to the surgeon: Improved tissue manipulation, reducing the breaking of sutures and increase the feeling of telepresence, among others. To design and develop a haptic system, the measurement of forces can be implemented based on two approaches: Direct and indirect force sensing. MIS performed with surgical robots, imposes many technical constraints to measure forces, such as: Miniaturization, need of sterilization or materials compatibility, making it necessary to rely on indirect force sensing. Based on mathematical models of the components involved in an intervention and indirect force sensing techniques, a global perspective on how to address the problem of measurement of tool-tissue interaction forces is presented.

Keywords: Surgical robotics, Haptic feedback, Indirect force sensing, Machine learning, Data fusion, Mathematical models


Antelis, J.M., Montesano, L., Giralt, X., Casals, A., Minguez, J., (2012). Detection of movements with attention or distraction to the motor task during robot-assisted passive movements of the upper limb Engineering in Medicine and Biology Society (EMBC) 34th Annual International Conference of the IEEE , IEEE (San Diego, USA) , 6410-6413

Robot-assisted rehabilitation therapies usually focus on physical aspects rather than on cognitive factors. However, cognitive aspects such as attention, motivation, and engagement play a critical role in motor learning and thus influence the long-term success of rehabilitation programs. This paper studies motor-related EEG activity during the execution of robot-assisted passive movements of the upper limb, while participants either: i) focused attention exclusively on the task; or ii) simultaneously performed another task. Six healthy subjects participated in the study and results showed lower desynchronization during passive movements with another task simultaneously being carried out (compared to passive movements with exclusive attention on the task). In addition, it was proved the feasibility to distinguish between the two conditions.

Keywords: Electrodes, Electroencephalography, Induction motors, Medical treatment, Robot sensing systems, Time frequency analysis, Biomechanics, Cognition, Electroencephalography, Medical robotics, Medical signal detection, Medical signal processing, Patient rehabilitation, Attention, Cognitive aspects, Desynchronization, Engagement, Motivation, Motor learning, Motor task, Motor-related EEG activity, Physical aspects, Robot-assisted passive movement detection, Robot-assisted rehabilitation therapies, Upper limb