Staff member


Paul Verschure

Group Leader / ICREA Research Professor
Synthetic, Perceptive, Emotive and Cognitive Systems (SPECS)
pverschure@ibecbarcelona.eu
+34 93 401 19 89
Staff member publications

Pacheco Estefan, D., Sánchez-Fibla, M., Duff, A., Principe, A., Rocamora, R., Zhang, H., Axmacher, N., Verschure, P., (2019). Coordinated representational reinstatement in the human hippocampus and lateral temporal cortex during episodic memory retrieval Nature Communications 10, (1), 2255

Theoretical models of episodic memory have proposed that retrieval depends on interactions between the hippocampus and neocortex, where hippocampal reinstatement of item-context associations drives neocortical reinstatement of item information. Here, we simultaneously recorded intracranial EEG from hippocampus and lateral temporal cortex (LTC) of epilepsy patients who performed a virtual reality spatial navigation task. We extracted stimulus-specific representations of both item and item-context associations from the time-frequency patterns of activity in hippocampus and LTC. Our results revealed a double dissociation of representational reinstatement across time and space: an early reinstatement of item-context associations in hippocampus preceded a later reinstatement of item information in LTC. Importantly, reinstatement levels in hippocampus and LTC were correlated across trials, and the quality of LTC reinstatement was predicted by the magnitude of phase synchronization between hippocampus and LTC. These findings confirm that episodic memory retrieval in humans relies on coordinated representational interactions within a hippocampal-neocortical network.


Grechuta, K., Rubio Ballester, B., Espín Munne, R., Usabiaga Bernal, T., Molina Hervás, B., Mohr, B., Pulvermüller, F., San Segundo, R., Verschure, P., (2019). Augmented dyadic therapy boosts recovery of language function in patients with nonfluent aphasia Stroke 50, (5), 1270-1274

Background and Purpose- Evidence suggests that therapy can be effective in recovering from aphasia, provided that it consists of socially embedded, intensive training of behaviorally relevant tasks. However, the resources of healthcare systems are often too limited to provide such treatment at sufficient dosage. Hence, there is a need for evidence-based, cost-effective rehabilitation methods. Here, we asked whether virtual reality-based treatment grounded in the principles of use-dependent learning, behavioral relevance, and intensity positively impacts recovery from nonfluent aphasia. Methods- Seventeen patients with chronic nonfluent aphasia underwent intensive therapy in a randomized, controlled, parallel-group trial. Participants were assigned to the control group (N=8) receiving standard treatment or to the experimental group (N=9) receiving augmented embodied therapy with the Rehabilitation Gaming System for aphasia. All Rehabilitation Gaming System for aphasia sessions were supervised by an assistant who monitored the patients but did not offer any elements of standard therapy. Both interventions were matched for intensity and materials. Results- Our results revealed that at the end of the treatment both groups significantly improved on the primary outcome measure (Boston Diagnostic Aphasia Examination: control group, P=0.04; experimental group, P=0.01), and the secondary outcome measure (lexical access-vocabulary test: control group, P=0.01; experimental group, P=0.007). However, only the Rehabilitation Gaming System for aphasia group improved on the Communicative Aphasia Log ( P=0.01). The follow-up assessment (week 16) demonstrated that while both groups retained vocabulary-related changes (control group, P=0.01; experimental group, P=0.007), only the Rehabilitation Gaming System for aphasia group showed therapy-induced improvements in language ( P=0.01) and communication ( P=0.05). Conclusions- Our results demonstrate the effectiveness of Rehabilitation Gaming System for aphasia for improving language and communication in patients with chronic aphasia suggesting that current challenges faced by the healthcare system in the treatment of stroke might be effectively addressed by augmenting traditional therapy with computer-based methods. Clinical Trial Registration- URL: https://www.clinicaltrials.gov . Unique identifier: NCT02928822.

Keywords: Aphasia, Embodied training, Neurological rehabilitation, Virtual reality


Blithikioti, C., Miquel, L., Batalla, A., Rubio, B., Maffei, G., Herreros, I., Gual, A., Verschure, P., Balcells-Oliveró, M., (2019). Cerebellar alterations in cannabis users: A systematic review Addiction Biology Online Version of Record before inclusion in an issue

Cannabis is the most used illicit substance in the world. As many countries are moving towards decriminalization, it is crucial to determine whether and how cannabis use affects human brain and behavior. The role of the cerebellum in cognition, emotion, learning, and addiction is increasingly recognized. Because of its high density in CB1 receptors, it is expected to be highly affected by cannabis use. The aim of this systematic review is to investigate how cannabis use affects cerebellar structure and function, as well as cerebellar‐dependent behavioral tasks. Three databases were searched for peer‐reviewed literature published until March 2018. We included studies that focused on cannabis effects on cerebellar structure, function, or cerebellar‐dependent behavioral tasks. A total of 348 unique records were screened, and 40 studies were included in the qualitative synthesis. The most consistent findings include (1) increases in cerebellar gray matter volume after chronic cannabis use, (2) alteration of cerebellar resting state activity after acute or chronic use, and (3) deficits in memory, decision making, and associative learning. Age of onset and higher exposure to cannabis use were frequently associated with increased cannabis‐induced alterations. Chronic cannabis use is associated with alterations in cerebellar structure and function, as well as with deficits in behavioral paradigms that involve the cerebellum (eg, eyeblink conditioning, memory, and decision making). Future studies should consider tobacco as confounding factor and use standardized methods for assessing cannabis use. Paradigms exploring the functional activity of the cerebellum may prove useful as monitoring tools of cannabis‐induced impairment.

Keywords: Behavior, Cannabis use, Cerebellum, Cognitive function, Structure


Ballester, B. R., Maier, M., Duff, A., Cameirão, M., Bermúdez, S., Duarte, E., Cuxart, A., Rodríguez, S., San Segundo Mozo, R. M., Verschure, P., (2019). A critical time window for recovery extends beyond one-year post-stroke Journal of neurophysiology 122, (1), 350-357

The impact of rehabilitation on post-stroke motor recovery and its dependency on the patient's chronicity remain unclear. The field has widely accepted the notion of a proportional recovery rule with a "critical window for recovery" within the first 3-6 mo poststroke. This hypothesis justifies the general cessation of physical therapy at chronic stages. However, the limits of this critical window have, so far, been poorly defined. In this analysis, we address this question, and we further explore the temporal structure of motor recovery using individual patient data from a homogeneous sample of 219 individuals with mild to moderate upper-limb hemiparesis. We observed that improvement in body function and structure was possible even at late chronic stages. A bootstrapping analysis revealed a gradient of enhanced sensitivity to treatment that extended beyond 12 mo poststroke. Clinical guidelines for rehabilitation should be revised in the context of this temporal structure. NEW & NOTEWORTHY Previous studies in humans suggest that there is a 3- to 6-mo "critical window" of heightened neuroplasticity poststroke. We analyze the temporal structure of recovery in patients with hemiparesis and uncover a precise gradient of enhanced sensitivity to treatment that expands far beyond the limits of the so-called critical window. These findings highlight the need for providing therapy to patients at the chronic and late chronic stages.

Keywords: Motor recovery, Neuroplasticity, Neurorehabilitation, Stroke recovery, Virtual reality


Maier, Martina, Rubio Ballester, Belén, Duff, Armin, Duarte Oller, Esther, Verschure, P., (2019). Effect of specific over nonspecific VR-based rehabilitation on poststroke motor recovery: A systematic meta-analysis Neurorehabilitation and Neural Repair 33, (2), 112-129

Background. Despite the rise of virtual reality (VR)-based interventions in stroke rehabilitation over the past decade, no consensus has been reached on its efficacy. This ostensibly puzzling outcome might not be that surprising given that VR is intrinsically neutral to its use—that is, an intervention is effective because of its ability to mobilize recovery mechanisms, not its technology. As VR systems specifically built for rehabilitation might capitalize better on the advantages of technology to implement neuroscientifically grounded protocols, they might be more effective than those designed for recreational gaming. Objective. We evaluate the efficacy of specific VR (SVR) and nonspecific VR (NSVR) systems for rehabilitating upper-limb function and activity after stroke. Methods. We conducted a systematic search for randomized controlled trials with adult stroke patients to analyze the effect of SVR or NSVR systems versus conventional therapy (CT). Results. We identified 30 studies including 1473 patients. SVR showed a significant impact on body function (standardized mean difference [SMD] = 0.23; 95% CI = 0.10 to 0.36; P = .0007) versus CT, whereas NSVR did not (SMD = 0.16; 95% CI = −0.14 to 0.47; P = .30). This result was replicated in activity measures. Conclusions. Our results suggest that SVR systems are more beneficial than CT for upper-limb recovery, whereas NSVR systems are not. Additionally, we identified 6 principles of neurorehabilitation that are shared across SVR systems and are possibly responsible for their positive effect. These findings may disambiguate the contradictory results found in the current literature.

Keywords: Stroke, Paresis, Virtual reality, Rehabilitation, Occupational therapy, Review


Sánchez-Fibla, M., Forestier, S., Moulin-Frier, C., Puigbò, J. Y., Verschure, P., (2019). From motor to visually guided bimanual affordance learning Adaptive Behavior Article first published online

The mechanisms of how the brain orchestrates multi-limb joint action have yet to be elucidated and few computational sensorimotor (SM) learning approaches have dealt with the problem of acquiring bimanual affordances. We propose a series of bidirectional (forward/inverse) SM maps and its associated learning processes that generalize from uni- to bimanual interaction (and affordances) naturally, reinforcing the motor equivalence property. The SM maps range from a SM nature to a solely sensory one: full body control, delta SM control (through small action changes), delta sensory co-variation (how body-related perceptual cues covariate with object-related ones). We make several contributions on how these SM maps are learned: (1) Context and Behavior-Based Babbling: generalizing goal babbling to the interleaving of absolute and local goals including guidance of reflexive behaviors; (2) Event-Based Learning: learning steps are driven by visual, haptic events; and (3) Affordance Gradients: the vectorial field gradients in which an object can be manipulated. Our modeling of bimanual affordances is in line with current robotic research in forward visuomotor mappings and visual servoing, enforces the motor equivalence property, and is also consistent with neurophysiological findings like the multiplicative encoding scheme.

Keywords: Affordances, Bimanual affordances, Goal babbling, Interlimb coordination, Motor equivalence, Sensorimotor learning


Rubio Ballester, Belen, Duff, Armin, Maier, Martina, Cameirao, Monica, Bermudez, Sergi, Duarte, Esther, Cuxart, Ampar, Rodriguez, Susana, Verschure, Paul F. M. J., (2019). Revealing an extended critical window of recovery post-stroke bioRxiv (pre-print server) , 458745

The impact of rehabilitation on post-stroke motor recovery and its dependency on the patient's chronicity remain unclear. The existence and regularity of a, so called, proportional recovery rule across a range of functional deficits and therapies supports the notion that functional interventions have little or no impact beyond spontaneous recovery rates in a 'critical window of recovery' which lasts from 3 to 6 months post-stroke. In this meta-analysis, we apply a bootstrap analysis method to assess the overall impact of a specific VR-based rehabilitation protocol for the upper extremities on a homogeneous sample of 219 individuals with hemiparesis at various stages post stroke. Our analysis uncovers a precise gradient of sensitivity to treatment that expands more than one year beyond the limits of the so-called 'critical window of recovery'. These findings redefine the limits of the so-called 'critical window of recovery' and suggest that stroke-derived plasticity mechanisms do facilitate functional recovery even at the chronic and late chronic stage.


Grechuta, Klaudia, Ulysse, Laura, Rubio Ballester, Belen, Verschure, Paul, (2019). Self beyond the body: task-relevant distal cues modulate performance and body ownership bioRxiv (pre-print server) , 361022

The understanding of Body Ownership (BO) largely relies on the Rubber Hand Illusion (RHI) where synchronous stroking of real and Rubber Hands (RH) leads to an illusion of ownership of RH provided physical, anatomical, postural and spatial plausibility of the two body-parts. RHI also occurs during visuomotor synchrony, in particular, when the visual feedback of virtual arm movements follows the trajectory of the instantiated motor command. Hence BO seems to result from a bottom-up integration of afferent and efferent proximal multisensory evidence, and top-down prediction of both externally and self-generated signals, which occurs when the predictions about upcoming sensory signals are accurate. In motor control, the differential processing of predicted and actual sensory consequences of self-generated actions is addressed by, the so-called, Forward Model (FM). Based on an efference copy or corollary discharge, FM issues predictions about the sensory consequences of motor commands and compares them with the actual outcome. The discrepancies (Sensory Prediction Errors, SPEs) are used to correct the action on the consecutive trial and provide new estimates of the current state of the body and the environment. Here, we propose that BO might be computed by FMs, and therefore, it might depend on their consistency, specifically, in contexts where the sensory feedback is self-generated. Crucially, to reduce SPE, FMs integrate both proximal (proprioceptive) and distal (vision, audition) sensory cues relevant to the task. Thus, if BO depends on the consistency of FMs, it would be compromised by the incongruency of not only proximal but also distal cues. To test our hypothesis, we devised an embodied VR-based task where action outcomes were signaled by distinct auditory cues. By manipulating the cues with respect to their spatiotemporal congruency and valence, we show that distal feedback which violates predictions about action outcomes compromises both BO and performance. These results demonstrate that BO is influenced by not only efferent and afferent cues which pertain to the body itself but also those arising outside of the body and suggest that in goal-oriented tasks BO might result from a computation of FM.


Feitosa, J. A., Stefano Filho, C. A., Casseb, R. F., Camargo, A., Martins, B. S. G., Ballester, B. R., Omedas, P., Verschure, P., Oberg, T. D., Min, L. L., Castellano, G., (2019). Complex network changes during a virtual reality rehabilitation protocol following stroke: A case study NER 2019 9th International IEEE/EMBS Conference on Neural Engineering , IEEE (San Francisco, USA) , 891-894

Stroke is one of the main causes of disabilities caused by injuries to the human central nervous system, yielding a wide range of mild to severe impairments that can compromise sensorimotor and cognitive functions. Although rehabilitation protocols may improve function of stroke survivors, patients often reach plateaus while undergoing therapy. Recently, virtual reality (VR) technologies have been paired with traditional rehabilitation aiming to improve function recovery after stroke. Aiming to better understand structural brain changes due to VR rehabilitation protocols, we modeled the brain as a graph and extracted three measures representing the network's topology: degree, clustering coefficient and betweenness centrality (BC). In this single case study, our results indicate that all metrics increased on the ipsilesional hemisphere, while remaining about the same at the contrale-sional site. Particularly, the number of functional connections increased in the lesion area overtime. In addition, the BC displayed the highest variations, and in brain regions related to the patient's cognitive and motor impairments; hence, we argue that this measure could be regarded as an indicative for brain plasticity mechanisms.


Vouloutsi, V., Grechuta, K., Verschure, P., (2019). Evaluation of the facial expressions of a humanoid robot Biomimetic and Biohybrid Systems 8th International Conference, Living Machines 2019 (Lecture Notes in Computer Science) , Springer International Publishing (Nara, Japan) 11556, 365-368

Facial expressions are salient social features that crucial in communication, and humans are capable of reading the messages faces convey and the emotions they display. Robots that interact with humans will need to employ similar communication channels for successful interactions. Here, we focus on the readability of the facial expressions of a humanoid robot. We conducted an online survey where participants evaluated emotional stimuli and assessed the robot’s expressions. Results suggest that the robot’s facial expressions are correctly recognised and the appraisal of the robots expressive elements are consistent with the literature.

Keywords: Emotion recognition, Facial expressions, Human-robot interaction


Blancas-Muñoz, M., Vouloutsi, Vasiliki, Zucca, R., Mura, Anna, Verschure, P., (2018). Hints vs distractions in intelligent tutoring systems: Looking for the proper type of help ARXIV Computer Science, (Human-Computer Interaction), 1-4

The kind of help a student receives during a task has been shown to play a significant role in their learning process. We designed an interaction scenario with a robotic tutor, in real-life settings based on an inquiry-based learning task. We aim to explore how learners' performance is affected by the various strategies of a robotic tutor. We explored two kinds of(presumable) help: hints (which were specific to the level or general to the task) or distractions (information not relevant to the task: either a joke or a curious fact). Our results suggest providing hints to the learner and distracting them with curious facts as more effective than distracting them with humour.

Keywords: Computer Science, Human-Computer Interaction


Fischer, Tobias, Puigbò, Jordi-Ysard, Camilleri, Daniel, Nguyen, Phuong D. H., Moulin-Frier, Clément, Lallée, Stéphane, Metta, Giorgio, Prescott, Tony J., Demiris, Yiannis, Verschure, P., (2018). iCub-HRI: A software framework for complex human-robot interaction scenarios on the iCub humanoid robot Frontiers in Robotics and AI , 5, (22), Article 22

Generating complex, human-like behaviour in a humanoid robot like the iCub requires the integration of a wide range of open source components and a scalable cognitive architecture. Hence, we present the iCub-HRI library which provides convenience wrappers for components related to perception (object recognition, agent tracking, speech recognition, touch detection), object manipulation (basic and complex motor actions) and social interaction (speech synthesis, joint attention) exposed as a C++ library with bindings for Java (allowing to use iCub-HRI within Matlab) and Python. In addition to previously integrated components, the library allows for simple extension to new components and rapid prototyping by adapting to changes in interfaces between components. We also provide a set of modules which make use of the library, such as a high-level knowledge acquisition module and an action recognition module. The proposed architecture has been successfully employed for a complex human-robot interaction scenario involving the acquisition of language capabilities, execution of goal-oriented behaviour and expression of a verbal narrative of the robot's experience in the world. Accompanying this paper is a tutorial which allows a subset of this interaction to be reproduced. The architecture is aimed at researchers familiarising themselves with the iCub ecosystem, as well as expert users, and we expect the library to be widely used in the iCub community.

Keywords: Robotics, iCub Humanoid, YARP, Software architecture, C++, Python, Java, Human-robot interaction


Freire, I. T., Moulin-Frier, C., Sanchez-Fibla, M., Arsiwalla, X. D., Verschure, P., (2018). Modeling the formation of social conventions in multi-agent populations ARXIV Computer Science, (Multiagent Systems), 1-30

In order to understand the formation of social conventions we need to know the specific role of control and learning in multi-agent systems. To advance in this direction, we propose, within the framework of the Distributed Adaptive Control (DAC) theory, a novel Control-based Reinforcement Learning architecture (CRL) that can account for the acquisition of social conventions in multi-agent populations that are solving a benchmark social decision-making problem. Our new CRL architecture, as a concrete realization of DAC multi-agent theory, implements a low-level sensorimotor control loop handling the agent's reactive behaviors (pre-wired reflexes), along with a layer based on model-free reinforcement learning that maximizes long-term reward. We apply CRL in a multi-agent game-theoretic task in which coordination must be achieved in order to find an optimal solution. We show that our CRL architecture is able to both find optimal solutions in discrete and continuous time and reproduce human experimental data on standard game-theoretic metrics such as efficiency in acquiring rewards, fairness in reward distribution and stability of convention formation.

Keywords: Computer Science, Multiagent Systems


Puigbò, J. Y., Maffei, G., Herreros, I., Ceresa, M., González Ballester, M. A., Verschure, P. F. M. J., (2018). Cholinergic behavior state-dependent mechanisms of neocortical gain control: A neurocomputational study Molecular Neurobiology 55, (1), 249-257

The embodied mammalian brain evolved to adapt to an only partially known and knowable world. The adaptive labeling of the world is critically dependent on the neocortex which in turn is modulated by a range of subcortical systems such as the thalamus, ventral striatum, and the amygdala. A particular case in point is the learning paradigm of classical conditioning where acquired representations of states of the world such as sounds and visual features are associated with predefined discrete behavioral responses such as eye blinks and freezing. Learning progresses in a very specific order, where the animal first identifies the features of the task that are predictive of a motivational state and then forms the association of the current sensory state with a particular action and shapes this action to the specific contingency. This adaptive feature selection has both attentional and memory components, i.e., a behaviorally relevant state must be detected while its representation must be stabilized to allow its interfacing to output systems. Here, we present a computational model of the neocortical systems that underlie this feature detection process and its state-dependent modulation mediated by the amygdala and its downstream target the nucleus basalis of Meynert. In particular, we analyze the role of different populations of inhibitory interneurons in the regulation of cortical activity and their state-dependent gating of sensory signals. In our model, we show that the neuromodulator acetylcholine (ACh), which is in turn under control of the amygdala, plays a distinct role in the dynamics of each population and their associated gating function serving the detection of novel sensory features not captured in the state of the network, facilitating the adjustment of cortical sensory representations and regulating the switching between modes of attention and learning.

Keywords: Acetylcholine, Inhibitory network, Neocortical circuits, Neuromodulation


Zamora, R., Korff, S., Mi, Q., Barclay, D., Schimunek, L., Zucca, R., Arsiwalla, X. D., Simmons, R. L., Verschure, P., Billiar, T. R., Vodovotz, Y., (2018). A computational analysis of dynamic, multi-organ inflammatory crosstalk induced by endotoxin in mice PLoS Computational Biology 14, (11), e1006582

Bacterial lipopolysaccharide (LPS) induces an acute inflammatory response across multiple organs, primarily via Toll-like receptor 4 (TLR4). We sought to define novel aspects of the complex spatiotemporal dynamics of LPS-induced inflammation using computational modeling, with a special focus on the timing of pathological systemic spillover. An analysis of principal drivers of LPS-induced inflammation in the heart, gut, lung, liver, spleen, and kidney to assess organ-specific dynamics, as well as in the plasma (as an assessment of systemic spillover), was carried out using data on 20 protein-level inflammatory mediators measured over 0-48h in both C57BL/6 and TLR4-null mice. Using a suite of computational techniques, including a time-interval variant of Principal Component Analysis, we confirm key roles for cytokines such as tumor necrosis factor-α and interleukin-17A, define a temporal hierarchy of organ-localized inflammation, and infer the point at which organ-localized inflammation spills over systemically. Thus, by employing a systems biology approach, we obtain a novel perspective on the time- and organ-specific components in the propagation of acute systemic inflammation.


Arsiwalla, Xerxes D., Verschure, Paul, (2018). Measuring the complexity of consciousness Frontiers in Neuroscience 12, (424), Article 424

The grand quest for a scientific understanding of consciousness has given rise to many new theoretical and empirical paradigms for investigating the phenomenology of consciousness as well as clinical disorders associated to it. A major challenge in this field is to formalize computational measures that can reliably quantify global brain states from data. In particular, information-theoretic complexity measures such as integrated information have been proposed as measures of conscious awareness. This suggests a new framework to quantitatively classify states of consciousness. However, it has proven increasingly difficult to apply these complexity measures to realistic brain networks. In part, this is due to high computational costs incurred when implementing these measures on realistically large network dimensions. Nonetheless, complexity measures for quantifying states of consciousness are important for assisting clinical diagnosis and therapy. This article is meant to serve as a lookup table of measures of consciousness, with particular emphasis on clinical applicability. We consider both, principle-based complexity measures as well as empirical measures tested on patients. We address challenges facing these measures with regard to realistic brain networks, and where necessary, suggest possible resolutions. We address challenges facing these measures with regard to realistic brain networks, and where necessary, suggest possible resolutions.

Keywords: Consciousness in the Clinic, Computational neuroscience, Complexity measures, Clinical Neuroscience, Measures of consciousness


Moulin-Frier, C., Fischer, T., Petit, M., Pointeau, G., Puigbo, J., Pattacini, U., Low, S. C., Camilleri, D., Nguyen, P., Hoffmann, M., Chang, H. J., Zambelli, M., Mealier, A., Damianou, A., Metta, G., Prescott, T. J., Demiris, Y., Dominey, P. F., Verschure, P. F. M. J., (2018). DAC-h3: A proactive robot cognitive architecture to acquire and express knowledge about the world and the self IEEE Transactions on Cognitive and Developmental Systems 10, (4), 1005-1022

This paper introduces a cognitive architecture for a humanoid robot to engage in a proactive, mixed-initiative exploration and manipulation of its environment, where the initiative can originate from both the human and the robot. The framework, based on a biologically-grounded theory of the brain and mind, integrates a reactive interaction engine, a number of state-of-the art perceptual and motor learning algorithms, as well as planning abilities and an autobiographical memory. The architecture as a whole drives the robot behavior to solve the symbol grounding problem, acquire language capabilities, execute goal-oriented behavior, and express a verbal narrative of its own experience in the world. We validate our approach in human-robot interaction experiments with the iCub humanoid robot, showing that the proposed cognitive architecture can be applied in real time within a realistic scenario and that it can be used with naive users.

Keywords: Autobiographical Memory., Biology, Cognition, Cognitive Robotics, Computer architecture, Distributed Adaptive Control, Grounding, Human-Robot Interaction, Humanoid robots, Robot sensing systems, Symbol Grounding


Santos-Pata, D., Verschure, P., (2018). Human vicarious trial and error is predictive of spatial navigation performance Frontiers in Behavioral Neuroscience 12, Article 237

When learning new environments, rats often pause at decision points and look back and forth over their possible trajectories as if they were imagining the future outcome of their actions, a behavior termed “Vicarious trial and error” (VTE). As the animal learns the environmental configuration, rats change from deliberative to habitual behavior, and VTE tends to disappear, suggesting a functional relevance in the early stages of learning. Despite the extensive research on spatial navigation, learning and VTE in the rat model, fewer studies have focused on humans. Here, we tested whether head-scanning behaviors that humans typically exhibit during spatial navigation are as predictive of spatial learning as in the rat. Subjects performed a goal-oriented virtual navigation task in a symmetric environment. Spatial learning was assessed through the analysis of trajectories, timings, and head orientations, under habitual and deliberative spatial navigation conditions. As expected, we found that trajectory length and duration decreased with the trial number, implying that subjects learned the spatial configuration of the environment over trials. Interestingly, IdPhi (a standard metric of VTE) also decreased with the trial number, suggesting that humans benefit from the same head-orientation scanning behavior as rats at spatial decision-points. Moreover, IdPhi captured exclusively at the first decision-point of each trial, was correlated with trial trajectory duration and length. Our findings demonstrate that in VTE is a signature of the stage of spatial learning in humans, and can be used to predict performance in navigation tasks with high accuracy.

Keywords: Deliberation, Habitual, Hippocampus, Navigation, Spatial decision-making


Truschzinski, M., Betella, A., Brunnett, G., Verschure, P., (2018). Emotional and cognitive influences in air traffic controller tasks: An investigation using a virtual environment? Applied Ergonomics 69, 1-9

Air traffic controllers are required to perform complex tasks which require attention and high precision. This study investigates how the difficulty of such tasks influences emotional states, cognitive workload and task performance. We use quantitative and qualitative measurements, including the recording of pupil dilation and changes in affect using questionnaires. Participants were required to perform a number of air traffic control tasks using the immersive human accessible Virtual Reality space in the "eXperience Induction Machine". Based on the data collected, we developed and validated a model which integrates personality, workload and affective theories. Our results indicate that the difficulty of an air traffic control task has a direct influence on cognitive workload as well as on the self-reported mood; whereas both mood and workload seem to change independently. In addition, we show that personality, in particular neuroticism, affects both mood and performance of the participants.

Keywords: Air traffic control, Mood, Personality, Virtual reality, Workload


Pacheco, D., Verschure, P. F. M. J., (2018). Long-term spatial clustering in free recall Memory 26, (6), 798–806

We explored the influence of space on the organisation of items in long-term memory. In two experiments, we asked our participants to explore a virtual environment and memorise discrete items presented at specific locations. Memory for those items was later on tested in immediate (T1) and 24 hours delayed (T2) free recall tests, in which subjects were asked to recall as many items as possible in any order. In experiment 2, we further examined the contribution of active and passive navigation in recollection dynamics. Results across experiments revealed a significant tendency for participants to consecutively recall items that were encountered in proximate locations during learning. Moreover, the degree of spatial organisation and the total number of items recalled were positively correlated in the immediate and the delayed tests. Results from experiment 2 indicated that the spatial clustering of items was independent of navigation types. Our results highlight the long-term stability of spatial clustering effects and their correlation with recall performance, complementing previous results collected in immediate or briefly delayed tests.

Keywords: Free recall, Spatial clustering, Spatial memory, Spatial navigation, Virtual reality


Arsiwalla, Xerxes, Signorelli, Camilo M., Puigbo, Jordi-Ysard, Freire, Ismael, Verschure, P., (2018). Are brains computers, emulators or simulators? Biomimetic and Biohybrid Systems 7th International Conference, Living Machines 2018 (Lecture Notes in Computer Science) , Springer International Publishing (Paris, France) 10928, 11-15

There has been intense debate on the question of whether the brain is a computer. If so, that challenge is to show that all cognitive processes can be described by algorithms running on a universal Turing machine. By extension that implies consciousness is a computational process. Both Penrose and Searle have vehemently argued against this view, proposing that consciousness is a fundamentally non-computational process. Even proponents of the brain as a computer metaphor such a Dennett agree that the organizational architecture of the brain is unlike any computing system ever conceived, possibly alluding to non-classical computational processes. The latter class of processes veer away from any program that can be encoded by Church’s lambda calculus. In fact, such a program would have to be based on non-classical logic (either semi-classical or quantum). But quantum logic or machines that might implement them typically are not meant for solving the same type of problems that a classical computer solves (nor are they necessarily faster for any given problem). We will argue that machines implementing non-classical logic might be better suited for simulation rather than computation (a la Turing). It is thus reasonable to pit simulation as an alternative to computation and ask whether the brain, rather than computing, is simulating a model of the world in order to make predictions and guide behavior. If so, this suggests a hardware supporting dynamics more akin to a quantum many-body field theory.


Maier, M., Low, S. C., Ballester, B. R., Bañuelos, N. L., Oller, E. D., Verschure, P., (2018). Depression modulates attentional processing after stroke Converging Clinical and Engineering Research on Neurorehabilitation III (Biosystems and Biorobotics) 4th International Conference on NeuroRehabilitation (ICNR2018) , Springer, Cham (Pisa, Italy) 21, 702-706

Depression is a common sequela after stroke and has severe implications on a patient’s life. Post-stroke depression has been linked to cognitive impairment, but the mechanisms that lead to this deficit are not well understood. We tested 18 chronic stroke patients with depression in a psychophysical task to evaluate their attentional processing under varying cognitive loads. We found that the level of depression had no effect on the unconscious, bottom-up components of attentional processing but did influence the top-down ones. These results support the notion that depression might act like an additional cognitive load, impeding the conscious processes and responses although the information has been unconsciously processed.


Moulin-Frier, C., Puigbò, J. Y., Arsiwalla, X. D., Sanchez-Fibla, M., Verschure, P., (2018). Embodied artificial intelligence through distributed adaptive control: An integrated framework ICDL-EpiRob 2017 7th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics , IEEE (Lisbon, Portugal) , 324-330

In this paper, we argue that the future of Artificial Intelligence research resides in two keywords: integration and embodiment. We support this claim by analyzing the recent advances in the field. Regarding integration, we note that the most impactful recent contributions have been made possible through the integration of recent Machine Learning methods (based in particular on Deep Learning and Recurrent Neural Networks) with more traditional ones (e.g. Monte-Carlo tree search, goal babbling exploration or addressable memory systems). Regarding embodiment, we note that the traditional benchmark tasks (e.g. visual classification or board games) are becoming obsolete as state-of-the-art learning algorithms approach or even surpass human performance in most of them, having recently encouraged the development of first-person 3D game platforms embedding realistic physics. Building on this analysis, we first propose an embodied cognitive architecture integrating heterogeneous subfields of Artificial Intelligence into a unified framework. We demonstrate the utility of our approach by showing how major contributions of the field can be expressed within the proposed framework. We then claim that benchmarking environments need to reproduce ecologically-valid conditions for bootstrapping the acquisition of increasingly complex cognitive skills through the concept of a cognitive arms race between embodied agents.

Keywords: Cognitive Architectures, Embodied Artificial Intelligence, Evolutionary Arms Race, Unified Theories of Cognition


Santos-Pata, D., Escuredo, A., Mathews, Z., Verschure, P., (2018). Insect behavioral evidence of spatial memories during environmental reconfiguration Biomimetic and Biohybrid Systems 7th International Conference, Living Machines 2018 (Lecture Notes in Computer Science) , Springer International Publishing (Paris, France) 10928, 415-427

Insects are great explorers, able to navigate through long-distance trajectories and successfully find their way back. Their navigational routes cross dynamic environments suggesting adaptation to novel configurations. Arthropods and vertebrates share neural organizational principles and it has been shown that rodents modulate their neural spatial representation accordingly with environmental changes. However, it is unclear whether insects reflexively adapt to environmental changes or retain memory traces of previously explored situations. We sought to disambiguate between insect behavior in environmental novel situations and reconfiguration conditions. An immersive mixed-reality multi-sensory setup was built to replicate multi-sensory cues. We have designed an experimental setup where female crickets Gryllus Bimaculatus were trained to move towards paired auditory and visual cues during primarily phonotactic driven behavior. We hypothesized that insects were capable of identifying sensory modifications in known environments. Our results show that, regardless of the animal’s history, novel situation conditions did not compromise the animals performance and navigational directionality towards a new target location. However, in trials where visual and auditory stimuli were spatially decoupled, the animals heading variability towards a previously known position significantly increased. Our findings showed that crickets can behaviorally manifest environmental reconfiguration, suggesting the encoding for spatial representation.

Keywords: Insect, Memory, Navigation, Spatial representation


Freire, Ismael, Puigbo, J., Arsiwalla, Xerxes, Verschure, Paul, (2018). Modeling the opponent’s action using control-based reinforcement learning Biomimetic and Biohybrid Systems 7th International Conference, Living Machines 2018 (Lecture Notes in Computer Science) , Springer International Publishing (Paris, France) 10928, 179-186

In this paper, we propose an alternative to model-free reinforcement learning approaches that recently have demonstrated Theory-of-Mind like behaviors. We propose a game theoretic approach to the problem in which pure RL has demonstrated to perform below the standards of human-human interaction. In this context, we propose alternative learning architectures that complement basic RL models with the ability to predict the other’s actions. This architecture is tested in different scenarios where agents equipped with similar or varying capabilities compete in a social game. Our different interaction scenarios suggest that our model-based approaches are especially effective when competing against models of equivalent complexity, in contrast to our previous results with more basic predictive architectures. We conclude that the evolution of mechanisms that allow for the control of other agents provide different kinds of advantages that can become significant when interacting with different kinds of agents. We argue that no single proposed addition to the learning architecture is sufficient to optimize performance in these scenarios, but a combination of the different mechanisms suggested is required to achieve near-optimal performance in any case.


Arsiwalla, X. D., Pacheco, D., Principe, A., Rocamora, R., Verschure, P., (2018). A temporal estimate of integrated information for intracranial functional connectivity Artificial Neural Networks and Machine Learning (Lecture Notes in Computer Science) 27th International Conference on Artificial Neural Networks (ICANN 2018) , Springer, Cham (Rhodes, Greece) 11140, 403-412

A major challenge in computational and systems neuroscience concerns the quantification of information processing at various scales of the brain’s anatomy. In particular, using human intracranial recordings, the question we ask in this paper is: How can we estimate the informational complexity of the brain given the complex temporal nature of its dynamics? To address this we work with a recent formulation of network integrated information that is based on the Kullback-Leibler divergence between the multivariate distribution on the set of network states versus the corresponding factorized distribution over its parts. In this work, we extend this formulation for temporal networks and then apply it to human brain data obtained from intracranial recordings in epilepsy patients. Our findings show that compared to random re-wirings of the data, functional connectivity networks, constructed from human brain data, score consistently higher in the above measure of integrated information. This work suggests that temporal integrated information may indeed be a good starting point as a future measure of cognitive complexity.

Keywords: Brain networks, Complexity measures, Computational neuroscience, Functional connectivity


Puigbò, J. Y., Arsiwalla, X. D., Verschure, P., (2018). Challenges of machine learning for living machines Biomimetic and Biohybrid Systems 7th International Conference, Living Machines 2018 (Lecture Notes in Computer Science) , Springer International Publishing (Paris, France) 10928, 382-386

Machine Learning algorithms (and in particular Reinforcement Learning (RL)) have proved very successful in recent years. These have managed to achieve super-human performance in many different tasks, from video-games to board-games and complex cognitive tasks such as path-planning or Theory of Mind (ToM) on artificial agents. Nonetheless, this super-human performance is also super-artificial. Despite some metrics are better than what a human can achieve (i.e. cumulative reward), in less common metrics (i.e. time to learning asymptote) the performance is significantly worse. Moreover, the means by which those are achieved fail to extend our understanding of the human or mammal brain. Moreover, most approaches used are based on black-box optimization, making any comparison beyond performance (e.g. at the architectural level) difficult. In this position paper, we review the origins of reinforcement learning and propose its extension with models of learning derived from fear and avoidance behaviors. We argue that avoidance-based mechanisms are required when training on embodied, situated systems to ensure fast and safe convergence and potentially overcome some of the current limitations of the RL paradigm.

Keywords: Avoidance, Neural networks, Reinforcement learning


Verschure, P., (2018). The architecture of mind and brain Living machines: A handbook of research in biomimetics and biohybrid systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 338-345

The components of a Living Machine must be integrated into a functioning whole, which requires a detailed understanding of the architecture of living machines. This chapter starts with a conceptual and historical analysis which from Plato brings us to nineteenth-century neuroscience and early concepts of the layered structure of nervous systems. These concepts were further captured in the cognitive behaviorism of Tolman and came to full fruition in the cognitive revolution of the second half of the twentieth century. Verschure subsequently describes the most relevant proposals of cognitive architectures followed by an overview of the few proposals stemming from modern neuroscience on the architecture of the brain. Subsequently, we will look at contemporary contenders that mediate between cognitive and brain architecture. An important challenge to any model of cognitive architectures is how to benchmark it. Verschure proposes the Unified Theories of Embodied Minds (UTEM) benchmark which advances from Newell’s classic Unified Theories of Cognition benchmark.

Keywords: Architecture, Mind, Brain, Organization, System, Virtualization, Abstraction layers


Pfeifer, R., Verschure, P., (2018). The challenge of autonomous agents: Pitfalls and how to avoid them The Artificial Life Route to Artificial Intelligence (ed. Steels, L., Brooks, R.), Taylor & Francis Group (New York, USA) 9, 237-266

Traditional symbol processing AI has been criticized on many grounds. Well known criticisms concern, among others, brittleness, lack of learning and generalization capacity, lack of fault and noise tolerance, neural implausibility, and the inability to perform in real time. More recently there has been much discussion about situatedness, grounding, and the frame problem.


Verschure, P., (2018). A chronology of Distributed Adaptive Control Living machines: A handbook of research in biomimetics and biohybrid systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 346-360

This chapter presents the Distributed Adaptive Control (DAC) theory of the mind and brain of living machines. DAC provides an explanatory framework for biological brains and an integration framework for synthetic ones. DAC builds on several themes presented in the handbook: it integrates different perspectives on mind and brain, exemplifies the synthetic method in understanding living machines, answers well-defined constraints faced by living machines, and provides a route for the convergent validation of anatomy, physiology, and behavior in our explanation of biological living machines. DAC addresses the fundamental question of how a living machine can obtain, retain, and express valid knowledge of its world. We look at the core components of DAC, specific benchmarks derived from the engagement with the physical and the social world (the H4W and the H5W problems) in foraging and human–robot interaction tasks. Lastly we address how DAC targets the UTEM benchmark and the relation with contemporary developments in AI.

Keywords: Distributed Adaptive Control, Problem of priors, Symbol grounding problem, Convergent validation, Foraging, brain, Architecture, system


Vouloutsi, Vasiliki, Verschure, P., (2018). Emotions and self-regulation Living Machines: A Handbook of Research in Biomimetic and Biohybrid Systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 327-337

This chapter takes the view that emotions of living machines can be seen from the perspective of self-regulation and appraisal. We will first look at the pragmatic needs to endow machines with emotions and subsequently describe some of the historical background of the science of emotions and its different interpretations and links to affective neuroscience. Subsequently, we argue that emotions can be cast in terms of self-regulation where they provide for a descriptor of the state of the homeostatic processes that maintain the relationship between the agent and its internal and external environment. We augment the notion of homeostasis with that of allostasis which signifies a change from stability through a fixed equilibrium to stability through continuous change. The chapter shows how this view can be used to create complex living machines where emotions are anchored in the need fulfillment of the agent, in this case considering both utilitarian and epistemic needs.

Keywords: Emotion, Motivation, Needs, Appraisal, Self-regulation, Homeostasis, Allostasis, Human–robot interaction, James–Lange theory


This chapter discusses basic concepts from control theory and machine learning to facilitate a formal understanding of animal learning and motor control. It first distinguishes between feedback and feed-forward control strategies, and later introduces the classification of machine learning applications into supervised, unsupervised, and reinforcement learning problems. Next, it links these concepts with their counterparts in the domain of the psychology of animal learning, highlighting the analogies between supervised learning and classical conditioning, reinforcement learning and operant conditioning, and between unsupervised and perceptual learning. Additionally, it interprets innate and acquired actions from the standpoint of feedback vs anticipatory and adaptive control. Finally, it argues how this framework of translating knowledge between formal and biological disciplines can serve us to not only structure and advance our understanding of brain function but also enrich engineering solutions at the level of robot learning and control with insights coming from biology.

Keywords: Feedback control, Feed-forward control, Supervised learning, Unsupervised learning, Reinforcement, Learning, Classical conditioning, Operant conditioning, Reflex, Anticipatory reflex


Lepora, Nathan, Verschure, P., Prescott, T. J., (2018). A roadmap for Living Machines research Living machines: A handbook of research in biomimetics and biohybrid systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 26-50

This roadmap identifies current trends in biomimetic and biohybrid systems together with their implications for future research and innovation. Important questions include the scale at which these systems are defined, the types of biological systems addressed, the kind of principles sought, the differences between biologically based and biologically inspired approaches, the role in the understanding of living systems, relevant application domains, common benchmarks, the relation to other fields, and developments on the horizon. We interviewed and collated answers from experts who have been involved a series of events organized by the Convergent Science Network. These answers were then collated into themes of research. Overall, we see a field rapidly expanding in influence and impact. As such, this report will provide information to researchers and scientific policy makers on contemporary biomimetics and its future, together with pointers to further reading on relevant topics within this handbook.

Keywords: Biomimetics, Biohybrid, Bio-inspiration, Biologically inspired, Roadmap, Living machines, policy


Verschure, P., (2018). Capabilities Living machines: A handbook of research in biomimetics and biohybrid systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 211-217

This chapter introduces the “Capabilities” section of the Handbook of Living Machines. Where the previous section considered building blocks, we recognize that components or modules do not automatically make systems. Hence, in the remainder of this handbook, the emphasis is toward the capabilities of living systems and their emulation in artifacts. Capabilities often arise from the integration of multiple components and thus sensitize us to the need to develop a system-level perspective on living machines. Here we summarize and consider the 14 contributions in this section which cover perception, action, cognition, communication, and emotion, and the integration of these through cognitive architectures into systems that can emulate the full gamut of integrated behaviors seen in animals including, potentially, our own capacity for consciousness.

Keywords: Action, Cognition, Cognitive architecture, Communication, Consciousness, Emotion, Perception


Freire, I. T., Arsiwalla, X. D., Puigbò, J. Y., Verschure, P., (2018). Limits of multi-agent predictive models in the formation of social conventions Frontiers in Artificial Intelligence and Applications (ed. Falomir, Z., Gibert, K., Plaza, E.), IOS Press (Amsterdam, The Netherlands) Volume 308: Artificial Intelligence Research and Development, 297-301

A major challenge in cognitive science and AI is to understand how intelligent agents might be able to predict mental states of other agents during complex social interactions. What are the computational principles of such a Theory of Mind (ToM)? In previous work, we have investigated hypotheses of how the human brain might realize a ToM of other agents in a multi-agent social scenario. In particular, we have proposed control-based cognitive architectures to predict the model of other agents in a game-theoretic task (Battle of the Exes). Our multi-layer architecture implements top-down predictions from adaptive to reactive layers of control and bottom-up error feedback from reactive to adaptive layers. We tested cooperative and competitive strategies among different multi-agent models, demonstrating that while pure RL leads to reasonable efficiency and fairness in social interactions, there are other architectures that can perform better in specific circumstances. However, we found that even the best predictive models fall short of human data in terms of stability of social convention formation. In order to explain this gap between humans and predictive AI agents, in this work we propose introducing the notion of trust in the form of mutual agreements between agents that might enhance stability in the formation of conventions such as turn-taking.

Keywords: Cognitive Architectures, Game Theory, Multi-Agent Models, Reinforcement Learning, Theory of Mind


Verschure, P., Prescott, T. J., (2018). A living machines approach to the sciences of mind and brain Living Machines: A Handbook of Research in Biomimetic and Biohybrid Systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 15-25

How do the sciences of mind and brain—neuroscience, psychology, cognitive science, and artificial intelligence (AI)—stand in relation to each other in the 21st century? This chapter proposes that despite our knowledge expanding at ever-accelerating rates, our understanding of the relationship between mind and brain is, in some important sense, becoming less and less. An increasing explanatory gap can only be bridged by a multi-tiered and integrated theoretical framework that recognizes the value of developing explanations at different levels, combining these into cross-level integrated theories, and directly contributing to new technologies that improve the human condition. Development of technologies that instantiate principles gleaned from the study of the mind and brain, or biomimetic technologies, is a key part of the validation process for scientific theories of mind and brain. We call this strategy for the integration of science and engineering a Living Machines approach. Following this path can lead not only to better science, and useful engineering, but also a richer view of human experience and of relationships between science, engineering, and art.

Keywords: Convergent validation, Multi-tiered theories, Paradigms in cognitive science, Philosophy of science, Physical models, Reductionism


Prescott, T. J., Verschure, P. F. M. J., (2018). Living machines: An introduction Living Machines: A Handbook of Research in Biomimetic and Biohybrid Systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 3-14

Biomimetics is the development of novel technologies through the distillation of principles from the study of biological systems. Biohybrid systems are formed by at least one biological component—an already existing living system—and at least one artificial, newly engineered component. The development of either biomimetic or biohybrid systems requires a deep understanding of the operation of living systems, and the two fields are united under the theme of “living machines”—the idea that we can construct artifacts that not only mimic life but share some of the same fundamental principles. This chapter sets out the philosophy and history underlying this Living Machines approach and sets the scene for the remainder of this book.

Keywords: Biohybrids, Biological principles, Biomimetics, History of technology, Living machines, Technology ethics


Vouloutsi, V., Halloy, J., Mura, A., Mangan, M., Lepora, N., Prescott, T. J., Verschure, P., (2018). Preface Biomimetic and Biohybrid Systems (ed. Vouloutsi, Vasiliki, Halloy, José, Mura, Anna, Mangan, Michael, Lepora, Nathan, Prescott, T. J., Verschure, P.), Springer International Publishing (Lausanne, Switzerland) 10928, V-VII

Arsiwalla, X. D., Signorelli, C. M., Puigbo, J. Y., Freire, I. T., Verschure, P., (2018). What is the physics of intelligence? Frontiers in Artificial Intelligence and Applications (ed. Falomir, Z., Gibert, K., Plaza, E.), IOS Press (Amsterdam, The Netherlands) Volume 308: Artificial Intelligence Research and Development, 283-286

In the absence of a first-principles definition, the concept of intelligence is often specified in terms of its phenomenological functions as a capacity or ability to solve problems autonomously. Whenever an agent, biological or artificial, possesses this ability, it is considered intelligent, otherwise not. While this description serves as a useful correlate of intelligence, it is far from a principled explanation that provides a general, yet precise definition along with predictions of mechanisms leading to intelligent behavior. We do not want an explanation to depend on any functionality that itself might be a consequence of intelligence. A possible conceptualization of a function-free approach might be to formulate the concept in terms of dynamical information complexity. This constitute a first step towards a statistical mechanics theory of intelligence. In this paper, we outline the steps towards a physics-based definition of intelligence.

Keywords: Complexity, Information Theory, Physics of Intelligence


Vouloutsi, Vasiliki, Halloy, José, Mura, Anna, Mangan, Michael, Lepora, Nathan, Prescott, T. J., Verschure, P., (2018). Biomimetic and Biohybrid Systems 7th International Conference, Living Machines 2018, Paris, France, July 17–20, 2018, Proceedings , Springer International Publishing (Lausanne, Switzerland) 10928, 1-551

This book constitutes the proceedings of the 7th International Conference on Biomimetic and Biohybrid Systems, Living Machines 2018, held in Paris, France, in July 2018. The 40 full and 18 short papers presented in this volume were carefully reviewed and selected from 60 submissions. The theme of the conference targeted at the intersection of research on novel life-like technologies inspired by the scientific investigation of biological systems, biomimetics, and research that seeks to interface biological and artificial systems to create biohybrid systems.

Keywords: Artificial neural network, Bio-actuators, Bio-robotics, Biohybrid systems, Biomimetics, Bipedal robots, Earthoworm-like robots, Robotics, Decision-making, Tactile sensing, Soft robots, Locomotion, Insects, Sensors, Actuators, Robots, Artificial intelligence, Neural networks, Motion planning, Learning algorithms


Prescott, T. J., Lepora, Nathan, Verschure, P., (2018). Living machines: A handbook of research in biomimetics and biohybrid systems Oxford Scholarship , 1-623

Biomimetics is the development of novel technologies through the distillation of ideas from the study of biological systems. Biohybrids are formed through the combination of at least one biological component—an existing living system—and at least one artificial, newly engineered component. These two fields are united under the theme of Living Machines—the idea that we can construct artifacts that not only mimic life but also build on the same fundamental principles. The research described in this volume seeks to understand and emulate life’s ability to self-organize, metabolize, grow, and reproduce; to match the functions of living tissues and organs such as muscles, skin, eyes, ears, and neural circuits; to replicate cognitive and physical capacities such as perception, attention, locomotion, grasp, emotion, and consciousness; and to assemble all of these elements into integrated systems that can hold a technological mirror to life or that have the capacity to merge with it. We conclude with contributions from philosophers, ethicists, and futurists on the potential impacts of this remarkable research on society and on how we see ourselves.

Keywords: Novel technologies, Biomimetics, Biohybrids, Living systems, Living machines, Biological principles, Technology ethics, Societal impacts


Maffei, Giovanni, Herreros, Ivan, Sanchez-Fibla, Marti, Friston, Karl J., Verschure, Paul F. M. J., (2017). The perceptual shaping of anticipatory actions Proceedings of the Royal Society B , 284, (1869)

Humans display anticipatory motor responses to minimize the adverse effects of predictable perturbations. A widely accepted explanation for this behavior relies on the notion of an inverse model that, learning from motor errors, anticipates corrective responses. Here, we propose and validate the alternative hypothesis that anticipatory control can be realized through a cascade of purely sensory predictions that drive the motor system, reflecting the causal sequence of the perceptual events preceding the error. We compare both hypotheses in a simulated anticipatory postural adjustment task. We observe that adaptation in the sensory domain, but not in the motor one, supports the robust and generalizable anticipatory control characteristic of biological systems. Our proposal unites the neurobiology of the cerebellum with the theory of active inference and provides a concrete implementation of its core tenets with great relevance both to our understanding of biological control systems and, possibly, to their emulation in complex artefacts.

Keywords: Active inference, Cerebellum, Computational model, Motor control, Perceptual learning


Pacheco, D., Sánchez-Fibla, M., Duff, A., Verschure, P. F. M. J., (2017). A spatial-context effect in recognition memory Frontiers in Behavioral Neuroscience 11, Article 143

We designed a novel experiment to investigate the modulation of human recognition memory by environmental context. Human participants were asked to navigate through a four-arm Virtual Reality (VR) maze in order to find and memorize discrete items presented at specific locations in the environment. They were later on tested on their ability to recognize items as previously presented or new. By manipulating the spatial position of half of the studied items during the testing phase of our experiment, we could assess differences in performance related to the congruency of environmental information at encoding and retrieval. Our results revealed that spatial context had a significant effect on the quality of memory. In particular, we found that recognition performance was significantly better in trials in which contextual information was congruent as opposed to those in which it was different. Our results are in line with previous studies that have reported spatial-context effects in recognition memory, further characterizing their magnitude under ecologically valid experimental conditions.

Keywords: Context effects, Recognition memory, Spatial behavior, Spatial memory and navigation, Virtual reality


Hindriks, Rikkert, Schmiedt, Joscha, Arsiwalla, Xerxes D., Peter, Alina, Verschure, Paul F. M. J., Fries, Pascal, Schmid, Michael C., Deco, Gustavo, (2017). Linear distributed source modeling of local field potentials recorded with intra-cortical electrode arrays PLoS ONE 12, (12), e0187490

Planar intra-cortical electrode (Utah) arrays provide a unique window into the spatial organization of cortical activity. Reconstruction of the current source density (CSD) underlying such recordings, however, requires “inverting” Poisson’s equation. For inter-laminar recordings, this is commonly done by the CSD method, which consists in taking the second-order spatial derivative of the recorded local field potentials (LFPs). Although the CSD method has been tremendously successful in mapping the current generators underlying inter-laminar LFPs, its application to planar recordings is more challenging. While for inter-laminar recordings the CSD method seems reasonably robust against violations of its assumptions, is it unclear as to what extent this holds for planar recordings. One of the objectives of this study is to characterize the conditions under which the CSD method can be successfully applied to Utah array data. Using forward modeling, we find that for spatially coherent CSDs, the CSD method yields inaccurate reconstructions due to volume-conducted contamination from currents in deeper cortical layers. An alternative approach is to “invert” a constructed forward model. The advantage of this approach is that any a priori knowledge about the geometrical and electrical properties of the tissue can be taken into account. Although several inverse methods have been proposed for LFP data, the applicability of existing electroencephalographic (EEG) and magnetoencephalographic (MEG) inverse methods to LFP data is largely unexplored. Another objective of our study therefore, is to assess the applicability of the most commonly used EEG/MEG inverse methods to Utah array data. Our main conclusion is that these inverse methods provide more accurate CSD reconstructions than the CSD method. We illustrate the inverse methods using event-related potentials recorded from primary visual cortex of a macaque monkey during a motion discrimination task.


Santos-Pata, D., Zucca, R., Low, S. C., Verschure, P. F. M. J., (2017). Size matters: How scaling affects the interaction between grid and border cells Frontiers in Computational Neuroscience , 11, Article 65

Many hippocampal cell types are characterized by a progressive increase in scale along the dorsal-to-ventral axis, such as in the cases of head-direction, grid and place cells. Also located in the medial entorhinal cortex (MEC), border cells would be expected to benefit from such scale modulations. However, this phenomenon has not been experimentally observed. Grid cells in the MEC of mammals integrate velocity related signals to map the environment with characteristic hexagonal tessellation patterns. Due to the noisy nature of these input signals, path integration processes tend to accumulate errors as animals explore the environment, leading to a loss of grid-like activity. It has been suggested that border-to-grid cells' associations minimize the accumulated grid cells' error when rodents explore enclosures. Thus, the border-grid interaction for error minimization is a suitable scenario to study the effects of border cell scaling within the context of spatial representation. In this study, we computationally address the question of (i) border cells' scale from the perspective of their role in maintaining the regularity of grid cells' firing fields, as well as (ii) what are the underlying mechanisms of grid-border associations relative to the scales of both grid and border cells. Our results suggest that for optimal contribution to grid cells' error minimization, border cells should express smaller firing fields relative to those of the associated grid cells, which is consistent with the hypothesis of border cells functioning as spatial anchoring signals.

Keywords: Border cells, Error minimization, Grid cells, Navigation, Path integration


Moulin-Frier, C., Puigbò, J.-Y., Arsiwalla, Xerxes D., Martì Sanchez-Fibla, M., Verschure, Paul F. M. J., (2017). Embodied artificial intelligence through distributed adaptive control: An integrated framework 7th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-Epirob 2017) , IEEE (Lisbon, Portugal) , 1-8

In this paper, we argue that the future of Artificial Intelligence research resides in two keywords: integration and embodiment. We support this claim by analyzing the recent advances of the field. Regarding integration, we note that the most impactful recent contributions have been made possible through the integration of recent Machine Learning methods (based in particular on Deep Learning and Recurrent Neural Networks) with more traditional ones (e.g. Monte-Carlo tree search, goal babbling exploration or addressable memory systems). Regarding embodiment, we note that the traditional benchmark tasks (e.g. visual classification or board games) are becoming obsolete as state-of-the-art learning algorithms approach or even surpass human performance in most of them, having recently encouraged the development of first-person 3D game platforms embedding realistic physics. Building upon this analysis, we first propose an embodied cognitive architecture integrating heterogenous sub-fields of Artificial Intelligence into a unified framework. We demonstrate the utility of our approach by showing how major contributions of the field can be expressed within the proposed framework. We then claim that benchmarking environments need to reproduce ecologically-valid conditions for bootstrapping the acquisition of increasingly complex cognitive skills through the concept of a cognitive arms race between embodied agents.

Keywords: Cognitive Architectures, Embodied Artificial Intelligence, Evolutionary Arms Race, Unified Theories of Cognition


Puigbò, Jordi-Ysard, Gonzalez-Ballester, Miguel Ángel, Verschure, Paul , (2017). Behavior-state dependent modulation of perception based on a model of conditioning Biomimetic and Biohybrid Systems 6th International Conference, Living Machines 2017 (Lecture Notes in Computer Science) , Springer International Publishing (Standfor, USA) 10384, 387-393

The embodied mammalian brain evolved to adapt to an only partially known and knowable world. The adaptive labeling of the world is critically dependent on the neocortex which in turn is modulated by a range of subcortical systems such as the thalamus, ventral striatum and the amygdala. A particular case in point is the learning paradigm of classical conditioning where acquired representations of states of the world such as sounds and visual features are associated with predefined discrete behavioral responses such as eye blinks and freezing. Learning progresses in a very specific order, where the animal first identifies the features of the task that are predictive of a motivational state and then forms the association of the current sensory state with a particular action and shapes this action to the specific contingency. This adaptive feature selection has both attentional and memory components, i.e. a behaviorally relevant state must be detected while its representation must be stabilized to allow its interfacing to output systems. Here we present a computational model of the neocortical systems that underlie this feature detection process and its state dependent modulation mediated by the amygdala and its downstream target, the nucleus basalis of Meynert. Specifically, we analyze how amygdala driven cholinergic modulation these mechanisms through computational modeling and present a framework for rapid learning of behaviorally relevant perceptual representations.


Marco, S., Gutiérrez-Gálvez, A., Lansner, A., Martinez, D., Rospars, J. P., Beccherelli, R., Perera, A., Pearce, T. C., Verschure, P. F. M. J., Persaud, K., (2014). A biomimetic approach to machine olfaction, featuring a very large-scale chemical sensor array and embedded neuro-bio-inspired computation Microsystem Technologies , 20, (4-5), 729-742

Biological olfaction outperforms chemical instrumentation in specificity, response time, detection limit, coding capacity, time stability, robustness, size, power consumption, and portability. This biological function provides outstanding performance due, in a large extent, to the unique architecture of the olfactory pathway, which combines a high degree of redundancy and efficient combinatorial coding, with unmatched chemical information processing mechanisms. The last decade has seen important advances in the understanding of the computational primitives underlying the functioning of the olfactory system. The EU-funded Project NEUROCHEM (Bio-ICT-FET- 216916) developed novel computing paradigms and biologically motivated artefacts for chemical sensing, taking its inspiration from the biological olfactory pathway. To demonstrate this approach, a biomimetic demonstrator has been built that features a very large-scale sensor array (65,536 elements) using conducting polymer technology which mimics the olfactory receptor neuron layer. It implements derived computational neuroscience algorithms in an embedded system that interfaces the chemical sensors and processes their signals in real-time. This embedded system integrates abstracted computational models of the main anatomic building blocks in the olfactory pathway: the olfactory bulb, and olfactory cortex in vertebrates (respectively, antennal lobe and mushroom bodies in the insect). For implementation in the embedded processor, an abstraction phase has been carried out in which their processing capabilities are captured by algorithmic solutions implemented in software. Finally, the algorithmic models are tested in mixed chemical plumes with an odour robot having navigation capabilities.


Ziyatdinov, Andrey, Calvo, Jose Maria Blanco, Lechon, Miguel, Bermudez i Badia, Sergi, Verschure, Paul F. M. J., Marco, Santiago, Perera, Alexandre, (2011). Odour mapping under strong backgrounds with a metal oxide sensor array Olfaction and Electronic Nose: Proceedings of the 14th International Symposium on Olfaction and Electronic Nose AIP Conference Proceedings (ed. Perena Gouma, SUNY Stony Brook), AIP (New York City, USA) 1362, (1), 232-233

This work describes the data from navigation experiments with the mobile robot, equipped with the sensor array of three MOX gas sensors. Performed four series of measurements aim to explore the capabilities of sensor array to build the odour map with one or two odour sources in the wind tunnel space. It was demonstrated that the method based on Independent Component Analysis (ICA) is able to discriminate two odour sources, that in future can be used in the surge-and-cast robot navigation algorithm.

Keywords: Mobile robots, Data acquisition, MIS devices, Chemioception