Synthetic, Perceptive, Emotive and Cognitive Systems (SPECS)


About

SPECS uses synthetic methods to study and synthesize the neuronal, psychological and behavioural principles underlying perception, emotion, and cognition.

SPECS activities are organized around three complementary dimensions:

• Theory of mind and brain
• Biomimetic real-world artefacts
• Brain repair and quality of life technologies

SPECS is also very much involved in the development of scientific co-operation in the field of Biomimetics and Neurotechnology, as well as in Educational and Outreach activities.

Cognitive Systems Laboratory

The Cognitive Systems Laboratory is a multidisciplinary environment that supports research in the following areas:

  • Distributed Adaptive Control
  • Multi-robot exploration and coordination
  • Classical conditioning, operant conditioning and learning models based on the Distributed Adaptive Control framework, which has become a standard in the field of artificial intelligence and behavior-based robotics (McFarland and Bosser, 1993; Hendriks-Jansen, 1996; Arkin, 1998; Pfeifer and Scheier, 1999; Clancey 1996; Cordeschi, 2002).

Robotic Systems Laboratory

The Robotic Systems Laboratory is a multidisciplinary environment that supports research in the following areas:

  • Classical conditioning, operant conditioning and learning models based on the Distributed Adaptive Control framework, which has become a standard in the field of artificial intelligence and behavior-based robotics
  • Multi-robot exploration and coordination
  • Navigation in human and animal behavior
  • Implementation in robots of brain models of the hippocampus, cerebellum, thalamus/cortex
  • Rule learning VR robots/avatars
  • Fast and reliable insect-based visual navigation models for flying vehicles
    Investigation of the neuronal substrates of chemical sensing and their application to odor discrimination and localization

Hybrid Systems laboratory (HLB)

The HLB is primarily involved in the development, implementation, and analysis of machine-brain-machine interfaces.
The interdisciplinary nature of the study of hybrid systems lies at the intersection of different research areas, namely:

  • computational neuroscience
  • electronics
  • robotics
  • artificial intelligence
  • neuromorphic engineering

The HLB was involved in the ReNaChip FP7 project, whose overarching goal is to build s neuroprosthetic neuromorphic chip recovering a learning function lost in the aged cerebellum.

Digital Heritage

By using advanced digital humanities technologies, and making it accessible online, we can conserve, develop and preserve the memory of Europe’s cultural heritage, and in particular the Holocaust, for future generations.

Existing memorial sites or museums offer a traditional historiographical approach. We propose to use virtual and augmented reality techniques to reconstruct sites of WW-II crimes and their interrelated structures. SPECS’s approach combines virtual and augmented reality with integrated databases of graphical reconstructions and historical sources to allow us to actively explore and try to comprehend the incomprehensible: the massive scale of the crimes Nazi Germany perpetrated on the world and the depth of the destruction and suffering it caused.

The SPECS research group has been pioneering this approach over the last 15 years and grounded it in its fundamental research in psychology and neuroscience. In collaboration with the Bergen-Belsen memorial site and Prof. Habbo Knoch, this paradigm has been elaborated to conserve and present the history of the Bergen Belsen concentration camp.

Educational Robotics

Technology evolves and advances faster than ever in all aspects of our society. Thus, it is important that the next generations of students learn as much as possible about emerging technology and stay competitive.

SPECS contributes to the education of the next generations by combining platforms for training and outreach activities, facilitating multidisciplinary education and innovation by sharing the value of convergent science, excellence, and societal impact. We have developed Educational Robotics programs for students of the primary and secondary school, as well as courses to train teachers and young adults.

Interaction Technology

There is a growing interest in understanding creativity from a more neuroscientific point of view, so to say, to disclose the neural basis of creativity we will need great insights on how the brain elaborates the process of human thought.

Our approach to understanding the process of creativity is to use Art & Technology to create high impact, sophisticated man-machine interaction tools.

  • Narrative in interactive mixed reality environment
  • Multimedia installations: affect-based self-generated media content

Mixed-reality lab

The Mixed-reality lab serves a threefold research agenda:

  • Understand human behavior in a mixed-reality context
  • Build mixed-reality applications based on neurobiological understanding and methodologies see iqr and Brainx3
  • Test neurobiological models by deploying them in control of mixed-reality systems

Neuro-Rehabilitation

Over the past 15 years, SPECS has been developing science-based technology tools to drive perceptual, cognitive, affective and motor systems of the brain to facilitate functional recovery after damage. By means of novel interaction paradigms such as Virtual Reality or music therapy, and based on the Distributed Adaptive Control theory of mind and brain DAC developed by Paul Verschure, SPECS studies the brain and the mechanisms underlying loss of function and its rehabilitation and recovery after stroke, and other brain diseases (see Verschure Conf Proc IEEE Eng Med Biol Soc. 2011, Mónica S. Cameirão et al. Restor Neurol Neurosci 2011 and Stroke 2012 )

Psychophysiology lab

The Psychophysiology lab studies how humans react to various uni- and multisensory signals – visual, auditory and tactile stimuli. We assess human responses at different levels using subjective ratings, behavioral data, physiological and brain wave recordings. This data helps us to understand human perception and cognition mechanisms, with particular stress on the novel methods for diagnosis and treatment of various brain disorders (chronic pain, migraine, autism, depression, Alzheimer’s disease).

  • affective chronometry (such parameters as the rise time to peak and the recovery time of the emotional waveform)
  • multisensory perception (sound, vision, touch)
  • multisensory interactions for emotional stimuli (custom sound and video databases are created)
  • sonification of EEG signals
  • neurofeedback using mixed reality environments processing of eye-gaze in autistic children


Staff

Paul Verschure | Group Leader / ICREA Research Professor
Anna Mura | Senior Researcher
Xerxes Arsiwalla | Postdoctoral Researcher
Daniel Pacheco | Postdoctoral Researcher
Belén Rubio Ballester | Postdoctoral Researcher
Vicky Vouloutsi | Postdoctoral Researcher
Riccardo Zucca | Postdoctoral Researcher
Pedro Omedas Morera | Senior Technician
Maria Blancas Muñoz | PhD Student
Klaudia Grechuta | PhD Student
Sock Low | PhD Student
Martina Maier | PhD Student
Jordi-Ysard Puigbò Llobet | PhD Student
Dina Urikh | PhD Student
Ismael Tito Freire González | Research Assistant
Héctor López Carral | Research Assistant
Giovanni Maffei | Research Assistant
Kacper Antoni Metzler | Research Assistant
Diogo Pata Santos | Research Assistant
Santiago Brandi | Laboratory Technician
Alejandro Escuredo Chimeno | Laboratory Technician
Antoni Gurguí Valverde | Laboratory Technician
Enrique Martínez Bueno | Laboratory Technician
Sytse Baldwin Wierenga | Laboratory Technician
Guillem Arias Bedmar | Masters Student
Sacha Brouté | Masters Student
Hélène Marie Coupé | Masters Student
Violeta De Azpiazu Castillo | Masters Student
Javier De la Torre Costa | Masters Student
Adrián Fernández Amil | Masters Student
Lucas Lorenzo Peña | Masters Student
Elise Olivier | Masters Student
Svenja Pieritz | Masters Student
Christopher Rauser | Masters Student
Jenny Robayo Gómez | Masters Student
Anna Ruiz Ferrer | Masters Student
Jorge Verdú Añón | Masters Student
Yan Rong Zhang | Masters Student

News/Jobs

Nobel Laureate is special guest speaker at IBEC event
05/09/17

IBEC welcomed Prof. Edvard Moser, Nobel Prize in Medicine or Physiology 2014, as the keynote speaker in a special event to mark the move of ICREA professor and ERC grantee Prof. Paul Verschure to the institute.


New research group boosts neuroengineering focus at IBEC
03/07/17

The Institute for Bioengineering of Catalonia (IBEC) gains a world-renowned neuroscientist and psychologist with the move this week of ICREA professor Paul Verschure and his Synthetic Perceptive, Emotive and Cognitive Systems group (SPECS) from the Universitat Pompeu Fabra to the institute.


Projects

EU-funded projects
CDAC  The role of consciousness in adaptive behavior: A combined empirical, computational and robot-based approach (2014-2019) ERC Advanced Grant Paul Verschure
iC-ACCESS  Accessing Campscapes: Inclusive Strategies for Using European Conflicted Heritage (2016-2019) HERA Joint Research Programme Uses of the Past, REFLECTIVE-1-2014 Paul Verschure
socSMCs  Socialising Sensori-Motor Contingencies (2015-2018) Future Emerging Technologies (FET), H2020 Paul Verschure
WYSIWYD  What You Say Is What You Did (2014-2017) FP7-ICT-2013-10, grant agreement n° 612139 Paul Verschure
EASEL  Expressive Agents for Symbiotic Education and Learning (2013-2017) FP7-ICT-2013-10, grant agreement n° 611971 Paul Verschure
CSNII  Convergent Science Network of Biomimetics and Neurotechnology (2013-2016) FP7-ICT-601167 Paul Verschure
National projects
DAC-CHM Distributed Adaptive Control of Consciousness in Humans and Machines Paul Verschure
INSOCO (2015-2018) Paul Verschure
SANaR Smart Autonomous Neuro-Rehabilitation System MINECO, Retos Investigación 2013 Paul Verschure
TECNIO (2016-2019) Generalitat of Catalonia Paul Verschure

Publications


Moulin-Frier, C., Fischer, T., Petit, M., Pointeau, G., Puigbo, J., Pattacini, U., Low, S. C., Camilleri, D., Nguyen, P., Hoffmann, M., Chang, H. J., Zambelli, M., Mealier, A., Damianou, A., Metta, G., Prescott, T. J., Demiris, Y., Dominey, P. F., Verschure, P. F. M. J., (2018). DAC-h3: A proactive robot cognitive architecture to acquire and express knowledge about the world and the self IEEE Transactions on Cognitive and Developmental Systems in press

This paper introduces a cognitive architecture for a humanoid robot to engage in a proactive, mixed-initiative exploration and manipulation of its environment, where the initiative can originate from both the human and the robot. The framework, based on a biologically-grounded theory of the brain and mind, integrates a reactive interaction engine, a number of state-of-the art perceptual and motor learning algorithms, as well as planning abilities and an autobiographical memory. The architecture as a whole drives the robot behavior to solve the symbol grounding problem, acquire language capabilities, execute goal-oriented behavior, and express a verbal narrative of its own experience in the world. We validate our approach in human-robot interaction experiments with the iCub humanoid robot, showing that the proposed cognitive architecture can be applied in real time within a realistic scenario and that it can be used with naive users.

Keywords: Autobiographical Memory., Biology, Cognition, Cognitive Robotics, Computer architecture, Distributed Adaptive Control, Grounding, Human-Robot Interaction, Humanoid robots, Robot sensing systems, Symbol Grounding


Puigbò, J. Y., Maffei, G., Herreros, I., Ceresa, M., González Ballester, M. A., Verschure, P. F. M. J., (2018). Cholinergic behavior state-dependent mechanisms of neocortical gain control: A neurocomputational study Molecular Neurobiology 55, (1), 249-257

The embodied mammalian brain evolved to adapt to an only partially known and knowable world. The adaptive labeling of the world is critically dependent on the neocortex which in turn is modulated by a range of subcortical systems such as the thalamus, ventral striatum, and the amygdala. A particular case in point is the learning paradigm of classical conditioning where acquired representations of states of the world such as sounds and visual features are associated with predefined discrete behavioral responses such as eye blinks and freezing. Learning progresses in a very specific order, where the animal first identifies the features of the task that are predictive of a motivational state and then forms the association of the current sensory state with a particular action and shapes this action to the specific contingency. This adaptive feature selection has both attentional and memory components, i.e., a behaviorally relevant state must be detected while its representation must be stabilized to allow its interfacing to output systems. Here, we present a computational model of the neocortical systems that underlie this feature detection process and its state-dependent modulation mediated by the amygdala and its downstream target the nucleus basalis of Meynert. In particular, we analyze the role of different populations of inhibitory interneurons in the regulation of cortical activity and their state-dependent gating of sensory signals. In our model, we show that the neuromodulator acetylcholine (ACh), which is in turn under control of the amygdala, plays a distinct role in the dynamics of each population and their associated gating function serving the detection of novel sensory features not captured in the state of the network, facilitating the adjustment of cortical sensory representations and regulating the switching between modes of attention and learning.

Keywords: Acetylcholine, Inhibitory network, Neocortical circuits, Neuromodulation


Pacheco, D., Verschure, P. F. M. J., (2018). Long-term spatial clustering in free recall Memory Article in press

We explored the influence of space on the organisation of items in long-term memory. In two experiments, we asked our participants to explore a virtual environment and memorise discrete items presented at specific locations. Memory for those items was later on tested in immediate (T1) and 24 hours delayed (T2) free recall tests, in which subjects were asked to recall as many items as possible in any order. In experiment 2, we further examined the contribution of active and passive navigation in recollection dynamics. Results across experiments revealed a significant tendency for participants to consecutively recall items that were encountered in proximate locations during learning. Moreover, the degree of spatial organisation and the total number of items recalled were positively correlated in the immediate and the delayed tests. Results from experiment 2 indicated that the spatial clustering of items was independent of navigation types. Our results highlight the long-term stability of spatial clustering effects and their correlation with recall performance, complementing previous results collected in immediate or briefly delayed tests.

Keywords: Free recall, Spatial clustering, Spatial memory, Spatial navigation, Virtual reality


Ballester, Rubio Belén, Nirme, Jens, Camacho, Irene, Duarte, Esther, Rodríguez, Susana, Cuxart, Ampar, Duff, Armin, Verschure, F. M. J. Paul, (2017). Domiciliary VR-based therapy for functional recovery and cortical reorganization: Randomized controlled trial in participants at the chronic stage post stroke JMIR Serious Games 5, (3), e15

Background: Most stroke survivors continue to experience motor impairments even after hospital discharge. Virtual reality-based techniques have shown potential for rehabilitative training of these motor impairments. Here we assess the impact of at-home VR-based motor training on functional motor recovery, corticospinal excitability and cortical reorganization. Objective: The aim of this study was to identify the effects of home-based VR-based motor rehabilitation on (1) cortical reorganization, (2) corticospinal tract, and (3) functional recovery after stroke in comparison to home-based occupational therapy. Methods: We conducted a parallel-group, controlled trial to compare the effectiveness of domiciliary VR-based therapy with occupational therapy in inducing motor recovery of the upper extremities. A total of 35 participants with chronic stroke underwent 3 weeks of home-based treatment. A group of subjects was trained using a VR-based system for motor rehabilitation, while the control group followed a conventional therapy. Motor function was evaluated at baseline, after the intervention, and at 12-weeks follow-up. In a subgroup of subjects, we used Navigated Brain Stimulation (NBS) procedures to measure the effect of the interventions on corticospinal excitability and cortical reorganization. Results: Results from the system?s recordings and clinical evaluation showed significantly greater functional recovery for the experimental group when compared with the control group (1.53, SD 2.4 in Chedoke Arm and Hand Activity Inventory). However, functional improvements did not reach clinical significance. After the therapy, physiological measures obtained from a subgroup of subjects revealed an increased corticospinal excitability for distal muscles driven by the pathological hemisphere, that is, abductor pollicis brevis. We also observed a displacement of the centroid of the cortical map for each tested muscle in the damaged hemisphere, which strongly correlated with improvements in clinical scales. Conclusions: These findings suggest that, in chronic stages, remote delivery of customized VR-based motor training promotes functional gains that are accompanied by neuroplastic changes. Trial Registration: International Standard Randomized Controlled Trial Number NCT02699398 (Archived by ClinicalTrials.gov at https://clinicaltrials.gov/ct2/show/NCT02699398?term=NCT02699398&rank=1)

Keywords: Stroke, Movement disorder, Recovery of function, neuroplasticity, Transcranial magnetic stimulation, Physical therapy, Hemiparesis, Computer applications software


Maffei, Giovanni, Herreros, Ivan, Sanchez-Fibla, Marti, Friston, Karl J., Verschure, Paul F. M. J., (2017). The perceptual shaping of anticipatory actions Proceedings of the Royal Society B 284, (1869)

Humans display anticipatory motor responses to minimize the adverse effects of predictable perturbations. A widely accepted explanation for this behavior relies on the notion of an inverse model that, learning from motor errors, anticipates corrective responses. Here, we propose and validate the alternative hypothesis that anticipatory control can be realized through a cascade of purely sensory predictions that drive the motor system, reflecting the causal sequence of the perceptual events preceding the error. We compare both hypotheses in a simulated anticipatory postural adjustment task. We observe that adaptation in the sensory domain, but not in the motor one, supports the robust and generalizable anticipatory control characteristic of biological systems. Our proposal unites the neurobiology of the cerebellum with the theory of active inference and provides a concrete implementation of its core tenets with great relevance both to our understanding of biological control systems and, possibly, to their emulation in complex artefacts.

Keywords: Active inference, Cerebellum, Computational model, Motor control, Perceptual learning


Pacheco, D., Sánchez-Fibla, M., Duff, A., Verschure, P. F. M. J., (2017). A spatial-context effect in recognition memory Frontiers in Behavioral Neuroscience 11, Article 143

We designed a novel experiment to investigate the modulation of human recognition memory by environmental context. Human participants were asked to navigate through a four-arm Virtual Reality (VR) maze in order to find and memorize discrete items presented at specific locations in the environment. They were later on tested on their ability to recognize items as previously presented or new. By manipulating the spatial position of half of the studied items during the testing phase of our experiment, we could assess differences in performance related to the congruency of environmental information at encoding and retrieval. Our results revealed that spatial context had a significant effect on the quality of memory. In particular, we found that recognition performance was significantly better in trials in which contextual information was congruent as opposed to those in which it was different. Our results are in line with previous studies that have reported spatial-context effects in recognition memory, further characterizing their magnitude under ecologically valid experimental conditions.

Keywords: Context effects, Recognition memory, Spatial behavior, Spatial memory and navigation, Virtual reality


Hindriks, Rikkert, Schmiedt, Joscha, Arsiwalla, Xerxes D., Peter, Alina, Verschure, Paul F. M. J., Fries, Pascal, Schmid, Michael C., Deco, Gustavo, (2017). Linear distributed source modeling of local field potentials recorded with intra-cortical electrode arrays PLoS ONE 12, (12), e0187490

Planar intra-cortical electrode (Utah) arrays provide a unique window into the spatial organization of cortical activity. Reconstruction of the current source density (CSD) underlying such recordings, however, requires “inverting” Poisson’s equation. For inter-laminar recordings, this is commonly done by the CSD method, which consists in taking the second-order spatial derivative of the recorded local field potentials (LFPs). Although the CSD method has been tremendously successful in mapping the current generators underlying inter-laminar LFPs, its application to planar recordings is more challenging. While for inter-laminar recordings the CSD method seems reasonably robust against violations of its assumptions, is it unclear as to what extent this holds for planar recordings. One of the objectives of this study is to characterize the conditions under which the CSD method can be successfully applied to Utah array data. Using forward modeling, we find that for spatially coherent CSDs, the CSD method yields inaccurate reconstructions due to volume-conducted contamination from currents in deeper cortical layers. An alternative approach is to “invert” a constructed forward model. The advantage of this approach is that any a priori knowledge about the geometrical and electrical properties of the tissue can be taken into account. Although several inverse methods have been proposed for LFP data, the applicability of existing electroencephalographic (EEG) and magnetoencephalographic (MEG) inverse methods to LFP data is largely unexplored. Another objective of our study therefore, is to assess the applicability of the most commonly used EEG/MEG inverse methods to Utah array data. Our main conclusion is that these inverse methods provide more accurate CSD reconstructions than the CSD method. We illustrate the inverse methods using event-related potentials recorded from primary visual cortex of a macaque monkey during a motion discrimination task.


Santos-Pata, D., Zucca, R., Low, S. C., Verschure, P. F. M. J., (2017). Size matters: How scaling affects the interaction between grid and border cells Frontiers in Computational Neuroscience 11, Article 65

Many hippocampal cell types are characterized by a progressive increase in scale along the dorsal-to-ventral axis, such as in the cases of head-direction, grid and place cells. Also located in the medial entorhinal cortex (MEC), border cells would be expected to benefit from such scale modulations. However, this phenomenon has not been experimentally observed. Grid cells in the MEC of mammals integrate velocity related signals to map the environment with characteristic hexagonal tessellation patterns. Due to the noisy nature of these input signals, path integration processes tend to accumulate errors as animals explore the environment, leading to a loss of grid-like activity. It has been suggested that border-to-grid cells' associations minimize the accumulated grid cells' error when rodents explore enclosures. Thus, the border-grid interaction for error minimization is a suitable scenario to study the effects of border cell scaling within the context of spatial representation. In this study, we computationally address the question of (i) border cells' scale from the perspective of their role in maintaining the regularity of grid cells' firing fields, as well as (ii) what are the underlying mechanisms of grid-border associations relative to the scales of both grid and border cells. Our results suggest that for optimal contribution to grid cells' error minimization, border cells should express smaller firing fields relative to those of the associated grid cells, which is consistent with the hypothesis of border cells functioning as spatial anchoring signals.

Keywords: Border cells, Error minimization, Grid cells, Navigation, Path integration


Moulin-Frier, C., Puigbò, J.-Y., Arsiwalla, Xerxes D., Martì Sanchez-Fibla, M., Verschure, Paul F. M. J., (2017). Embodied artificial intelligence through distributed adaptive control: An integrated framework 7th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-Epirob 2017) , IEEE (Lisbon, Portugal) , 1-8

In this paper, we argue that the future of Artificial Intelligence research resides in two keywords: integration and embodiment. We support this claim by analyzing the recent advances of the field. Regarding integration, we note that the most impactful recent contributions have been made possible through the integration of recent Machine Learning methods (based in particular on Deep Learning and Recurrent Neural Networks) with more traditional ones (e.g. Monte-Carlo tree search, goal babbling exploration or addressable memory systems). Regarding embodiment, we note that the traditional benchmark tasks (e.g. visual classification or board games) are becoming obsolete as state-of-the-art learning algorithms approach or even surpass human performance in most of them, having recently encouraged the development of first-person 3D game platforms embedding realistic physics. Building upon this analysis, we first propose an embodied cognitive architecture integrating heterogenous sub-fields of Artificial Intelligence into a unified framework. We demonstrate the utility of our approach by showing how major contributions of the field can be expressed within the proposed framework. We then claim that benchmarking environments need to reproduce ecologically-valid conditions for bootstrapping the acquisition of increasingly complex cognitive skills through the concept of a cognitive arms race between embodied agents.

Keywords: Cognitive Architectures, Embodied Artificial Intelligence, Evolutionary Arms Race, Unified Theories of Cognition


Puigbò, Jordi-Ysard, Gonzalez-Ballester, Miguel Ángel, Verschure, Paul F. M. J., (2017). Behavior-state dependent modulation of perception based on a model of conditioning Biomimetic and Biohybrid Systems: Living Machines 2017 (Lecture Notes in Computer Science) (ed. Mangan, M., Cutkosky, M., Mura, A., Verschure, P., Prescott, T., Lepora, N.), Springer, Cham 10384, 387-393

The embodied mammalian brain evolved to adapt to an only partially known and knowable world. The adaptive labeling of the world is critically dependent on the neocortex which in turn is modulated by a range of subcortical systems such as the thalamus, ventral striatum and the amygdala. A particular case in point is the learning paradigm of classical conditioning where acquired representations of states of the world such as sounds and visual features are associated with predefined discrete behavioral responses such as eye blinks and freezing. Learning progresses in a very specific order, where the animal first identifies the features of the task that are predictive of a motivational state and then forms the association of the current sensory state with a particular action and shapes this action to the specific contingency. This adaptive feature selection has both attentional and memory components, i.e. a behaviorally relevant state must be detected while its representation must be stabilized to allow its interfacing to output systems. Here we present a computational model of the neocortical systems that underlie this feature detection process and its state dependent modulation mediated by the amygdala and its downstream target, the nucleus basalis of Meynert. Specifically, we analyze how amygdala driven cholinergic modulation these mechanisms through computational modeling and present a framework for rapid learning of behaviorally relevant perceptual representations.



(See full publication list in ORCID)

Equipment

  • EXperience Induction Machine (XIM), an immersive room equipped with a number of sensors and effectors that have been constructed to conduct experiments in mixed-reality.
  • Robotics lab
  • Codi-Bot, the musical robot that teaches you how to program
  • iqr: simulator for large-scale neural systems
  • Collective machine cognition: Autonomous dynamic mapping and planning using a hybrid team of aerial and ground-based robots
  • Humanoid robots: iCub
  • Quality of Life Technologies

Collaborations

In 2014, SPECS created the spin-off company “Eodyne“.