The Robotics Institute

RI | Seminar | April 1

Robotics Institute Seminar, April 1
Time and Place | Seminar Abstract | Speaker Biography | Speaker Appointments

Multimodal Signal Processing with Application to Neuroprostheses and Biometrics

Richard Reilly

Senior Lecturer

University College Dublin

 

 

Time and Place

Mauldin Auditorium (NSH 1305)
Refreshments 3:15 pm
Talk 3:30 pm

Abstract

 

Combining information is a vital element of comprehension and understanding. Multimodal signal processing allows us to fuse information from multiple information signals and sources for benefit in medical diagnosis, cognitive neuroscience and biometrics. With current pressure within the healthcare system to deal with an increasingly aging population and with demands for increased security, multimodal signal processing will become more important in the future.

 

This talk will overview recent work in three areas neural engineering (multisensory integration and neuroprostheses) and biometrics. It will also point to other multimodal aspects in these and other areas which are expected to be important in the future.

 

1.         Biometrics:

Multimodal biometrics provide the most adaptive approach to person verification and identification. We review existing methods employed to carry out audio-visual integration, for speaker identification. Additionally, we examine classifier combination theory and its’ applications towards person recognition, such as the fusion of audio and face experts, or indeed, the fusion of several face and/or several audio experts. We will also highlight which fusion techniques are more suited towards speaker identification, a multi-class problem, and which are suited towards speaker verification/authentication. Results are provided on existing and newly acquired multimodal databases.

 

2.         Neuroprostheses:   Multisensory Integration:

Everyday tasks involve the seemingly automatic integration of information from multiple sensory modalities. For example, driving a car involves the synthesis of a number of sensory activities: visual (seeing the road), auditory (hearing the car engine and passing traffic), somatosensory (feeling the steering wheel) and motor (depressing the accelerator). The combination of inputs from different senses can function to reduce perceptual ambiguity and also enhance stimulus detection. Despite the fundamental role that sensory integration plays in performance and perception, how and when information from separate sensory modalities comes together in the human neocortex is an unsolved problem. We overview new analysis of multisensory (audio and visual) integration from a nonlinear dynamic viewpoint. It is hoped this approach will become useful in aiding the diagnosis and monitoring of schizophrenia.

 

3.         Neuroprostheses: Monitoring and Analysis of Attention

The ability of humans to extract information from the environment and select a response, to maintain focus for prolonged periods of time on even monotonous tasks, and to identify thought processes and actions in accordance with our goals is facilitated by the attention system of the brain. Selective attention is examined with regard to the use of its characteristic electrophysiology in a Brain Computer Interface, for the severely disabled. The results of a preliminary dependent BCI design involving Steady-State Visual Evoked Potentials (SSVEPs) are first presented. In this BCI subjects shifted gaze towards one of two visual stimuli in a real-time gaming environment to make selections. An independent BCI involving SSVEPs and real-time biofeedback is developed: the Visual-Spatial Attention Control (V-SAC) BCI. Real-time experiments in which left/right spatial attention is classified by extracting (SSVEPs) will be described. In a further BCI study some critical issues of the use of SSVEPs in the V-SAC BCI are addressed through an offline analysis of high density EEG data. A comparison is made between using stimulation frequencies within and outside the alpha band and the effects behavioural performance on BCI accuracy is examined.

 

 

Speaker Biography

 

Richard Reilly received his B.E., M.Eng.Sc. and Ph.D. Degrees all in Electronic Engineering, from the National University of Ireland.  In 1988 he joined Space Technology Ireland and the Dept. de Recherche Spatiale (CNRS group) in Paris, developing DSP-based on-board experimentation for the NASA satellite WIND. In 1990, he joined the National Rehabilitation Hospital and in 1992 became a Post-Doctoral Research Fellow at University College Dublin, focusing on signal processing for speech and gesture recognition. Since 1996, Dr Reilly has been on the academic staff in the Department of Electronic and Electrical Engineering at University College, Dublin. He is currently Senior Lecturer and researches into neurological signal processing and multimodal signal processing.

 

Dr Reilly was the 1999/2001 Silvanus P. Thompson International Lecturer for the IEE. In 2004 he was awarded a US Fulbright Award for research collaboration into multisensory integration with the Nathan Kline Institute for Psychiatric Research, New York. In 2005 he is been based collaborating with the Cognitive Neuroscience Department at the University of Barcelona.

 

Dr Reilly is an Associate Editor for IEEE Transactions on Multimedia and also a reviewer for IEEE Transactions on Biomedical Engineering, IEEE Transactions on Neural Systems and Rehabilitation Engineering, IEEE Transactions on Industrial Electronics, Journal of Applied Signal Processing, Signal Processing and IEE Proceedings Vision, Image & Signal Processing.

 

 

Speaker Appointments

For appointments, please contact Louise Ditmore (lditmore+@cs.cmu.edu).

The Robotics Institute is part of the School of Computer Science, Carnegie Mellon University.