Faiyaz Doctor

Physiological and affect aware computing for intelligent pervasive environments


The ever growing number of embedded and networked enabled physical devices collectively termed as the ‘Internet of Things (IoT) has become an enabler for facilitating richer context awareness, personalisation through integration of intelligence, into everyday consumer devices. The global IoT market is expected to hit $7.1 trillion by 2020 (IDC) and is projected to drive the circulation and use of some 50 billion connected devices. This influx of personal, mobile and wearable devices will open up many new research opportunities focusing on human centered technology with a particular focus on extending the contextual sensing and smart processing capabilities of ubiquitous systems and smart environments. Consequentially there will be need for researchers and industry to develop more unobtrusive and natural communication between computing artefacts and users and make systems more aware and reactive to user needs. This would involve expanding traditional sensing and behaviour modelling modalities to include signals from the body and the interpretation of the user’s affective, emotive and cognitive states.

Affective computing (AC) is concerned with emotional interactions performed with and through computers. It is defined as “computing that relates to, arises from, or deliberately influences emotions” (R. Picard, MIT Media Lab). AC seeks to facilitate research through the recognition and modelling of human affective states (e.g., happiness, sadness, etc.), cognition (e.g., frustration, boredom, etc.) and motivation, as represented by speech, facial expressions, physiological signals, and neurocognitive performance. Physiological Computing (PC) relates to more generic computational systems that incorporates and utilizes physiological information (e.g., as in computer-human interaction). Practical applications of AC and PC based systems seek to achieve a positive impact on our everyday lives by monitoring, recognising and acting on our physiological signals, speech, facial expressions and gestures. Integrating these sensing modalities into intelligent pervasive computing systems will reveal a far richer picture of how our fleeting emotional responses, changing moods, feelings and sensations, such as pain, touch, tastes and smells, are a reaction to or influence how we implicitly or explicitly interact with the environment and increasingly the connected computing artefacts within.

The integration and use of AC and PC raise new challenges for signal processing, machine learning and Computational Intelligence (CI). Probabilistic and fuzzy systems provide a useful methodology for addressing some of the fundamental research challenges in AC/PC, where data sources such as: body signals (e.g. heart rate, brain waves, skin conductance and respiration), facial features, speech and human kinematics are characterized by ambiguity and uncertainty as well as being subject and context dependent. Other key areas of CI research, such as evolutionary algorithms, neural networks and hybrid systems also provide useful methodologies for addressing these challenges. As we develop better ways of using this data pervasively and in context of different, diverse data sources, it will create highly complex information rich scenarios where data frequently changes, can be conflicting and users’ goals evolve as they learn and redefine their objectives, perceptions and desires. Here the use of new self-learning and adaptive, cognitive computing systems will provide a means of intelligently capturing and understanding the relationship between people and their increasingly pervasive digital environment (J. E. Kelly et al, IBM Research).

PhD research topics areas:



Some related publications:


My full list of publications on Goggle Scholar

My ResearchGate profile