Welcome to NINA

Neurometrics INdicators for ATM

Enter

Overview


The project

NINA is a 27 month project co-funded in the SESAR WPE framework, started in September 2013. It will bring together three highly qualified partners with complementary competencies coming from academia, research institution and SME: DeepBlue, Sapienza Università di Roma and ENAC.
NINA intends to exploit these results and innovations to monitor the cognitive state of ATM operators and identify appropriate actions to support their activity, using sensors and a combination of neurometrics (measures of the brain’s electrical activity) and other neurophysiologic measures. These measure, regarding eye movements, blink frequency and their duration recorded in electrooculogram (EOG), electrocardiographic data (EKG) and Galvanic Skin Response data (GSR), will be collected through a non-invasive bonnet equipped with sensors, that in an industrial version of the device could be substituted with a cap connected through a wi-fi link.

The objectives

Monitor the level of cognitive workload of ATM operators in a realistic ATC context, through a combination of neurometrics and physiologic measures, characterizing the relation between the measured variables and the work performances.

Verify the existence of identifiable neurophysiologic conditions associated to the levels of cognitive control (e.g. Skills-Rules-Knowledge) defined by well-established cognitive models, using a combination of neurometrics and neurophysiologic measures.

Identify, prototype and evaluate the main elements of a simple adaptive interface in which adaptation is triggered by the described measures, and based on the key principles of an ecological interface (at the Skills, Rules and Knowledge levels of the cognitive control).
Validate the above results in realistic conditions through real ATM simulation facilities.

Sesar wpe EU Logo HALA

The content of this website is produced under the EC contract 12-120610-C4. It is the property of the NINA Consortium and shall not be distributed or reproduced without the formal approval of the NINA Steering Committee. The content of this web site is protected by copyright.

Context


The SESAR programme settled the path for an overall productivity improvement in ATM in its target Concept, in order to develop a modernised air traffic management system for Europe able to meet future capacity and air safety needs.

The common point of the identified necessary key enablers is an increase of automation that will support, and in some long term case even completely replace, human tasks. Increasing automation in conflict detection and resolution, and for data gathering and management, will drastically reduce the controllers’ task load. This will consent to meet the new efficiency necessities and let human operators manage a higher number of tasks, while shifting their roles towards more strategic ones.

Context_medium_sizeProblems regarding the interaction between human and automation have been identified, studied and reported in the human factor literature since the Eighties.
The adequateness of the interaction with the humans controlling the overall conduct of a task was a crucial point of study. The results show that human controllers may face difficulties when particular conditions lead to situations that exceed the capabilities of the automatic equipment: in these cases, humans can quickly pass from a condition of low to one of very high workload. An inverted U-shaped function between mental workload and pilots’ performance can be assumed: if the mental workload is too low or too high, the performance is not acceptable.

In NINA a recent line of research is explored: the use of neurometrics, i.e. the measurement of the brain’s electrical activity.
By collecting data showing the “internal” attentive and cognitive state of the operator (i.e. information regarding the vigilance state, the level of cognitive workload and the type of cognitive activity he is engaged in), NINA will model the activity of the controllers. The adoption of Ecological Interfaces Design to create interfaces that adapt to the mental state of the operator can lessen mental workload, particularly when dealing with unforeseen and unfamiliar events under increased psychological pressure.


Sesar wpe EU Logo HALA

The content of this website is produced under the EC contract 12-120610-C4. It is the property of the NINA Consortium and shall not be distributed or reproduced without the formal approval of the NINA Steering Committee. The content of this web site is protected by copyright.

 

Outcomes


NINA will deliver the following main tools and technologies:

NINA proposes a new approach to determine the appropriate degree of automation: developing a toolbox to monitor the internal state of the ATCOs through a combination of neurophysiologic measures (e.g. brain activity, eye movements, blink frequency and duration etc.).
The use of neurometrics makes possible a bidirectional communication between the machine and the human. This will consent to flexibly determine the best degree of automation considering not only the current traffic situation, but also the controller’s internal status.

References:
Borghini, G., Aricò, P., Graziani, I., Pozzi, S., Napoletano, L., Imbert, J.P., Granger, G., Benhacene, R., Babiloni, F. A neurophysiological training evaluation metric for Air Traffic Management. 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), At Chicago, 2014.

Click here to see the image gallery of the EEG recording sessions held in Toulouse, France at ENAC premises.

The project will develop an integrated set of physiological indicators to monitor and measure workload, fatigue, and other well-established cognitive concepts (like the Skill-Rule-Knowledge taxonomy of behaviour levels).
An objective physiological measurement will in turn cross-validate these cognitive concepts, allowing their quantitative measurement or clarifying some definitions. One of the expected results is, for instance, the possibility to trace the boundary between different levels of performances or to detect fatigue by using physiological measurements.
The development of a set of neurophysiologic measures and its integration and mapping onto concepts derived from cognitive psychology will serve as instruments for designing an adaptive interface.
The interface will be based on the key principles of an ecological interface, differentiating tasks at the skills, rules and knowledge levels of the cognitive control (as input from the previous element). The adaptation will be triggered by the described measures and by the current traffic situation.

The following list shows a selection of references:

Borghini, G., Aricò, P., Di Flumeri, G., Graziani, I., Colosimo, A., Salinari, S., Babiloni, F., Granger, G., Imbert, J-P., Benhacene, R., Golfetti, A., Bonelli, S., Pozzi, S. (2015). Skill, Rule and Knowledge-based Behaviors Detection during Realistic ATM Simulations by Means of ATCOs’ Brain Activity Fifth SESAR Innovation Days.

Aricò, P., Borghini, G., Di Flumeri, G., Colosimo, A., Graziani, I., Granger, G., Imbert, J-P., Benhacene, R., Terenzi, M., Pozzi, S., and Babiloni, F., (2015). Reliability over time of EEG-based mental workload evaluation during Air Traffic Management (ATM) tasks. 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

Borghini, G., Aricò, P., Ferri, F., Graziani, I., Pozzi, S., Napoletano, L., Imbert, J-P., Granger, G., Benhacene, R., and Babiloni, F. (2014). A neurophysiological training evaluation metric for Air Traffic Management. 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

Borghini, G., Aricò, P., Salinari, S., Graziani, I., Pozzi, S., Napoletano, L., Terenzi, M., Imbert, J-P., Granger, G., Benhacene, R., and Babiloni, F. (2014). Analysis of neurophysiological signals for the training and mental workload assessment of ATCOs. The 4th SESAR Innovation Days, Madrid.

Borghini, G., Aricò, P., Babiloni, F., Granger, G., Imbert, J-P., Benhacene, R., Napoletano, L., Pozzi, S. (2013). NINA: Neurometrics Indicators for ATM. Third SESAR Innovation Days.

Aricò, P., Borghini, G., Graziani, I., Imbert, J-P., Granger, G., Benhacene, R., Pozzi, S., Napoletano, L., Di Flumeri, G., Colosimo, A., and Babiloni, F. AIR-TRAFFIC-CONTROLLERS (ATCO): NEUROPHYSIOLOGICAL ANALYSIS OF TRAINING AND WORKLOAD ATCO. Italian Journal of Aerospace Medicine.

Di Flumeri, G., Borghini, G., Aricò, P., Pozzi, S., Bonelli, S., Golfetti, A., Colosimo, A., and Babiloni, F. Mental workload evaluation of ATCOs during ecological ATM scenarios. Italian Journal of Aerospace Medicine,

Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., Babiloni, F., 2012. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci Biobehav Rev. doi:10.1016/j.neubiorev.2012.10.003

Borghini, G., Aricò, P., Astolfi, L., Toppi, J., Cincotti, F., Mattia, D., Cherubino, P., Vecchiato, G., Maglione, A.G., Graziani, I., Babiloni, F., 2013. Frontal EEG theta changes assess the training improvements of novices in flight simulation tasks. Conf Proc IEEE Eng Med Biol Soc 2013, 6619–6622. doi:10.1109/EMBC.2013.6611073

Aricò, P., Borghini, G., Graziani, I., Fumihiko, T., Sun, Yu., Bezerianos, A., Thakor, N., Cincotti, F., Babiloni, F. Towards a Multimodal Bioelectrical Framework for the Online Mental Workload Evaluation. 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), At Chicago, 2014.

Aricò, P., Aloise, F., Schettini, F., Salinari, S., Mattia, D., Cincotti, F., 2014. Influence of P300 latency jitter on event related potential-based brain-computer interface performance. J Neural Eng 11, 035008. doi:10.1088/1741-2560/11/3/035008

Bonelli, S., Napoletano, L., 2013. The Usability Evaluation of a Touch Screen in the Flight Deck. In Cases on Usability Engineering: Design and Development of Digital Products, IGI Global. Copyright © 2013. 28 pages.

Bonelli, S., Napoletano, L., 2011. Flying with complexity. Proc. of the 9th ACM SIGCHI Conference on Computer-Human Interaction, New York.

Dreyer, D., Napoletano, L., 2011. ALICIA: All conditions operations and innovative cockpit infrastructure. Proceedings of the Aerodays 2011 Conference, Madrid.


 

Sesar wpe EU Logo HALA

The content of this website is produced under the EC contract 12-120610-C4. It is the property of the NINA Consortium and shall not be distributed or reproduced without the formal approval of the NINA Steering Committee. The content of this web site is protected by copyright.

 

 

Consortium


NINA brings together three highly qualified partners from academia, research institution and SME, and with complementary competencies regarding academic and scientific research, experimentation and prototypical development. All the members have proven experience in relevant research project in ICT, Aviation, ATM or other domains and in collaborating with other partners in EU R&D Projects:

DEEPBLUE
logo_dblue_300_66
Contact: Simone Pozzi
SAPIENZA UNIVERSITÀ DI ROMA
l_Uniroma1 (2)
Contact: Fabio Babiloni
ENAC
Enac (3)
Contact: Georges Mykoniatis

Work programme

NINA is structured in five work packages, each of which organised in activities.

Two of the WPs are technical, namely WP1 (Definition of Cognitive User State) and WP2 (Adaptive Interface Solutions), and represent the core of the NINA project. WP3 will be dedicated to Validation.
Parallel to these three are WP0 (Management) and WP4 (Interactions and Dissemination).


Sesar wpe EU Logo HALA

The content of this website is produced under the EC contract 12-120610-C4. It is the property of the NINA Consortium and shall not be distributed or reproduced without the formal approval of the NINA Steering Committee. The content of this web site is protected by copyright.

 

Dissemination


Dissemination materials and activities

Presentation

Click here to download the project final presentation.

Papers and articles

For papers and articles on the project, see the outcomes page.

Poster

Two posters are available: SID 2013 and SID 2014.

Gallery

Watch NINA project image gallery.

Video

Visit NINA’s channel on YouTube.


 

Sesar wpe EU Logo HALA

The content of this website is produced under the EC contract 12-120610-C4. It is the property of the NINA Consortium and shall not be distributed or reproduced without the formal approval of the NINA Steering Committee. The content of this web site is protected by copyright.

 

 

Contacts


Message sent.
Error! Please validate your fields.

Coordinator


Toggle Map
Deep Blue
Piazza Buenos Aires, 20
00198 - Roma, ITALY