Home

A collection of papers about concepts for supporting human cognition in complex tasks : human factors/ ergonomics, cognitive modelling, cognitive architecture, mental workload, situation awareness, cognitive error, skill and training, interface design, automation.

For general information about this new site, see the About page.

INTRODUCTION

Many aspects of the way in which human beings do complex dynamic tasks are not well understood, and the mechanisms underlying this behaviour are not adequately accounted for by current models of cognitive processes. So there is not a firm basis for designing to help a person doing such tasks.

The aims of this site are to :

- point out some of these issues,

- make available some past papers,

- 'circulate' some draft papers.

The papers available here are indicated by a red access link.

This Home page is in five main sections :

Case studies of behaviour in complex dynamic tasks, the evidence I referred to when making practical or theoretical modelling suggestions.

My papers arranged in topic groups, those papers that I thought when I retired would be worth retaining.

Two big summary integration papers.

The relation of my work to that of Jens Rasmussen.

Papers in chronological order.

The papers are in these main topic groups :

Cognitive processes in complex tasks :

- Case studies of process operation.

- Working storage.

- Cognitive operations.

- Organisation of knowledge and organisation of behaviour.

Skill and Workload.

Practical implications :

- Displays, training.

- Model limitations.

- Automation.

Knowledge elicitation.

Notes on terms :

In the UK 'ergonomics' includes all aspects of designing for human use, not just biomechanics.

I use the word 'skill' to indicate level of ability at doing a task, from beginner to expert, not with the limited meaning of a specific type of task processing.

These papers were written 30-50 years ago, but I think they raise issues many of which are still not adequately accounted for.

The interface technologies and the tasks required may have changed dramatically over the decades, but the human cognitive mechanisms investigated in the 1960s and 1970s are still in operation. Human neural processes for sensing/ perceiving / thinking/ remembering have not changed.

It is interesting to find the 'Ironies of Automation' paper being actively discussed in many new areas of application where computers are being expected to take over formerly human tasks. Baxter et al in 2012 celebrated the 30th anniversary of the paper and described how the ironies apply to financial market trading and cloud computing as well as aviation. More recently, my eldest nephew tells me this paper is widely quoted in the field of autonomous vehicles, and his wife has met it in her work on automated protein structure recognition.

Also now that 'AI' is taking over some cognitive components of office jobs, 'cognitive automation' rather than control automation, and is being introduced with much enthusiasm but little thought about the wider effects, some of the same concerns arise. There is a paper here on the topic, written over 30 years ago so it is primitive in many ways, but still it did flag that there are issues.

PAPERS

CASE STUDIES OF COGNITIVE PROCESSES IN COMPLEX TASKS DONE WITHIN TIME LIMITS

Many of the papers below include generalisations about cognitive behaviour, which are the basis for comments on practical implications, or about how these cognitive processes can best be modelled. There is not space in most of the papers to include the specific evidence on which the generalisations are based.

The following papers included here do summarise the results from detailed studies of people doing real complex dynamic tasks under time constraints :

Analysis of verbal protocols from a process control task.

Working memory in air-traffic control.

Complex tasks, including planning and multi-tasking by pilots.

Diagnostic skill in process operation.

Problems in the assessment of mental load, when there are several strategies for doing a task (the data are from air-traffic control).

The studies of air-traffic control and flying were done in France by former students and colleagues of Jacques Leplat.

Studies by Ted Crossman's group :

There were several process operator case studies done by Ted Crossman’s group when he was in Oxford. Ted is someone who is not as well known as he should be, but he was not interested in writing reports on what had been done - he was always more interested in exploring the next idea. At the time I was at Oxford (1962-4) he had 3 research assistants : John Beishon, John Cooke, and me. John Beishon and John Cooke were mature students with previous careers, and both got a D.Phil. from the work they did at that time. These studies include more evidence about what a process operator can do, which also needs be accounted for in models of how people act in a complex dynamic environment. Especially as these tasks use abilities which were not needed in the tasks studied above.

The following case studies are all reprinted in :

Edwards, E. and Lees, F.P., The Human Operator in Process Control. Tayor & Francis London 1974,

copies of which are readily available through on-line listings of pre-owned book dealers, such as Abebooks and Alibris.

This book also includes many process operation case studies by other groups.

The following three papers add evidence about controlling a slow response system (in the process I studied, changes to a control were responded to in the process immediately).

Crossman, E.R.F.W., Cooke, J.E. Manual control of slow-response systems. 1962 (extract showing effect of practice on control effectiveness).

Crossman, E.R.F.W., Cooke, J.E., Beishon, R.J. Visual attention and the sampling of displayed information in process control. 1974. (extract - section on sampling by paper mill operators - I was the one who walked round after the operator noting down everything he did and the information he had access to).

In Beishon's bakery study (done after he left Oxford), the operator was not only controlling slow response systems, he was also controlling several at the same time. And there were external interrupts. So this was mainly a study of multi-tasking. Changes happened in minutes on the three bakery ovens, while events happened over much longer periods in the five furnaces control task I studied, and multi-tasking for pilots can change within seconds.

Beishon, R.J. An analysis and simulation of an operator's behaviour in controlling continuous baking ovens. 1967.

After this bakery study, John Beishon went on to organise the furnace power control study reported in my papers. My research was done within the context of this prior bakery study, so I decided to include the bakery study in full here.

There are also two papers by me in Edwards & Lees :

- a paper included below, Analysis of verbal protocols from a process control task,

- a paper describing more specifics about the furnace power control task simulation and a little about the cognitive processes :

Bainbridge, L., Beishon, R.J., Hemming, J. and Splaine, M. (1968). A study of real-time human decision making using a plant simulator. Operational Research Quarterly, Special Conference Issue, 19, 91-106. reprinted : In E. Edwards and F.P. Lees (Eds.) (1974). The Human Operator in Process Control, London: Taylor and Francis. pp.91-104.

The previous famous collection of detailed case studies is :

Sinaiko, H.W. Selected Papers on Human Factors in the Design & Use of Control Systems, Dover, 1961.

Also still available. The papers that have stayed in my memory are studies of flying and radar watch-keeping.

These tasks may no longer exist in the same form. But these studies still tell us a great deal about what human beings can do when expected to do difficult tasks with little or inadequate help, our limitations, and what we need for support.

MY MAIN PAPERS

Case studies of process operation

(1971) The influence of display type on decision making.

Differences in cognitive processes when using analogue and digital displays.

In Institute of Electrical Engineers Conference Publication No. 80, Displays, pp. 209-215.

(1974) Analysis of verbal protocols from a process control task.

A summary of the cognitive processes of operators controlling the electricity supply to electric-arc steel-making furnaces.

In Edwards, E. and Lees, F.P. (eds.) The Human Operator in Process Control, Taylor and Francis Ltd, London, pp. 146-158.

- - -

General cognitive processes

Working storage

(1975) Working memory in air-traffic control.

The capacity and content of working memory in a complex task.

Unpublished report.

Department of Psychology, University of Reading.

(1975) The representation of working storage and its use in the organisation of behaviour.

In Singleton, W.T. and Spurgeon, P. (eds.) Measurement of Human Resources, Taylor and Francis, London, pp. 165-183.

Cognitive operations

(Note : I use the word 'skill' to indicate level of ability at doing a task, from beginner to expert, not with the limited meaning of a specific type of task processing.)

(1981) Mathematical equations or processing routines ?

The reasons for the need to replace models based on engineering concepts with models based on cognitive processes.

In Rasmussen, J. and Rouse, W.B. (eds.) Human Detection and Diagnosis of System Failures. NATO Conference Series III : Human Factors, Vol. 15. Plenum Press, New York, pp. 259-286.œ

(1984) Diagnostic skill in process operation.

Proceedings of the 1984 International Conference on Occupational Ergonomics, Volume 2 : Reviews. May 7-9, Toronto, Canada, pp. 1-10.

(1989) The relation between the categories in 'Types of Skill' and in the 'Skill- Rule-Knowledge based' schema.

Problems with the semantics of describing different types of cognitive process.

Unpublished report.

Department of Psychology, University College London.

(1992) Mental models in cognitive skill : the case of industrial process operation.

An overview of a proposed cognitive architecture for complex tasks.

In Rogers, Y., Rutherford, A. and Bibby, P. (eds.) Models in the Mind. Academic Press, London, pp. 119-143.

Organisation of knowledge and organisation of behaviour

(1988) Types of Representation.

Different representations, and the implications for cognitive processing.

In Goodstein, L.P., Anderson, H.B. and Olsen, S.E. (eds.) Tasks, Errors and Mental Models. Taylor and Francis Ltd., London, pp. 70-91.

(1993) Types of hierarchy imply types of model.

Different ways of organising complex knowledge structures, and the processes required for handling them.

Ergonomics, Special Issue on Cognitive Processes in Complex Tasks, 36 (11), 1399-1412.

(1997) The change in concepts needed to account for human behaviour in complex dynamic tasks.

IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 27, 351-359.

- - -

The interrelation of skill and workload

(Note : I use the word 'skill' to indicate level of ability at doing a task, from beginner to expert, not with the limited meaning of a specific type of task processing.)

(1974) Problems in the assessment of mental load.

The adaptation of cognitive processes to task demands and mental capacity : some reasons for the lack of correlation between objective and subjective mental workload.

Le Travail Humain, 37 (2), 279-302.

(1978) Forgotten alternatives in skill and workload.

Changes in cognitive processes with the development of skill, and the implications for mental workload.

Ergonomics, 21, 169-185.

Simultaneously published as :

(1977) Possibilités oubliées en matière d'habilété et de charge de travail. Le Travail Humain, 40, 203-224.

(1989) Development of skill, reduction of workload.

This may be referred to as the 'Types of Skill' paper.

In Bainbridge, L. and Ruiz Quintanilla, S.A. (eds.), Developing Skills with Information Technology, Wiley, pp. 87-116.

(There’s another 1989 paper by me in this book, which reviews the training methods mentioned in the book. It’s a compilation/ integration/review of work by others, no interesting contribution from me.)

(1993) Difficulties and errors in complex dynamic tasks.

Unpublished discussion paper.

Department of Psychology, University College London.

- - -

Practical implications

(1991) Multiplexed VDT display systems.

Problems in the design of multi-format display systems for complex tasks.

In Weir, G.R.S. and Alty, J.L. (eds.) Human-Computer Interaction and Complex Systems. Academic Press, London, pp. 189-210.

(1993) Planning the training of a complex skill.

Integrating the many types and levels of knowledge and skill involved.

Le Travail Humain, Special Issues in Honour of Jacques Leplat, 56 (2/3), 211-232.

Automation

(1983) Ironies of automation.

Increasing levels of automation can increase, rather than decrease, the problems of supporting the human operator.

Automatica, 19, 775-779 (conference proceedings)

First reprinted in : (1987) Rasmussen, J., Duncan, K. and Leplat, J. (eds.) New Technology and Human Error, Wiley, Chichester, pp. 276-283.

Reprinted in several other places, which I haven’t kept track of as I don’t own the copyright. Many web copies, including German and Spanish translations, so no doubt there are others. It even has a Wikipedia entry !

(1990) Will expert systems solve the operator's problems ?

The 'ironies of automation' concepts applied to cognitive automation.

In Roe, R.A., Antalovitz, M. and Dienes, E. (eds.), Proceedings of the Workshop on Technological Change Process and its Impact on Work. September 9-13, Siofok, Hungary, pp. 197-218.

Translation published as : (1991) Les systèmes experts resoudront-ils tous les problèmes des operateurs ? In Neboit, M. and Fadier, E. (eds.) Proceedings of the Colloquium on Facteurs Humains de la Fiabilité et de la Securité des Systèmes Complexes. April, INRS, Nancy, France, pp. 17-26.

Problems with Modelling

(1988) Multiple representations or 'good' models.

How and what do models communicate ?

In Patrick, J. and Duncan, K.D. (eds.) Training, Human Decision Making and Control. Elsevier/ North Holland, Amsterdam. pp.1-11.

(1992) Is your simulation valid ? helpful ? for what purpose ?

Using the word 'simulation' to mean any type of modelling. More issues added to those in the previous paper.

prepared for proposed but never published book : Cacciabue, C., Pavard, B. and Hoc, J-M. (eds.) Simulation in Dynamic Environments.

- - -

Knowledge elicitation

(1985) Inferring from verbal reports to cognitive processes.

In Brenner, M., Brown, J. and Canter, D. (eds.) The Research Interview : uses and approaches. Academic Press, London, pp. 201-215.

(1986) Asking Questions and accessing knowledge.

A rather cryptic review of issues.

Future Computing Systems, 1, 143-149.

EXTENDED REVIEWS OF COGNITIVE PROCESSES IN COMPLEX TASKS

(1995) Processes underlying human performance.

A review of the general principles underlying cognitive human factors/ ergonomics.

in Garland, D.J., Hopkin, V.D. and Wise, J.A. (eds). Aviation Human Factors, Erlbaum, November 1998.

I. Using an interface, the bases of classic HF/E.

II. Complex tasks.

III. Mental workload, learning, errors.

References.

These sections have been combined in a single 74 page pdf, which can be downloaded here.

This is the version printed in the above book.

I was unable to provide a version for the second edition of the book, as I had retired.

There is a paper called

Processes underlying Human Performance

by Lisanne Bainbridge and Michael Dorneich

in the Handbook of Aviation Human Factors 2nd edition, CRC Press, Taylor & Francis Group 2009

which I have not seen. I only recently discovered the existence of this paper, and managed to contact Michael Dorneich. He didn't know I didn't know, and says he tried to keep the replacement in the same style. I thank him for having taken over the revision.

- - -

(1993, 2022) Building up behavioural complexity from a cognitive processing element.

A cognitive architecture to account for situation awareness and the choice of adaptive context-sensitive behaviour, using cognitive goals, an overview, and meta-knowledge.

Unpublished report (much revised from the 1990s version), presented in sections.

There are 3 main groups of topics in this review of the suggested mechanism :

1. Introduction.

Basic element, sources of data which meet cognitive needs.

2. The cognitive processing element.

Meeting an information need :

3a. by finding it in the environment.

3b. from a stored knowledge base.

3c. by working through a 'routine',

or referring to the result of using a routine elsewhere.

3d. more on knowledge bases.

Choosing what to do next.

4. Sequences of activity, introduction to the 'overview'.

5. 'Sequencers'.

6. Working storage.

Choosing how to do it.

7. Choosing the method used to meet a task need :

using meta-knowledge, implications for mental workload.

8. Learning and modes of processing : notes on issues and possibilities.

9. Final comments.

References.

RELATION TO JENS RASMUSSEN'S WORK

From the Wiki page on my Ironies paper, I followed up a link to a review by B. Strauch.

I am grateful to him for collating and commenting on the influence of that paper.

But I was horrified to see that Strauch says my work was done within the context of Rasmussen’s. That could not be further from the case. Most of my working life I was well known for pointing out that Rasmussen's ideas are inadequate. I was an applied cognitive psychologist, and learned my trade (methods and assumptions) working in Ted Crossman’s team in 1962-4. Rasmussen first published on operator characteristics later in the 60s. My concepts for cognitive modelling originated during my Ph.D. research, thesis published 1972. While doing that I was a lowly member of staff in a minor university, not in the loop for receiving reports from foreign research labs. I immediately recognised, when I first read about Rasmussen’s model, that it did not include the mechanisms needed to account for the cognitive behaviour I had observed in my own study.

He used engineering-like input-output models and crucially did not include mechanisms which can generate, or remind designers to think about supporting (a) building up an overview of the changing task situation, (b) top-down processing - actively seeking the information needed in thinking, (c) anticipatory control - thinking what will happen in the future and what to do about it. He did a good detailed study of maintenance technicians. So his cognitive models described people doing a static non-repeating task using a context-free strategy (first published in a Risø report 1973), and do not include the cyclic overview-building (situation awareness) mechanisms used in context-sensitive control of changing dynamic systems and under time constraints. His models may be easy for people used to an engineering approach to understand, but they do not account for complex cognitive behaviour.

They are also similar to ideas in the post-Broadbent era of experimental psychology. Broadbent took his models from control theory, while we in Crossman's group already knew that those ideas were inadequate to account for real-life behaviour. Experimental psychology at that time tried to do 'context-free' experiments, which for us was to ignore the key determinants of real behaviour.

Perhaps it's another irony that I should be described as working within Jens Rasmussen’s ideas, while actually during my later working life I was noted for disagreeing with him, emphasising that Rasmussen’s models do not include essential concepts. This is also true of the Ironies paper, as one of the key issues in that paper is that people taking over from failed dynamic systems have not had the time or opportunity to build up an overview of the task situation. Just displaying an overview is not sufficient, as it is built up during and structured relative to the person's (top-down) task thinking. Operators of automated systems also may not have had opportunities to learn/ practice how to build up the overview, which is an essential part of familiar cognitive skills.

A smaller point : throughout this site there are papers in which I make it clear that I use the word 'skill' with a different meaning : I use the word 'skill' to indicate level of ability at doing a task from beginner to expert (as the word is used in the UK and in psychology), not with the limited meaning of a specific type of task processing.

It would be rather an over-statement to say that my later working life was dedicated to pointing out that Rasmussen's ideas are inadequate, as I hope my work contributed more positively rather than just criticising, but I certainly did criticise often. This site includes a continuing sequence of papers showing that Jens Rasmussen’s ideas are not sufficient for modelling complex cognitive processes in dynamic tasks. Though I was often too polite to mention him by name in print.

Papers in which the difference in approach to cognitive modelling is most explicit :

(1984) Diagnostic skill in process operation.

(1989) Development of skill, reduction of workload.

This may be referred to as the 'Types of Skill' paper.

(1989) The relation between the categories in 'Types of Skill' and in the 'Skill- Rule-Knowledge based' schema.

(1993) Types of hierarchy imply types of model.

(1997) The change in concepts needed to account for human behaviour in complex dynamic tasks.

See also the note before Types of Representation. It was generous of the editors to ask me to contribute to a book in honour of Jens, when I was already well known for criticising his work. But there was a small group of topics on which I agreed with him.