1. Introduction

This review was originally written in 1993, to bring together my ideas on a proposed cognitive processing mechanism. In 1994 more material was added on the evidence for the mechanism, and the paper was divided into separate parts (computers and web-sites at that time had very small capacity by modern standards). This division has the disadvantage that the close integration of themes - the cognitive mechanism, the data from which it was developed, and the wider implications for what might be done by the mechanism - was lost.

But the advantage is it is easy to access different issues.

The parts were not written to be read independently, and I plan to add a pdf which contains the whole review as a single unit.

Recently (2022) this review has been revised very considerably. Viewed from the distance of time, and not being so immersed in and familiar with the details, I could see that many issues needed more explanation, or some alteration and a wider perspective.

In this review, there are 3 main groups of topics.

1. Introduction.

Basic element, sources of data which meet cognitive needs.

2. The cognitive processing element.

Meeting an information need :

3a. by finding it in the environment.

3b. from a stored knowledge base.

3c. by working through a 'routine',

or referring to the result of using a routine elsewhere.

3d. more on knowledge bases.

Choosing what to do.

4. Sequences of activity, introduction to the 'overview'.

5. 'Sequencers'.

6. Working storage.

Choosing how to do it.

7. Choosing the method used to meet a task need :

using meta-knowledge, implications for mental workload.

8. Learning and modes of processing : some issues and possibilities.

9. Final comments.

References.

Section 1 : Introduction

Building up behavioural complexity from a cognitive processing element

Lisanne Bainbridge

November 1993, revised August 1994, November 1997, March - August 2022

This review proposes a mechanism to account for how someone doing a complex dynamic task chooses their behaviour (what best to do next, and how best to do it) so that it is appropriate to the current context. A simple cognitive processing element is built up into larger structures from which, in use, many aspects of flexibility in the organisation of behaviour emerge without being explicitly represented.

This mechanism has two essential aspects :

- the cognitive processing elements, which are discussed more fully here,

- the context, the multidimensional 'space' of dimensions and their values, within which the elements operate (and update the values). These 'dimensions' are not simply linear or scalar. The brain handles space and organisation in a 'Gestalt' way which is not captured by mathematics. And these dimensions are referred to, not only by the 'conditionals' of processing but also in choosing what to do and how to do it which would be most appropriate at this time.

The processing element suggested has the key features : a cognitive goal/need, with working storage for what meets the need, and links to methods for meeting the need. Behaviour is active and goal/need oriented, rather than occurring only in reaction to environmental events.

'Routines', which are independent from the cognitive needs which they meet, are themselves built from these processing elements. They built up local working storage, only relevant to and retained for this 'routine'.

'Routines' have associated meta-knowledge (part of the context in which the other elements operate) about their properties. This meta-knowledge is involved in the choice of working method for meeting a cognitive need, and in the control of mental workload and learning.

'Sequencers', which include conditional elements, determine the sequence in which the main cognitive needs are thought about. The working storage associated with the main cognitive needs in the 'sequencers' is maintained relatively continuously, and acts as an overview of the current state of the task. This overview is the context determining the optimum sequence of, and method used in, thinking.

Serial behaviour, and different modes of processing ('types of skill'), emerge from the contextual mechanism in operation in particular contexts, rather than being formally defined in advance.

When I was trying to analyse the verbal protocols collected from people doing the furnace power allocation task, I tried several methods of generating the same sequence of thinking as the operators, and this 'box element’ representation was the one that 'worked' (I now have no record of what else I tried !).

When I am writing, I keep notes of many questions which need to be answered. After developing this cognitive element, when I went through my pile of questions, I was surprised to find most could be answered using this mechanism without additions. This gave me some confidence in the validity of what I had done.

So - this element started as a notational convenience for describing the results, and was later reified into a cognitive processing mechanism when it turned out to provide answers to many questions about how cognitive processing might be done.

Parts of this mechanism have been described in many of the papers on this site. This is an attempt to integrate the material, with a focus on the cognitive element rather than on operator behaviour/ operator modelling/ support design etc.

So the main aim of this review is to pull together and integrate the material on the nature of the proposed cognitive processing element. I didn’t think it was necessary to repeat all the content already written elsewhere. Where relevant I have just provided links to the other papers.

The main papers which describe the mechanism more briefly are :

1975 The representation of working storage, and its use in the organisation of behaviour.

1989 Development of skill, reduction of workload.

1992 Mental models in cognitive skill.

- - -

Main topics and attitudes

'Cognition’ is what is done between use of senses and use of muscles, the thinking between perception and action. Although, as will be obvious from the findings discussed here, it does not always happen in an information-decision-action sequence.

Cognitive behaviour is often complex. I find it fascinating to ask whether there is some relatively simple mechanism from which this complex behaviour could be built up, to give a parsimonious account of its flexibility.

The aim of this paper is to describe such a mechanism, called a 'cognitive processing element', and to explore how much of cognitive behaviour it can account for in complex dynamic tasks such as industrial process operation or air-traffic control.

Extensive analyses of behaviour in these tasks suggests that their key features are as shown in Table 1.1. For each of the key task features, the Table suggests some cognitive processes involved and the underlying mechanisms needed. The aim of this review is to account for many of these features of behaviour. This review gives some of the evidence for these features, more evidence can be found in Bainbridge, 1981, 1992, 1993.

Table 1.1 : Key Features of Complex Dynamic Tasks

In my opinion, the nature of behaviour in complex dynamic tasks means that the focus in accounting for it needs to be on the underlying organisation of behaviour, and on how the behaviour can be flexible in dealing with situations which change in detail from one instance to another.

The focus here is also on cognitive processes in well practised cognitive skill. Some of the mechanisms needed to account of learning are mentioned in Section 8, but the mechanisms proposed here are not sufficient to deal with all the processes involved in learning, or in the problem solving needed to deal with new situations.

The most frequently used element in many methods of representing cognitive processes is an (if condition - then action) pair. When combined together, such elements often only make up undifferentiated chains or networks, without any further structure. They may be sufficient to account for simple laboratory tasks and intellectual games, but they are inadequate to account for the behaviours indicated in Table 1.1.

To account for cognitive behaviour in tasks such as industrial process control or air-traffic control, it is not necessarily adequate to expand a cognitive model used to account for simple tasks. A simple model may not contain concepts sufficient either for understanding the behaviour or to suggest human factors/ ergonomic designs which would support this behaviour.

This review (and papers throughout this site) proposes that many of the underlying mechanisms can be formed from a richer but still simple cognitive processing element. This cognitive element was first applied, and has been used most extensively, in developing a (paper-and-pencil) simulation of the cognitive processes of an experienced steelworks furnace power supply controller. (The task is described in Bainbridge et al, 1968.) This review will also discuss other examples of complex human behaviour which need to be accounted for, especially when they provide a briefer example, or when they suggest modelling needs which were not required to account for the furnace operator data.

Verbal reports were collected from groups of inexperienced university students and experienced steelworks furnacemen, who were asked to give a running commentary ('verbal protocol') about what they were thinking when controlling a simulation of the task. One of these spoken reports was chosen for detailed analysis, leading to an attempt to 'model' the sequence of topics thought about. As the main data analysed came from verbal protocols, so the analysis has all the limitations of this type of data. This problem, and the analytic techniques used, are discussed in more detail in Bainbridge (1985, 1990). The mechanism proposed here is necessarily somewhat 'serial' in nature, as it comes from analysis of and attempts to mimic the content of language reports (which have to be sequential), so the actual underlying mechanisms are probably very different. Especially as the brain does its processing in very different ways from most computer 'mechanisms', with much parallel and Gestalt pattern processing, which the mechanism used here, like most other cognitive models, barely mentions.

The analysis of the verbal protocol was built up into a paper-and-pencil simulation of the operator's cognitive behaviour (presented in full in Bainbridge, 1972 : Ph.D. thesis, Figures 7.2.1 to 7.3.3). Paper-and-pencil simulation was used, because I did not want, when thinking about the types of mechanism underlying cognitive behaviour, to be constrained to the facilities available in any particular programming language (and before 1972 they were primitive). So the emphasis here is on the types of mechanism needed, rather than on a specific way of realising them.

This review also makes some points about additional mechanisms which are needed to account for the evidence on human cognition, but which might not otherwise be needed as part of a general cognitive device. The review finally discusses extensions and expansions to the model : how its knowledge mechanisms could account for aspects of behaviour which this approach to modelling was not originally devised for, particularly in the control of mental workload and learning, and some features of behaviour which it does not account for and for which additional mechanisms are needed.

In this approach to modelling there are three key theoretical concepts as well as the cognitive element :

complexity : the processes underlying the contextual mechanism are themselves built up from a basic cognitive processing element with potential for adaptability, i.e. complexity emerges from simplicity.

context : complex behaviour involves building up an overview of understanding of the task situation and what to do about it. This overview acts as the context for doing the behaviour : for choosing what behaviour to do next, and for choosing how to do the behaviour. The behaviour done updates the overview, which then determines the next best behaviour, and so on.

meta-knowledge : working methods are chosen on the basis of meta-knowledge about their properties.

The word 'complex' has already been used with two meanings :

1. as in complex tasks, in which the person may need to integrate simultaneous responsibilities for several activities, each of which may not be simple. This use of the word is implicitly defined by Table 1.1 above.

2. as in complexity theories, in which complex behaviour and its organisation emerge from simple components.

The aim of this review is related to both meanings : to account for human behaviour in complex tasks in a way that has the characteristics of a complexity theory. However, I hope it will be clear that what is said in this review results from data analyses rather than being driven by commitment to a particular theoretical principle. As mentioned above, the mechanisms which will be described were first used simply as a notational convenience for representing the furnace operator's behaviour. They were only reified into basic mechanisms when I found that they gave a parsimonious and internally consistent account of the behaviour, and could also answer questions which they had not originally been devised to account for. The length of this review is due, not to the difficulty of explaining the proposed mechanism, which is quite simple, but of describing the behaviour it is supposed to account for, and how it accounts for it.

From another perspective, a key focus of this review is the way in which this cognitive processing element (Section 2) may account for the flexibility of complex behaviour. This element, including the meta-knowledge (Section 7) describing the properties of a group of elements (a 'routine'), may account for the big variety in a person's response to the combination of task circumstances and the person’s abilities, attitudes and experience in the current moment. These factors affect the details of choosing : what needs to be done (Sections 2-3), what working method is used to do it (Sections 4, 5, 6), and what 'mode of processing' is used to realise the behaviour (Section 8).

In my previous papers, the focus has been on accounting for, or designing to support, cognitive behaviour in complex dynamic tasks, and those accounts made use of assumptions about the nature of the underlying cognitive mechanisms which were based on my conclusions from the furnace operator simulation. The aim of the present paper is the inverse, to make the nature and potential of the proposed mechanisms as clear as possible.

- - -

The sequence of topics

In this account, cognitive processes are built up from processing elements. This review describes the reasons for suggesting these mechanisms, the detailed nature of the proposed element and the structures built up from it, and the aspects of behaviour which it is claimed can be accounted for by it, in the following sections :

The proposed processing element consists of a cognitive goal/ need, with working storage for what meets the need, and links to methods for meeting the need (Section 2).

'Routines', for meeting the main cognitive needs, are themselves built from elements (Section 3c).

'Routines' may refer to knowledge bases (brief discussion in Sections 3b and 3d).

'Sequencers', which include conditional elements, determine the sequence in which the main cognitive needs are thought about (Sections 4-5).

The working storage associated with the main cognitive needs and routines in the 'sequencers' is maintained relatively continuously (Section 6). This acts as an overview of the current state of the task, which acts as the context for choosing the optimum sequence of, and method used in, thinking (see Bainbridge, 1975, 1978).

'Routines' have associated meta-knowledge about their properties. The choice is made between alternative methods of meeting a cognitive need, on the basis of meta-knowledge about the methods. The place of this choice mechanism in the control of mental workload is discussed in Section 7.

This meta-knowledge is also part of the choice mechanism used in learning and in the use of different modes of processing. This paper will mention some of the issues but will not give a complete account, either of problems in learning which need to be accounted for, or of a mechanism which could explain all the processes in learning (Section 8).

Final comments are in Section 9.

- - -

The furnace power control task, and some early results, were described in :

Bainbridge, L., Beishon, R.J., Hemming, J. and Splaine, M. (1968). A study of real-time human decision making using a plant simulator. Operational Research Quarterly, Special Conference Issue, 19, 91-106.

reprinted in : In E. Edwards and F.P. Lees (Eds.) (1974). The Human Operator in Process Control, London: Taylor and Francis, pp.91-104.