Processes underlying human performance : II. Complex tasks
This is the second part of this chapter on cognitive processes underlying complex tasks. The parts were not written to be read independently.
I. Using the interface : the bases of classic HF/E.
II. Complex tasks :
Topics in this section :
A. Sequences of transforms
B. Language processing
1. written instructions
2. language understanding
C. Inference and diagnosis
D. Working storage
1. short term memory
2. the overview in working storage
content
form in which material is retained
practical implications
E. Planning, multi-tasking and problem solving
1. planning
pre-planning
on-line adaptation of plans
2. multi-tasking
a possible mechanism
practical implications
3. problem solving
F. Knowledge
knowledge and representation
an optimum format ?
III. Mental workload, learning, and errors.
Processes underlying human performance : II. Complex tasks
Lisanne Bainbridge
Department of Psychology, University College London
1995
Published in Garland, D.J., Hopkin, V.D. and Wise, J.A. (eds) Aviation Human Factors, Erlbaum, November 1998.
Using an interface for a simple task entails the functions of distinguishing between stimuli, integrating stimuli, naming, comparing, and choosing and making simple actions [see the previous section]. When the interface is well designed, these functions can be carried out by decision making, integration, and recoding processes. These processes use knowledge about the alternatives which may occur, their distinguishing features, probabilities and costs, and the translations to be made.
Figure 23 : Flight strips describing two aircraft, AAL419 and DAL152, which are flying at the same flight level (310 hundred feet [4th box from left]) from fix OTK [2nd box] to fix LEESE7 [6th box]. DAL1152 is estimated to arrive at LEESE7 two minutes after AAL419 (18-16 [3rd box, top row]), and is travelling faster (T783-T746 [1st box, 3rd row]).
[These printed paper flight strips were an intermediate technology for flight controllers. In the earliest days they worked with raw radar without labels, and wrote their own flight strips.
So controllers had well over-learned this location coding. There were considerable problems with designing new displays because the eye-movements etc. used in reading and sorting these strips were so integral to an expert controllers' activities.]
Doing a more complex task uses more complex knowledge, in more complex functions and processes. For example, suppose an air-traffic controller is given the two flight strips in Figure 23. Commercial aircraft fly from one fix point to another. These two aircraft are flying at the same level (31,000 ft) from fix OTK to fix LEESE7. DAL1152 is estimated to arrive at LEESE7 two minutes after AAL419 (18-16), and is travelling faster (783-746). So DAL1152 is closing on AAL419 relatively fast and the controller needs to take immediate action, to tell one of the aircraft to change flight level. The person telling the aircraft to change level is doing more than simply recoding the given information. They use strategies for searching the displays and for comparing the data about the two aircraft, plus a simple dynamic model of how an aircraft changes position in time, to build up a mental picture of the relative positions of the aircraft, with one overtaking the other so a collision is possible. They then use a strategy for optimising the choice of which aircraft to instruct to change.
The overall cognitive functions or goals involved are :
- to understand what is happening,
- and to plan what to do about it.
In complex dynamic tasks these two main cognitive needs are met by subsidiary cognitive functions such as :
- infer/ review present state.
- predict/ review future changes/ events.
- review/ predict task performance criteria.
- evaluate acceptability of present or future state.
- define sub-tasks (task goals) to improve acceptability.
- review available resources/ actions, and their effects.
- define possible (sequences of) actions (and enabling actions) and predict their effects.
- choose action / plan.
- formulate execution of action plan (including monitor the effects of actions, which may involve repeating all the above).
These cognitive functions are interdependent. They are not carried out in a fixed order but are used as necessary. Lower level cognitive functions implement higher level ones. At the lowest levels, the functions are fulfilled by cognitive processes such as searching for the information needed, discrimination, integration, and recoding. So the processing is organised within the structure of cognitive goals/ functions.
An overview is built up in working storage by carrying out these functions. This overview represents the person's understanding of the current state of the task and their thinking about it. The overview provides the data the person uses in later thinking, as well as the criteria for what best to do next and how best to do it. There is a cycle : processing builds up the overview, which determines the next processing, which updates the overview, and so on, see Figure 24. Figure 22 [previous section] showed an alternative representation of context, as nested rather than cyclic. (For more about this mechanism, see Bainbridge 1993a.)
Figure 24 : The cycle of processing. Cognitive activity builds up the person's understanding and planning in working storage [top left circle]. The contents of working storage then affect the choice of the next best behaviour to carry out, and how to do it, and so on.
The main cognitive processes discussed in the previous section were decision making, integrating stimuli, and recoding. Additional modes of processing are needed in complex tasks, such as :
- carrying out a sequence of recoding transformations, and temporarily storing intermediate results in working memory.
- building up a structure of inference, an overview of the current state of understanding and plans, in working storage, using a familiar working method.
- using working storage to mentally simulate carrying out a cognitive or physical strategy.
- deciding between alternative working methods on the basis of meta-knowledge.
- planning and multi-tasking.
- developing new working methods.
These complex cognitive processes are not directly observable. The classic experimental psychology method, which aims to control all except one or two measured variables, and to vary one or two variables so their effects can be studied, is well suited to investigating discrimination and recoding processes. It is not well suited to investigating cognitive activities in which many inter-related processes may occur without any observable behaviour.
Studying these tasks involves special techniques : case studies, videos, verbal protocols, or distorting the task in some way, perhaps slowing it down or making the person do extra actions to get information (Wilson and Corlett, 1995). Both setting up and analysing the results of such studies can take years of effort. The results tend to be as complex as the processes studied, so they are difficult to publish in the usual formats. Such studies do not fit well into the conventions about how research is done, so there are unfortunately not many of this type. However, the rest of this section will give some evidence about the nature of complex cognitive processes, to support the general claims made so far. The sub-sections are on : sequences; language understanding; inference and diagnosis; working storage; planning, multi-tasking, and problem solving; and knowledge.
A. Sequences of transforms
After decision making, integrating and recoding, the next level of complexity in cognitive processing is carrying out a sequence of recoding translations or transforms. The result of one step in the sequence acts as the input to the next step, and so has to be kept temporarily in working memory. Here the notion of recoding needs to be expanded to include transforms such as simple calculations and comparisons, and conditions leading to alternative sequences.
Note that in this type of processing the goal of the behaviour, the reason for doing it, is not included in the description of how it is done. Some people call this type of processing 'rule based'. There are two typical working situations in which behaviour is not structured relative to goals.
When a person is following instructions which do not give them any reason for why they have to do each action, then they are using this type of processing. This is usually not a good way of presenting instructions as, if anything goes wrong, the person has no reference point for identifying how to correct the problem.
The second case can arise in a stable environment, in which behaviour can be done in the same way each time. If a person has practised often, the behaviour may be done without needing to check it, or to think out what to do or how to do it (see below). Such over-learned sequences give a very efficient way of behaving, in the sense of using minimal cognitive effort. But if the environment does change, then over-learning is maladaptive and can lead to errors (see next section on learning and errors).
B. Language processing
This section will cover two issues : using language to convey information and instructions, and the processes involved in language understanding. Although language understanding is not the primary task of either pilot or air traffic controller, it does provide simple examples of some key concepts in complex cognitive processing.
1. Written instructions
Providing written instructions is often thought of as a way of making a task easy, but this is not guaranteed. Reading instructions involves interpreting the words in order to build up a plan of action. The way the instructions are written may make this processing more or less difficult. Video recorder operating manuals are notorious for this.
Various techniques have been used for measuring the difficulty of processing different sentence types. Some typical results are (Savin and Perchonock, 1965) :
Such data suggest that understanding negatives and passive involves two extra and separate processes. This suggests it is best in general to use active positive forms of sentence. But when a negative or restriction is the important message, it should be the most salient and come first. For example, 'No smoking' is more effective than 'Smoking is not permitted'. Though using a simple form of sentence does not guarantee that a message makes good sense. I recently enjoyed staying in a hotel room with a notice on which the large letters said:
Do not use the elevator during a fire.
Read this notice carefully.
Connected prose is not necessarily the best format for showing alternatives in written instructions. Spatial layout can be used to show the groupings and relations between phrases, by putting each phrase on a separate line, by indenting to show items at the same level, and by using flow diagrams to show the effect of choice between alternatives ( e.g. Oborne, 1995, Chapter 4). When spatial layout is used to convey meaning in written instructions, it is a code and should be used consistently, as discussed in the previous section.
Instructions also need to be written from the point of view of the reader : 'if you want to achieve this, then do this'. Instruction books are often written the other way round : 'if you do this, then this happens'. The second approach requires from the reader much more understanding, searching and planning to work out what to do. Note that the effective way of writing instructions is goal oriented. In complex tasks, methods of working are in general best organised in terms of what is to be achieved; this will be discussed again below.
2. Language understanding
In complex tasks, many of the cognitive processes and knowledge used are only possible because the person has considerable experience of the task.
Language understanding is the chief complex task studied by experimental psychologists (e.g. Ellis, 1993), because it is easy to find experts to test. Some examples illustrate processes which are also used in other complex tasks.
When someone is listening to or reading language, each word evokes learned expectations. For example :
The : can only be followed by : a descriptor, or a noun.
The pilot : depending on the context, either :
1. will be followed by the word 'study', or :
2. a. evokes general knowledge (scenarios) about aircraft or ship pilots.
2. b. can be followed by :
2. b. i. a descriptive clause, containing items relevant to living things/ animals/ human beings/ pilots, or
2. b. ii. a verb, describing possible actions by pilots.
Each word leads to expectations about what will come next, each constrains the syntax (grammar) and semantics (meaning) of the possible next words. To understand the language, a person needs to know the possible grammatical sequences, the semantic constraints on what words can be applied to what types of item, and the scenarios. During understanding, a person's working storage contains the general continuing scenario, the structure of understanding built up from the words received so far, and the momentary expectations about what will come next. (Many jokes depend on not meeting these expectations.)
The overall context built up by a sequence of phrases can be used to disambiguate alternative meanings, such as :
The Inquiry investigated why
the pilot turned into a mountain.
or : In this fantasy story
the pilot turned into a mountain.
The knowledge base/ scenario is also used to infer missing information. For example :
The flight went to Moscow.
The stewardess brought her fur hat.
Answering the question 'Why did she bring her fur hat ?' involves knowing that stewardesses go on flights and about the need for and materials used in protective clothing, which are not explicitly mentioned in the information given.
Understanding language does not necessarily depend on the information being presented in a particular sequence. Although it requires more effort, we can understand someone whose first language uses a different word order from English, such as :
The stewardess her fur hat brought.
We do this by having a general concept that a sentence consists of several types of unit (noun phrases, verb phrases, and so on) and we make sense of the input by matching it to the possible types of unit. This type of processing can be represented as being organised by a 'frame with slots', where the frame co-ordinates the slots for the types of item expected, which are then instantiated in a particular case, as in :
|
|
Noun phrase Verb Noun phrase
The stewardess brought her fur hat
(As language has many alternative sequences, this is by no means a simple operation, Winograd 1972.)
The understanding processes used in complex control and operation tasks show the same features that are found in language processing.
The information obtained evokes both general scenarios and specific moment to moment expectations. The general context, and additional information, can be used to decide between alternative interpretations of the given information. A structure of understanding is built up in working storage. Frames or working methods suggest the types of information the person needs to look for to complete their understanding. These items can be obtained in a flexible sequence. And knowledge is used to infer whatever is needed to complete the understanding but which is not supplied by the input information.
There is an important addition in control/ operation tasks, which is that the structure of understanding is built up in order to influence the state of the external world, to try to get it to behave in a particular way.
C. Inference and diagnosis
To illustrate these cognitive processes in an aviation example, this section uses an imaginary example so the presentation can be short. Later sections describe real evidence on pilot and air traffic controller behaviour which justifies the claims made here.
Suppose that an aircraft is in flight and the 'engine oil low' light goes on. What might be the pilot's thoughts ? The pilot needs to infer the present state of the aircraft (cognitive functions here are indicated by italics). This involves considering alternative hypotheses which could explain the light, such as that there is an instrument fault, or there genuinely is an engine fault, and then choosing between the hypotheses according to their probability (based on previous experience of this or other aircraft) or by looking for other evidence which would confirm or disprove the possibilities. The pilot could predict the future changes which will occur as a result of the chosen explanation of events. Experienced people's behaviour in many dynamic tasks is future oriented. A person takes anticipatory action, not to correct the present situation, but to ensure that predicted unacceptable states or events do not occur. Before evaluating the predictions for their acceptability, the pilot needs to review the task performance criteria, such as the relative importance of arriving at the original destination quickly, safely or cheaply. The result of comparing the predictions with the criteria will be to define the performance needs to be met. It is necessary to review the available resources, such as the state of the other engines or the availability of alternative landing strips. The pilot can then define possible alternative action sequences and predict their outcomes. A review of action choice criteria, which includes the task performance criteria plus others such as the difficulty of the proposed procedures, is needed as a basis for choosing an action sequence / plan, before beginning to implement the plan. Many of these cognitive functions must be based on incomplete evidence, for example about future events or the effects of actions, so risky decision making is involved.
A pilot who has frequently practised these cognitive functions may be able to carry them out 'automatically', without being aware of the need for intermediate thought. And an experienced pilot may not be aware of thinking about the functions in separate stages, for example (predict + review criteria + evaluation) may be done together.
Two modes of processing have been used in this example :
- 'automatic' processing (i.e. recoding),
- using a known working method which specifies what thinking to carry out.
Other modes of processing will be suggested later. The mode of processing needed to carry out a function depends on the task situation and the person's experience (see next section on learning). An experienced person's knowledge of the situation may enable them to reduce the amount of thinking they have to do, even when they do need to think things out explicitly. For example, it may be clear early in the process of predicting the effects of possible actions that some will be not acceptable and so need not be explored further (see below on planning).
Nearly all the functions and processing mentioned have been supplied from the pilot's knowledge base. In the example, the warning light evoked working methods for explaining the event and for choosing an action plan, as well as knowledge about the alternative explanations of events and suggestions of relevant information to look for. The combination of (working method + knowledge referred to in using this method + mental models for predicting events) is the scenario. Specific scenarios may be evoked by particular events, or by particular phases of the task (phases of the flight).
This account of the cognitive processes is goal oriented. The cognitive functions or goals are the means by which the task goals are met, but are not the same as them. Task and personal goals act as constraints on what it is appropriate and useful to think about when fulfilling the cognitive goals.
The cognitive functions and processing build up a structure of data (in working storage) which describes :
- the present state and the reasons for it,
- predicted future changes,
- task performance and action choice criteria,
- resources available,
- possible actions,
- evaluations of the alternatives,
- the chosen action plan.
This data structure is an overview which represents the results of the thinking and deciding done so far, and provides the data and context for later thinking. As an example, the result of reviewing task performance criteria is not only an input to evaluation, it could also affect what is focused on in inferring the present state, or in reviewing resources, or in action choice. The overview ensures that behaviour is adapted to its context.
The simple example above described reaction to a single unexpected event. Normally flying and air traffic control are ongoing tasks. For example, at the beginning of shift an air traffic controller has to build up their understanding of what is happening and what actions are necessary, from scratch. After this, each new aircraft which arrives is fitted into the controller's ongoing mental picture of what is happening in the airspace; the thinking processes do not start again from the beginning. Aircraft usually arrive according to schedule and are expected, but the overview needs to be updated and adapted to changing circumstances (see below on planning and multi-tasking).
There are two groups of practical implications of these points.
One is that cognitive task analysis should focus on the cognitive functions involved in a task, rather than simply pre-specifying the cognitive processes by which they are met.
The second is that designing specific displays for individual cognitive functions may be unhelpful. A person doing a complex task meets each function within an overall context, the functions are interdependent, and the person may not think about them in a pre-specified sequence. Giving independent interface support to each cognitive function, or sub-task within a function, could make it more difficult for the person to build up an overview which interrelates the different aspects of their thinking.
Diagnosis
The most difficult cases of inferring what underlies the given evidence may occur during fault diagnosis. A fault may be indicated by a warning light or, for an experienced person, by a device not behaving according to expectations. Like any other inference, fault diagnosis can be done by several modes of cognitive processing, depending on the circumstances. If a fault occurs frequently, and has unique symptoms, it may be possible to diagnose the fault by visual pattern recognition, i.e. pattern on interface indicates fault identity (e.g. Marshall, Scanlon, Shepherd and Duncan, 1981). This is a type of recoding. But diagnosis can also pose the most difficult issues of inference, for example by reasoning based on the physical or functional structure of the device (e.g. Hukki and Norros, 1993).
In-flight diagnosis may need to be done at speed. Experienced people can work rapidly using 'recognition primed decisions', in which situations are assigned to a known category with a known response, on the basis of similarity. The processes involved in this are discussed by Klein (1989). The need for rapid processing emphasises the importance of training for fault diagnosis.
Amalberti (1992, Expt. 4) studied fault diagnosis by pilots. Two groups of pilots were tested, one group were experts on the Airbus, the other group were experienced pilots beginning their training on the Airbus. They were asked to diagnose two faults specific to the Airbus, and two general problems. In 80% of responses, the pilots gave only one or two possible explanations. This is compatible with the need for rapid diagnosis. Diagnostic performance was better on the Airbus faults, which the pilots had been specifically trained to watch out for, than on the more general faults.
One of the general problems was a windshear on take-off. More American than European pilots diagnosed this successfully. American pilots are more used to windshear as a problem, so are more likely to think of this as a probable explanation of an event. People's previous experience is the basis for the explanatory hypotheses they suggest.
In the second general fault there had been an engine fire on take-off, during which the crew forgot to retract the landing gear, which made the aircraft unstable when climbing. Most of the hypotheses suggested by the pilots to explain this instability were general problems with the aircraft, or were related to the climb phase. Amalberti suggested that when the aircraft changed the phase of flight, from take-off to climb, the pilots changed their scenario providing the appropriate events, procedures, mental models and performance criteria for use in thinking. Their knowledge about the previous phase of flight became less accessible, and so was not used in explaining the fault.
D. Working Storage
The inference processes build up the contextual overview or situation awareness in working storage. This is not the same as short-term memory, but short-term memory is an important limit to performance, and will be discussed first.
1. Short-term memory
Figure 25 shows some typical data on how much is retained in short-term memory after various time intervals. Memory decays over about 30 seconds, and is worse if the person has to do another cognitive task before being tested on what they can remember.
Figure 25 : Decay in short-term memory over a short period of time. Memory decays more if there is a more complicated other task to be done while the item is in memory (adapted from Posner and Rossman, 1965).
This memory decay is important in the design of computer based display systems in which different display formats are called up in sequence on a screen. Suppose the user has to remember an item from one display, for use with an item on a second display. Suppose that the second display format is not familiar, so the person has to search for the second item : this search may take about 25 seconds. The first item must then be recalled after doing the cognitive processes involved in calling up the second display and searching it. The memory data suggest that the person may have forgotten the first item on 30% of occasions.
The practical implication is that, to avoid this source of errors, it is necessary to have sufficient display area so that all the items used in any given cognitive processing can be displayed simultaneously. Minimising non-task-related cognitive processes is a general HF/E aim, to increase processing efficiency. In this case it is also necessary in order to reduce errors. This requirement emphasises the need to identify what display items are used together, in a cognitive task analysis.
2. The overview in working storage
Although there are good reasons to argue that the cognitive processes in complex dynamic tasks build up a contextual overview of the person's present understanding and plans (Bainbridge 1993a), not much is known about this overview. This section will make some points about its capacity, its content, and the way items are stored.
Figure 26 : Number of aircraft remembered, and number of items remembered about each aircraft, by air-traffic controllers with three different levels of experience and at three different levels of workload (data in personal communication from Bisseret, based on Bisseret, 1970).
Capacity
Bisseret (1970) asked air traffic area controllers, after an hour of work, what they remembered about the aircraft they had been controlling. Three groups of people were tested : trainee controllers, people who had just completed their training, and people who had worked as controllers for several years. Figure 26 shows the number of items recalled. The experienced controllers could remember on average 33 items. This is a much larger figure than the 7+/-2 chunk capacity for static short-term memory (Miller 1956) or the 2 items capacity of running memory for arbitrary material (Yntema and Mueser, 1962). Evidently a person's memory capacity is improved by doing a meaningful task and by experience. A possible reason for this will be given below.
Content
Bisseret also studied which items were remembered. The most frequently remembered items were flight level (33% of items remembered), position (31%) and time at fix (14%). Leplat and Bisseret (1965) had previously identified the strategy the controllers used in conflict identification (checking whether aircraft are a safe distance apart). The frequency with which the items were remembered matches the sequence in which they were thought about : the strategy first compared aircraft flight levels, then position, then time at fix, and so on.
Figure 27 : Number of items remembered about aircraft with different status for the air-traffic controller (data from Sperandio, 1970).
[e.g. top : For a/c not in conflict with others, and not yet in radio contact, the controllers remembered 1 item about more than 40% of the a/c, 3 items about another 20% of the a/c.
e.g. bottom : For a/c in conflict with others, for which the controllers had decided what to do but not yet made the action, the controllers remembered more than 6 items about more than 50% of the a/c in this category. While, for a/c which they have already done something about, they remember 4 or 5 items about 80% of those.]
Sperandio (1970) studied another aspect (Figure 27). He found that more items were remembered about aircraft involved in conflict than ones which were not. For non-conflict aircraft, more was remembered about aircraft which had been in radio contact. For conflict aircraft, more was remembered about aircraft on which action had been taken, and most was remembered about aircraft for which an action had been chosen but not yet made.
These results might be explained by two classic memory effects.
One is the rehearsal or repetition mechanism by which items are maintained in short-term memory. The more frequently the item or aircraft has been considered by the controllers when identifying potential collisions and acting on them, the more likely it is to be remembered.
The findings about aircraft in conflict could be explained by the recency effect, that items which have been rehearsed most recently are more likely to be remembered.
These rehearsal and recency mechanisms make good sense as mechanisms for retaining material in real as well as in laboratory tasks.
The form in which material is retained
The controllers studied by Bisseret op cit remembered aircraft in pairs or threes : 'there are two flying towards DIJ, one at level 180, the other below at 160', 'there are two at level 150, one passed DIJ towards BRY several minutes ago, the other should arrive at X at 22', or 'I've got one at level 150 which is about to pass RLP and another at level 170 which is about 10 min behind'. The aircraft were not remembered by their absolute positions, but in relation to each other. Information was also remembered relative to the future, many of the errors put the aircraft too far ahead.
These sorts of data suggest that, while rehearsal and recency are important factors, the items are not remembered simply by repeating the raw data, as in short term memory laboratory experiments. What is remembered is the outcome of working through the strategy for comparing aircraft for potential collisions. The aircraft are remembered in terms of the key features which bring them close together, whether they are at the same level, or flying towards the same fix point, etc.
A second anecdotal piece of evidence is that air traffic controllers talk about 'losing the picture' as a whole, not piecemeal. This implies that their mental representation of the situation is an integrated structure. It is possible to suggest that experienced controllers remember more because they have better cognitive skills for recognising the relations between aircraft, and the integrated structure makes items easier to remember.
The only problem with this integrated structure is that the understanding, predictions and plans can form a 'whole' which is so integrated and self-consistent that it becomes too strong to be changed. People may then only notice information which is consistent with their expectations, and it may be difficult to change the structure of inference if it turns out to be unsuccessful or inappropriate (this rigidity in thinking is called 'perceptual set').
Some practical implications
Some points have already been made about the importance of short-term memory in display systems. The interface also needs to be designed to support the person in developing and maintaining their overview. It is not yet known whether an overview can be obtained directly from an appropriate display, or whether the overview can only be developed by actively understanding and planning the task, with a good display enhancing this processing but not replacing it. It is important, in display systems in which the data needed for the whole task are not all displayed at the same time, to ensure there is a permanent overview display and it is clear how the other possible displays are related to it.
Both control automation (replacing the human controller) and cognitive automation (replacing the human planner, diagnoser, and decision maker) can cause problems with the person's overview. A person who is expected to take over manual operation or decision making will only be able to make informed decisions about what to do after they have built up an overview of what is happening. This may take 15-30 minutes to develop. So the system design needs to allow for this sort of delay before a person can take over effectively (Bainbridge, 1983). Also the data above show that a person's ability to develop a wide overview depends on experience. This means that, to be able to take over effectively from an automated system, they need to practise building up this overview. So practise opportunities should be allowed for, in the allocation of functions between computer and person, or in other aspects of the system design such as refresher training.
E. Planning, multi-tasking and problem solving
Actions in complex dynamic tasks are not simple single units. A sequence of actions may be needed, and it may be necessary to deal with several responsibilities at the same time. Organisation of behaviour is an important cognitive function, which depends on and is part of the overview. This section will be in three inter-related parts, on : planning future sequences of action; multi-tasking - dealing with several concurrent responsibilities including sampling; and problem solving - devising a method of working when a suitable one is not known.
1. Planning
It may be more efficient to think out what to do in advance - if there is a sequence of actions to carry out, or multiple constraints to satisfy, or it would be more effective to anticipate events. Alternative actions can be considered and the optimum ones chosen, and the thinking is not done under time pressure. The planning processes may use working storage, for testing the alternatives by mental simulation, and for holding the plan as part of the overview.
In aviation, an obvious example is preflight planning. Civilian pilots plan their route in relation to predicted weather. Military pilots plan their route relative to possible dangers and the availability of evasive tactics. In high speed low level flight there is not time to think out what to do during the flight, so the possibilities need to be worked out beforehand. The plan then needs to be implemented, and adjusted if changes in circumstances make this necessary. So this section will be in two parts, on preplanning and on-line revision of plans.
Figure 28 : Stages of planning by pilots (translated with permission from Amalberti, 1992). This figure shows the differences between novice and expert pilots.
[X axis : time during planning, Y axis : type of planning activity. Continuous lines on right show how 4 pilots in each group changed from one planning activity to another over time.]
Preplanning
Figure 28 shows results from a study of preflight planning by Amalberti (1992, Expt. 2). Pilots thought out the actions to take at particular times or geographical points. Planning involves thinking about several alternative actions, and choosing the best compromise given several constraints. Some of the constraints the pilots consider are the level of risk of external events, the limits to manoeuvrability of the aircraft, and their level of expertise to deal with particular situations, as well as the extent to which the plan can be adapted, and what to do if circumstances mean that major changes in plan are needed.
Amalberti studied 4 novice pilots, who were already qualified but at the beginning of their careers, and 4 experts. The cognitive aims considered during planning are listed on the left of Figure 28. Each line on the right represents one pilot, and shows the sequence in which he thought about the cognitive functions.
The results show that novice pilots took longer to do their planning, and that each of the novice pilots returned to reconsider at least one point he had thought about earlier. Verbal protocols collected during the planning showed that novices spent more time mentally simulating the results of proposed actions to explore their consequences.
The experts did not all think about the cognitive functions in the same sequence, but only one of them reconsidered an earlier point. Their verbal protocols showed they prepared fewer responses to possible incidents than the novices.
One of the difficulties with planning is that later in planning the person may think of problems which mean that parts of the plan already devised need to be revised. Planning is an iterative process. The topics are interdependent, for example the possibility of incidents may affect the best choice of route to or from the objective. What is chosen as the best way of meeting any one of the aims may be affected by, or affect, the best way of meeting other aims. As the topics are interdependent, there is no one optimum sequence for thinking about them. The results suggest that experts have the ability, when thinking about any one aspect of the flight, to take into account its implications for other aspects, so it does not need to be revised later.
The experts have better knowledge about the scenario, about possible incidents and levels of risk. They know more about what is likely to happen, so they need to prepare fewer alternative responses to possible incidents. The experts also know from experience the results of alternative actions, including the effects of actions on other parts of the task, so they do not need to mentally simulate making the actions to check their outcomes. They also have more confidence in their own expertise to deal with given situations. All these are aspects of their knowledge about the general properties of the things they can do, how risky these are, how good they are at them, and so on. This 'meta knowledge' was introduced in the section above on actions, and is also central to multi-tasking and in workload and learning (see next section).
On-line adaptation of plans
In the second part of Amalberti's study, the pilots carried out their mission plan in a high fidelity simulator. The main flight difficulty was that they were detected by radar. The pilots responded immediately to this. The response had been preplanned, but had to be adapted to details of the situation when it happened. The novice pilots showed much greater deviations from their original plan than the experts. Some of the young pilots slowed down before the point at which they expected to be detected, as accelerating was the only response they knew for dealing with detection. This acceleration led to a deviation from their planned course, so they found themselves in an unanticipated situation. They then made a sequence of independent reactive short-term decisions, because there was not time to consider the wider implications of each move. The experts made much smaller deviations from their original plan, and were able to return to the plan quickly. The reason for this was that they had not only preplanned their response to radar, they had also thought out in advance how to recover from deviations from their original plan. Again experience, and therefore training, plays a large part in effective performance.
In situations in which events happen less quickly, people may be more effective in adapting their plans to changing events at the time. The current best model for the way that people adapt their plans to present circumstances is probably the opportunistic planning model of Hayes-Roth and Hayes-Roth (1979, see also Hoc, 1988).
2. Multi-tasking
If a person has several concurrent responsibilities, each of which involves a sequence of activities, then inter-leaving these sequences is called multi-tasking. Doing this involves an extension of the processes mentioned under planning. Multi-tasking involves working out in advance what to do, combined with opportunistic response to events and circumstances at the time.
Figure 29 : Multi-tasking by a pilot during one phase of a mission (translated with permission from Amalberti, 1992). This figure shows that the activities are inter-leaved.
[Above - hierarchy of tasks and sub-tasks.
Below - each horizontal line represents a particular activity, double headed arrows show the times during which the pilot did that activity. Sub-tasks which he returns to several times : motor safety, slope control, communications with AirTrafficControl.]
Examples of multi-tasking
Amalberti (1992, Expt.1) studied military pilots during a simulated flight. Figure 29 shows part of his analysis, of one pilot's activities during descent to low level flight. The bottom line in this Figure is a time line.
The top part of the figure describes the task as a hierarchy of task goals and sub-goals.
The parallel double-headed arrows beneath represent the time which the pilot spent on each of the activities. These arrows are arranged in five parallel lines which represent the five main tasks in this phase of flight : maintain engine efficiency at minimum speed; control angle of descent; control heading; deal with air traffic control; and prepare for the next phase of flight.
Figure 29 shows how the pilot allocated his time between the different tasks.
Other principal tasks which occurred in other phases of flight were : keep to planned timing of manoeuvres; control turns; check safety.
Sometimes it is possible to meet two goals with one activity. The pilot does not necessarily complete one sub-task before changing to another. Indeed this is not often not possible in a control task, in which states and events develop over time. Usually the pilot does one thing at a time. However, it is possible for him to do two tasks together when they use different cognitive processing resources. For example, controlling descent, which uses eyes + motor co-ordination, can be done at the same time as communicating with air traffic control, which uses hearing + speech (see also below on workload - in the third section of this chapter).
Some multi-tasking examples are difficult to describe in a single figure. For example, Reinartz (1989), studying a team of three nuclear power plant operators, found they might work on nine to ten different goals at the same time. Other features of multi-tasking have been observed by Benson (1990):
- Multi-tasking may be planned ahead (a process operator studied by Beishon, 1974, made plans for up to 1.5 hours ahead). These plans are likely to be partial, and incomplete in terms of timing and detail. Planned changes in activity may be triggered by times or events. When tasks are done frequently, much of the behaviour organisation may be guided by habit.
- Executing the plan. Interruptions may disrupt planned activity. The preplan is incomplete, and actual execution depends on details of the situation at the time. Some tasks may be done when they are noticed in passing (Beishon op cit first noticed this, and called it serendipity.) This is opportunistic behaviour. The timing of activities of low importance may not be preplanned, but may be fitted in spare moments. The remaining spare moments are recognised as spare time.
- Effects of probabilities and costs. In a situation which is very unpredictable, or when the cost of failure is high, people may make the least risky commitment possible. If there is a high or variable workload, people may plan to avoid increasing their workload, and use different strategies in different workload conditions (see below on workload - in the third section of this chapter).
A possible mechanism
Sampling is a simple example of multi-tasking in which people have to monitor several displays to keep track of changes on them. Mathematical sampling theory has been used as a model for human attention in these tasks. In the sampling model, the frequency of attending to an information source is related to the frequency of changes on that source. This can be a useful model of how people allocate their attention when changes to be monitored are random, as in straight and level flight, but this model is not sufficient to account for switches in behaviour in more complex phases of flight.
Amalberti op cit made some observations about switching from one task to another. He found that :
- Before changing to a different principal task the pilots review the normality of the situation, by checking that various types of redundant information are compatible with each other.
- Before starting a task that will take some time, they ensure that they are in a safe mode of flight. For example, before analysing the radar display, they check that they are in the appropriate mode of automatic pilot.
- While waiting for feedback about one part of the task, pilots do part of another task, which they know is short enough to fit into the waiting time.
- When doing high risk high workload tasks, pilots are less likely to change to another task.
These findings suggest that, at the end of a sub-section of a principal task, the pilots check that everything is alright. They then decide (not necessarily consciously) what next to devote effort to, by combining their preplan with meta-knowledge about the alternative tasks, such as how urgent they are, how safe or predictable they are, how difficult they are, how much workload they involve, and how long they take (see below on workload - in third section of paper).
Practical implications
Multi-tasking can be preplanned, and involves meta-knowledge about alternative behaviours. Both planning and knowledge develop with experience, which underlines the importance of practice and training.
The nature of multi-tasking also emphasises the difficulties which could be caused by task specific displays. If a separate display was used for each of the tasks combined in multi tasking, then the user would have to call up a different display, and perhaps change coding vocabularies, each time they changed to a different main task. This would require extra cognitive processing and extra memory load, and could make it difficult to build up an overview of the tasks considered together. This suggests an extension to the point made in the section above on working storage. All the information used in all the principal tasks which may be interleaved in multi-tasking needs to be available at the same time, and easily cross-referenced. If this information is not available, then co-ordination and opportunistic behaviour may not be possible.
3. Problem solving
A task is familiar to a person who knows :
- the appropriate working methods,
- the associated reference knowledge about the states which can occur,
- the constraints on allowed behaviour,
- the scenarios, mental models, etc. which describe the environmental possibilities within which the working methods must be used.
Problem solving is the general term for the cognitive processes a person uses in an unfamiliar situation, which they do not already have an adequate working method or reference knowledge for dealing with. Planning and multi-tasking are also types of processing which are able to deal with situations which are not the same each time. However, both take existing working methods as their starting point, and either think about them as applied to the future, or work out how to interleave the working methods used for more than one task. In problem solving, a new working method is needed.
There are several ways of devising a new working method. Some are less formal techniques which do not use much cognitive processing, such as trial-and-error, or asking for help. There are also techniques which should not need much creativity, such as reading an instruction book.
People may otherwise use one of three techniques for suggesting a new working method. Each of these uses working methods recursively, it uses a general working method to build up a specific working method.
1. Categorisation. This involves grouping the problem situation with similar situations for which a working method is available. The working method which applies to this category of situation can then be used. This method is also called 'recognition primed decision making'. The nature of 'similarity' and the decisions involved are discussed by Klein (1989).
2. Case-based reasoning. This involves thinking of a known event (a 'case') which is similar or analogous to the present one, and adapting the method used then for use in the present situation. This is the reason why stories about unusual events circulate within an industry. They provide people in the industry with exemplars for what they could do themselves if a similar situation arose, or with opportunities to think out for themselves what would be a better solution.
3. Reasoning from basic principles. In the psychological literature, the term 'problem solving' may be restricted to a particular type of reasoning in which a person devises a new method of working by building it up from individual components (e.g. Eysenck and Keane, 1990, Chapters 11, 12). This type of processing may be called 'knowledge based' by some people.
A general problem solving strategy consists of a set of general cognitive functions, which have much in common with the basic cognitive functions in complex dynamic tasks (see introduction to this section). Problem solving for example could involve understanding the problem situation, defining what would be an acceptable solution, and identifying what facilities are available. Meeting each of these cognitive needs can be difficult, because the components need to be chosen for their appropriateness to the situation and then fitted together. This choice could involve : identifying what properties are needed from the behaviour; searching for components of behaviour which have the right properties (according to the meta-knowledge which the person has about them); and then combining them into a sequence.
The final step in developing a new working method is to test it, either by mental simulation, or by trial-and-error. This mental simulation could be similar to the techniques used in planning and multi-tasking. So working storage may be used in problem solving in two ways : to hold both the working method for building up a working method and the proposed new method; and to simulate carrying out the proposed working method to test whether its processing requirements and outputs are acceptable.
F. Knowledge
Knowledge is closely involved in all modes of cognitive processing. Even in simple tasks, it provides the probabilities, utilities and alternatives considered in decision making, and the translations used in recoding. In complex tasks it provides the working methods and reference knowledge used in thinking about cognitive functions, and the meta-knowledge.
Different strategies may use different types of reference knowledge. For example, a strategy for diagnosing faults by searching the physical structure of the device uses one type of knowledge, while a strategy which relates symptoms to the functional structure of the device uses another. The reference knowledge may include scenarios, categories, cases, mental models, performance criteria, and other knowledge about the device the person is working with. Some knowledge may be used mainly for answering questions, for explaining why events occur or actions are needed. This basic knowledge may also be used in problem solving.
There are many interesting fundamental questions about how these different aspects of knowledge are structured, inter-related and accessed (Bainbridge, 1993b), but these issues are not central to this chapter. The main questions here are the relation between the type of knowledge and how it can best be displayed, and what might be an optimum general display format.
Knowledge and representation.
Any display for a complex task can show only a sub-set of what could be represented. Ideally, the display should make explicit the points which are important for a particular purpose, and provide a framework for thinking. The question of which display format is best for representing what aspect of knowledge has not yet been thoroughly studied, and most of the recommendations about this are assumptions based on experience (Bainbridge, 1988). For example, the following formats are often found :
Figure 30 : Relations between different types of knowledge in a small part of a pilot's knowledge base (adapted from Bainbridge, 1991).
'Part-of' relations, 'is-a/category'' relations, and cause-effect relations (shown by the thick arrows), are all interrelated.
Knowledge-display recommendations raise three sorts of question.
One arises because each aspect of knowledge is one possible 'slice' from the whole body of knowledge. All the types of knowledge are interrelated, but there is not a simple one-to-one relation between them. Figure 30 illustrates some links between different aspects of knowledge. Any strategy is unlikely to use only one type of knowledge, or to have no implications for aspects of thinking which use other types of knowledge. It might mislead the user to show different aspects of knowledge with different and separate displays which are difficult to cross-refer between, as this might restrict the thinking about the task. Knowledge about cross links is difficult to display, and is gained by experience. This emphasises training.
A second question is concerned with salience. Visual displays emphasise (make more salient) the aspects which can easily be represented visually. (For example, see the discussion at the end of the third section of this paper on the limitations of Figures 22 and 24 as models of behaviour.) It might be unwise to make some aspects of knowledge easy to take in simply because they are easier to display, rather than because they are important in the task. There are vital types of knowledge which are not easy to display visually, such as the associations used in recoding, or the categories, cases, scenarios, and meta-knowledge used in complex thinking. These are all learned by experience. The main approach to supporting non-visual knowledge is to provide the user with reminder lists about the alternatives (see below on cued recall). Display design and training are interdependent, as they are each effective at providing different types of knowledge. It could be useful to develop task analysis techniques which identify different aspects of knowledge, as well as to do more research on how types of knowledge, and the links between them, can best be presented.
The third issue about all these multiple possible display formats repeats the questions raised previously about efficient use of codes. If a user was given all the possible display types listed above, each of which would use different codes, possibly with different display formats using the same code with different meanings (for example a network with nodes could be used to represent physical, functional or hierarchical relations between items), the different codes might add to the user's difficulties in making cross connections between different aspects of knowledge.
An optimum format ? [cued recall]
These issues suggest the question : is there one or a small number of formats which subsume or suggest the others ? This is a question which has not yet been much studied.
A pilot study (Brennan, 1987) asked people to explain an event, given either a mimic or a cause-effect diagram of the physical device involved. The people tested either did or did not already know how the device worked. The results suggested that people who did not know how the device worked were most helped by a cause-effect representation (which does show how it worked), while experts were best with the mimic representation.
Contextual cues can greatly aid memory performance (e.g. Eysenck and Keane, 1990, Chapter 6). A cue is an aid to accessing the items to be recalled. The reason for expert performance with mimic displays might be that the icons and flow links on this type of display not only give direct evidence about the physical structure of the device, they also act as cues to or reminders about other knowledge the person has about the device - they evoke other parts of the scenario. This is an example from only one type of cognitive task, but it does point to the potential use of contextual cued recall in simplifying display systems. Cued recall can however only be effective with experienced people, who can recognise the cues and know what they evoke.
Other sections of this chapter :
III. Workload, learning, errors
Access to other papers through the Home page
©2021 Lisanne Bainbridge
Comments
Post a Comment