Processes underlying human performance : III. Mental workload, learning, errors
This is the third of three sections of a chapter on cognitive processes underlying complex tasks. The sections were not written to be read independently :
Topics in this section :
A. Mental Workload
1. Single or multi-channel processing : focussed attention; parallel processing.
2. Factors influencing processing capacity : capacities of different processing resources; extrinsic and intrinsic stressors; individual differences; practical implications.
3. Response to overload : increasing efficiency; changing strategy; practical implications.
B. Learning
1. Changes within a mode of processing : physical skills; perceptual skills ; recodings; familiar working methods; developing new methods.
2. Learning processes : repetition; meta-knowledge and feedback; independent goals-means; change to another mode of processing.
3. Training implications : simple processes, complex processes; training as part of system design.
C. Difficulties and errors [brief]
Conclusion : integrating concepts; difficulty of HF/E; modelling.
Processes underlying human performance : III. Mental workload, learning, errors
Lisanne Bainbridge
Department of Psychology, University College London
1995
Published in Garland, D.J., Hopkin, V.D. and Wise, J.A. (eds) Aviation Human Factors, Erlbaum, November 1998.
Workload, learning, and errors are all aspects of the efficiency of cognitive processing. There are limits to human processing capacities but these are difficult to define, because of the adaptability of human behaviour. As a result of learning, processing becomes more efficient and adapted to what is required. As efficiency increases, so mental workload may decrease. Error rates can be affected by both expertise and workload, and errors are closely involved in the processes of learning. There is a huge wealth of material which could be discussed, so the aim here is only to give a brief survey.
A. Mental Workload
There is a large number of issues involved in accounting for mental workload and how it is affected by different aspects of a task. This section will mention three main topics : whether people can only do one task at a time; factors affecting processing capacity; and the ways in which people typically respond to overload.
1. Single or multi-channel processing
Many types of evidence, including the example of multi-tasking in Figure 29 [Section II], show that people usually do one task at a time. This section will look at how people attend to one source of stimuli among many, and under what circumstances people can do more than one task at a time. As usual, the findings show how adaptable human beings are, and that there is not yet a full account of the processes involved.
Focused attention
People have the ability to pick out one message against a background of others, visual or auditory. Studies show however that a person does not only process one of the stimulus sources, but takes in enough about the other possible signals to be able to separate them. This chapter has already used the notion of 'depth' of processing, as in discrimination, recoding, sequences of recoding, and building up an overview. This notion is also involved here. Separation of two signal sources requires the least processing if they can be discriminated by physical cues, such as listening to a high voice while a low voice also speaks, or reading red lettering against a background of green letters. The sorts of factors discussed in Section I on discrimination affect how easy it is to do this separation. If stimuli cannot be distinguished by physical cues, then 'deeper' processing may be involved. For example, Gray and Wedderburn (1960) found that messages presented to the ears as :
left ear : mice 5 cheese
right ear : 3 eat 4
were heard as 354 mice eat cheese
In that case, the items might be grouped by recognising their semantic category.
In some tasks 'deeper' processing for meaning may be needed, that is, building up an overview.
Try reading the bold words in this passage [this was originally done using colour, which is more effective] :
It is important that the subject man be car pushed house slightly boy beyond hat his shoe normal candy limits horse of tree competence pen for be only in phone this cow way book can hot one tape be pin certain stand that snaps he with is his paying teeth attention in to the the empty relevant air task and hat minimal shoe attention candy to horse the tree second or peripheral task.
(from Lindsay and Norman, 1972)
Note that if the cue used becomes ineffective, this is disconcerting. It then takes time, and a search for clues about what would be effective, before the person can orient to a new cue and continue with the task. There is also an interplay of 'depths' of processing : when the physical cue becomes inadequate for following the message, then the reader uses continuity of meaning as a basis for finding a new physical cue. This account fits in with several points made earlier. The person is using active attention for what they want to take in, not passive reception of signals. The task setting provides the cue which can be used to minimise the effort needed to distinguish between signal sources. This cue then acts as a perceptual frame for searching for relevant inputs.
The concept of 'depth' of processing was first introduced by Craik and Lockhart (1972) to explain results in some memory experiments. The word 'depth' is in inverted commas here to distinguish it from depth in the organisation of behaviour, as in goal/ sub-goal, etc.
Parallel processing
The criteria defining whether or not people are able to do two tasks at the same time have so far proved elusive to identify.
Figure 16 [Section I] shows that, after high levels of practice, choice time is not affected by number of alternatives. Such tasks are said to be 'automated', or to require no 'conscious' attention. They can be done at the same time as something else, unless both tasks use the same peripheral resources such as vision or hand movement. Wickens (e.g. 1984) has done a series of studies showing that people can use different peripheral resources at the same time. People can also learn to do some motor tasks so that movements are monitored by feel rather than visually; then movements can be made at the same time as looking at or thinking about something else.
In practice the possibility of multiple processing means that care is needed in designing tasks. One might, for example, think it would reduce unnecessary effort for an air traffic controller to have the flight strips printed out, rather than expecting the controller to write the strips by hand. However if the controller, while writing, is simultaneously thinking out how the information fits into their overview, then printing the flight strips might deprive them of useful attention and thinking time.
Whether or not two tasks which both involve 'central' processing can be done at the same time is less clear. This is partly because what is meant by 'central' processing has not been clearly defined. People can do two tasks at the same time if the tasks are processed by different areas of the brain, for example a music task and a language task (Allport, Antonis and Reynolds, 1972), though both tasks need to be simple and perhaps done by recoding. Going to 'deeper' levels of processing, there does seem to be a limit to the extent to which people can build up distinct overviews for two different tasks at the same time. Whether or not an overview is needed to do a task may be part of the question. As a couple of anecdotal examples : people playing multiple chess games may have very good pattern recognition skills and so react to each game by recognition primed decisions as they return to it, rather than having to keep in mind a separate and continuing overview for each of the games they are playing. Most experienced drivers can drive and hold a conversation on a different topic at the same time, when the driving task is simple, but they stop talking when the driving task becomes more difficult.
This is an area in which it is challenging to identify the limits to performance, and it is probably beyond the competence of HF/E at the moment, either to define the concepts, or to investigate and measure the processing involved. Fortunately in practice the issue can often be simplified. When predicting performance, the conservative strategy is to assume that people cannot do two tasks at the same time. This will always be the worst case performance.
2. Factors influencing processing capacity
The amount of mental work a person can do in a given time is not a simple quantity to specify. If it is assumed that a person can only do one thing at a time, then every factor which increases the time taken to do a unit task will decrease the number of those tasks which can be done in a given time interval, and so decrease performance capacity. So every factor in interface design might affect performance capacity.
Focusing on performance time emphasises performance measures of workload effects. Other important measures of workload are physiological, such as the rate of secretion of stress chemicals, and subjective, such as moods and attitudes. Any factor could be considered a 'stressor' if its effect is that performance levels, stress hormone secretion rates, or subjective feelings, deteriorate. The approach in this section will be to indicate some key general topics, rather than to attempt a full review.
The points made here are concerned with : the capacities of different mental processes; extrinsic and intrinsic stressors; individual differences; and practical implications.
Capacities of different cognitive resources
Different aspects of cognitive processing have different capacities. For a review of processing limits see Sage (1981). The capacity of different processes may be affected differently by different factors. Figure 31 shows time-of-day effects on performance in four tasks : serial search, verbal reasoning (working memory) speed, immediate retention, and alertness. The different performance trends in these tasks suggests that each task uses a different cognitive resource which responds differently to time-of-day stress. It is difficult to make reliable analyses of these differences, but some other tasks in which performance may differ in this way are coding and syllogisms (Folkhard, 1990).
Figure 31 : Time-of-day effects show a different pattern in four different tasks, suggesting that these tasks all use different processing resources (Folkhard, 1990)
Extrinsic and intrinsic stressors
Extrinsic stressors are stressors which apply to any person working in a particular environment, whatever task they are doing. Time-of-day, as in Figure 31, is extrinsic in this sense. Some other extrinsic stressors which can affect performance capacity are noise, temperature, vibration, fumes, and organisational culture.
Intrinsic stressors are factors which are local to a particular task. All the HF/E factors which affect performance speed or accuracy come in this category.
The effect of task difficulty interacts with motivation. Easy tasks may be done better with high motivation, while difficult tasks are done better at lower levels of motivation. This can be explained by assuming that stressors affect a person's 'arousal' level, and that there is an inverted-U relation between arousal level and performance, see Figure 32.
Figure 32 : An 'inverted U' relation between arousal level and task performance, and the effect of various stressors (not all stressors can be accounted for in this way).
Measures of stress hormones and of workforce attitudes show that several factors to do with the pacing of work, and the amount of control over their work that a person feels they have, can be stressors (e.g. Johansson, Aronsson and Linström, 1978). Such aspects are of more concern in repetitive manufacturing jobs than in work such as flying or air traffic control.
Individual differences
Individual differences affect a person's capacity for a task, and their willingness to do it. Aspects of individual differences fall into at least five groups.
1. Personality. Many personality dimensions, such as extroversion/ introversion, sensitivity to stimuli, need for achievement or fear of success, and preference for facts/ ideas or for regularity/ flexibility, can affect a person's response to a particular task.
2. Interests and values. A person's interests and values affect their response to various factors in the task and the organisational climate, which influence their willingness and commitment to do or learn a given task. People differ in their response to incentives or disincentives such as money, status, or transfer to a job which does not use their skills.
3. Talent. Different people have different primary senses, different cognitive styles, and different basic performance abilities (e.g. Fleishman, 1975). For example, very few of us have the ability to fly high-speed aircraft.
4. Experience. The rest of us may be able to develop higher levels of performance though practice. Even the few who can fly high-speed aircraft have millions spent on their training. The effects of training on cognitive capacities will be discussed more in the section below on learning.
5. Non-work stressors. There may be non-work stressors on an individual which affect their ability to cope with their work, such as illness, drugs, or home problems.
Practical implications
There are so many factors affecting the amount of effort any particular individual is able or willing to devote to a particular task at a particular time, that performance prediction might seem impossible. Actually the practical ways of dealing with this variety are familiar. There are two groups of issues, in HF/E design and in performance prediction.
Nearly all HF/E design recommendations are based on measures of performance capacity. Any factor which has a significant effect on performance should be improved, as far as is economically justifiable. Design recommendations could be made about all the intrinsic and extrinsic factors mentioned above, and individual differences might be considered in selection.
However it is easier to predict that a design change will improve performance than to predict the size of the improvement. Numerical performance predictions may be needed in order to assess whether a task can be done in the time available, or with the available people, or to identify the limits to speed or accuracy on which design investment should best be concentrated. Obviously it is not practical to include all the possible effective factors when making such predictions. Three simplifying factors can reduce the problem.
One is that, while smaller performance changes may give important clues about how to optimise design, from the point of view of performance prediction these factors may only be important if they make an order of magnitude difference to performance. Unfortunately our data relevant to this issue are far from complete.
The second point is that only conservative performance predictions are needed. For these purposes it may be valid to extrapolate from performance in simple laboratory tasks in which people with no relevant expertise react to random signals, which is the worst case. To predict minimum levels of performance, it may not be necessary to include the ways in which performance can improve when experienced people do tasks in which they know the redundancies, can anticipate, etc.
The third point is that, in practice, many of the techniques for performance prediction which have been devised have the modest aim of matching expert judgements about human performance in a technique which can be used by someone less expert, rather than attempting high levels of accuracy or validity.
3. Response to overload
If people doing a simple task have too much to do, they only have the options of omitting parts of the task or of accepting a lower level of accuracy in return for higher speed (Figure 18 [Section I]). People doing more complex tasks may have more scope for responding to increased workload while maintaining acceptable task performance. This section will discuss : increasing efficiency; changing strategy; and practical implications.
Increasing efficiency
Complex tasks often offer the possibility of increasing the efficiency with which a task is done. For example, Sperandio (1972) studied the radio messages of air traffic approach controllers. He found that when they were controlling one aircraft they spent 18% of their time in radio communication. When there were nine aircraft, they spent 87% of their time on the radio. In simple models of mental workload :
total workload = workload in one task x number of tasks
Evidently that relation does not apply here, or the controllers would spend 9 x 18 = 162% of their time on the radio.
Sperandio found that the controllers increased the efficiency of their radio messages in several ways. There were fewer pauses between messages. Redundant and unimportant information were omitted. And conversations were more efficient : the average number of conversations per aircraft decreased but the average number of messages per conversation increased, so fewer starting and ending procedures were necessary.
Changing strategy
The controllers studied by Sperandio op cit did not only alter the efficiency of their messages, the message content also altered. The controllers used two strategies for bringing aircraft into the airport (this is a simplification so the description can be brief). One strategy was to treat each aircraft individually. The other was to standardise the treatment of aircraft by sending them all to a stack at a navigation fix point, from which they could all enter the airport in the same way. When using the individual strategy, the controllers asked an aircraft about its height, speed, and heading. In the standard strategy they more often told an aircraft what height and heading to use. The standard strategy requires less cognitive processing for each aircraft. Sperandio found that the controllers changed from using only the individual strategy when there were three or fewer aircraft, to using only the standard strategy when there were eight or more aircraft. Expert controllers changed their strategy at lower levels of workload. Sperandio argued that the controllers change to a strategy which requires less cognitive processing, in order to keep the total amount of cognitive processing within achievable limits, see Figure 33A.
Figure 33 : The effect of changing task strategy on the mental workload experienced, and task performance achieved (Figure A from Sperandio, 1972). The figure is a simplification, in practice the use of methods overlap so there are not discontinuities.
The relation between task performance and workload is therefore not the same in mental work as it is in physical work. In physical work, conservation of energy ensures there is a monotonic relation between physical work and task performance. In mental workload, if there are alternative working methods for meeting given task demands, then there is not necessarily a linear relation between the task performance achieved and the amount of mental work needed to achieve it. By using different methods, the same amount of mental effort can achieve different amounts of task performance, see Figure 33B.
Figure 34 : Adaptation of choice of strategy to achieve a balance between task demands and mental capacities (Bainbridge, 1974).
In choosing an optimum working method, two adaptations are involved. A person must choose a method which meets the task demands (upper feedback loop in figure). The person must also choose a method which maintains mental workload at an acceptable level (lower loop). Whichever method is chosen will affect both the task performance achieved and the mental workload experienced, as indicated in Figure 34.
There needs to be a mechanism for this adaptive choice of working method. This is another contextual effect which could be based on meta-knowledge. Suppose that the person knows, for each method, both how well it meets various task demands and what mental workload demands it poses. The person could then compare this meta-knowledge with the demands of the task and mental context, to choose the best method for the circumstances (Bainbridge, 1978).
Practical implications
This flexibility of working method has several practical implications. It is not surprising that many studies have found no correlation between task performance and subjective experience of mental workload. There are also problems with predicting mental workload, similar to the problems of predicting performance capacity mentioned above.
A person can only use several alternative working methods if the performance criteria do not strictly constrain what method must be used. For example, in air traffic control, safety has much higher priority than the costs of operating the aircraft. Task analysis could check that alternative methods are possible, and perhaps what these methods are (it may not be possible to pre-define all methods, see Section II on problem solving and below on learning).
Adaptive use of working methods suggests that strategy specific displays should not be provided, as they could remove the possibility of this flexibility for dealing with varying levels of workload. It could also be useful to train people to be aware of alternative methods and of the use of meta-knowledge in choosing between them.
When decision support systems are introduced with the aim of reducing workload, it is necessary to consider a wider situation. Decision support systems can increase rather than decrease mental workload, if the user does not trust the decision support system and so frequently checks what it is doing (Moray, Hiskes, Lee and Muir, 1995).
B. Learning
Learning is another potentially huge topic. All the expertise of psychology on learning, of HF/E on training, and of educational psychology on teaching cognitive skills and knowledge, could be included. As this chapter focuses on cognitive processes, this section will primarily discuss cognitive skill and knowledge. The coverage will only attempt a brief mention of some key topics, which indicate how learning inter-relates with other aspects of cognitive processing rather than being a separate phase of performance.
This section will use the word 'skill' in the sense in which it is used in psychology and in British industry. There are two key features of skilled behaviour in this sense :
Processing can be done with increased efficiency, either because special task related abilities have been developed which would not be expected from the average person, or because no unnecessary movements or cognitive processing are used.
And behaviour is adapted to the circumstances. Choices, about what best to do next and how to do it, are adapted to the task and personal context.
In this general sense, any type of behaviour and any mode of cognitive processing can be skilled, so it can be confusing to use the word 'skill' as the name for one mode of processing.
This section will be in three main parts, brief notes on : changes in behaviour with experience; learning processes; and relations between mode of processing and appropriate training method.
1. Changes within a mode of processing
This section will briefly survey the modes of processing which have formed one framework of this chapter, and indicate the ways in which each can change by introducing new aspects of processing or losing inefficient ones. This is a summary of points made before and is by no means complete. Learning can also lead to changes from one mode of processing to another, discussed later.
Physical movement skills. By carrying out movements in a consistent environment, people can learn :
- which movement has which effect (i.e. they develop their meta-knowledge about movements, Figure 20 [Section I]). This means they do not need to make exploratory actions, and their movements do not oscillate around the target. People can then act with increased speed, accuracy and co-ordination, and can reach to the correct control or make the correct size of action without checking.
- to use kinaesthetic rather than visual feedback.
- the behaviour of a moving target, so its movements can be anticipated.
Changes in performance may extend over very long periods. For example Crossman (1959) studied people doing the manually dexterous task of rolling cigars, and found that performance continued to improve until people had made about five million items.
Perceptual skills : discriminations and integrations. People learn :
- the discriminations, groupings, and size, shape and distance inferences to make.
- the probabilities and biases to use in decision making.
- the appropriate items to attend to.
- the eye movements needed to locate given displays.
Recodings. The person connects from one item to another by association, without intermediate reasoning. These associations may need to be learned as independent facts, or there may be some general rule underlying a group of recodings, such as 'choose the control with its location opposite to the location of the display'. Many people need a large number of repetitions before they can learn arbitrary associations.
Sequence of recodings. Two aspects of learning may be involved :
- When a sequence is the same each time, so that the output of one recoding and the input of the next recoding are consistent, then a person may learn to 'chunk' these recodings together, to carry them out as a single unit without using intermediate working memory.
- When a goal/ function can be met in the same way each time, then choosing a working method which is adapted to circumstances is not necessary. A previously flexible working method may then reduce to a sequence of transforms which does not include goals or choice of working method.
Familiar working methods. People need to learn :
- appropriate working method(s).
- the reference knowledge needed during use of each method. When this reference knowledge has been learned while using the method, then it may be accessed automatically, without having to think out explicitly what knowledge is needed in a particular situation.
- how to build up an integrated overview.
- meta-knowledge about each working method, which is used in choosing the best method for a given context.
Planning and multi-tasking. People can become more skilled in planning and multi-tasking. They can learn a general method for dealing with a situation, and the subsidiary skills for dealing with parts of it (Samurçay and Rogalski, 1988).
Developing new working methods. The process of developing new working methods can itself be more or less effective. Skill here lies in taking an optimum first approach to finding a new working method. There are several possible modes of processing for doing this.
Recognition primed decisions. People can only make recognition primed decisions about which working method to use once they have learned the categories of working method. Several aspects of learning are involved :
- the features defining a category, and how to recognise that an instance has these features so is a member of the category.
- the members of a category, and their properties (such as, for each category of situation : what to do in it).
- how to adapt a category method to specific circumstances.
Case based reasoning. Cases (or, more distant from a particular task, analogies) provide examples as a basis for developing the knowledge or working method needed. To be able to do this, people need to know :
- cases,
- how to recognise which case is appropriate to which circumstances.
- how to adapt the method used in one case to different circumstances.
Reasoning from basic principles. For this sort of reasoning, people need to have acquired an adequate base of knowledge about the task and the device(s) they are using, with associated meta-knowledge. The same type of knowledge may also be used for explaining both events and actions.
2. Learning processes
Little is know about how changes in processing take place during learning, except for the very simplest. Similar processes may be involved in developing and maintaining physical and cognitive skills. This section will indicate some mechanisms : repetition; meta-knowledge and feedback; independent goals-means; and changing modes of processing.
Repetition
Repetition is crucial for acquiring and maintaining skills. The key aspects are that, each time a person repeats a task : some aspects of the environment are the same as before, and knowledge of results is given. This knowledge of results has two functions : it gives information about how and how well the task was done, and it acts as a reward.
Meta-knowledge and feedback
As described in Section I on movement execution, learning of motor skills involves learning both how to do an action and meta-knowledge about the action. Actions have associated expectations about their effect (meta-knowledge). Feedback about the actual effect provides information which can be used to refine the choice made next time (Figure 20 [Section I]). So, during learning, feedback is used both to revise the present action and to revise the next action.
Choosing an action instruction on the basis of meta-knowledge is a similar process to choosing the working method used to maintain mental workload at an acceptable level. The choice of working method involves checking meta-knowledge about each method, to find which method has the properties best suited to the present situation.
A similar process is also involved when developing a new cognitive working method : a person develops a working method, hoping (on the basis of a combination of meta-knowledge and mental simulation) that it will give the required result, and then revises the method on the basis of feedback about the actual effectiveness of what they do.
Independent goals-means
In coping with mental workload, and in developing cognitive processes while learning, several working methods may be used for meeting the same function/ goal. Also the same behaviour may be used to meet several goals. So the link between goal and means must be flexible. The goal and means are independent in principle although, after learning, particular working methods may become closely linked to particular goals. In the section above on workload, the goal-means link was described as a point at which a decision between working methods is made on the basis of meta-knowledge.
It is generally the case (Sherrington, 1906) that behaviour at one level of organisation transfers information about the goal to be met, and constraints on how it should be met, to the lower levels of behaviour organisation by which the goal is met, but not detailed instructions about how to meet it. How to carry out the function is decided locally, in the context at the time. As behaviour is not dictated from above, but has local flexibility, human beings are not by nature well suited to following standardised procedures.
Changes in the mode of processing
Learning does not only lead to changes within a given mode of processing. A person may also change to a different mode of processing. If the task is consistent, then a person can learn to do the task in a more automatic way, that is by using a simpler mode of processing. Inversely, when there is no fully developed working method or knowledge for meeting a given goal/ function then it is necessary to devise one. So the possibility or need for developing a simpler or more complex mode of processing depends both on a person's experience with the task, and on the amount and types of regularity in the task. It may be possible through learning to change from any mode of processing to any other mode of processing, but two types of change are most typical : from more complex to simpler processing, or vice versa.
Someone may start a new task by developing a working method. But once they have had an opportunity to learn the regularities in the task, the processing may become simpler. If the task and environment are sufficiently stable, the person may learn that making a choice between methods to meet a goal, or to search for appropriate knowledge, are not necessary. In familiar stable situations, the working method may become so standardised that the person using it is not aware of goals or choices.
Alternatively, someone may start by learning parts of a task, and gradually become able to organise them together into a wider overview, or become able to choose behaviour which is compatible with several cognitive functions. These changes depend on changes in processing efficiency. When someone first does a complex task, they may start at the lowest levels of behaviour organisation, learning components of the task which will eventually be simple but which at first require all the person's problem solving, attention and other processing resources. As the processing for doing these sub-tasks becomes simpler with learning, this releases processing capacity. This capacity can then be used for taking in larger segments of the task at the same time, so the person can learn about larger regularities in the task.
In general any cognitive function, and any sub-goal involved in meeting it, may be met by any mode of processing, depending on the person's experience with the task, and the details of the circumstances at the moment. A task can become 'automated' or flexible at any level of behaviour organisation, depending on the repetitions or variety of situations experienced. So in some tasks a person may learn to do the perceptual-motor components automatically but have to rethink the task each time at a higher level, as in a professional person using an office computer. In other tasks, 'higher' levels of behaviour organisation such as planning may become automated while lower levels remain flexible, as in driving to work by the same route every day. It is not necessarily the case that 'higher' levels of behaviour organisation are only done by more complex modes of processing such as problem solving, or vice versa.
As any of the main cognitive functions in a task could become so standardised that they are done automatically or unconsciously, this is the origin of so-called 'short cuts' in processing. Inversely, at any moment, a change in the task situation, such as a fault, may mean that what could previously be done automatically now has no associated standard working method, so problem solving is needed to find one. At any time, or at any point in the task, there is the potential for a change in the mode of processing. So care is needed, if an interface design strategy is chosen of providing displays which support only one mode of processing.
3. Some training implications
Gagné (e.g. 1977) first suggested the concept that different modes of processing are best developed by different training methods. It is not appropriate to survey these methods here, but some general points link to the general themes of this chapter.
Simple processes
Training for simple processes needs to :
- maximise the similarity to the real task (the transfer validity) of discriminations, integrations and recodings which are learned until they become automatic, by using high-fidelity simulation.
- minimise the need for changes in mode of processing during learning, by presenting the task in a way which needs little problem solving to understand.
- ensure that trainees retain a feeling of mastery, as part of their meta-knowledge about the task activities, by avoiding training methods in which errors are difficult to recover from, and by only increasing the difficulty of the task at a rate such that trainees continue to feel in control.
Complex processes
Tasks which involve building up an overview and using alternative strategies need more than simple repetition if they are to be learned with least effort. The status of errors is different in learning complex tasks. In training for simple discriminations, recodings, and motor tasks, the emphasis is on minimising the number of errors made, so that wrong responses do not get associated with the inputs. By contrast, when learning a complex task, an 'error' can have positive value as a source of information about the nature and limits of the task. So in learning complex tasks, the emphasis should be more on exploring the possibilities without negative consequences, in order to develop a variety of working methods and wide knowledge of the task alternatives. Flexibility might be encouraged by giving trainees :
- guided discovery exercises, in which the aim is to explore the task rather than to achieve given aims.
- recovery exercises in which people practise recovering from non-optimal actions.
- problem solving and planning exercises, with or without real time pressures.
- opportunities to share the discoveries made with other trainees.
- practise with considering alternative working methods, and with assessing the criteria for choosing between them.
- practise with thinking about alternative 'hypotheses' for the best explanation of events, or the best action.
- practise with multi-tasking.
- practise with using different methods for developing working methods, and with the case examples and recognition categories used.
A feature of cognitive skill is having a knowledge base which is closely linked to the cognitive processing which uses it, so that the knowledge is appropriately organised and easy to access. This suggests that knowledge is best learned as part of doing the task, not separately.
Training as part of system design
This chapter has mentioned several ways in which training needs interact with the solutions chosen for other aspects of the system design :
- the quality of interface or procedure design and the need for training may be inversely related.
- skills are lost if they are not maintained by practice, so the amount of continuing training needed may be related to the extent of automation.
C. Difficulties and errors
Errors occur when people are operating at the limits of modes of processing. Errors result from misuse of normally effective processes. The concept of relating error types to modes of processing was first suggested by Rasmussen (1982), though the scheme suggested here is somewhat different.
There are several points, which the approach to complex tasks taken in this chapter suggests should be added to most error schemes.
Firstly, the notion of 'error' needs to be expanded. In some simple tasks such as recoding it is possible to be wrong. But in control tasks and in complex tasks it is useful to think in terms of difficulty, or lowered effectiveness, rather than focusing on being wrong. For example, Amalberti's novice pilots (Figure 28 [Section II]) were already qualified. They completed the task, they just did it less effectively than the more experienced pilots. So, as a basis for supporting people doing complex tasks, it is useful to look at factors which make the task more difficult, as well as factors which slow behaviour down or increase errors.
Secondly, many error schemes assume that task behaviour can be broken down into small independent units, each of which may be right or wrong. In Probabilistic Risk Assessment or Human Reliability Assessment techniques, behaviour is segmented into separate units. A probability of error is assigned to each unit, and the total probability of human error for the combined units is calculated by addition or multiplication. But this chapter has stressed that human behaviour in complex tasks does not consist of independent units. The components of complex behaviour are organised into an integrated interdependent structure. This means that, while PRA/HRA techniques are useful for practical purposes, any attempt to increase their fundamental validity while retaining an 'independent units' model of behaviour is doomed to failure (Hollnagel, 1993).
Thirdly, as the processes of building up and using an overview are often not included in models of human processing, the related errors are also often not discussed, so they will be the focus here. This section will briefly suggest some of the ways in which performance can be weaker (for examples, see Bainbridge, in press).
Discriminations
Decisions made under uncertainty cannot always be right, and are more likely to be wrong if the evidence on which they are based is ambiguous or incomplete. Incorrect expectations about probabilities, and incorrect biases about payoffs can also increase error rates. People make errors such as : misattributing risk, importance or urgency; ignoring a warning which is frequently a false alarm; or seeing what they expect to see. Some people when under stress refuse to make decisions involving uncertainty.
Recodings
There are many sorts of error which can be attributed to mis-translations. Sometimes the person does not know the coding involved. People are more likely to make coding errors when they have to remember which specific code translation to use in which circumstances. Difficult codes are often ambiguous or inconsistent. The salience of some stimuli may give improper emphasis to them or to their most obvious meaning.
Sequences
The items which need to be retained in working memory during a sequence of behaviour may be forgotten within half a minute, if other task processing distracts or interrupts the rehearsal needed to remember the items.
In an overlearned sequence, monitoring/ supervision of parts of the activity may be omitted. This can lead to 'slips' in performance, or to rigid behaviour which causes difficulties when the environment changes and adaptive behaviour is needed.
Overview and behaviour organisation
There may be errors in organising the search for information. People may only attend to part of the task information, fail to keep up-to-date with changes in the environment, or look at details without taking an overall view. They may not get information which there is a cost on getting. They may only look for information which confirms their present interpretation of the situation ('confirmation bias'). In team work, people may assume without checking that another member of the team, particularly someone with higher status, has done something which needed doing.
There may also be errors in the allocation of time between tasks, which may lead to omissions or repetitions. People may react to events rather than anticipating events and how to deal with them. They may not apply available strategies in a systematic way. They may shift between sub-tasks, without relating them to the task as a whole ('thematic vagabonding', Doerner, 1987). They may break the task down into sub-problems in an inadequate way, or fail to devise intermediate sub-goals, or they may continue to do parts of the task which they know how to do ('encystment', Doerner op cit ). Under high workloads, people may delay decisions in the hope that it will be possible to catch up later, or they may cycle through thinking about the task demands without taking any action.
The person’s overview influences their biases about what will happen, and what to do about it. If the overview is incorrect this can lead to inappropriate behaviour or expectations. People who have completed a sub-task, and so completed a part of their own overview, may fail to tell other members of the team about this. Once people have built up a complete and consistent overview, it may be difficult to change it when it turns out to be inadequate ('perceptual set'). The overview may also be lost completely if a person is interrupted.
Use of knowledge
People's knowledge of all types may be incomplete or wrong, so they make incorrect inferences or anticipations. There may be problems with assumed shared knowledge in a team, if team members change.
A person may have an incorrect or incomplete representation of the device they are using. For example, they may not know the correct causalities or interactions, or they may not be able to represent correctly the development of events over time. Or someone may use an inappropriate category in recognition primed decisions or in case based reasoning.
Knowledge about probabilities may be incorrect, or used wrongly. People may be under or over confident. They may have a 'halo effect', attributing the same probabilities to unrelated aspects. They may give inappropriate credence to information or instructions from people of higher status. Different social groups, for example unions, management, and the general public, may have different views on the risks and payoffs of particular scenarios.
This list of human weaknesses should not distract from two important points. One is that people can be good at detecting their errors and recovering from them, if they are given an interface and training which enable them to do this. So design to support recovery should be included in cognitive task analysis.
The second point is that care is needed with the attribution of responsibility for faults. Although it may be a given individual who makes an error, the responsibility for that error may be attributed elsewhere, to poor equipment or poor system design (training, workload, allocation of function, teamwork, organisational culture).
CONCLUSION
There are several integrative concepts in this chapter.
Cognitive goals : in complex tasks people use cognitive goals when implementing task goals. A person's cognitive goals are important in organising their behaviour, in directing attention to parts of the task, in choosing the best method for meeting a given goal, and in developing new working methods. The cognitive goals might be met in different ways in different circumstances, so the goals and the processes for meeting them can be independent. For example, flying an aircraft involves predicting the weather, and this may be done in different ways before and during the flight.
Contextual overview : in complex tasks people build up an overview of understanding and planning which then acts as the context for later activity. The overview provides data, expectations and values, and the criteria for deciding what would be the next best thing to do and how to do it.
Goal-means independence and meta-knowledge : Meta-knowledge is knowledge about knowledge, such as the likelihood of alternative explanations of what is happening, or the difficulty of carrying out a particular action. Alternative working methods have associated with them meta-knowledge about their properties. Decisions about how best to meet a particular aim are based on meta-knowledge, and are involved in :
- adapting behaviour to particular circumstances,
- the control of multi-tasking and mental workload,
- learning.
Modes of processing : As well as using different working methods, people may use different modes of processing, such as knowing the answer by association or thinking out a new working method. The mode of processing used varies from moment to moment, depending on the task and the person's experience.
Modelling human behaviour
Basing HF/E on an analysis of behaviour into small independent units fits well with a 'sequential stages' concept of the underlying structure of human behaviour. But a sequential stages model does not include many of the key features of complex tasks such as flying and air-traffic control. Complex behaviour is better described by a contextual model, in which processing builds up an overview which determines what processing is done next and how, which in turn updates the overview, and so on. In this mechanism for behaviour organisation, choices about what to do and how to do it depend on details of the immediate situation interacting with the individual's nature and previous experience.
The aspects missing from many sequential stages models are :
- the goal oriented nature of behaviour, and the independence of goals from the means by which they are met.
- the continuing overview.
- the flexible sequencing of cognitive activity, and the organisation of multi-tasking.
- the knowledge base, and the resulting predictions, anticipations and active search for information which are part of top-down processing ['top down' means starting from knowledge rather from environmental stimuli].
Some of these aspects require a fundamental change in the nature of the model used. The most important aspect to add is the overview, as all cognitive processes are done within the context provided by this overview, and the sequence in which they are done is determined by what is in the overview.
A simple version of a contextual model has been suggested in Figure 22 [end of Section I] and Figure 24 [beginning Section II]. These figures can act as an aide-memoire about contextual processing, but any small diagram can only indicate some features of what could be involved. These simple figures do not make explicit important aspects such as :
- risky decision making and the effects of biases.
- goal orientation of behaviour.
- typical sequences of activity.
- different modes of processing, including devising new working methods.
- use of meta-knowledge.
Perhaps the most important disadvantage of the one page contextual model will be felt by people who are concerned with tasks which are entirely sequential, rather than cyclic as in flying or air-traffic control. But I would argue that, although dependencies may define the order in which some parts of a task are done, it could still be useful, when designing to support sequential tasks, to consider the task sequence as a frame for structuring the overt behaviour, while the underlying order of thinking about task aspects may be more varied (cp. Figure 28 [Section II]).
The difficulty of HF/E
Contextual processing underlies two types of difficulty for HF/E.
One group of issues is concerned with HF/E techniques. As indicated above, the overview suggests the need for several additions to HF/E techniques :
- consider the codings used in the task as a whole, rather than for isolated sub-tasks.
- orient cognitive task analysis towards the cognitive goals or functions to be met, as an intermediary between the task goals and the cognitive processing. (Analysing either goals or working methods alone is necessary but not sufficient.)
- design the interface, training, and allocation of function between people and machines, to support the person's development and use of the contextual overview, alternative strategies, and the processes involved in the development of new working methods.
- extend human error schemes to include difficulties with the overview and with the organisation of sequences of behaviour.
The second group of issues is concerned with a fundamental complexity problem in human behaviour and therefore in HF/E. Human behaviour is adapted to the particular circumstances in which it is done. This does not make it impossible to develop a general model of human behaviour, but it does make it impossible to predict human behaviour in detail. Predicting human behaviour is like weather prediction : it is not possible to be right, but it is possible to be useful. Any HF/E answer is always going to be context sensitive. The continuing complaints of HF/E practitioners, that researchers do not provide them with what they need, are a consequence of the fundamental nature of human behaviour. Specific tests of what happens in specific circumstances will always be necessary. What models of human behaviour can provide is, not the details, but the key issues to focus on when doing such tests or when developing and applying HF/E techniques.
Other sections of this Chapter
I. Using an interface, the bases of classic HF/E.
Access to other papers via the Home page
©2021 Lisanne Bainbridge
Comments
Post a Comment