4. Sequences of activity

Topics in this section :

4.1. identifying the determinants of the sequence of 'routines'

4.2. the main ’routines’ and sequential determinants in the furnace control task.

4.3. representing the sequence of thinking in relatively conventional flow diagrams ('box element' representation in Section 5).

4.4. introduction to a cyclic/overview model (more on this in Sections 5-6).

Summary of main points in Section 4.

Section 4 : Sequence of activities, introduction to the contextual overview.

Building up behavioural complexity from a cognitive processing element

Lisanne Bainbridge

How are the sequences of activity built up? After working through one 'routine’, how does the person decide what type of activity to do next ? What determines the sequence in which topics are thought about ?

Finding this needed a different type of analysis.

The verbal reports give direct evidence about what the operators thought about their task.

How these were analysed is described in Section 2, also the reasons for using the 'box element' way of representing the findings.

The operators talk about a sequence of topics. A change in topic can be identified by an analyst using natural language understanding and knowledge of the task. Example in Table 3c.1.

But the operator does not say why he changes from one topic/ focus of attention, to another.

So how could these reasons for change be identified, what determines his sequence of behaviour ?

This Section shows how this can be found, by analysis of some specific examples.

There are repeated references to the following papers on this site in this section, as they include more examples and discussion of these issues.

On techniques for analysing verbal protocols

1985 Inferring from verbal reports to cognitive processes

The key results from that analysis

1974 Analysis of verbal protocols from a process control task

Types of 'skill' in cognitive processes

1989 Development of skill, reduction of workload

Basic concepts used in modelling the results

1981 Mathematical Equations or processing routines

1993 Types of hierarchy imply types of model

1997 The change in concepts need to account for human behaviour in complex dynamic tasks

4.1. Identifying the sequencing determinants

In verbal protocols, a speaker often just changes topic without mentioning why they are now thinking about something different (e.g. Table 3c.1). If the determining factors which affect the choice of topic are not explicitly mentioned, someone analysing the protocol has to use other means to identify what determines the choice.

The process by which this analysis was done for the furnace power supply verbal protocol was described in Bainbridge (1972, 1985). Suppose that, in the protocol, topic A is sometimes followed by topic B and sometimes by topic C, and one wants to find out what it is that determines which of these happens. The analyst can look at all the factors, in the process state and in the operator's recent behaviour, which might differ in the two circumstances. And find which type of operator’s behaviour consistently happened within given values of these contextual items.

(Tables 1 and 2 in Bainbridge, 1985 show a version of this type of analysis, in a simpler situation but one in which there are more instances to support the conclusion.)

An example from the furnace power control data :

Table 4.1 : Step-change in power usage, and protocol phrases after it (Bainbridge, 1972, Table 5.5b)
(Numbers in lower half indicate sequence of phrases)

Table 4.1 shows the only instance of this type of analysis from the steelworks protocol which is simple enough to fit onto one page. The table summarises what the operator does after a step change (sudden large change) in the amount of electric power being used. The top half of the table shows aspects of the process state displayed on the process interface. The bottom half of the table shows the topics the operator talked about after the change (the numbers indicate the sequence of phrases).

The protocol phrases clearly fall into two groups :

- the first two phrases listed in the table are concerned with making an action,

- the other phrases find out what has happened.

(There are 2 instances when he does both.)

So the question is : what determines whether, after a step change, the operator thinks about making an action, or about identifying what has happened ?

The operator’s general type of behaviour (whether he reviews or acts) is not obviously related to whether the Size of the change or the new D value is positive or negative. (Though the details of what he does within those general types of behaviour may be.)

What about within limits around a general target value ?

Table 4.2 shows the results for whether the new T (total power usage) value after a step change is close to 50 (the 'target' value) or far away.

Table 4.2 : activity related to T (total power usage) value

If the T value is close to 50, he usually reviews, if further away from 50, he acts.

Table 4.3 shows whether the operator’s next thinking is related to how close the new Discrepancy meter reading is to 0 (0 Discrepancy = no problems). If the discrepancy is larger he acts, if it is smaller he reviews what is the current state of the furnaces and the possible future events and actions :

Table 4.3 : activity related to D (discrepancy meter) value

There’s very little difference between the T and D results, and that size of difference could easily happen by chance. So this evidence doesn’t show clearly whether the operator reacts T or D. (Other evidence suggested he was more interested in T.)

There are two instances when the operator did both activities. They are late in the test period and when the control error is high. So this might indicate that this operator (an experienced furnace shop manager controlling a new proposed task on a simulator) might be learning about the task, and his behaviour might change if the learning time had been longer.

The choice could be represented as relative to the T value (in box element notation) by Figure 4.1.

(The lines before T and after 50 are absolute value symbols.)

Figure 4.1 : The operator's reaction to change in Total Demand (T) might be represented by this sort of mechanism.

How the use of the two groups of phrases (thinking about action or identifying state) depends on the size of the new D (discrepancy meter) value, particularly on how far it is from 0, could be represented by a very similar Figure.

Note that the 'alright' assessments (how close T is to 50 or D is to 0) would probably have been made by visual judgement while reading the display, from the angle of the display pointer on the scale, rather than by reading the display followed by a separate calculation. They are represented as separate steps in this Figure, but might not be in actual processing.

4.2. The main 'routines' and sequential determinants in the furnace control task

The subsection above used examples of two of the 'routines' identified from the furnace power control verbal report : choosing an action or reviewing the state. Over a dozen main 'routines' were needed to describe the operator's behaviour, see Table 3c.2.

The factors which affect the change from one routine to another were identified as described in Section 4.1, by :

- dividing the verbal report into sections concerned with different activities.

- bringing together the evidence from all the different exemplars of the same choice point, and identifying the factors which are different related to what is done next.

So this is a completely different type of analysis from the way the 'routines' were identified.

The left A column of Table 4.4 below lists the main 'routines' found in the furnace control task, see Table 3c.2.

The method of analysis described in Section 4.1 was used to identify the factors which determined the sequence of topics in the furnace operator's verbal protocol - when there enough instances of the sequence to feel justified that the sequence identified was valid (analyst’s assumption !).

This analysis led to the list of determinants of the sequence of behaviour in Table 4.4, column B.

So these two columns were found by independent analyses.

Table 4.4 : Data Items found by Main Routines (A), and determining sequence of behaviour (B). Adapted from Bainbridge, 1974, Table 1.

As the items found by the main thinking routines, and the items found by an independent analysis of different data to find what affects the sequence of behaviour, are roughly the same, I suggested that the items found by the main routines provide the context within which the best next behaviour is chosen. This context has been called the person’s 'overview'. There is more about this overview, and the main behaviour sequences, later below and in Sections 5-6.

In the analysis, the 'routines' were found before the items determining the sequence they were thought about. The order in which the two analyses were done leads to linking the two types of evidence in this way.

But actually a complex task works the other way round : there are items a person doing a complex task needs to know in order to choose what best to do next. So the person needs 'routines' for finding the values of those items.

4.3. In what sequence are the topics thought about ?

The results of this analysis, what determines the choices between activities, and what the operator does in these circumstances, are represented in this section by relatively conventional flow diagrams as used in the early 70s.

There’s a summary diagram of the sequence of behaviour in Figure 4.2, and this is expanded in Figure 4.3. Both these diagrams are simplifications showing the main issues. The full detail of the behaviour is shown by the 'box element' versions, some of which are shown in Section 5.

Greatly simplifying, in general for the furnace operator studied in detail, his main behaviour was determined by the control state, which he cycled back to checking after each main section of thinking.

This is a control task, and the operator cycles between assessing the use of electrical power, which he was controlling, and other thinking about the task.

Figure 4.2 : Summary of main sequences of operator's activities. Bainbridge, 1974, Figure 3.

shape codes :

diamond : conditionals, decision point

rectangle : operation, thinking

If the control state was acceptable (coded 0 in the figure, left column in the figure) the operator reviewed the state of the process.

He does this in a cycle : does some reviewing of the furnace conditions, checks the control state, does further reviewing of furnace states, and so on.

If the control state was trending to needing action (coded 1/2 in the figure, middle column), he reviewed the actions available, and if he had more time, chose the best next action.

Again he goes through a cycle : does some thinking about what would be a good action, again checks the control state, if still nothing is urgent then he considers more criteria for how to choose a good action, returns to checking the control state, and so on.

If action was needed (coded 1 in the Figure, right column) he made the action he had previously chosen during his reviewing activities.

Note that in both left and mid columns the operator starts to consider the stage review/ action choice, then re-checks the control state, then looks at another topic in the stage review/ action choice, and so on.

This leads to a sort of speed-accuracy trade-off - if the operator, after the next control state check, needs to change to a different type of activity, he has already found the most important information needed by that next activity.

Figure 4.3 below gives more detail about the sequence of items the operator considers about each topic. It fills in more detail about what is being done in the boxes in Figure 4.2 but has been made more clear by not including the constant referring back to assessing the control state.

Predicting events is important in this task. The operator had to control the amount of electric power used in a given half-hour (so as not to overload the local power generating facilities, 5 electricity-powered steel furnaces can use a huge amount of power - about what is used by a small city). The five furnaces use different amounts of power in different stages of the steel making process. A change to a stage that needs more power could cause problems in the future, while a change to a stage that needs less power could compensate for a present power over-usage. When action was needed to reduce power, he had to choose which furnace could take a power reduction with least effect on the quality of steel it made (first choice would be furnaces which are early in the melt stage).

Figure 4.3 : More detail about the operator's thinking. Bainbridge 1974, Figure 2

This figure has been simplified by omitting the check of the process state (top left box) between each step of thinking.

The right (0) column of the previous figure (4.3) is the bottom left of Figure 4.4.

The mid (1/2) column of Figure 4.3 is the right column of Figure 4.4.

Figure 4.3 gives more detail than Figure 4.2 about the sequence of characteristics the operator considers when reviewing the plant state and actions available.

4.4. Introduction to the overview and a cyclic contextual model

The evidence suggests that the operator was continuously checking and revising his knowledge of what was happening on the plant, what might happen in the future, and what he might do about it.

Because the two columns of Table 4.4 are similar, it is suggested that the top 'head' boxes of the main 'routines' ’store' the information used in the sequencing decisions. These head boxes form an 'overview' which acts as the context for deciding what is the optimum topic to think about next.

The operator goes through a cycle of thinking, which builds up an overview of the task situation (what the furnaces are and will be doing, and what best to do about it, Figures 4.2 and 4.3), which is referred to in deciding what to do next (Table 4.4).

More detail on this in Sections 5-6.

It is evident that a contextual rather than sequential model is needed.

This could be summarised in this cycle figure :

Figure 4.4 : The basic contextual processing cycle. (Bainbridge, 1997, Figure 1)

This contextual model is more complex than a feedback mechanism, as the operator is actively choosing to check the task acceptability. And reviewing the task situation is not just a matter of finding how far the value of one variable deviates from a required value. In the furnace power control task it involved reviewing the general overall state of the furnaces (lower left section of Figure 4.3), which was information used in reviewing future events and choosing the best action available (right column of Figure 4.3).

Here is a slightly more detailed diagram of this cycle, which also includes reminders about the two ways the person receives information from the environment. In many real-life tasks, actively searching for the information needed is much more important and frequent than just reacting to signals.

Figure 4.5 : A simple representation of the contextual cycle in relation to the knowledge base and the environment. (Bainbridge, 1997, Figure 2)

A sequential model of cognitive processing, or contextual model ?

This representation of the basic nature of cognitive processing, as an interplay between context (in the overview) and processing, contrasts with sequential models of processing, which may be used in experimental psychology and in human factors/ ergonomics. In sequential models :

- processing proceeds in one direction only, from activation of the senses to motor response,

- each stage of processing reacts only to the output of previous stages,

- the task goals are given.

A sequential representation does not include a contextual overview, and cannot account for

- active search for information,

- thinking about topics in a flexible sequence,

- thinking out what the optimum goals should be.

So a sequential model is unable to account for some important features of behaviour in complex dynamic tasks. Hollnagel, 1993, has an interesting discussion of these issues.

In the contextual account here, the main cognitive needs fall into two main groups, concerned with working out what is going on, and with working out what to do about it. These are not synonymous with processing sensory inputs and producing motor outputs. The processes required to understand what is going on may include actions (for example in process operation - walking to another part of the building to find some necessary information, or calling upon other people who have specific expertise). And devising a response can include getting information (for example about what actions are available) (Bainbridge, 1981, 1993a). This sort of flexibility can be dealt with by the cognitive element and 'sequencers' (see next section) approach.

There are tasks (such as the experimental psychology laboratory tasks much used at the original time of this research, which attempted to exclude context effects as far as possible) in which behaviour (after learning to do the task) may be adequately represented by a sequential model.

Personally I think it would be much more interesting (and much more difficult) to study what happens during the learning phase of a laboratory task, before results are ever collected. I remember an experience as a student. Behavioural psychology was very powerful at the time (late 50s). The theory represented rats as whizzing through a maze, perhaps stopping at each corner for a moment to check the environmental cues. We were given a rat in a lab class, and put it in a maze. It promptly stood on its hind legs and looked at and sniffed us with interest. I never bothered with behavioural theory again. Much the same things happen with human beings in experimental situations. As a 'subject’ doing a laboratory task, the person has first to learn what aspects of behaviour are required in this situation. Experimental results are only collected after this initial learning phase. Studying how people do that learning would be much more difficult, but would be much more relevant to understanding human behaviour in real-life situations.

In my view it would be both more parsimonious and more interesting to have only one model for both simple (sequential) and complex (contextual) tasks. This would be possible if the model which accounts for complex situations could also account for simple ones : in the present case, if the contextual model could behave like a serial model in the appropriate circumstances.

For an example showing the emergence of serial behaviour after learning a complex task, see Amalberti’s (1992) findings from a study of pre-flight planning by inexperienced and experienced pilots, shown in Figure 28 of Bainbridge (1995 II).

In fact, a possible mechanism for this has already been mentioned here. Section 3c.3 Figure 3c.4 describes the two-arrow box, which symbolises a mechanism by which, if the data required is not already available in working storage, then a 'routine' is carried out to find it. When the 2-arrow box is in a conditional, that indicates that when the information on which the decision is based is already available, the decision is made automatically/ unconsciously. While if the information needed is not available, the relevant 'routine' to find it has to be carried out. So that Section describes how this two-arrow box accounts for the way in which, in a new situation for which an overview has not yet been built up, the operator has to do some thinking (perhaps when learning to do a task, a person even has to do some problem solving about what to do). While once an overview has been built up, all the reference information needed is already available, so the person goes straight to the required activity. As can be described by a sequential processing model.

So after much learning of a task the behaviour appears as serial, as much of the support processing is no longer needed. Standard sequences of behaviour are generated by, or emerge from, a contextual mechanism in certain circumstances, even though they are not (necessarily) explicitly represented in it. For more discussion of this, see Bainbridge, 1993a, and especially discussions of learning, see e.g. Bainbridge (1978, 1989), and Section 8.

The remaining Sections of this review discuss :

5. The cycles of activity, as described in 'box element' notation.

6. Working storage and the overview.

7. Deciding how to do the next activity, when there are alternative methods for meeting a goal. It has been suggested this refers to meta-knowledge about its properties stored with each 'routine’. This decision also refers to the task overview. It has also been suggested that this process is used in a person’s control of their mental workload.

8. Some comments on how all this underlies learning the different types of cognitive activity (perceptual-motor skills, familiar cognitive skills, problem solving skills).

- - -

Summary of main points in Section 4

* A complex dynamic task involves several main classes of cognitive need.

* The results of meeting the main cognitive needs (carrying out the main 'routines') are maintained in working storage as an overview of the state of the task. This information is available for reference by other processing, it is part of the context in which later processing is done.

* The main cognitive needs may be thought about in a very varied order. This sequence is decided by reference to the overview/ context of the task situation.

* The order in which the cognitive needs are met is not rigid, but serial processing can be an emergent property from a parallel representation when a task is sufficiently repetitive.

Access to other papers on this site via the Home page

In this review, there are 3 main groups of topics.

1. Introduction.

Basic element, sources of data which meet cognitive needs.

2. The cognitive processing element.

Meeting an information need :

3a. by finding it in the environment.

3b. from a stored knowledge base.

3c. by working through a 'routine',

or referring to the result of using a routine elsewhere.

3d. more on knowledge bases.

Choosing what to do.