Process model comprehension: a human view #BPM2009

For the second half of the morning, I elected to attend a tutorial on a human view of process model comprehension by Jan Mendling of Humboldt-Universität zu Berlin and Hajo Reijers of Eindhoven University of Technology. Obviously a lot of other people are interested too, since we had to move to a much larger room before beginning.

This started with a definition of process model quality based on the SeQual Framework: syntactic, semantic, pragmatic and other measures of quality. Then, some basics on how human memory works:

• External stimuli pass through immediate sensory memory to short-term working memory to long-term memory that represents the knowledge that we maintain.
• Dual coding theory states that we process visual information differently depending on whether it is textual or graphical: with text, we tend to hear the words in our head and process them through auditory channels rather than visual channels.
• Cognitive load theory states that we can only hold a maximum of seven things in working memory at one time.
• Cognitive fit theory, which looks at how different types of stakeholders interact with the same information differently.

Having covered some of the theory around how we process information, we looked at some of the practical examples of how novices and experts view process models; in this case, “expert” may refer to either a subject matter expert or a process modeling expert. The selection of the visual representation – the “language” – does not have much of a difference on comprehension, assuming that all of these languages are flow-oriented, such as EPC, Petri Nets or BPMN. There are, however, a number of factors that do impact comprehension:

• Model complexity (this seems a pretty obvious conclusion to me, but I guess it needed to be proven 🙂 ), including complex operators and some clever but obscure model optimization.
• Layout/topology and coloring; these are considered secondary notation characteristics in that they don’t change the model, just its visual appearance.
• Text labels, that is, the understandability of text labels within process step.
• Purpose, that is, whether the process model is for execution, training or to meet specific certification requirements.

There are different methods of measuring process model comprehension while viewing a model: how accurately can people respond to questions about the model; how long does it take them to answer those questions; how much mental effort is expended to reach those answers, which is done by asking the subjects how hard it was for them. There are also different measures of how well that process model is remembered when it is removed from the subject: recall of process characteristics such as how many start events exist in the model; retention of the business meaning of tasks in the model; transfer of the entire model, measured by questions such as how the model could be simplified.

There are several implications of this process model comprehension research:

• Modeling tools should enforce structured models, analyze correctness (which is well understood in the research community and available in open source tools, but poorly represented in commercial products), and provide different views on the model for different stakeholders.
• With respect to training, abstract modeling knowledge is useful, but familiarity with a specific technique/language is less important
• Adopt 7PMG modeling guidelines: use as few elements as possible in the model, minimize the routing paths for each element (which can be counter to the first recommendation, since it may result in a complex gateway being split into two simpler gateways), use one start and one end event, keep the model as structured as possible, avoid OR routing elements in favor of AND and XOR elements, use verb-object text labeling style, and decompose a model with more than 50 elements to subprocesses (my sense, as well as Reijers’, is that this should be a lower number, such as 20-30, although their research shows a definite advantage at 50).

The relative importance of these factors are unknown, and further research is required to determine where to best invest time and money, e.g., is it better to invest in training or model decomposition? It should be possible for modeling tools to suggest some of the 7MPG guidelines (or similar guidelines for model improvement) when a model violates the rules, although none of them do; commercial products focus on optimizing the model from a business process standpoint, not model comprehension.

There are a number of reference papers supporting their research; I found this to be a fascinating tutorial with a great deal of practical applicability. I would highly recommend that anyone doing process modeling (and maybe some of the modeling vendors) should review the 7PMG paper, which I link to above, since it contains a lot of great ideas for creating better process models.

Update: I have a new link to the 7PMG paper from the authors which is a pre-print version with guidance on prioritizing the guidelines. It’s in place, above.