Category: process

business process management

Discovering Reference Models by Mining Process Variants Using a Heuristic Approach #BPM2009

Chen Li of University of Twente gave the last presentation of the conference on process variant mining. We heard yesterday in the tutorial on flexibility about process variants; one issue with process variants is that there needs to be some way to identify which of the variants are important enough to update the original process model. The paper describes a heuristic search algorithm for determining which of the variants are important, by considering both the similarity to the original process model and the most common changes amongst the variants, e.g., the same new step is added to almost all process variants.

Since the process can be varied at runtime, the new process model doesn’t have to be perfect, it just has to be better than the original one. In general, the more activities contained in a candidate model and the more that its structure matches that of the variants, the better it is: they have created a fitness function that combines these two parameters and calculates how good a specific candidate model is. The search tree used to find the optimal candidate process model generates all potential candidates by changing one activity at a time, calculating the best fit, then replacing the original with the candidate if it is better than the original. This continues until no better candidate model can be calculated, or until you reach your maximum search distance (which would be set in order to bound computations).

The algorithm was evaluated with simulations, indicating that the most important changes tend to be performed at the beginning of the search.

Divide-and-Conquer Strategies for Process Mining #BPM2009

In the first of two papers in the final session of the conference, Josep Carmona of Universitat Politecnica de Catalunya presented on process mining calculation strategies. The theory of regions shows how to derive a Petri net representation of a process model from the process log, which shows the transition between states, but it’s very computationally expensive. This paper deals with ways of making that computation less expensive in order to deal effectively with large logs.

First is a decompositional strategy, which partitions the regions in a way that allows the identification of a set of state machines that cover all the events, then uses parallel composition to assemble the state machines into a Petri net.

The second approach is a higher-level divide-and-conquer strategy, where the event log is recursively partitioned by event class until the log sections are small enough to use other techniques. The clustering of the events is the key thing here: first, compute the causal dependency graph, then use spectral graph theory to find clusters of highly related events that will be partitioned off into their own section of the event log.

What they’ve seen in experiments using this technique is that there is a significant computational improvement (from minutes to seconds) from the decompositional approach, and that the divide-and-conquer approach allows for the processing of event logs that are just too large for other techniques.

You can get Genet, the tool that they developed to do this, here.

Discovering process models from unlabelled event logs #BPM2009

Diogo Ferriera of Universidade Tecnica de Lisboa presented a paper on process discovery based on unlabelled event logs: where the events in the log are only identified by the specific task, not by the process instance. Consider that a process instance may be executed via multiple paths through the process model, resulting in a different sequence of events logged: although you might know all possible paths through the model, you don’t know which one that any given instance followed. Also consider that processes will be executing simultaneously, so their events are intermingled.

Taking a probabilistic approach, you can take the event sequence and the source sequence (i.e., you know both the events and the instances that created them) and generate a matrix of probabilities that any given event will follow another within the same source instance: that’s some fairly standard math. He then took the event sequence and the matrix (which now represents a priori knowledge about how events interrelated), and did a fairly magic-looking calculation that calculated the source sequence based on that information.

The problem, of course, is that you don’t have the magic matrix, you only have the event sequence: initialize the matrix to something, then use the event sequence and the matrix to estimate the source sequence, then use the event sequence and the estimated source sequence to estimate the matrix. Wash, rinse, repeat until the matrix converges. You could initialize the matrix randomly, but that would take a while to converge (or would converge to a local maximum); instead, Ferreira pulled a rabbit out of his hat by stating that the matrix can be initialized with the transition probabilities present in the event sequence, that is, as if the event sequence were generated from a single source. As the paper states:

Even if x [event sequence] is the result of interleaving a number of sources, their underlying behaviour will be present in M+ [probability matrix] since consistent behaviour will stand out with stronger transition probabilities than the spurious effects of random interleaving. Therefore, M+ is a good initial guess for the estimation of M.

Amazingly, this works. Or maybe not so amazing, because I suppose there would be no research paper if this didn’t work. As the number of sources (instances) increases, the accuracy approaches that of when both the event and source sequence are known; as the number of overlapping sources increases, the accuracy drops to about 60% (by the time that you reach 10 overlapping instances), then flattens out.

There are a number of use cases for this: preprocessing for other process mining algorithms, or as a labeling mechanism to find the source instance when it is unknown. Or just if you want to show off some cool math at parties.

Tutorial: enabling flexibility in process-aware information systems #BPM2009

Manfred Reichert of Ulm University and Barbara Weber of University of Innsbruck presented a tutorial on the challenges, paradigms and technologies involved in enabling flexibility in process-aware information systems (PAIS). Process flexibility is important, but you have to consider both build time flexibility (how to quickly implement and configure new processes) and run time flexibility (how to deal with uncertainty and exceptional cases during execution), as well as their impact on process optimization.

We started by looking at the flexibility issues inherent in the imperative approach to BPM, where pre-defined process models are deployed and executed, and the execution logs monitored (in other words, the way that almost all BPMS work today). As John Hoogland discussed this morning, there are a number of flexibility issues at build time due to regional process variations or the lack of a sufficient information about decisions to build them into the process model. There’s also flexibility issues in the run time, mostly around exception handling and the need for ad hoc changes to the process. As all this rolls back in to the process analyst through the execution monitoring, it can be used to optimize the process model, which requires flexibility in evolving the process model and impacting work in progress. The key problem is that there are way too many variants in most real-life processes to realistically model all of them: there needs to be a way to model a standard process, then allow user-driven configuration (either explicitly or based on the instance parameters) at run time. The Provop approach presented in the tutorial allows for selective enabling and disabling of process steps in a master model based on the instance conditions, with a lot of the research based on the interaction between the parameters and the soundness of the resultant models.

Late binding and late modeling approaches use a pre-specified business process with one or more placeholder activities, then the placeholder activities are replaced with a process fragment at run time either from a pre-determined set of process fragments or a process fragment assembled by the user from existing activity templates (the latter is called the “pockets of flexibility” approach, a name that I find particularly descriptive).

Up to this point, the focus has been on changes to the process model to handle variability that are part of normal business, but possibly not considered exceptions. Next, we looked at runtime exception handling, such as the failure or unavailability of a web service that causes the normal process to halt. Exceptions that are expected (anticipated) can be handled with compensation, with the compensation events and handler built into the process model; unexpected exceptions may be managed with ad hoc process changes to that executing instance. Ad hoc process changes can be a bit tricky: they need to be done at a high level of abstraction in order to make it understandable to the user making the change, yet the correctness of the changes must be validated before continuing. This ability needs to be constrained to a subset of the users, and the users who can make the changes may require some assistance to do this correctly.

This was a good tutorial, but I wanted to catch the process mining session so skipped out at the break and missed the last half.

John Hoogland keynote: Change in Control #BPM2009

John Hoogland, CEO of Pallas Athena, gave the opening keynote this morning, discussing how BPM has changed since the early days of workflow and painfully customized EAI to today’s landscape of BPM and SOA. He also contrasted what he sees as the old style of assembly line-style control flow unique to US and UK companies, and the more “one and done” style prevalent in Europe.

He went through a montage of BPM vendor websites – pointing out that the smaller the company, the clearer the message – and noted that the key message on all of those sites in 2009 is agility. The promise of all of these vendors is that change is key, and that this change can be done by the business, without IT. BPM doesn’t bring flexibility: it brings control. It gives more flexibility than the processes within legacy applications, but is inherently less flexible than a process that is not under the control of a system. Agility is more than flexibility, it’s about change but in a controlled environment: change in control.

He did a closer look at many of the vendors’ messages, pointing out the nonsense of much of the marketing speak about how easy and beneficial that BPM is. Agility in that context means that the process can be re-modeled; in fact, that’s often the only point of agility that the BPM vendors allow (although I would argue that integration of rules management creates significantly more agility without process model changes).

From a modeling standpoint, having users involved in process modeling does bring agility through faster and better design decisions. Separating process flow from legacy applications also provides agility. However, these processes need to be released within an IT infrastructure, serve the needs of the entire enterprise, and tend to be fairly complex: all recipes for killing agility. Even looking at the vendors’ architecture diagrams of BPM, it is always shown as a very small component within a much larger IT ecosystem; this is typical of what I see in large implementations, where the BPM part of an enterprise project is just a tiny piece of the whole project, even though it is labeled as a BPM project. Other factors also impede agility: the issue of migrating work in progress when the process model changes, including changes to related services. The conclusion: although the vendors make process model changes look easy in the demo, it’s not that easy in reality.

Complexity of process models is also an issue. In practice, exception handling tends to cause process models to balloon far beyond the “happy path” that you model on day 1, as do regional differences, handling of unexpected external events, and interfaces between departments and with systems. For example, you really don’t want separate process models for each region, you want a single model that can handle the regional differences, but that creates both a more complex model plus a nightmare of change control. However, if business users have the ability to change their local processes through parameters without having to change the enterprise-wide process model, things become much simpler. What parameters should be available for change on a local level, then? Hoogland suggests rules/decisioning, prioritization, resource allocation including role assignment, deadlines/SLAs and other factors. This requires an explicit step during the process design to design what parameters have to be able to be changed by the business users on the fly without changing the process model (I am going through exactly this exercise now with a client, and completely agree with his assessment), and ensure that the BPMS is capable of providing those as parameters under direct control of the business.

He ended up with some final thoughts about how the market is confused by vendors and analysts leaping onto new concepts and techniques too quickly, and how we waste time and energy on non-issues such as BPM standards.

He mentioned my post yesterday about the process model comprehension tutorial, and agrees with my assessment that the vendors have a huge amount to learn from academic research. I’m beginning to think that my true calling is as an evangelist bringing the word of new and useful academic BPM research to the BPM vendor community, although I have no idea how to monetize that. 🙂

There was a good Q&A following, with discussions on the potential value of bringing academics and vendors closer together; he admitted that a year ago, if asked to attend this conference, he would have been concerned only with whether there would be customers here. Now – hopefully due to some of my prodding of the vendors encouraging them to attend – he understands that vendors have a lot to gain from seeing what the academic BPM community is doing. There’s definitely monetary issues: academics need funding in order to continue research, and much of the research is still sufficiently early that a significant amount of research and development would still be required by the vendors to bring it to market. Many small vendors just can’t afford their own pet researchers such as IBM and SAP have, and are very focused on getting a product out this year as opposed to investing in the long-term potential from BPM research.

There must be a way to have funding flow from BPM vendors to BPM researchers without having a vendor try to control the speed and direction of the research, and without even having a specific researcher tied to a specific vendor. Is there a place for a virtual institute of BPM research spanning the many great BPM research institutions, funded by a consortium of vendors, with all having access to the end results? I’m thinking that for less than the cost of having a booth at a Gartner conference, a BPM vendor could contribute significantly to BPM research and reap the rewards.

Managing Processes #BPM2009

I attended the session on managing processes in the afternoon: this is a standard session made up of the presentation of three papers.

Extending BPM Environments of your choice with performance related Decision Support, by Mathias Fritzsche, Michael Picht, Wasif Gilani, Ivor Spence, John Brown and Peter Kilpatrick. The focus was on annotation of processes across multiple modeling environments to allow for combination of process monitoring data regardless of the execution environments; this is of specific use for end-to-end processes that span multiple environments such as a BPMS, an ERP system and outsourced processes.

Business Process-based Resource Importance Determination, by Stefan Fenz, Andreas Ekelhart and Thomas Neubauer. This is motivated by risk analysis, since risk calculations will be incorrect if there is insufficient information about the resources attached to specific business processes, and how those resources are used by other processes across the organization. This seems pretty straightforward: each resource and its importance to a business process and in the larger organizational context are required as input, as well as the process model in Petri Net form and the resource’s usage within the process; the research determined a calculation method for global importance for resources. The applicability, however, is key: the importance of the resources (these are automated resources, such as mail servers) ties into risk management and business continuity planning.

Case Study and Maturity Model for Business Process Management Implementation, by Michael Rohloff. I admit to losing a bit of focus during this presentation, so no notes, but I’ve given you the link to the paper so you can read it yourself. 🙂

The session was very sparsely attended, which means that most people are either at the tutorial on “A Business Process is more than a Workflow” or are skipping school to take advantage of the lovely weather.

Process model comprehension: a human view #BPM2009

For the second half of the morning, I elected to attend a tutorial on a human view of process model comprehension by Jan Mendling of Humboldt-Universität zu Berlin and Hajo Reijers of Eindhoven University of Technology. Obviously a lot of other people are interested too, since we had to move to a much larger room before beginning.

This started with a definition of process model quality based on the SeQual Framework: syntactic, semantic, pragmatic and other measures of quality. Then, some basics on how human memory works:

  • External stimuli pass through immediate sensory memory to short-term working memory to long-term memory that represents the knowledge that we maintain.
  • Dual coding theory states that we process visual information differently depending on whether it is textual or graphical: with text, we tend to hear the words in our head and process them through auditory channels rather than visual channels.
  • Cognitive load theory states that we can only hold a maximum of seven things in working memory at one time.
  • Cognitive fit theory, which looks at how different types of stakeholders interact with the same information differently.

Having covered some of the theory around how we process information, we looked at some of the practical examples of how novices and experts view process models; in this case, “expert” may refer to either a subject matter expert or a process modeling expert. The selection of the visual representation – the “language” – does not have much of a difference on comprehension, assuming that all of these languages are flow-oriented, such as EPC, Petri Nets or BPMN. There are, however, a number of factors that do impact comprehension:

  • Model complexity (this seems a pretty obvious conclusion to me, but I guess it needed to be proven 🙂 ), including complex operators and some clever but obscure model optimization.
  • Layout/topology and coloring; these are considered secondary notation characteristics in that they don’t change the model, just its visual appearance.
  • Text labels, that is, the understandability of text labels within process step.
  • Purpose, that is, whether the process model is for execution, training or to meet specific certification requirements.

There are different methods of measuring process model comprehension while viewing a model: how accurately can people respond to questions about the model; how long does it take them to answer those questions; how much mental effort is expended to reach those answers, which is done by asking the subjects how hard it was for them. There are also different measures of how well that process model is remembered when it is removed from the subject: recall of process characteristics such as how many start events exist in the model; retention of the business meaning of tasks in the model; transfer of the entire model, measured by questions such as how the model could be simplified.

There are several implications of this process model comprehension research:

  • Modeling tools should enforce structured models, analyze correctness (which is well understood in the research community and available in open source tools, but poorly represented in commercial products), and provide different views on the model for different stakeholders.
  • With respect to training, abstract modeling knowledge is useful, but familiarity with a specific technique/language is less important
  • Adopt 7PMG modeling guidelines: use as few elements as possible in the model, minimize the routing paths for each element (which can be counter to the first recommendation, since it may result in a complex gateway being split into two simpler gateways), use one start and one end event, keep the model as structured as possible, avoid OR routing elements in favor of AND and XOR elements, use verb-object text labeling style, and decompose a model with more than 50 elements to subprocesses (my sense, as well as Reijers’, is that this should be a lower number, such as 20-30, although their research shows a definite advantage at 50).

The relative importance of these factors are unknown, and further research is required to determine where to best invest time and money, e.g., is it better to invest in training or model decomposition? It should be possible for modeling tools to suggest some of the 7MPG guidelines (or similar guidelines for model improvement) when a model violates the rules, although none of them do; commercial products focus on optimizing the model from a business process standpoint, not model comprehension.

There are a number of reference papers supporting their research; I found this to be a fascinating tutorial with a great deal of practical applicability. I would highly recommend that anyone doing process modeling (and maybe some of the modeling vendors) should review the 7PMG paper, which I link to above, since it contains a lot of great ideas for creating better process models.

Update: I have a new link to the 7PMG paper from the authors which is a pre-print version with guidance on prioritizing the guidelines. It’s in place, above.

Prof. Dr. August-Wilhelm Scheer keynote: BPM 3.0 #BPM2009

Yesterday was a great day of workshops here at BPM2009 in Ulm, Germany, and today is the start of the conference proper. After some introduction, we heard from Dr. August-Wilhelm Scheer of IDS Scheer (recently acquired by Software AG); I’ve heard Dr. Scheer speak at ARIS ProcessWorld in 2007 and 2008, and I was curious to see how his presentation to an academic audience would differ from the more commercial/enterprise audience of the user conferences. First big difference: no saxophone. 🙁

His topic was “BPM 3.0”, that is, a look at the future of BPM. He started with a historical view: how BPM has evolved over the past 25 years, and how the organizational scope of BPM has expanded from project-based implementations to blanketing the enterprise. BPM today has enormous strategic importance: you can’t do any sort of complex risk analysis without considering processes, and governance is based in a large part on appropriate process management.

Looking forward, we need to consider empowering the user by, for example, giving them ways to mash up their own view of processes by combining multiple data sources into a dashboard that is specific to their needs, or customizing how they search for information within processes. We also need to leverage the power of the community: he used the examples of IDS Scheer’s social media, including their community site and YouTube channel, although there are obviously more direct social applications within process design and execution.

There was a bit too much emphasis on the ARIS products: he illustrated each point with ARIS, which is fair, but went into a great deal of detail about how it works and the customers using it, which felt like a bit of a sales pitch.

Micro workflow gestural analysis #BPM2009 #BPMS2’09

Ben Jennings of University College London had the last presentation slot of the day, wherein he classified a duck using both hierarchical classification and protoype theory. He was successful using both methods, although identified the inherent flaws in hierarchical classification.

The point, however, is about the nature of classification systems: hierarchical classification systems can lead to issues due to errors when creating the taxonomy, whereas prototype methods have more dynamic boundaries.

His research is around gestural analysis of micro workflows: essentially a sort of process discovery, wherein the actions of people in unstructured expertise-driven processes (that is, processes that involve domain experts) are analyzed within a social context in order to establish a reputational representation of the agents that can be used to find the best person to assign to a task. In other words, rather than having a process designer manually pre-determine who does which task, knowledge about the task and the people who might best perform that task can be determined as required based on the human social behavior around past process instances.

Gestural analysis in this context considers both active gestures – such as the explicit creation of content – and passive gestures – such as who a person emails in order to help solve a problem, or blog commenting behavior. It considers both the things that people create, and the people who create them.

This has the greatest value in tasks that have high complexity and high uncertainty, wherein an adhocracy tends to apply, although these tasks may be embedded within a more traditional structured process.