DecisionCAMP 2019: Industry use cases in airport gate allocation, financial risk monitoring, and composite material design

The Decision Model for Gate Allocation. Silvie Spreeuwenberg, Librt

Day 3 of DecisionCAMP 2019 started with three use cases from industry. First, Silvie Spreeuwenberg presented on decision models for allocating airport gates, specifically at Schiphol airport in Amsterdam. Although gate plans are made a day in advance based on flight schedules, they change constantly due to early arrivals, late departures and other unexpected disruptions to the schedule. Any given day, there are 50-100 gate changes one hour before an aircraft arrival; although this was seen as a disruption, this could also be considered an opportunity for optimization.

Day-ahead decision model for assigning gates. From Silvie Spreeuwenberg’s presentation.

There were a lot of rules used for the planning and reassignment that had more to do with preferences than actual optimization; they really wanted to drive towards the objective of optimizing asset usage and therefore airport capacity. There are a lot of factors involved, such as having sufficient gate area capacity to handle the number of passengers for a flight, or having buses available to offload flights that can’t be assigned a gate. They have created a policy for aircraft stand allocation which includes some identifiable decision tables, although these are just at the strategy documentation phase.

Definitely a complex problem that has applicability at every major airport around the world.

A hybrid implementation of multi-channel, multi-modal, high volume financial risk monitoring. Martijn Tromm and Marten Schokking, Oracle

Marten Schokking and Martijn Tromm presented a use case from Rabobank using decision management and machine learning for customer risk assessment in terms of KYC (know your client) and AML (anti-money laundering). This is used during client onboarding, but also during periodic reviews as well as reviews triggered by specific events. There are scoring rules that use data input from a variety of sources, including client information from a CRM, interview responses and policies.

Risk model for customer risk assessment. From Marten Schokking and Martijn Tromm’s presentation.

There are government regulations requiring that this be done for all clients at certain times. A triggering event, such as a change in the customer’s circumstances, will cause a customer interview and other data analysis to recalculate the risk; this may result in a more detailed manual review of the risk. At this point, there is still a lot of employee work which is creating a challenge in completing the customer risk assessments within the regulatory deadlines; they are looking at how to automate the basic assessment using machine learning in order to reduce the manual work required.

The risk model has been built using Oracle Policy Automation rules engine integrated with the Siebel CRM. They are reusing rules across channels where possible, and the use of natural language in the rules definition helps with traceability to the policies. They are continuing to innovate with rules, such as having context-driven rules based on user behavior on specific channels, and having a fast two-day turnaround for rule changes related to certain types of policy changes. The ability to predict the impact of policy changes based on actual data allows for operational planning to accommodate those changes.

The Role of DMN and BPMN in the Design of Composite Materials. Dario Campagna, ESTECO

Dario Campagna presented on how the COMPOSELECTOR project is integrating material modeling and business process management in a decision support system for composite material design; this type of design can have complex requirements, business decisions and simulation workflows. Using application cases from Dow, Airbus and Goodyear, they modeled the business flow using BPMN and DMN. ESTECO, which creates software tools for engineering design, is a contributor to the COMPOSELECTOR project.

Subprocess in Dow flow showing decision invocation. From Dario Campagna’s presentation.

While BPMN is used to model the flow at the business level, DMN decision tables are used to make decisions on the class of materials and manufacturing process, then on the simulation workflows to use based on business and engineering KPIs. DMN provides the link from the business layer to the engineering layer, then to the simulation layer. Using DMN provides a higher level of consistency in decision-making, which leads to better design and lower costs.

Decision table used to select simulation workflow. From Dario Campagna’s presentation.

We saw a brief video of a demo of the system in use: a business-level manager selects high-level parameters and KPIs for the proposed design; this selects one or more simulation models for the material design, which is then confirmed or decided by an engineer; the results of the simulation are passed back to the manager for final decision-making. This has the effect of integrating the business and technical sides of the design process, and include modeling and simulation results in the business-level (human) decisions in a standardized way.

CamundaCon 2019 breakout: DMN and BPMN for reusable survey forms at Indiana Farm Bureau

Sowmya Raghunathan and Corinna Cohn presented on a claims intake implementation that uses BPMN and DMN in an interesting way: driving the intake forms used by a claims administrator when gathering first notice of loss (FNOL) information from a claimant. The idea is that the claims admin doesn’t need to have any information about the claim type, and the claimant doesn’t get asked any irrelevant questions, because the form always presents the next best question based on previous responses: a wizard-like model, but driven by BPMN and DMN.

As the application and technical architects at Indiana Farm Bureau Insurance, they were able to give us a good view of how they use the tools for this: BPMN for orchestrating the DMN and UI communication as well as storing the responses, DMN for defining the questions and question/response mapping, and a UI component for implementing the survey forms. They consider this a headless application, but of course, it does surface via the form UI; from a Camunda process standpoint, however, it is decoupled piece of the architecture that interfaces with the claims system.

Technical architecture of the survey DMN/BPMN system

We saw a demo of one of the claim forms at work, where the previous questions and responses can be seen, and changes to the previous responses may cause changes to subsequent questions based on the DMN decision tables behind the scenes. They use a couple of DMN tables just as configuration tables for the UI for the questions and options (e.g., radio buttons versus free-form responses), then a Next Question decision table to determine the next question based on the previous response: this table is based on a directed acyclic graph that links questions (nodes) via answers (links), which allows for easy re-navigation of the graph if an earlier response is changed.

DMN decision table for FNOL questions related to auto claim

BPMN is used to navigate and determine the next question in a dynamic question subprocess, and if the survey can be exited; once sufficient information has been collected, the FNOL is initiated in the claims systems.

Dynamic Questions BPMN subprocess

The use of DMN means that the questions can be changed very easily since they’re not embedded in the code; this means that they can be created and modified by business analysts rather than requiring developers to code these into the UI directly.

FNOL BPMN process

The entire framework is reusable, and could be quickly reconfigured to be used for any type of survey. That’s great, because a few years ago, I saw a very similar use case for this in a clinical situation for a stroke assessment questionnaire: in a hospital setting, when someone arrives at an emergency department and is suspected of having had a stroke, there are standard questions to ask in order to evaluate the patient’s condition. At the time, I thought that this would be a perfect use case for DMN and BPMN, although that was beyond the scope of the project at that time.

A match made in BPMN/DMN heaven: @bpmswatch joining @Trisotech

Trisotech recently announced that Bruce Silver – who writes and teaches the gold standard Method & Style books and courses on BPMN and DMN, and who has forgotten more about BPMN than most people ever learned – is joining Trisotech as a principal consultant. Congrats all around, although Bruce may regret this when he’s needed at Trisotech Montreal headquarters in January when it’s -30C. Winking smile

Bruce even has his first post on the Trisotech blog, about practical DMN basics. Essential reading for getting started with DMN.

Disclosure: Trisotech is a consulting client of mine. I’m not being paid for writing this post, I just like these guys because they’re smart and do great work. You can read about my relationship with vendors here.

bpmNEXT 2019 demos: automation services with @Trisotech and @bpmswatch

The day started with my keynote on rolling your own digital automation platform using BPM and microservices, which set the stage for the two demos and the round table discussion that followed.

Business Automation as a Service, with Denis Gagne of Trisotech

Denis demoed a new product release from Trisotech, their business automation as a service platform: competing with services such as Zapier and IFTTT but with better process and decision management, and more complex service types available for integration. He showed creating a service built on a Twitter trigger, using BPMN to model the orchestration and FEEL as the scripting language in script activities, and incorporating a machine learning sentiment score and a decision service for categorizing the results, with the result displayed in the color of a flashing smart light bulb. Every service created exposes an Open API and REST API by default, and is deployed as a self-contained microservice. He showed a more complex example of marketing automation that extracts data from an input form, uses a geo-locator to find the customer location, uses a DMN decision model to assign to a sales team based on geography and other form parameters, then creates a lead in Microsoft Dynamics CRM. He finished up with an RPA task example that included the funniest execution of an “I am not a robot” CAPTCHA ever. Key point here is that Trisotech has moved from a pure modeling vendor into the execution space, integrated with any Open API service, and deployable across a number of different cloud platforms using standard protocols. Looking forward to playing around with this.

Business-Composable Services for the Mortgage Industry, with Bruce Silver of Method and Style

Bruce showed the business automation services that he’s created using Trisotech’s platform for the mortgage industry. Although he started looking at decision services around how to determine if someone should be approved for a mortgage (or how large of a mortgage), process was also required to do things like handle mapping and validation of data. Everything is driven by a standard application form and a standard set of underwriting rules used in the US mortgage industry, although this could be modified to suit other markets with different rules. The DMN rules are written in business-readable language, allowing them to be changed by non-developers. The BPMN process does the data validation and mapping before invoking the underwriting decision service. The entire process can be published as a service to be called from any environment, such as a web app used by underwriters inside a financial company or by an online prequalification review done directly by the consumer. The plan is to make these models and services available to see what the adoption is like, to help highlight the value and drive the usage of BPMN and DMN in practice.

Industry Round Table: The Coming Impact of Decision Services and Machine Learning on Business Automation

We finished the morning of day 2 with a discussion that included three of the earlier demo presenters: Denis Gagne, Bruce Silver and Scott Menter. They each gave a short talk on how decision services and machine learning are changing the automation landscape. Some ideas discussed:

  • It’s still up in the air whether DMN will “cross the chasm” and become generally used (to the same degree as, for example, BPMN); this means that vendors need to fully support it, potentially as an execution as well as requirements language.
  • Having machine learning algorithms expressed as DMN can improve transparency of decisions, which is essential in some jurisdictions (e.g., GDPR). There is a need for “explainable AI”.
  • The population using DMN is lower than BPMN, and the skill level is higher, although still well within the capabilities of data-focused business people who are comfortable with formulas and expression languages.
  • There’s a distinction between symbolic (rules-based) and sub symbolic (neural network) AI algorithms in terms of what they can do and how they perform; however, sub symbolic AI is less of a black box in terms of decision transparency.
  • If we here at bpmNEXT aren’t thinking about the ethics of automation, who will? Consider the labor disruption of automation, or decisions that make a choice involving the value of life (the AI “trolley problem”), or old norms used as training data to create biased machine learning.
  • We’re still in a culture of having people at a certain skill level (e.g., surgeons, pilots) make their own decisions, although they might be advised by AI. How soon before we accept automated decisions at that level?
  • Individually-targeted decisions are happening now by what is presented to specific people through platforms like Google Search and Amazon. How is our behavior being controlled by the limited set of options presented to us?
  • The closer that a technology gets to the end effect, the more responsibility that the creator of the technology needs to take in how it is used.
  • Machine learning may be the best way to discover the best transparent decision logic from human action (unfortunately that will also include the human biases), allowing for people to understand how and why specific decisions are made.
  • When AI is a black box, it needs to be understood as being a black box, so that adequate constructs can be created around it for testing and usage.

Great discussion and audience participation, and a good follow-on from the two demos that showed decision services in action.

Camunda BPM 7.5: CMMN, BPMN element templates, and more

I attended an analyst briefing earlier today with Jakob Freund, CEO of Camunda, on the latest release of their product, Camunda BPM 7.5. This includes both the open source version available for free download, and the commercial version with the same code base plus a few additional features and professional support. Camunda won the “Best in Show” award at the recent bpmNEXT conference, where they demonstrated combining DMN with BPMN and CMMN; the addition of the DMN decision modeling standard to BPMN and CMMN modeling environments is starting to catch on, and Camunda has been at the front edge of the wave to push that beyond modeling into implementation.

They are managing to keep their semi-annual release schedule since they forked from Activiti in 2013: version 7.0 (September 2013) reworked the engine for scalability, redid the REST API and integrated their Cockpit administration module; 7.1 (March 2014) focused on completing the stack with performance improvements and more BPMN 2.0 support; 7.2 (November 2014) added CMMN execution support and a new tasklist; 7.3 (May 2015) added process instance modification, improved authorization models and tasklist plugins; and 7.4 (November 2015) debuted the free downloadable Camunda Modeler based on the web application, and added DMN modeling and execution. Pretty impressive progression of features for a small company with only 18 core developers, and they maintain their focus on providing developer-friendly BPM rather than a user-oriented low-code environment. They support their enterprise editions for 18 months; of their 85-90 enterprise customers, 25-30% are on 7.4 with most of the rest on 7.3. I suspect that a customer base of mostly developers means that customers are accustomed to the cycle of regression testing and upgrades, and far fewer lag behind on old versions than would be common with products aimed at a less technical market.

Today’s 7.5 release marches forward with improvements in CMMN and BPMN modeling, migration of process instances, performance and user interface.

Oddly (to some), Camunda has included Case Management Model & Notation (CMMN) execution in their engine since version 7.2, but has only just added CMMN modeling in 7.5: previously, you would have used another CMMN-compliant modeler such as Trisotech’s then imported the model into Camunda. Of course, that’s how modeling standards are supposed to work, but a bit awkward. Their implementation of the CMMN modeler isn’t fully complete; they are still missing certain connector types and some of the properties required to link to the execution engine, so you might want to wait for the next version if you’re modeling executable CMMN. They’re seeing a pretty modest adoption rate for CMMN amongst their customers; the messaging from BPMS vendors in general is causing a lot of confusion, since some claim that CMMN isn’t necessary (“just use ad hoc tasks in BPMN”), others claim it’s required but have incomplete implementations, and some think that CMMN and BPMN should just be merged.

Camunda 7.5 element templatesOn the BPMN modeling side, Camunda BPM 7.5 includes “element templates”, which are configurable BPMN elements to create additional functionality such as a “send email” activity. Although it looks like Camunda will only create a few of these as samples, this is really a framework for their customers who want to enable low-code development or encapsulate certain process-based functionality: a more technical developer creates a JSON file that acts as an extension to the modeler, plus a Java class to be invoked when it is executed; a less technical developer/analyst can then add an element of that type in the modeler and configure it using the properties without writing code. The examples I saw were for sending email and tweets from an activity; the JSON hooked the modeler so that if a standard BPMN Send Task was created, there was an option to make it an Email Task or Tweet Task, then specify the payload in the additional properties that appeared in the modeler (including passed variables). Although many vendors provide a similar functionality by offering things such as Send Email tasks directly in their modeling palettes, this appears to be a more standards-based approach that also allows developers to create their own extensions to standard activities. A telco customer is using Camunda BPM to create their own customized environments for in-house citizen developers, and element templates can significantly add to that functionality.

Camunda 7.5 instance migration 4 - manual mapping of unrecognized steps between modelsThe process instance migration feature, which is a plugin to the enterprise Cockpit administration module but also available to open source customers via the underlying RESET and Java APIs, helps to solve the problem of what to do with long-running processes when the process model changes. A number of vendors have solutions for this, some semi-automated and some completely manual. Camunda’s take on it is to compare the existing process model (with live instances) against the new model, attempt to match current steps to new steps automatically, then allow any step to be manually remapped onto a different step in the new model. Once that migration plan is defined, all running instances can be migrated at once, or only a filtered (or hand-selected) subset. Assuming that the models are fairly similar, such as the addition or deletion of a few steps, this should work well; if there are a lot of topology changes, it might be more of a challenge since there could need to roll back instance property values if instances are migrated to an earlier step in the process.

They have also improved multi-tenancy capabilities for customers who use Camunda BPM as a component within a SaaS platform, primarily by adding tenant identifier fields to their database tables. If those customers’ customers – the SaaS users – log in to Cockpit or a similar admin UI, they will only see their own instances and related objects, without the developers having to create a custom restricted view of the database.

Camunda 7.5 process duration reportThey’ve released a simple process instance duration report that provides a visual interface as well a downloadable data. There’s not a lot here, but I assume this means that they are starting to build out a more robust and accessible reporting platform to play catch-up with other vendors.

Lastly, I saw their implementation of external task handling, another improvement based on customer requests. You can see more of the technical details here and here: instead of a system task calling an Camunda 7.5 external-task-patternexternal task asynchronously then wait for a response, this creates a queue that an external service can poll for work. There are some advantages to this method of external task handling, including easier support for different environments: for example, it’s easier to call a Camunda REST API from a .NET client than to put a REST API on top of .NET; or to call a cloud-based Camunda server from behind a firewall than to let Camunda call through your firewall. It also provides isolation from any scaling issues of the external task handlers, and avoids service call timeouts.

Camunda BPM 7.5

There’s a public webinar tomorrow (June 1) covering this release, you can register for the English one here (11am Eastern time) and the German one here (10am Central European time).

Bruce Silver Now Stylish With DMN As Well As BPMN

I thought that Bruce Silver’s blog had been quiet for a while: turns out that he moved to a new, more representative domain name, and my feed reader wasn’t updating from there. He’s rebranding his business, including his blog, under Method & Style, mirroring the title of his popular book and training BPMN Method and Style , and now his new book and training options for DMN: DMN Method and Style: The Practitioner’s Guide to Decision Modeling with Business Rules .

His blog has a ton of new content on DMN, starting with a great piece that compares the path of the DMN standard with that of BPMN, which is considerably more mature. He discusses the five key elements of DMN, then goes into each of those in detail in the next five posts: Decision Requirements Diagrams, Decision Tables, FEEL (a new expression language developed for DMN), Boxed Expressions and the Metamodel and Schema. It’s really interesting to read his analysis comparing the evolution of the two standards: there was a time when everyone thought that BPMN was just about the visual notation, but to make it really useful, the interchange format and execution semantics have to come along at some point. Still, it’s useful to get started in DMN now with DRDs and decision tables, since that at least makes the decision models explicit instead of being buried in text requirements.

Once you’ve brushed up on his posts covering the five key elements, you can also read about conformance levels that vendor can choose to implement, and what didn’t make it into DMN 1.1, which is the first real version of the standard.

He doesn’t pull any punches in his discussion, and is not very complimentary on some aspects of the standard and how some vendor choose to implement it. Just as he is with BPMN. Smile

bpmNEXT 2014 Thursday Session 1: Intelligence And A Bit More BPMN

Harsh Jegadeesan of SAP set the dress code bar high by kicking off the Thursday demos in a suit jacket, although I did see Thomas Volmering and Patrick Schmidt straightening his collar before the start. He also set a high bar for the day’s demo by showing how to illuminate business operations with intelligent process intelligence. He discussed a scenario of a logistics hub (such as Amazon’s), and the specific challenges of the hub operations manager who has to deal with inbound and outbound flights, and sorting all of the shipments between them throughout the day. Better visibility into the operations across multiple systems allows problems to be detected and resolved while they are still developing by reallocating the workforce. Harsh showed a HANA-based hub operations dashboard, where the milestones for shipments demark the phases of the value chain: from arrival to ground handling to warehouse to outbound buffer to loading and takeoff. Real-time information is pulled from each of the systems involved, and KPIs show; drill downs can show the lower level aggregate or even individual instance data to determine what is causing missed KPIs – in the demo, shipments from certain other hubs are not being unloaded quickly enough. But more than just a dashboard, this allows the hub operations manager to add a task directly in the context of the problem and assign it (via an @mention) to someone else, for example, to direct more trucks to unload the shipments. The dashboard can also make recommendations, such as changing the flights for specific shipments to improve the overall flow and KPIs. He showed a flight map view of all inbound and outbound flights, where the hub operations manager can click on a specific flight and see the related data. He showed the design environment for creating the intelligent business operations process by assembling SAP and non-SAP systems using BPMN, mapping events from those systems onto the value chain phases (using BPAF where available), thereby providing visibility into those systems from the dashboard; this builds a semantic data mart inside HANA for the different scenarios to support the dashboard but also for more in-depth analytics and optimization. They’ve also created a specification for Process Façade, an interface for unifying process platforms by integrating using BPMN, BPAF and other standards, plus their own process-based systems; at some point, expect this to open up for broader vendor use. Some nice case studies from process visualization in large-scale enterprises.

Dominic Greenwood of Whitestein on intelligent process execution, starting by defining an intelligent process: it has experiences (acquired data), knowledge (actionable information, or analytical interpretation of acquired data), goals (adoptable intentions, or operationally-relevant behavioral directives), plans (ways to achieve goals through reusable action sequences, such as BPMN processes) and actions (result of executing plans). He sees intelligent process execution as an imperative because of the complexity of real-world processes; processes need to dynamically adapt, and process governance needs to actively apply constraints in this shifting environment. An intelligent process controller, or reflective agent, passes through a continuous cycle of observe, comprehend, deliberate, decide, act and learn; it can also collaborate with other intelligent process controllers. He discussed a case study in transportation logistics – a massively complex version of the travelling salesman problem – where a network of multi-modal vehicles has to be optimized for delivery of goods that are moved through multiple legs to reach their destinations. This involves knowledge of the goods and their specific requirements, vehicle sensors of various types, fleet management, hub/port systems, traffic and weather, and personnel assignments. DHL in Europe is using this to manage 60,000 orders per day, allocated between 17,500 vehicles that are constantly in motion, managed by 300 dispatchers across 24 countries with every order changing at least once while en route. The intelligent process controllers are automating many of the dispatching decisions, providing a 25-30% operational efficiency boost and a 12% reduction in transportation costs. A too-short demo that just walked through their process model to show how some of these things are assigned, but an interesting look into intelligent processes, and a nice tie-in to Harsh’s demonstration immediately preceding.

Next up was Jakob Freund of camunda on BPMN everywhere; camunda provides an open-source BPM framework intended to be used by Java developers to incorporate process automation into their applications, but he’s here today to talk about an open-source toolkit in Javascript that provides a framework for developers and a BPMN web modeler, all published on GitHub. The first iteration is kicking off next week, and the web modeler will be available later this year. Unlike yesterday’s demonstrators who firmly expressed the value of no-code BPM implementations, Jakob jumped straight into code to show how to use the Javascript classes to render BPMN XML as a graphical diagram and add annotations around the display of elements. He showed how these concepts are being used in their cockpit process monitoring product; it could also be used to demonstrate or teach BPMN, making use of functions such as process animation. He demonstrated uploading a BPMN diagram (as XML) to their camunda community site; the site uses the Javascript libraries to render the diagram, and allows selecting specific elements in the diagram and adding comments, which are then seen via a numeric indicator (indicating the number of comments) attached to the elements with comments. He demonstrated some of the starting functionality of the web modeler, but there’s a lot of work to do there still; once it’s released, any developer can download the code and embed that web modeler into their own applications.

We finished the first morning session with Keith Swenson of Fujitsu on letting go of control: we’re back on the topic of agents, which Keith initially defined as autonomous, goal-directed software that does something for you, before pointing out that that describes a lot of software today. He expanded that definition to mean something more…human-like. A personal assistant that can coordinate your communications with those of other domains. These type of agents do a lot of communication amongst themselves in a rules-based dynamic fashion, simplifying and reducing the communication that the people need to do in order to achieve their goals. The key to determining what the personal assistants should be doing is to observe emergent behavior through analytics. Keith demonstrated a healthcare scenario using Cognoscenti, an open-source adaptive case management project; a patient and several different clinicians could set goals, be assigned tasks, review documents and other activities centered around the patient’s care. It also allows the definition of personal assistants to do specific rules-based actions, such as cloning cases and synchronizing documents between federated environments (since both local and cloud environments may be used by different participants in the same case), accepting tasks, and more; copying between environments is essential so that each participant can have their information within their own domain of control, but with the ability to synchronize content and tasks. The personal assistants are pretty simple at this point, but the concept is that they are helping to coordinate communications, and the communications and documents are all distributed via encrypted channels so safer than email. A lot of similarities with Dominic’s intelligent process controllers, but on a more human scale. As many thousand of these personal assistant interactions occur, patterns will begin to emerge of the process flows between the people involved, which can then be used to build more intelligence into the agents and the flows.

bpmNEXT 2014: BPMN MIWG Demo

The BPMN Model Interchange Working Group is all about (as you might guess from the name) interchanging BPMN models between different vendors’ products: something that OMG promised with the BPMN standard, but which never actually worked out of the box due to contradictions in the standard and misinterpretations by some vendors. To finish off Wednesday morning at bpmNEXT, we have a live demo involving 12 different tools with participants in different locations, with Denis Gagne of Trisotech (who chairs the working group) and Falko Menge of camunda (who heads up the test automation subgroup) on the stage, a few others here on the sidelines, some at the OMG meeting in Reston, and some in their offices in France and Poland.

To start, different lanes of the process were designed by four different people on IBM Blueworks Live, Activiti, camunda and W4; each then exported their process models and saved to Dropbox. Denis switched back and forth between the different screens (they were all on a Google Hangout) to show us what was happening as the proceeded, and we could see the notifications from Dropbox as the different files were added. In the second stage, Bonitasoft was used to augment the Blueworks Live model, itp-commerce edited the Activiti model, and Signavio edited the camunda model. In the third stage, ADONIS was used to merge together the lanes created in several of the models (I lost track of which ones) into a single process model, and Yaoqiang used to merge the Signavio and camunda models. Then, the Trisiotech Visio BPMN modeler was used to assemble the ADONIS and Yaoqiang models into the final model with multiple pools. At the end, the final model was imported into a number of different tools: the Trisotech web modeler, the Oracle BPM modeler, the environment from camunda, and directly into to the W4 execution engine (without passing through a modeling environment). Wow.

The files exchanged were BPMN XML files, and the only limitations of which tool to use when was that some only support a single pool so had to be used at the earlier stages where each tool was only modeling a single lane or pool. This is how BPMN was supposed to work, but the MIWG has found some number of inconsistencies with the standard and also some issues with the vendors’ tools that had to be corrected.

They have developed a number of test cases that cover the descriptive and analytic classes within BPMN, and automated tools to test the outcome of different vendors’ modelers. Over 20 BPMN modelers have been tested for import, export and roundtrip capabilities; if you’re a BPMS vendor supporting BPMN 2.0 (or claiming to), you should be on this list because there are a lot of us who just aren’t going to write our own XSLT to translate your models into something that can be read by another tool. If you’re a process designer using a BPMS, shame your vendor into participating because it creates a more flexible and inclusive environment for your design and modeling efforts.

This is hugely valuable work that they’re doing in the working group; note that you don’t have to be an OMG member to get involved, and the BPMN MIWG would love to have others join in to help make this work even better.

We’re off for lunch and a break now, then back for six more sessions this afternoon. Did I mention how awesome bpmNEXT is?

ACM Workshop at BPM2012: BPMN Smackdown by @swensonkeith

In the last portion of the ACM workshop at BPM 2012, we had a couple of short non-research papers, the first of which was by Keith Swenson, in which he posits that BPMN is incompatible with ACM. He starts by saying that it’s not a critique of BPMN in particular, but of any two-dimensional flow diagram notation. He also makes a distinction between production case management and adaptive case management – a distinction that I find to be a bit artificial since I don’t think that there’s a hard line between them – where PCM systems have developers creating systems for people to use, whereas ACM has people doing the work themselves. The distinction between PCM and ACM has created a thin, rarified slice of what remains defined as ACM: doctors and lawyers are favorite examples, and it is self-evident that you’re not going to get either doctors or lawyers to draw event-driven BPMN models with the full set of 100+ elements for their processes, or to follow rigidly defined processes in order to accomplish their daily tasks. Instead, their “processes” should be represented as checklists, so that users can completely understand all of the tasks, and can easily modify the process as required.

He states that drawing a diagram (such as BPMN) requires a level of abstract thinking that is common with developers but not with end users, hence BPMN is really a programming language. Taking all of that together, you can see where he’s coming from, even if you disagree: if a system uses BPMN to model processes, most people will not understand  how BPMN models work [if they are drawn in full complexity by developers, I would add], therefore won’t modify them; if all users can’t modify the process, then it’s not ACM. Furthermore, creating a flow model with temporal dependencies where no such dependencies exist in reality hinders adaptability, since people will be forced to follow the flow even if there is another way to accomplish their goals that might be more appropriate in a particular context.



My problem with this is that BPMN has been used by developers to create complex flow models because both the language and their organization allows them to, but that’s not the only way to use it. You can use a limited subset of BPMN to create flow models – in cases where flow models are appropriate, such as when there are clear temporal dependencies – that are understandable by anyone involved in those processes. You can create a BPMN diagram that is a collection of ad hoc tasks that don’t have temporal dependencies, which is semantically identical to a checklist. You can create alternative views, so that a model may be viewed in different forms by different audiences. In other words, just like Jessica Rabbit, BPMN isn’t bad, it’s just drawn that way.

Process Modeling With BPMN

I’m sitting in on Bruce Silver’s online BPMN course this week: this is the same as his onsite course, just using remote classroom tools to allow him to present and demonstrate to us, then get our feedback using typed chat. It’s a combination of lecture and hands-on, using a 60-day license for the business edition of the itp-commerce BPMN Visio add-in that is included with the course. The course runs 11am-3:30pm (Eastern) for three straight days, which took a bit of schedule juggling to be able to attend most of it; not sure if he is recording this for us to access after the course, which would be a nice benefit especially for those doing the certification exam. I use a lot of BPMN with my customers in my role as a process architect, but Bruce’s knowledge of the standard and its usage far outweigh mine, and I’m sure that I will learn a lot in addition to providing a review of the course for my readers.

He’s using the itp-commerce Visio tool, in spite of the hefty price tag ($975 for the Business Edition, $1,535 for the Professional Edition that also includes serialization; the free edition does not include full BPMN 2.0 support), because it natively supports Bruce’s methodology and style validation, which he covers in his book BPMN Method and Style and uses in this course. There are other Visio add-ons for BPMN 2.0 modeling, including one from Trisotech on the Business Process Incubator site that I’ve been using lately since it has a full-featured (but branded) version that customers can use for free, or the full non-branded version for the price of a BPI premium membership. Visio 2010 supports BPMN natively, but not the 2.0 version – if you’re a big Microsoft Visio customer, you might want to start agitating with Microsoft to include that via a service pack, since their current line seems to be that there isn’t sufficient demand for it yet. Bruce and I both believe that BPMN 2.0 support will become a significant differentiator for modeling products by the end of 2011, and Microsoft really needs to get on board with this if they’re going to be a player in the BPMN 2.0 market. There are some nice features in the itp-commerce tool that we didn’t cover in the course, such as simulation and BPMN 2.0 interchange, but many of those are available in lower-cost alternatives: I think that this is a race to the bottom price-wise, since Microsoft will eventually just include all of this in Visio natively.

He started with some basic definitions of BPMN and how it differs from flowcharts – especially in the area of collaboration, extra-process events and exception handling – highlighting the notions of standardization and of the hierarchical view that allows for inclusion of expandable subprocesses, rather than trying to put everything on one enormous top-level process model. He also covered how BPMN creates a bridge between business analysts who are creating these models, and developers who are making them executable, including the BPM systems that make the models directly executable without a lot of coding. He also discussed what’s not in the BPMN standard, such as user interface for human steps, data models, dynamic work assignments, rules, KPIs and anything to do with the higher-level strategy and goals. Although you may see some of these implemented in a BPMS, those will be done in a proprietary manner, and learning how to do that in that particular tool won’t be transferrable to other tools.

As I often do when I’m presenting a quick look at BPMN in a client presentation, he talked about the full BPMN 2.0 standard, with its new support for choreography and conversation diagrams, execution semantics and an XML schema for model interchange between tools, and highlighted that it’s possible to use the descriptive and analytic subclasses (subsets) of the standard if you don’t need to learn all 100 elements of the standard: the descriptive is for business analysts to be able to model processes as documentation, and the analytic is a minimum subset required to model executable processes.

Bruce keeps bringing it back to the value and benefits of BPMN: why it’s important both in terms of its modeling capabilities, and in the role as a standard for widespread understanding. There are a lot of BPMN detractors, but I don’t see the problem if you don’t try to shove the entire standard down the throats of business people: using the descriptive subclass (plus a few more elements), I’m able to have non-technical business people understand the notation in about 5 minutes, although it would take them a little bit longer to be able to create their own diagrams.

After an hour or so of initial presentation to provide the necessary background, Bruce shared his screen and had us all start up Visio with the itp-commerce add-in, and we started modeling some BPMN. As those of you familiar with BPMN know, there are only three main objects in a BPMN diagram: activities, gateways and events. The fun stuff comes with all the adornments that you can add to those three basic objects to indicate a huge variety of functionality.  We started off with a high-level straight-through order process, then added gateways for the exception paths. We started to learn some of the guidelines from Bruce’s style guide, such as using a gateway not to indicate work but only as a question testing the output state of the previous activity (which I always do), and using a separate end event for each distinct end state (which I rarely do but probably will start, since you can label the end events with the states). I also learned a standard Visio trick for moving the placement of the text on a connector using the Text Block tool, which allows you to snug labels of flows leaving a gateway right up to the gateway regardless of the connector length – cool! There were some great questions from the attendees, such as whether you can eliminate the gateway symbol and just have the labeled flows leaving the preceding activity, as you might in traditional flowcharting; in BPMN, that would denote have all of the paths be executed in parallel, not have one path or the other executed, so that’s not a legal representation of an exclusive OR gateway. Gateways can create a lot of confusion, because in spite of how they are often referred to as “decisions”, the decision is actually made in the previous activity, and the gateway just tests the result of that decision.

A great deal of day 1 alternated between some short presentations (a couple of slides each) on concepts, then exercises that allowed us to model those in diagrams ourselves, reinforcing the concepts immediately. While we were doing the modeling, Bruce would talk about other information about the concept, such as explaining some of the benefits and rules of pools while we were adding pools and lanes to our diagram, or the same for subprocess syntax. We saw some of the less-used but essential constructs such as ad hoc subprocesses, in which the contained activities don’t have a flow, and may be completed in any order (or not at all): this is how BPMN represents case management-style processes, for example, where the possible tasks are known but the order and applicability of any given task is not known. He also pointed out (and quizzed us on) common errors, such as having the same activity within a subprocesses and also in the process that calls it.

By the end of the first day, we had learned all of the Level 1 elements (effectively the BPMN 2.0 descriptive subclass), quite a bit of Bruce’s style guidelines around the use of those elements, and we were creating our own BPMN diagrams using those elements. At the start of day 2, after a recap, Bruce talked about having a BPMN method and style – whether it is his or not – so that there are standardized ways of using BPMN: in spite of it being a standard, it is possible to create diagrams that mean the same thing but look different, and having some standard usage makes it a more powerful communication tool within your organization. His method works toward four basic goals:

  • Structural consistency: a variety of the style points that he’s been covering, such as explicit end states and hierarchical composition
  • Readability: top-down traceability through levels of the process and subprocesses
  • Model completeness: diagram doesn’t require additional documentation to describe the process
  • Shareability with IT: models created by business analysts are aligned with the level 2 models used for executable processes

He then took us through the steps of his method for modeling processes that meets these goals; this was part of the essential intellectual property that he had to pass on to us (as opposed to the most standard BPMN on day 1), but too dense with slides and lecture rather than hands-on. Following that, he went through his BPMN style guides, which were also all lecture, but went much more quickly since these tended to be quick rules rather than larger concepts that we saw in the method section, and also we had covered a lot of these already in the exercises and the method. He did a blog post with a first cut of the rules and style of BPMN, both the standard BPMN rules and his style guidelines, plus a later post showing an example of reworking a process model to meet his style guidelines. The first is a great reference if you decide not to cough up for the itp-commerce product that will do the style validations for you; in reality, once you start using these for a while, they’ll become second nature and you won’t need to have them validated. He provided an updated list of the rules as part of the course, and has given me permission to republish, which I will do in a following post.

For the second half of day 2, we moved on to Level 2 BPMN elements (Analytic subclass) with more of the hands-on exercises on events: one of my favorite topics, since events are the most powerful yet the least understood of all BPMN elements. As Bruce pointed out, no one (not even him, and certainly not me) memorizes the table of 50 or so possible event permutations: for level 1 (descriptive subclass used by business analysts), you only need to know six of them (all start and end events), although I usually teach business analysts a couple of the intermediate events from level 2 as well. He suggests focusing on message, timer and error events, adding another nine to the six we’ve already seen; if you master these 15, then have to look up the others as required, you’re way ahead of most people using BPMN today.

Day 3 saw us still covering events via a combination of lecture and exercises; after timers on day 2, we moved on to message events and had a ton of great discussions on some of the finer points of BPMN usage (e.g., a script task that executes purely within the BPMS versus a service task that calls an external service). Message events are critical if you want to start modeling executable processes; intermediate message events are essential for automated messaging outside the process or organization, and boundary message events manage external events that modify or interrupt processes while in flight.  We also covered error events, and Bruce provided some supplementary information on other event types. Interestingly, Bruce is constantly reevaluating how BPMN can and should be used, with some changes over what he published in his book. He was a bit short on time for the last part of day 3 – the course timing definitely needs a bit of work – but we branched into splits and joins, went around iterations, and waded through multi-participant pools (which had an unfortunate effect on my brain).

He finished up with model validation using the itp-commerce add-in to Visio, which optionally validates against his style guide as well as the standard BPMN rules. As he puts it, any modeling tool that doesn’t provide validation against the BPMN specification is a toy, suitable only for drawing nice pictures. I suppose you could argue that after Bruce’s course, you will be able to validate automatically as you model so don’t need a tool to do it, but think of it as being like a spell-checker for process models: we all need a little help once in a while. 😉

He invited us all to go ahead and do the certification exam (no extra fee if done in the next 60 days), and showed one of the example multiple choice questions that had four possible answers, and received votes for all four of the answers from the class, showing that this is not quite as simple as it seems (yes, I got the right answer). If we pass that part, then we have to create a process model from one of our own processes of a specific level of complexity, following his method and style, and submit it for his review. Suffice it to say that certification via his BPMessentials exam will actually mean that you have mad BPMN skillz, it’s not just a certificate for showing up for the course.

Some potential improvements for the course:

  • It’s a bit hard to demo and talk at the same time, and Bruce could have captured screencams of some parts of the Visio demos to playback for us while he was discussing what we needed to do next, then just gone to Visio live for the finer points of demonstration; that would have made it easier for him to focus on describing what was happening rather than focusing on the actual drawing activity.
    • Some of the finer lecture points (such as going through the method and concepts) were a bit slow-moving, since Bruce would talk to one very dense slide for a number of minutes rather than having multiple slides with less information to absorb. Some restructuring of the slides would improve this, especially to show model snippets on the same page as the concept points, or possibly a much quicker summary to start, then return to the concepts later to reinforce.
    • The non-modeling exercises (e.g., defining the process scope given a specific scenario) didn’t work very well online, since there’s no fluid interaction with the participants, just the chat window with Bruce responding to the chat questions verbally when he sees them. In a regular classroom environment, he could ask for verbal solutions and write it out on a chart as they developed more collaboratively; here, all he could do was walk through the exercise and his solution. I’m not sure that a more interactive online collaboration tool would make a big dent in this problem; some things are just made for face-to-face (or at least audio) interaction. These sections could be enhanced by showing the process model solution at the same time as the exercise description – or better yet, a screencam – so that as he walks through it, he could point out how it manifests in the process.
    • It would be great to see a summary of the redundant elements in BPMN 2.0, with the preferred one (if one is preferred) indicated. For example, send/receive tasks are the same as intermediate throwing/catching message events except if you want to put boundary events (e.g., for error handling or timeouts) on the tasks in an executable process; a gateway is implied to be XOR if it has no marker; parallel split gateways and exclusive merge gateways are implied without showing the gateway. Although some of these are reflected in Bruce’s style guidelines, we just stumbled across some of them throughout the course.

    I definitely learned some of the finer points of BPMN that I didn’t already know, and I will be going back to some BPMN diagrams that I’m working on with clients and clean up the style a bit with what I’ve learned. With this being an online course, I could multitask with other activities during the parts that were review for me; for a BPMN newbie (the target audience), the pace would have been just about right.

    There are few people who have this depth of BPMN knowledge, and Bruce is the only one who I know who is doing this as a professional trainer: his is the only BPMN course that I recommend to my clients. He needs to work out a few bumps in how the online course works, but in general, I thought this was a great course, perfect for a business analyst who is already doing some process modeling but doesn’t know any BPMN, but also informative for those of us with some prior knowledge of BPMN.