CMMN – Column 2

My writing on the Trisotech blog: better analysis and design of processes

I’ve been writing some guest posts over on the Trisotech blog, but haven’t mentioned them here for a while. Here’s a recap of what I’ve posted over there the past few months:

In May, I wrote about designing loosely-coupled processes to reduce fragility. I had previously written about Conway’s Law and the problems of functional silos within an organization, but then the pandemic disruption hit and I wanted to highlight how we can avoid the sorts of cascading supply chain process failures that we saw early on. A big part of this is not having tightly-coupled end-to-end processes, but separating out different parts of the process so that they can be changed and scaled independently of each other, but still form part of an overall process.

In July, I helped to organize the DecisionCAMP conference, and wrote about the BPMN-CMMN-DMN “triple crown”: not just the mechanics of how the three standards work together, but why you would choose one over the other in a specific design situation. There are some particular challenges with the skill sets of business analysts who are expected to model organizations using these standards, since they will end up using more of the one that they’re most familiar with regardless of its suitability to the task at hand, as well as challenges for the understandability of multi-model representations that require a business operations reader of the models to be able to see how this BPMN diagram, that CMMN model and this other DMN definition all fit together.

In August, I focused on better process analysis using analytical techniques, namely process mining, and gave a quick intro to process mining for those who haven’t seen it in action. For several months now, we haven’t been able to do a lot of business “as is” analysis through job shadowing and interviews, and I put forward the idea that this is the time for business analysts to start learning about process mining as another tool in their kit of analysis techniques.

In early September, I wrote about another problem that can arise due to the current trend towards distributed (work from home) processes: business email compromise fraud, and how to foil it with better processes. I don’t usually write about cybersecurity topics, but I have my own in-home specialist, and this topic overlapped nicely with my process focus and the need for different types of compliance checks to be built in.

Then, at the end of September, I finished up the latest run of posts with one about the process mining research that I had seen at the (virtual) academic BPM 2020 conference: mining processes out of unstructured emails, and queue mining to see the impact of queue congestion on processes.

Recently, I gave a keynote on aligning intelligent automation with incentives and business outcomes at the Bizagi Catalyst virtual conference, and I’ve been putting together some more detailed thoughts on that topic for this months’ post. Stay tuned.

Disclosure: Trisotech is my customer, and I am compensated for writing posts for publication on their site. However, they have no editorial control or input into the topics that I wrote about, and no input into what I write here on my own blog.

Take Mike Marin’s CMMN survey: learn something and help CMMN research

CMMN diagram from OMG CMMN 1.0 specification document

Mike Marin, who had a hand in creating FileNet’s ECM platform and continued the work at IBM as chief architect on their Case Manager product, is taking a bit of time away from IBM to complete his PhD. He’s doing research into complexity metrics for the Case Management Model and Notation standard, and he really needs people to complete a survey in order to complete his empirical research. The entire thing will take 45-60 minutes, and can be completed in multiple sessions; 30-40 minutes of that is an optional tutorial, which you can skip if you’re already familiar with CMMN.

Here’s his invitation to participate (feel free to share with your process and case modeling friends):

We are conducting research on the Case Management Modeling and Notation (CMMN) specification and need your help. You don’t need to be familiar with CMMN to participate, but you should have some basic understanding of process technology or graphical modeling (for example: software modeling, data modeling, object modeling, process modeling, etc.), as CMMN is a new modeling notation. Participation is voluntary and no identifiable personal information will be collected.

You will learn more about CMMN with the tutorial; and you will gain some experience and appreciation for CMMN by evaluating two models in the survey. This exercise should take about 45 to 60 minutes to complete; but it can be done in multiple sessions. The tutorial is optional and it should take 30 to 40 minutes. The survey should take 15 to 20 minutes. You can consider the survey a quiz on the tutorial.

As an appreciation for your collaboration, we will donate $6 (six dollars) to a charity of your choice and we will provide you with early results of the survey.

You can use the following URL to take the tutorial and survey. The first page provides more information on the project.

http://cmmn.limequery.org/index.php/338792?lang=en

He wrote a more detailed description of the research over on BPTrends.

Mike’s a former colleague and a friend, but I’m not asking just because of that: he’s also a Distinguished Engineer at IBM and a contributor to standards and technology that make a huge impact in our field. Help him out, take the survey, and it will help us all out in the long run.

Camunda BPM 7.5: CMMN, BPMN element templates, and more

I attended an analyst briefing earlier today with Jakob Freund, CEO of Camunda, on the latest release of their product, Camunda BPM 7.5. This includes both the open source version available for free download, and the commercial version with the same code base plus a few additional features and professional support. Camunda won the “Best in Show” award at the recent bpmNEXT conference, where they demonstrated combining DMN with BPMN and CMMN; the addition of the DMN decision modeling standard to BPMN and CMMN modeling environments is starting to catch on, and Camunda has been at the front edge of the wave to push that beyond modeling into implementation.

They are managing to keep their semi-annual release schedule since they forked from Activiti in 2013: version 7.0 (September 2013) reworked the engine for scalability, redid the REST API and integrated their Cockpit administration module; 7.1 (March 2014) focused on completing the stack with performance improvements and more BPMN 2.0 support; 7.2 (November 2014) added CMMN execution support and a new tasklist; 7.3 (May 2015) added process instance modification, improved authorization models and tasklist plugins; and 7.4 (November 2015) debuted the free downloadable Camunda Modeler based on the bpmn.io web application, and added DMN modeling and execution. Pretty impressive progression of features for a small company with only 18 core developers, and they maintain their focus on providing developer-friendly BPM rather than a user-oriented low-code environment. They support their enterprise editions for 18 months; of their 85-90 enterprise customers, 25-30% are on 7.4 with most of the rest on 7.3. I suspect that a customer base of mostly developers means that customers are accustomed to the cycle of regression testing and upgrades, and far fewer lag behind on old versions than would be common with products aimed at a less technical market.

Today’s 7.5 release marches forward with improvements in CMMN and BPMN modeling, migration of process instances, performance and user interface.

Oddly (to some), Camunda has included Case Management Model & Notation (CMMN) execution in their engine since version 7.2, but has only just added CMMN modeling in 7.5: previously, you would have used another CMMN-compliant modeler such as Trisotech’s then imported the model into Camunda. Of course, that’s how modeling standards are supposed to work, but a bit awkward. Their implementation of the CMMN modeler isn’t fully complete; they are still missing certain connector types and some of the properties required to link to the execution engine, so you might want to wait for the next version if you’re modeling executable CMMN. They’re seeing a pretty modest adoption rate for CMMN amongst their customers; the messaging from BPMS vendors in general is causing a lot of confusion, since some claim that CMMN isn’t necessary (“just use ad hoc tasks in BPMN”), others claim it’s required but have incomplete implementations, and some think that CMMN and BPMN should just be merged.

On the BPMN modeling side, Camunda BPM 7.5 includes “element templates”, which are configurable BPMN elements to create additional functionality such as a “send email” activity. Although it looks like Camunda will only create a few of these as samples, this is really a framework for their customers who want to enable low-code development or encapsulate certain process-based functionality: a more technical developer creates a JSON file that acts as an extension to the modeler, plus a Java class to be invoked when it is executed; a less technical developer/analyst can then add an element of that type in the modeler and configure it using the properties without writing code. The examples I saw were for sending email and tweets from an activity; the JSON hooked the modeler so that if a standard BPMN Send Task was created, there was an option to make it an Email Task or Tweet Task, then specify the payload in the additional properties that appeared in the modeler (including passed variables). Although many vendors provide a similar functionality by offering things such as Send Email tasks directly in their modeling palettes, this appears to be a more standards-based approach that also allows developers to create their own extensions to standard activities. A telco customer is using Camunda BPM to create their own customized environments for in-house citizen developers, and element templates can significantly add to that functionality.

The process instance migration feature, which is a plugin to the enterprise Cockpit administration module but also available to open source customers via the underlying RESET and Java APIs, helps to solve the problem of what to do with long-running processes when the process model changes. A number of vendors have solutions for this, some semi-automated and some completely manual. Camunda’s take on it is to compare the existing process model (with live instances) against the new model, attempt to match current steps to new steps automatically, then allow any step to be manually remapped onto a different step in the new model. Once that migration plan is defined, all running instances can be migrated at once, or only a filtered (or hand-selected) subset. Assuming that the models are fairly similar, such as the addition or deletion of a few steps, this should work well; if there are a lot of topology changes, it might be more of a challenge since there could need to roll back instance property values if instances are migrated to an earlier step in the process.

They have also improved multi-tenancy capabilities for customers who use Camunda BPM as a component within a SaaS platform, primarily by adding tenant identifier fields to their database tables. If those customers’ customers – the SaaS users – log in to Cockpit or a similar admin UI, they will only see their own instances and related objects, without the developers having to create a custom restricted view of the database.

They’ve released a simple process instance duration report that provides a visual interface as well a downloadable data. There’s not a lot here, but I assume this means that they are starting to build out a more robust and accessible reporting platform to play catch-up with other vendors.

Lastly, I saw their implementation of external task handling, another improvement based on customer requests. You can see more of the technical details here and here: instead of a system task calling an external task asynchronously then wait for a response, this creates a queue that an external service can poll for work. There are some advantages to this method of external task handling, including easier support for different environments: for example, it’s easier to call a Camunda REST API from a .NET client than to put a REST API on top of .NET; or to call a cloud-based Camunda server from behind a firewall than to let Camunda call through your firewall. It also provides isolation from any scaling issues of the external task handlers, and avoids service call timeouts.

There’s a public webinar tomorrow (June 1) covering this release, you can register for the English one here (11am Eastern time) and the German one here (10am Central European time).

bpmNEXT 2016 demos: IBM, Orquestra, Trisotech and BPM.com

On the home stretch of the Wednesday agenda, with the last session of the four last demos for the day.

BPM in the Cloud: Changing the Playing Field – Eric Herness, IBM

IBM Bluemix process-related cloud services, including cognitive services leveraging Watson. Claims process demo that starts by uploading an image of a vehicle and passing to Watson image recognition for visual classification; returned values show confidence in vehicle classification, such as “car”, and sends any results over 90% to the Alchemy taxonomy service to align those — in the demo, Watson returned “cars” and “sedan” with more than 90% confidence, and the taxonomy service determined that sedan is a subset of cars. This allows routing of the claim to the correct process for the type of vehicle. If Watson has not been trained for the specific type of vehicle, the image classification won’t be determined with a sufficient level of confidence, and it will be passed to a work queue for manual classification. Unrecognized images can be used to add to classifier either as example of an existing classification or as a new classification. Predictive models based on Spark machine learning and analytics of past cases create predictions of whether claim should be approved, and the degree of confidence in that decision; at some point, as this confidence increases, some of the claims could be approved automatically. Good examples of how to incorporate cognitive computing to make business processes smarter, using cognitive services that could be called from any BPM system, or any other app that can call REST services.

Model, Generate, Compile in the Cloud and Deploy Ready-To-Use Mobile Process Apps – Rafael Bortolini and Leonardo Luzzatto, CRYO/Orquestra

Demo of Orquestra BPMS implementation for Rio de Janeiro’s municipal processes, e.g., business license requests. From a standard worklist style of process management, generate a process app for a mobile platform: specify app name and logo, select app functionality based on templates, then preview it and compile for iOS or Android. The .ipa or .apk files are generated ready for uploading to the Apple or Google app stores, although that upload can’t be automated. Full functionality to allow mobile user to sign up or login, then access the functionality defined for the app to request a business license. Although an app is generated, the data entry forms are responsive HTML5 to be identical to the desktop version. Very quick implementation of a mobile app from an existing process application without having to learn the Orquestra APIs or even do any real mobile development, but it can also produce the source code in case this is just wanted as a quick starting point for a mobile development project.

Dynamic Validation of Integrated BPMN, CMMN and DMN – Denis Gagné, Trisotech

Kommunicator tool based on their animation technology that animates models, which allows tracing the animation directly from a case step in the BPMN model to the CMMN model, or from a decision step to the DMN model. Also links to the semantic layer, such as the Sparx SOA architecture model or other enterprise architecture reference models. This allows manually stepping through an entire business model in order to learn and communicate the procedures, and to validate the dynamic behavior of the model against the business case. Stepping through a CMMN model requires selecting the ad hoc tasks as the case worker would in order to step through the tasks and see the results; there are many different flow patterns that can emerge depending on the tasks selected and the order of selection, and stages will appear as being eligible to close only when the required tasks have been completed. Stepping through a DMN model allows selecting the input parameters in a decision table and running the decision to see the behavior. Their underlying semantic graph shows the interconnectivity of all of the models, as well as goals and other business information.

Simplified CMMN – Lloyd Dugan, BPM.com

Last up is not a demo (by design), but a proposal for a simplified version of CMMN, starting with a discussion of BPMN’s limitations in case management modeling: primarily that BPMN treats activities but not events as first-class citizens, making it difficult to model event-driven cases. This creates challenges for event subprocesses, event-driven process flow and ad hoc subprocesses, which rely on “exotic” and rarely used BPMN structures and events that many BPMN vendors don’t even support. Moving a business case – such as an insurance claim – to a CMMN model makes it much clearer and easier to model; the more unstructured that the situation is, the harder it is to capture in BPMN, and the easier it is to capture in CMMN. Proposal for simplifying CMMN for use by business analysts include removing PlanFragment and removing all notational attributes (AutoComplete, Manual Activitation, Required, Repetition) that are really execution-oriented logic. This leaves the core set of elements plus the related decorators. I’m not enough of a CMMN expert to know if this makes complete sense, but it seems similar in nature to the subsets of BPMN commonly used by business analysts rather than the full palette.

BPM Cyber Monday: Camunda 7.2 Adds Tasklist And CMMN

I caught up with Jakob Freund and Daniel Meyer of camunda last week in advance of their 7.2 release; with 1,700 person-days of work invested in this April-November release cycle, this includes a new tasklist application, an initial implementation of the Case Management Model and Notation (CMMN) standard, developer accelerators particularly for non-Java developers, and performance and stability improvements. You can hear more about the new release and see a demo in their webinar on Wednesday this week, and read their blog post about it. [Update: you can see the webinar replay here and the slides here, no registration required.]

It’s been interesting to track the progress of camunda and Activiti after camunda forked from the Activiti project in early 2013, since they are targeting slightly different markets but still offer competitive solutions in the open source BPM space. Last week, I wrote about Activiti’s recent release of their BPM Suite, which includes end-user task list and forms interfaces and tools for for “citizen developers” (read: non-hard-core-Java developers); we are seeing some similar themes in the new camunda release, although camunda sticks closer to a true open source model by releasing pretty much all of the code as part of the open source project, while Activiti is making everything except the core engine part of their commercial product.

The biggest news of this release is the tasklist app, since this marks a completely new foray by camunda into end-user interfaces that remain part of the open source project. Developed in conjunction with their customer DAB bank, it’s a single-page JavaScript application that is usable out of the box but intended to be developer-friendly for customization. The 3-panel UI (inspired by MS Outlook, although I thought that might not be considered very inspiring by some) shows filters, the task list resulting from the selected filter, and the details of the selected task. Any of the three columns can be collapsed or expanded by the user: you might want to maximize the active task form while completing a task, for example, or collapse the form to see just the filters and task list for quickly sorting or triaging work.

Filters can be created using expressions based on drop-down lists of instance properties, and can be shared for use by other users or groups. Filters can also specify process variables so that those variables are visible in the task list and can be used for searching and sorting, which makes it easier to locate a specific task without having to click on each task to see the details. If permissions allow, other user’s tasks can be included in the task list. During our demo, they pointed out the ability to use keyboard controls to navigate through the task list, something that was suggested by the users: having seen many keyboard-centric users slowed down by having to use a mouse for controlling their screens, this was not a surprise to me, but I think that many software developers don’t think about the needs of the old-school keyboarding users.

They have had to improve their APIs to expose more information and functionality to support the tasklist app, which helps developers even if they are not using the tasklist. For the forms interface of an active task, they created the camunda JavaScript Forms SDK to simplify the connection to their REST APIs, allowing HTML form controls to be bound directly to process variables so that the SDK handles the interactions with the process engine as well as type conversions. This can be used for the forms interface even if you replace the tasklist app with your own HTML5 UI, and the SDK optionally integrates with AngularJS for additional features such as form validation. There are some nice page/process-flow control features for tasks: the option to use a Save button as well as Complete, which saves the task in place (with any instance data updates) but does not move it through the process flow; and the ability for a user completing a task to directly access the following task for this instance if it is assigned to them.

The task detail pane of the UI contains more than just the task form: it also has a history tab showing events, due dates and comments; a diagram tab highlighting the current task within the BPMN diagram; and a description tab that I didn’t see, but I assume can contain task instructions.

The other new major user-facing functionality is support for CMMN; 2015 is definitely going to be the year when we see the BPM vendors pile on here, and camunda is out in front with this. Like many other BPM vendors, camunda’s BPMN implementation does not support ad hoc activities – arguably, support for events and ad hoc activities provides most of what is required for case management – so they are using the Trisotech CMMN modeler instead of their own modeler, but executing on the same core camunda engine which exposes standard engine features such as REST APIs and scripting. Cases can be instantiated through the API directly by a web form or other event, or can be instantiated using a call from a structured process. In turn, cases can instantiate structured processes through a call from an activity. This covers all of the use cases along the structured/unstructured spectrum: completely pre-defined processes, pre-defined with ad hoc exceptions, ad hoc with pre-defined fragments, and completely ad hoc. Calls from BPMN that instantiate a CMMN case can be synchronous (i.e., wait for completion), or asynchronous.

Case tasks, once instantiated, will appear in the tasklist along with those from BPMN processes; however, note that ad hoc tasks will require some sort of custom UI to allow a user to instantiate them if they are not triggered automatically by events or other tasks. They have nothing on the out of the box tasklist app to do this, although I can envision that they might extend this in the future to allow a case owner or participant to see and trigger ad hoc case tasks.

CMMN is still pretty new, although the concepts of case management have been around for a long time; camunda has some customers testing and providing feedback on their CMMN implementation, and they are expecting requirements and capabilities to emerge as they get more practical experience with it. By providing an open source engine that supports CMMN, they also hope to contribute to the CMMN standard and its use cases in general since others can use their engine to test the standard.

Moving on to some of the developer and non-functional features in the new release, they are positioning their developer-friendly methods against “death by property panel” in zero-code BPM suites. The subject of many conversations amongst BPM practitioners, fully model-driven development BPM suites may not require Java coding but often are suitably complex that they require developer skills to use them; this can frustrate developers who are saddled with inefficient (to them) environments for creating applications. However, not every developer is a hard-core Java coder, and many organizations have access to a middle ground of people with development skills but who want to use lighter-weight languages, often regional IT resources embedded within business units. Where Activiti’s new citizen developer tools leans more to the less-technical, model-driven developer, camunda focuses on developers using scripting languages such as JavaScript, Groovy, Jython and JRuby by allowing these languages to be used in place of the expression language or custom Java code within their implementation of BPMN models, including listeners, script tasks and sequence flow conditions. Wrappers for templates, e.g., XSLT, allow these to be added for composing or mapping data. There’s a new API for implementing connectors, with SOAP and REST connectors included out of the box with 7.2. Finally, Java objects are now serialized in XML/JSON rather than byte streams, allowing other languages – and BPMN processes, via APIs – to introspect and consume them directly without converting to Java objects, and making it possible to easily combine Java with other languages within an implementation.

On the execution performance side, camunda likes to distinguish their product within the open source BPM field as being for high-load straight-through processing rather than just manual activities, where they define “high-load” as 10 process instances per second: they’ve been steadily improving execution engine performance since the split and are even working on implementing with alternative storage/computing strategies such as in-memory grids (I jokingly asked if we would see camunda on HANA any time soon). In tests on an earlier 7.x release, they were already 10-30x (that’s times, not percent) faster than Red Hat’s jBPM; 7.2 improves the caching and load balancing, and enhances the asynchronous history logging to allow the level of data logged to be configured by process class and activity. This helps provides the level of scalability that they need for their highest-volume customers, such as telcos, that may be executing more than 1,000 process instances per second.

The camunda open source community edition is available for download and they will be pushing out updates to their Enterprise subscription customers. Check out their upcoming webinar, or sign up for the webinar and watch the recording later.

Disclosure: camunda has been my customer during 2014 for services including a webinar, white paper and the keynote at their user conference. However, I have not been compensated in any way for researching and writing this review.