Camunda Community Day: @CamundaBPM technical sessions

I’m a few weeks late completing my report on the Camunda Community Day. The first part was on the community contributions and sessions, while the second half documented here is about Camunda showing new things that could be used by the community developers in the audience.

First up was Vladimirs Katusenoks, core developer on BPMN.io, with a presentation on bpmn-js: how it works, and how to extend it with custom functionality such as adding color to BPMN diagrams, which is a permitted extension to BPMN XML. His live coding presentation showed changing the colour of a shape background, either statically in code for the element class or by adding a colour picker to an individual element context palette; this was based on the bpmn-js core BPMN functionality, using bpmn-moddle to read/write into the metamodel and diagram-js to render it. There are a number of other bpmn-js examples on Github.

Next, Felix Müller discussed KPI management, expanding on his August blog post on the topic. KPI management is based on quantitative indicators for process cycle-time improvement, including cycle time and overdue time, plus definitions of the time period, unit of measure and calculation method. In Camunda, KPIs are defined in the Modeler, then monitored in Cockpit. He showed how to use the concept of element templates (that extend core definitions) to create custom fields on collaboration object (process) or individual tasks, e.g., KPI unit (hours, days, minutes) and KPI threshold (number). In Cockpit, this appears as a new tab for KPI Overview, showing a list of individual instances and target/current/average duration, plus an indicator of overdue status of the instance and any contained tasks; there is also a decorator bubble on the top right of the task on the process model to show the number of overdue instances on the aggregate model, or overdue status as a check mark or exclamation on individual models. The Cockpit modifications were done by creating a plug-in to display KPI statistics, which queries and calculates on the fly – a potential performance problem that might be improved through pre-aggregation of statistics. He also demonstrated how to modify this basic KPI model to include an expected duration as well as maximum duration. A good start, although I think there’s a lot more that’s needed here.

Thorsen Lindhauer, a Camunda core BPM developer, discussed how to contribute to the Camunda open source community, both at camunda.org (engine and desktop modeler, same as the commercial product) and bpmn.io (JS tools). Possible contributions include answering questions on forums; logging error reports; documenting ideas for new functionality; and working on code. Code contributions typically start by having a forum discussion about planned new functionality, then a decision is made on whether it will be core code (higher quality standards since it will become part of the commercial product, and will eventually be maintained by Camunda) versus a community extension; this is followed by ongoing development, merge and release cycles. Camunda is very supportive of community contributions, even if they don’t become part of the core product: community involvement is critical to the health of any open source project.

The last presentation of the community day was Daniel Meyer discussing the product roadmap. The next release, 7.6, will be on November 30 – they have a strict twice-yearly release cycle. This release includes updates to DMN, CMMN, BPMN, rolling updates, Cockpit features, and UI/UX in web apps; I have captured a few notes here but see the linked roadmap for a more complete and accurate description and the online documentation as it is rolled out.

  • DMN:
    • Simpler decision table editing with drop-down lists of comparison/range operators instead of having to remember FEEL or Juel syntax
    • Ability to add list of selection values (advanced mode still exists for full flexibility)
    • Decisions with literal expressions
    • DMN engine performance 4-6x faster
    • Support for decision requirements diagrams/graphs (DRD/DRG) that can link decision tables; visualization in Modeler and Cockpit are not there yet but the structures are supported – in my experience, this is typical of Camunda, which builds and releases the engine capabilities early then follows with the visualization, allowing for a quicker start for executable diagrams
  • CMMN:
    • Modeler now completely models CMMN including technical attributes such as listeners
    • Cockpit (visualization still incomplete although we saw a brief view) will allow linking models of same or different types
    • Engine feature and functionality improvements
  • Rolling updates allow Camunda process engine to be updated without shutdown: guaranteed backwards compatibility of database schema to allow database to be updated first, then roll updates of engines by taking each offline individually and allowing load balancer to reroute sessions.
  • BPMN:
    • BPMN conditional event supported
    • Improved modeling including labels, collapsing/expanding subprocesses to switch between view types, and field injections in property panel.
  • Cockpit:
    • More flexible/granular human task monitoring
    • New welcome page with links to apps (Cockpit, Tasklist, Admin), user profile, and frequent links
    • Batch operations (cancel, suspend, etc.) based on batch action capability built for instance migration
    • CMMN and DMN DRD visualization

Daniel discussed some other minor improvements based on customer feedback, plus plans for 2017, including a web modeler for collaborative BPMN, CMMN and DMN modeling via a SaaS offering and a future on-premise version. They finished the day with a poll and community feedback to establish priorities for future versions.

I stayed on for the second day, which is actually a separate conference: BPMCon for Camunda’s enterprise (commercial) customers. Rather, I stayed on for Neil Ward-Dutton’s keynote, then ducked out for most of the rest of day, which was in German. Neil’s keynote included results from workshops that he has done with executives on digital transformation, and how BPM can be used to create the bridges between the diverse parts of a digital business (internal to external, automated to people-centric), while tracking and coordinating the work that flows between the different areas.

Disclaimer: Camunda paid my travel expenses to attend both conference days. I was not compensated in any way for attending or for writing this post, and the opinions here are my own.

Camunda Community Day: community contributions

Two years ago, I attended Camunda’s open source community day, and gave the opening keynote at their enterprise user conference the following day. I really enjoyed my experience at the open source day, and jumped at the chance to attend again this year – and to visit Berlin.

The first day was the community day, where users of Camunda’s open source software version (primarily developers) talk about what they’re doing with it, plus some of the contributions that the community is making to the project and updates from Camunda on new features on the horizon. To break this up a bit – since I’m already a week after the conference and want to get something out there – I’ll cover the community sessions in this post, then the Camunda technical sessions and a bit about the enterprise conference in a later post.

The first presentation was by Oliver Hock of Videa Project Services, demonstrating robot control using a LEGO Mindstorms robot to solve a Rubix cube. He showed how they used BPMN to define movements and decision tables to determine the move logic, then automated the solution using Camunda BPM. Although you may never want to build a robot to solve a Rubix cube, there are a lot of other devices out there that, like the Mindstorms robot, are controlled via Java APIs; Hock’s design showed how these Java-enabled devices can make use of higher-level modeling constructs such as BPMN and decision tables.

Next up was Jan Galinski of Holisticon to show the Spring Boot community code extension – an example of how the community of Camunda open source users give back to the open source project for everyone’s benefit. Spring Boot is a microservices framework allowing for fast deployment of web applications with a minimal amount of overhead; the Spring Boot starter extension to Camunda allows for using Camunda without a Java application server to essentially provide Camunda apps as microservices. The extension, consisting of about 5,000 lines of code, has been developed over two years with 10 contributors, including both community and Camunda contributors. Galinski showed a live coding demo of replacing JBoss server with Spring Boot starter in a Camunda application to show how this works; he has also written a post on the Camunda community site on the 1.3.0 version of Camunda BPM Spring Boot for more technical details. Although granualar process apps such as this are easier from a devops perspective in terms of deployment and scalability, the challenge is that there is no single point of entry for an end user to look at a worklist (for example). We saw some methods for dealing with this, where a workload service collects information from individual process services with the help of the Camunda BPM Reactor plugin and aggregates them; a federated task list is under development to bring together tasks from multiple process servers into a single list, with a simple completion form. Galinski walked through the general architecture for this, and noted that they are working on making this an official extension. Update: Jan Galinski pointed out in the comments that it was Simon Zambrovski  (also of Holisticon) who did the portion on the presentation on cloud, universal tasklist and event processing — I missed the transition and his name in my hasty note-taking.

Jarl Friis of the Danish tax authority (SKAT) presented their use of the Camunda decision engine: they are using only decision services, and not the BPM capabilities, which likely makes them unusual as a Camunda customer. There are a couple of applications for them: first is to raise data quality in financial reporting to the IRS (for FATCA requirements), where they receive data from Danish financial institutions and have to process it into a specific XML format to send to the IRS. Although many of the data cleansing and transformation rules are in the XML schema definitions, some are not amenable to that format and are being defined in DMN decision tables instead. As this has rolled out, they see that decision tables give them an easier way to respond to annual rule changes, although their business people are not yet trained to make changes to the decision tables. That has resulted in developers having to make the decision table changes and test the results, which is one of the challenges that they have had to deal with: some of the developer test frameworks replicated the original decision table logic in code, which effectively tested the decision table implementation rather than the business logic. That test framework, of course, no longer worked when the decision table was changed, and Friis’ message to the audience was that organizations have to deal with challenges of ownership and responsibility for rules as well as rules testing.

Niall Deehan of Camunda gave a great presentation on on modeling anti-patterns: snaking models that are often used to fit models onto a single sheet of paper (instead, use model with happy path down the centre from left to right); inappropriate use of BPMN versus CMMN (e.g., voting scenarios); inappropriate use of BPMN versus process engine or Cockpit capabilities (e.g., service call with error exceptions for null pointer, bad response, service down); too many listeners on tasks (masks problems and pushes process logic into code, based on concept that analysts’ model should not be changed). He discussed some best practices for consistency: define the symbol set to be used by your analysts and lock down the modeler to remove elements that you don’t want people using; create and maintain your own best practices documentation; use model templates for commonly used activities; and proper training. I would love to see his presentation captured for replay: it was engaging and informative.

The last community presentation was Martin Schimak of plexiti, showing three community extensions used for automating testing of BPMN and CMMN models.  Assert checks and sets the status of tasks in order to drive a process instance through a test scenario. Process Test Coverage visualizes test process paths and checks process model coverage ratio, as covered by individual test methods and entire test classes (e.g., using mockito). Assert Scenario is for writing robust test suites for process models; this was not covered in Schimak’s demo due to time contraints, but you can read more about it on his blog.

Before we started on the Camunda technical presentations, the community award was presented by Camunda as a reward for contributions on extensions: this went to Jan Galinski from Holisticon. It’s really encouraging to see the level of engagement between Camunda and their open source community: Camunda obviously realizes that the community is an important contributor to the success of the enterprise version of the software and the company altogether, and treats them as trusted partners.

Disclaimer: Camunda paid my travel expenses to attend both conference days. I was not compensated in any way for attending or for writing this post, and the opinions here are my own.

BPM Cyber Monday: Camunda 7.2 Adds Tasklist And CMMN

I caught up with Jakob Freund and Daniel Meyer of camunda last week in advance of their 7.2 release; with 1,700 person-days of work invested in this April-November release cycle, this includes a new tasklist application, an initial implementation of the Case Management Model and Notation (CMMN) standard, developer accelerators particularly for non-Java developers, and performance and stability improvements. You can hear more about the new release and see a demo in their webinar on Wednesday this week, and read their blog post about it. [Update: you can see the webinar replay here and the slides here, no registration required.]

It’s been interesting to track the progress of camunda and Activiti after camunda forked from the Activiti project in early 2013, since they are targeting slightly different markets but still offer competitive solutions in the open source BPM space. Last week, I wrote about Activiti’s recent release of their BPM Suite, which includes end-user task list and forms interfaces and tools for for “citizen developers” (read: non-hard-core-Java developers); we are seeing some similar themes in the new camunda release, although camunda sticks closer to a true open source model by releasing pretty much all of the code as part of the open source project, while Activiti is making everything except the core engine part of their commercial product.

Camunda Tasklist - 3-panel UIThe biggest news of this release is the tasklist app, since this marks a completely new foray by camunda into end-user interfaces that remain part of the open source project. Developed in conjunction with their customer DAB bank, it’s a single-page JavaScript application that is usable out of the box but intended to be developer-friendly for customization. The 3-panel UI (inspired by MS Outlook, although I thought that might not be considered very inspiring by some) shows filters, the task list resulting from the selected filter, and the details of the selected task. Any of the three columns can be collapsed or expanded by the user: you might want to maximize the active task form while completing a task, for example, or collapse the form to see just the filters and task list for quickly sorting or triaging work.

Camunda Tasklist - Create new filterFilters can be created using expressions based on drop-down lists of instance properties, and can be shared for use by other users or groups. Filters can also specify process variables so that those variables are visible in the task list and can be used for searching and sorting, which makes it easier to locate a specific task without having to click on each task to see the details. If permissions allow, other user’s tasks can be included in the task list. During our demo, they pointed out the ability to use keyboard controls to navigate through the task list, something that was suggested by the users: having seen many keyboard-centric users slowed down by having to use a mouse for controlling their screens, this was not a surprise to me, but I think that many software developers don’t think about the needs of the old-school keyboarding users.

They have had to improve their APIs to expose more information and functionality to support the tasklist app, which helps developers even if they are not using the tasklist. For the forms interface of an active task, they created the camunda JavaScript Forms SDK to simplify the connection to their REST APIs, allowing HTML form controls to be bound directly to process variables so that the SDK handles the interactions with the process engine as well as type conversions. This can be used for the forms interface even if you replace the tasklist app with your own HTML5 UI, and the SDK optionally integrates with AngularJS for additional features such as form validation. There are some nice page/process-flow control features for tasks: the option to use a Save button as well as Complete, which saves the task in place (with any instance data updates) but does not move it through the process flow; and the ability for a user completing a task to directly access the following task for this instance if it is assigned to them.

Camunda Tasklist - Task within BPMN processThe task detail pane of the UI contains more than just the task form: it also has a history tab showing events, due dates and comments; a diagram tab highlighting the current task within the BPMN diagram; and a description tab that I didn’t see, but I assume can contain task instructions.

The other new major user-facing functionality is support for CMMN; 2015 is definitely going to be the year when we see the BPM vendors pile on here, and camunda is out in front with this. Like many other BPM vendors, camunda’s BPMN implementation does not support ad hoc activities – arguably, support for events and ad hoc activities provides most of what is required for case management – so they are using the Trisotech CMMN modeler instead of their own modeler, but executing on the same core camunda engine which exposes standard engine features such as REST APIs and scripting. Cases can be instantiated through the API directly by a web form or other event, or can be instantiated using a call from a structured process. In turn, cases can instantiate structured processes through a call from an activity. This covers all of the use cases along the structured/unstructured spectrum: completely pre-defined processes, pre-defined with ad hoc exceptions, ad hoc with pre-defined fragments, and completely ad hoc. Calls from BPMN that instantiate a CMMN case can be synchronous (i.e., wait for completion), or asynchronous.

Camunda CMMNCase tasks, once instantiated, will appear in the tasklist along with those from BPMN processes; however, note that ad hoc tasks will require some sort of custom UI to allow a user to instantiate them if they are not triggered automatically by events or other tasks. They have nothing on the out of the box tasklist app to do this, although I can envision that they might extend this in the future to allow a case owner or participant to see and trigger ad hoc case tasks.

CMMN is still pretty new, although the concepts of case management have been around for a long time; camunda has some customers testing and providing feedback on their CMMN implementation, and they are expecting requirements and capabilities to emerge as they get more practical experience with it. By providing an open source engine that supports CMMN, they also hope to contribute to the CMMN standard and its use cases in general since others can use their engine to test the standard.

Moving on to some of the developer and non-functional features in the new release, they are positioning their developer-friendly methods against “death by property panel” in zero-code BPM suites. The subject of many conversations amongst BPM practitioners, fully model-driven development BPM suites may not require Java coding but often are suitably complex that they require developer skills to use them; this can frustrate developers who are saddled with inefficient (to them) environments for creating applications. However, not every developer is a hard-core Java coder, and many organizations have access to a middle ground of people with development skills but who want to use lighter-weight languages, often regional IT resources embedded within business units. Where Activiti’s new citizen developer tools leans more to the less-technical, model-driven developer, camunda focuses on developers using scripting languages such as JavaScript, Groovy, Jython and JRuby by allowing these languages to be used in place of the expression language or custom Java code within their implementation of BPMN models, including listeners, script tasks and sequence flow conditions. Wrappers for templates, e.g., XSLT, allow these to be added for composing or mapping data. There’s a new API for implementing connectors, with SOAP and REST connectors included out of the box with 7.2. Finally, Java objects are now serialized in XML/JSON rather than byte streams, allowing other languages – and BPMN processes, via APIs – to introspect and consume them directly without converting to Java objects, and making it possible to easily combine Java with other languages within an implementation.

Camunda performanceOn the execution performance side, camunda likes to distinguish their product within the open source BPM field as being for high-load straight-through processing rather than just manual activities, where they define “high-load” as 10 process instances per second: they’ve been steadily  improving execution engine performance since the split and are even working on implementing with alternative storage/computing strategies such as in-memory grids (I jokingly asked if we would see camunda on HANA any time soon). In tests on an earlier 7.x release, they were already 10-30x (that’s times, not percent) faster than Red Hat’s jBPM; 7.2 improves the caching and load balancing, and enhances the asynchronous history logging to allow the level of data logged to be configured by process class and activity. This helps provides the level of scalability that they need for their highest-volume customers, such as telcos, that may be executing more than 1,000 process instances per second.

The camunda open source community edition is available for download and they will be pushing out updates to their Enterprise subscription customers. Check out their upcoming webinar, or sign up for the webinar and watch the recording later.

Camunda 7.2

Disclosure: camunda has been my customer during 2014 for services including a webinar, white paper and the keynote at their user conference. However, I have not been compensated in any way for researching and writing this review.

What’s Next In camunda – Wrapping Up Community Day

We finished the camunda community day with an update from camunda on features coming in 7.2 next month, and the future roadmap. camunda releases the community edition in advance of the commercial edition; this is the way that open source should work, but some commercial open source vendors switch that around so that the community version lags by as much as a full version.

The highlights of the 7.2 release are as follows:

  • CMMN-based case management engine, which includes the core activities (stages, human tasks, process tasks, case tasks, milestones and sentries), the base case instance and plan item lifecycle, and a CMMN model API and REST API on a common process engine. They demonstrated a basic case manager UI that can manage cases and the related tasks; I assume that this is really just a demo of what can be done rather than intended as production code. They also don’t have case modeling in their modeler yet, so it’s early times.
  • A variety of functions for speeding development: connectors (currently REST and SOAP), dataformats, templating and scripting (calling external scripts, currently Groovy or Javascript but with others to come)
  • New tasklist, updating the tasklist UI that they released just before announcing camunda as an open source project. It allows filters to be defined, including specifying who can see the results of a filter in addition to the search criteria; that filter then appears as a tab on the task list, in the color defined by the filter author. The sort order can’t currently be defined as part of the filter, but can be set on the general tasklist interface. This adds a third (left) column to the tasklist UI, which also shows the list of tasks and the form for the selected task. Still work to be done, but the new filters capability is a big step up, providing a conceptually similar (but much different graphically) functionality to the Brazos portal filters.

There were a list of other smaller enhancements and fixes, from platform support to performance improvements to new functions.

We also saw some work in progress from the labs. First of all, an update on bpmn.io, which I saw at bpmNEXT earlier this year: a BPMN viewer and web modeler.. The viewer allows embedding of a BPMN diagram into a web page, including adding annotations, overlays and markers on the diagram, via a Javascript API. Check out a live demo here, demonstrating a BPMN diff function based on two similar process diagrams. From the viewer, you can export the model to a file. You can also create BPMN diagrams from scratch or import from a file, either directly on their site or embedded in another page. The modeler is still bit basic, and doesn’t handle containers (pools, lanes, subprocesses) very well yet, but that’s all coming; keep up with new functionality on the bpmn.io blog.

Another lab project is the camunda BPM workbench, a debugging tool that allows inspection of the runtime state of processes alongside the process model, allowing breakpoints to be set in the process model (rather than in code). A console interface allows for interrogation and updating of the process variables as the developer steps through the process. The process model is displayed using the bpmn.io viewer.

At the end of all the roadmap sessions, the audience had a chance to say what was most important for them in terms of what will be implemented when; there were questions about case management, centralized model repositories, bulk runtime operations and other features.

A great half-day; this is the first time that I’ve attended an open source code community day, and it’s quite a different environment from a typical vendor conference. We’re about to enter the beer-drinking portion of the day so I will sign off for today; I’m giving the keynote at the main camunda user conference tomorrow morning, and not sure how much blogging that I’ll do during after that.

Disclaimer: camunda paid my travel expenses to be here today and tomorrow, and is providing a speaking fee for tomorrow’s keynote. I was not compensated for blogging, and the opinions here (and in my keynote) are my own.

camunda Community Day technical presentations

The second customer speaker at camunda’s community day was Peter Hachenberger from 1&1 Internet, describing how they use Signavio and camunda BPM to create their Process Platform, which is in turn used by their clients’ developers for building and executing automated processes. His presentation was primarily about the details of their technical implementation of the platform; they have built some fairly comprehensive tools for monitoring and managing executing processes, many of which are facilitated by changes that they made to the core process engine, including retry behavior, process ID generator, multiple business keys, an asynchronous process starter API, an extended REST API and a few new commands. Since camunda BPM is open source, any customer such as 1&1 can take a copy of the code and make changes to it, optionally returning them to the community if they are valuable to others. There’s a bit of danger in this, in that if you make changes to core functionality (such as the engine) rather than create an extension or plug-in, and those changes do not end up back in the community version, you’re not only on your own for future development on those components but may not be able to upgrade to future versions.

We had a number of short (10 minute) presentations from community members to discuss extensions that they are working on:

  • Grails plugin to add camunda functionality to Grails applications
  • OSGi module extension for greater flexibility and configurability at runtime, including sharing process engines as services
  • Elasticsearch extension to write camunda BPM history data to an elasticsearch cluster to allow full-text searching, enabling more comprehensive analytics
  • camunda mocking extensions for process testing with mockito
  • Cockpit plugin to add interactive graphs and some statistical calculations (e.g., aggregation, regression, min/max) for process monitoring directly on the camunda history database

Some of these extension projects were done by camunda employees, but great to see the external community contributions as well.

Australia Post at camunda Community Day

I am giving the keynote at camunda’s BPMcon conference tomorrow, and since I arrived in Berlin a couple of days early, camunda invited me to attend their community day today, which is the open source community meeting. Nice to see such a great turnout — 70 or 80 people? — and I definitely wasn’t the one to travel the furthest to get here, since today’s opening presentation is from Rob Parker of Australia Post. Australia has a lot of the same issues as Canada when it comes to nation-wide services such as post, since we both have a lot of geography and not a lot of population: this means that a lot of services have to be delivered at a fiscal loss to sparsely-populated areas, with specific rules about what percentage of the population has to be within a certain distance of a postal outlet.

Post offices in particular are hard-hit by digital disruption; Australia Post has seen their letter delivery service decline by 1 billion articles (and the related revenue), even though the number of addresses to cover has increased. However, they have seen their parcel delivery business increase, even though this is a competitive business with courier companies. They’re also offering a number of other products, such as electronic bill payment, digital mail delivery and even passport interviews, which has driven them to create a more integrated multi-channel/multi-product architecture to be able to quickly bring new products to market. They’re using camunda BPM for their order management processses, both for customer orders and service fulfillment orders. Customer order processes support the various customer channels, then drive out one or more service order processes to fulfill a customer order.

They decided to use BPM in order to externalize processes from applications, making for more agile development and better reusability. They picked camunda because they wanted “just enough technology”: that is, they wanted to add process management to their existing Java application development environment, not rewrite all of their apps in a proprietary, monolithic BPMS app dev environment. camunda BPM is used to implement the multiple service order processes that might be kicked off by any given customer order, with their overall architecture handling the communication between the two order management layers: the customer order layer as a consumer for the service order layer producer.

Parker went into a lot of detail of how they have implemented this architecture, putting their BPM usage into the context of their overall technical architecture, and walked through the general process model for their service order that instantiates a dispatcher process for each customer order, which in turn instantiates a subprocess for each line item in the order. They really want to implement all of this in camunda, but are still using TIBCO for the dispatching process while they work out some of the sticky bits such as how subprocess cancelations are propagated to the parent process. They are also having some challenges with handling process versions, considering that they run 7×24: they need a mapping table that takes these temporal anomalies into consideration, so that the process version in use may be tied to the order date for longer-running order processes. They also created a business dashboard by modifying Cockpit, the camunda IT operations dashboard, to remove all of the “dangerous” operations while exposing the work in progress, and adding some additional functions such as searching by a business key.

Parker ended up with their outcomes, some expected, some less so: basically, BPMN 2.0 is really working for them both for business-IT collaboration and model-driven development; this level of business-IT alignment means that error handling can be shared, with business handling business errors, and IT handling IT errors. They found that developers became productive very quickly since they were just adding some tools to their existing familiar Java application development environment, although some had to be gently reminded to use the BPM capabilities instead of writing code.

It was great to see the reactions and interactions of the camunda team during the presentation: Australia Post is a “do-it-themselves” open source user of camunda, and as Parker discussed some of the shortcomings, they were obviously taking some notes for future work. The presentation finished with him being presented as an award for the non-camunda person who contributed most to the community forum discussions, suggesting that you get out of open source what you put into it.

Developer-Friendly BPM

I gave a webinar today sponsored by camunda on developer-friendly BPM, discussing the myth of zero-code BPM. I covered the different paradigms of BPM development, that is, fully model-driven versus process models and code, with some pointers for how to evaluate the different approaches within your organization – it’s not a simple one-size-fits-all decision for most large companies with complex application development needs.

I also wrote a white paper on the topic, it will be posted on the camunda site soon, as well as a link to a replay of the webinar.

Update: the webinar replay is here and the white paper is here. Registration on the camunda site required for both.

In the meantime, you can see my slides here:

Jakob Freund of camunda co-presented with me, his slides are here:

camunda BPM 7.0 Refactors Engine And Process Administration

On August 31, camunda released camunda BPM platform 7.0 (community open source and enterprise editions), the first major release of the software since it was forked from the Activiti project in March, although there were nine community releases between the fork at the release of 7.0. I had the chance for a couple of briefings over a period following that with Daniel Meyer, the project lead, and promised that I’d actually get this post written in time for Christmas. 🙂

The 7.0 release contains a significant amount of new code, but their focus remains the same: a developer-friendly BPM platform rather than a tool positioned for use by end users or non-technical analysts. As I discussed in a recent webinar, BPMS have become model-driven application development environments, hence having a BPMS positioned explicitly for developers meets a large market segment, especially for complex core processes.

The basic tasklist and other modeler and runtime features are mostly unchanged in this version, but there are big changes to the engine and to Cockpit, the technical process monitoring/administration module. Here’s what’s new:

Cockpit:

  • Inspect/repair process instances, including retrying failed service calls.
  • Create instance variables at runtime, and update variable values.
  • Reassign human activities.
  • Send a link directly to a specific instance or view.
  • Create a business key corresponding to a line-of-business system variable, providing a fast and easy to search on LOB data.
  • Extensible via third-party plug-ins. The aim with Cockpit is to solve 80% of use cases, then allow plug-ins from consulting partners and customers to handle the remainder; they provide full instructions on how to develop a Cockpit plug-in.
  • Add tabs to the detailed views of process instance, e.g., a link to LOB or other external data.

Engine:

  • A new authorization framework (also manifesting in admin capabilities for users/groups/authorizations): this is a preview feature in 7.0, supporting only application, group, group membership authorization. In the future, this will be expanded to include process definition and instance authorization. Users can be maintained in an internal camunda database or using a direct link to LDAP.
  • A complete rewrite of the history/audit log, splitting the history and runtime databases, which is potentially a huge performance booster. Updates to the history are triggered from events on running instances, whereas previously, writing history records required querying and updating existing records for that instance. The history log can be redirected to a database that is shared by multiple process engines; since some amount of the Cockpit monitoring is done on the history database, this makes it easier to consolidate monitoring of multiple process engines if the history logs are redirected to the same database. The logs can also be written directly to an external database based on the new history event stream API. Writes to the history log are asynchronous, which also improves performance. At the time of release, they were seeing preliminary benchmarks of 10-20% performance improvement in the process engine, and a significant reduction in the runtime database index size.
  • There is some increase in the coverage of the BPMN 2.0 standard; their reference document shows  supported elements in orange with a link through on each element to the description and usage, including code snippets where appropriate. Data objects/stores are still not supported, nor are about half of the event types, but their track record is similar to most vendors in this regard.

Version 7.0 is all open source, but a consolidation release (7.1) is already in alpha and will contain some proprietary administration features in Cockpit not available in the open source version: bulk edit/restart of instances, complex search/filter across instances from different process definitions, and a process-level authorizations UI (although the authorization structure will be built into the common engine). camunda is pretty open about their development, as you might expect from an open source company; you can even read some quite technical discussions about design decisions such as a new Activity Instance Execution Model have been made in the process engine in order to improve performance.

In September, camunda released a cloud application for collaborating on process models, camunda share. This is not a full collaborative authoring environment, but a place to upload, view and discuss process models. The camunda team created it during their “ShipIt-Day”, where they are tasked with creating something awesome within 24 hours. There’s no real security, your uploaded model generates a unique URL that you can send to others, but it provides the option to anonymize the process model by removing labels if your process contains proprietary information. A cool little side project that could let you avoid sending around PDFs of your process models for review.

camunda’s business model is in providing and supporting the enterprise edition of the software, which includes some proprietary functions in Cockpit but is otherwise identical to the community open source edition, plus in consulting and training services to help you get started with camunda BPM. They provide a great deal of the effort behind the community edition, while encouraging and supporting platform extensions such as fluent testing, PHP developer support and enterprise integration via Apache Camel.

camunda BPM 7.0

Stick A (Open Source) Fork In It: camunda BPM Splits From Activiti

At the end of 2012, I had a few hints that things at Alfresco’s Activiti BPM group was undergoing some amount of transition: Tom Baeyens, the original architect and developer of Activiti (now CEO of the Effektif cloud BPM startup announced last week), was no longer leading the Activiti project and had decided to leave Alfresco after less than three years; and camunda, one of the biggest Activiti contributors (besides Alfresco) as well as a major implementation consulting partner, was making noises that Activiti might be too tightly tied to Alfresco’s requirements for document-centric workflow rather than the more general BPM platform that Activiti started as. I’m not in a position to judge how Alfresco was controlling the direction and release cycle of Activiti, who was making the biggest contribution to the open source efforts, or what was said behind closed doors, but obviously things reached a breaking point, and this week camunda announced that they are forking a new open source project from Activiti, to be known as camunda BPM.

This is big news in the world of open source BPM. There are a few players already – Activiti, BonitaSoft, jBPM and Processmaker, to name a few – and it’s not clear that there’s enough demand for open source BPM software to warrant another entrant. Also, there has to be some hard feelings between the parties here, and this is a small community where you can’t really afford to make enemies, because you never know who you’re going to end up working with in years to come. This parting of the ways is described as “sad” by both camunda in their announcement post and by Joram Barrez (current Activiti lead core developer) in his post, and puts Activiti and camunda in direct competition for both existing Activiti users and future business. Signavio, whose process modeler is deeply integrated with camunda BPM, issued a press release stating that the camunda BPM fork will be good for Signavio customers, and including a nice quote from Tom Baeyens; keep in mind that Signavio just provided the funding for Baeyens’ new startup. It’s like the Peyton Place of BPM.

Leaving the personal (and personnel) aspects aside, camunda BPM is offering some significant additional capabilities beyond what is available in Activiti, mostly through open-sourcing their previously proprietary Activiti add-ons. I had a briefing a couple of weeks ago with Jakob Freund, camunda’s CEO, to get caught up on what they’re doing. camunda is about 20 people now, founded 4-1/2 years ago and completely self-funded. That makes them a bit small for launching an enterprise software product – including the implementation and support aspects – but also not driven to unreasonable growth since they have no external investors to please. Having once grown a consulting company to about twice that size without external funding, I can understand the advantages of maintaining the organic growth: control to pick the projects and products that you want to build, and to hand-pick a great team.

camunda BPM, as with Activiti (and jBPM, for that matter) are not claiming to be zero-code BPM suites – some would argue that even those claiming to be, aren’t – but are BPM engines and capabilities intended to be embedded within line-of-business enterprise applications. They see the zero-coding market as being general tooling for non-strategic processes, and likely served equally well or better by outsourcing or cloud solutions (Effektif, anyone?); instead, camunda targets situations where IT is a competitive differentiator, and BPM is just part of the functionality within a larger application. That doesn’t mean that there’s nothing for the non-technical business analyst here: BPMN is used as a bridge for business-IT alignment, and camunda is bringing their previously proprietary BPMN round-tripping capabilities into the new open source project. Their BPMN plugin for Eclipse provides an easy-to-use modeler for business analysts, or round-tripping with Signavio, Adonis and other modeling tools; camunda blogged back in June 2012 about how to integrate several different BPMN modelers with camunda BPM, although they have a definite preference for Signavio.

camunda BPM is a complete open source BPM stack under an Apache License (except for Eclipse, the framework for the designer/developer UI, which uses the Eclipse Public License). The Community (open source) edition will always be the most up-to-date edition – note that some commercial open source vendors relegate their community edition to being a version behind the commercial edition in order to drive revenue – with the Enterprise (commercial) edition lagging slightly to undergo further testing and integrations. The only capabilities available exclusively in the Enterprise edition are WebSphere Application Server (WAS) integration and Cockpit Pro, a monitoring/administration tool, although there is a Cockpit Light capability in the Community edition. You can see a Community-Enterprise feature comparison here, and a more complete list here. Unless you’re tied to WAS from the start, or need quite a bit of support, the Community edition is likely enough to get you up and running initially, allowing for an easier transition from open source to commercial.

However, the question is not really whether camunda has some great contributions to make to the Activiti code base (they do), but whether they can sustain and build an open source fork of Activiti. They have some good people internally to provide vision – Daniel Meyer for the core process engine architecture, Bernd Rücker for a technical consulting/product management view, Jakob Freund for the business aspects of BPM – and a development team experienced with the Activiti and camunda code bases. They have showed significant leadership in the Activiti open source community and development, so are likely capable of running a camunda BPM open source community, but need to make sure that they dedicate enough resource to it to keep it vital. There is a German camunda community already, but that’s not the same as an open source community, and also is only in German, so they have some work to do there.

And then there’s the existing Activiti and camunda users. Existing camunda customers probably won’t be freaked out about the fork since the contributions important to them were being made by camunda anyway, but existing Activiti users (and prospects) aren’t just going to fall into camunda’s lap: they might be weighing the additional functionality against the bigger company, stable brand and existing community behind Activiti. Given some of the new UI features being rolled into Activiti from the Alfresco team, it’s fair to say that Alfresco will continue to innovate Activiti, and attempt to maintain their solid standing in the open source BPM market. There’s likely a small window for existing Acitiviti users to shift to camunda BPM if they want to: right now, the engine is identical and the migration will be trivial, but I expect that within six months, both sides will make enough changes to their respective projects that it will become a more significant effort. In other words, if you’re on Activiti or camunda now and are thinking of switching, do it now.

camunda could be ruffling a few feathers by declaring an open source fork rather than just rolling their proprietary offerings into the Activiti project; they might have been able to become a stronger influencer within the project by doing that, counteracting any (perceived) document-centric influence from Alfresco. Again, I’m not internal to either of the companies nor part of the Activiti open source community, so that’s just speculation.

Meanwhile, Alfresco remains officially silent on the whole business. Given that they had advance warning about this, that’s a pretty serious PR mistake.