camunda – Column 2

CamundaLocal Toronto 2024

When conferences end up in my back yard, I usually find a way to attend. Today, I’m at CamundaLocal Toronto, a one-day conference and workshop for Camunda customers, prospects and partners. Toronto is the hot spot for Canadian finance, and we can see the major bank towers from our perch at the top of the Bisha Hotel. I’ve worked for a long time as a process automation consultant in this region and there is really an overwhelming number of investment firms, bank, insurance and other related financial organizations within a short distance. Lots of representation from the big banks and other financials in attendance today.

Our host for the day is Lisa-Marie Fernandes, Camunda Strategic Account Executive, and after she introduced the day and gave some background on Camunda, we heard from Sathya Sethuraman, Field CTO, on the problems with siloed automation versus Camunda’s vision of universal process orchestration. Every technology company and CIO have been talking about the necessity of business transformation for years, and Sathya pointed out that process automation is a fundamental part of any sort of digital transformation. The problem arises with siloed automation: local automation within departments or even sub-departments, with no automated interaction between parts of the process. Although people in that department are convinced of the value of their local automation, customers deal with a much broader end-to-end process in order to do business with an organization. With siloed automation, that customer journey is fragmented and difficult, with the customer having to fill in the gaps of the process themselves through multiple phone calls, forms and emails, while re-entering their information and explaining their issues at each step. Contrast that with an organization that has end-to-end orchestration using a product such as Camunda to bind together these departmental processes into a full customer journey. The customer no longer has to deal with multiple processes (while trying to figure out what those processes are from the outside), and people inside the organization no longer have to make heroic efforts just to meet the customer’s needs.

Sathya pointed out the risks of broken (or missing) end-to-end automation, including the increased complexity caused by multiple isolated processes, and the disconnect between business and IT due to lack of a common language for process automation. From an organization’s standpoint, there are issues with efficiency and quality of internal processes, but the bigger issue is that of customer satisfaction: if a competitor has a fully-automated end-to-end process that makes it easier for the customer, then many customers will choose that path of least resistance when they are choosing who to give their business to. This, of course, is not new. However, with the changes that we’ve seen in the past four years due to the pandemic, work from home and shifting supply chains, more streamlined and automated processes have become a true competitive differentiator. He had an interesting slide showing a heatmap of the many process orchestration opportunities just in the consumer banking value chain: probably 50-60 distinct processes across customer management, retail, lending, cards, payment, risk, finance and accounting, and corporate management where many financial organizations have room for improvement.

Daniel Meyer, Camunda CTO, then gave us a product roadmap update. He kicked off with a view of the common reality of most large organizations, echoing Sathya’s points about islands of automation without end-to-end process orchestration. Many companies focus on improving the local automation — making their core banking systems better, for example — without considering how this is making the customer journey worse because there’s no integration or linkage between these disparate systems. Opening a bank account as a new customer? There are likely different systems and processes for know-your-client checks, credit checks, customer onboarding and account opening; when one isn’t connected to the others, the customer experiences delays and an increasingly negative view of the organization as they struggle to get the new account open. The missing link is end-to-end process orchestration, and this is the focus that Camunda has defined for their product over the past few years.

The challenges of end-to-end orchestration include endpoint diversity — it’s more difficult to integrate a heterogeneous set of endpoints since they have different interfaces and may be at different levels — and process complexity that goes beyond a simple sequence of steps. He showed where some of the customer use cases live on the endpoint diversity versus process complexity graph, and the sweet spot for Camunda in the top right quarter of that graph, where processes and the system integrations are both complex. Of course, those are also the mission-critical processes that are controlled by IT, even though business people may be involved in the design and requirements. In this context, their vision is Camunda as the universal process orchestrator: providing the capability of complex process flows as well as being able to integrate in human work, AI, business rules, microservices, RPA, IoT and APIs of all types. That being said, they still have quite a developer-centric bias.

Daniel spent a bit of time on some of the terminology and usage: process orchestration versus business process management (essentially the same when you’re talking about the automation side of BPM, and Camunda also uses the term “workflow engine” as the core of process automation), and who within an organization creates BPMN diagrams. This was a pretty technical audience, and there was a bit of discussion on the value of BPMN once you got down to the level of details that a non-technical process owner isn’t going to be looking at. I believe it’s still incredibly valuable as a graphical process-oriented development language even for those models that are not directly viewed by the business people: it provides a level of functionality as well as development guardrails (e.g., model validation) that can accelerate development and increase reusability while making it more accessible to less experienced developers.

Camunda, of course, is only one piece of the entire process improvement cycle, albeit a critical core piece of the automation. Most customers are also using some combination of process mining, more advanced business modeling, and business intelligence; these essential activities of discovery, design and analysis/improvement fit around Camunda’s process automation and low-code integration offerings. Some automation technologies, including AI, RPA and an event bus, are not part of the core Camunda platform but easily integrated to allow the best of breed components to be added. And although Camunda has recently provided a some front-end UI capabilities, it’s pretty rudimentary forms and most companies will be integrating Camunda into their existing application/UI development environment.

Daniel took us through the investment focus that Camunda has for product development in the areas of developer productivity, AI/ML, collaboration, business intelligence, low-code and universal connectivity. He also highlighted the availability of connectors in the Camunda Marketplace that can be pulled in and used in any model for additional connectivity beyond what is included in the out of the box product.

Camunda has a regular cycle of releases every six months, and he gave us a quick overview of what’s coming up in 8.5 in April, 8.6 in October, as well as future plans for 8.7 and beyond. This is all for systems based on the Zeebe architecture (V8+); not clear from this presentation what, if anything, they are doing for customers still on V7 aside from encouraging and assisting with migration to V8. As with many vendors that completely replatform, migrations to the new platform are likely much slower than they anticipated, although with the added complication that they are leaving behind their open source legacy with the shift to their modern Zeebe engine.

After lunch, there was a short customer presentation by Norfolk & Dedham insurance, who are using Camunda 7 and Spring Boot as an embedded engine invoked from their own UI portal for managing claims. They customized the Camunda Modeler to integrate their own applications and data sources, in order to accelerate development. In addition to the usual benefits from process automation and integration, the data generated by process instances is hugely valuable to them when looking at how to translate activities to more strategic actions when planning future improvements to their business operations. Now that their claims applications are in advanced testing stages, they are expanding the same technical stack to their underwriting applications, and are seeing a significant benefit from the reusability of components that they created during the claims project. They aren’t yet planning a V8 migration, although are very cognizant that this is likely in their future.

Following the customer presentation, Gustavo Mendoza, a Camunda senior sales engineer, gave us a technical demo of the V8 product stack to expand on some of the points that Daniel Meyer went through in his presentation earlier. We saw demos of the process and decision modelers, including the collaboration features, the UI forms builder for task handling, then the execution environment to see the user interaction with a running process as well as the Operate monitoring portal and integration with Slack.

The remainder of the afternoon was split into a hands-on live coding workshop for customers, and a partner workshop. A full and worthwhile day if you’re in the Camunda ecosystem.

Camunda has a couple of other CamundaLocal events coming up soon in Chicago and San Francisco, plus their main European conference in Berlin in May and their North American conference later in the year (September in New York, I think).

Voices in Tech panel

Edit with correction to above graphic: the panel is at 10am Eastern which is 3pm Central European time because we’re in that hellish period where North American clocks have moved forward but European ones haven’t.

I posted earlier this week on Mastodon that I’ve been taking a bit of a break but now getting back to things, and one of events that’s on my upcoming agenda is presenting on a panel Voices in Tech: Building Effective Automation Teams, hosted by Camunda and also sponsored by Infosys. This will take place online on March 15th, but you can head over to the link now and sign up. I will have the pleasure of reconnecting with co-panelists Uzma Khan of the Ontario Teachers’ Pension Plan, who I have known for many years, and Smriti Gupta of Infosys, who I shared the stage with at CamundaCon in Berlin last October. We will be joined by Ola Inozemtceva, a senior product marketing manager at Camunda, and the moderator will be Lana Ginns, product marketing manager at Camunda.

This is the 2023 version of the International Women’s Day panel that Camunda has been organizing for a few years now, and I really like that the focus is not on the fact that all of the panelists are women, but that we are “brilliant trailblazers in the tech world, who inspire people every day to redefine technology and how it can transform the world”. We’ll be discussing challenges and best practices with building high-performing orchestration teams, which ties in nicely with the series of video blogs that I’ve been doing lately for Trisotech on best practices in business automation.

I hope to see you there (virtually).

Camunda Platform 7.15: now low-code (-ish)

I had a quick briefing with Daniel Meyer, CTO of Camunda, about today’s release. With this new version 7.15, they are rebranding from Camunda BPM to Camunda Platform (although most customers just refer to the product as “Camunda” since they really bundle everything in one package). This follows the lead of other vendors who have distanced themselves from the BPM (business process management) moniker, in part because what the platforms do is more than just process management, and in part because BPM is starting to be considered an outdated term. We’ve seen the analysts struggle with naming the space, or even defining it in the same way, with terms like “digital process automation”, “hyperautomation” and “digitalization” being bandied about.

An interesting pivot for Camunda in this release is their new support for low-code developers — which they distinguish as having a more technical background than citizen developers — after years of primarily serving the needs of professional technical (“pro-code”) developers. The environment for pro-code developers won’t change, but now it will be possible for more collaboration between low-code and pro-code developers within the platform with a number of new features:

Create a catalog of reusable workers (integrations) and RPA bots that can be integrated into process models using templates. This allows pro-code developers to create the reusable components, while low-code developers consume those components by adding them to process models for execution. RPA integration is driving some amount of this need for collaboration, since low-code developers are usually the ones on the front-end of RPA initiatives in terms of determining and training bot functionality, but previously may have had more difficult integrating those into process orchestrations. Camunda is extending their RPA Bridge to add Automation Anywhere integration to their existing UIPath integration, which gives them coverage of a significant portion of the RPA market. I covered a bit of their RPA Bridge architecture and their overall view on RPA in one of my posts from their October 2020 CamundaCon. I expect that we will soon see Blue Prism integration to round out the main commercial RPA products, and possibly an open source alternative to appeal to their community customers.
DMN support, including DRD and decision tables, in their Cawemo collaborative modeler. This is a good way to get the citizen developers and business analysts involved in modeling decisions as well as processes.
A form builder. Now, I’m pretty sure I’ve heard Jakob Freund claim that they would never do this, but there it is: a graphical form designer for creating a rudimentary UI without writing code. This is just a preliminary release, only supporting text input fields, so isn’t going to win any UI design awards. However, it’s available in the open source and commercial versions as well as accessible as a library in bpmn.io, and will allow a low-code developer to do end-to-end development: create process and decision models, and create reusable “starter” UIs for attaching to start events and user activities. When this form builder gets a bit more robust in the next version, it may be a decent operational prototyping tool, and possibly even make it into production for some simple situations.

They’ve also added some nice enhancements to Optimize, their monitoring and analytics tool, and have bundled it into the core commercial product. Optimize was first released mid-2017 and is now used by about half of their customers. Basically, it pumps the operational data exhaust out of the BPM engine database and into an elastic search environment; with the advent of Optimize 3.0 last year, they could also collect tracking events from other (non-Camunda) systems into the same environment, allowing end-to-end processes to be tracked across multiple systems. The new version of Optimize, now part of Camunda Platform 7.15, adds some new visualizations and filtering for problem identification and tracking.

Overall, there’s some important things in this release, although it might appear to be just a collection of capabilities that many of the all-in-one low-code platforms have had all along. It’s not really in Camunda’s DNA to become a proprietary all-in-one application development platform like Appian or IBM BPM, or even make low-code a primary target, since they have a robust customer base of technical developers. However, these new capabilities create an important bridge between low-code developers who have a better understanding of the business needs, and pro-code developers with the technical chops to create robust systems. It also provides a base for Camunda customers who want to build their own low-code environment for internal application development: a reasonably common scenario in large companies that just can’t fit their development needs into a proprietary application development platform.

Camunda Community Day: @CamundaBPM technical sessions

I’m a few weeks late completing my report on the Camunda Community Day. The first part was on the community contributions and sessions, while the second half documented here is about Camunda showing new things that could be used by the community developers in the audience.

First up was Vladimirs Katusenoks, core developer on BPMN.io, with a presentation on bpmn-js: how it works, and how to extend it with custom functionality such as adding color to BPMN diagrams, which is a permitted extension to BPMN XML. His live coding presentation showed changing the colour of a shape background, either statically in code for the element class or by adding a colour picker to an individual element context palette; this was based on the bpmn-js core BPMN functionality, using bpmn-moddle to read/write into the metamodel and diagram-js to render it. There are a number of other bpmn-js examples on Github.

Next, Felix Müller discussed KPI management, expanding on his August blog post on the topic. KPI management is based on quantitative indicators for process cycle-time improvement, including cycle time and overdue time, plus definitions of the time period, unit of measure and calculation method. In Camunda, KPIs are defined in the Modeler, then monitored in Cockpit. He showed how to use the concept of element templates (that extend core definitions) to create custom fields on collaboration object (process) or individual tasks, e.g., KPI unit (hours, days, minutes) and KPI threshold (number). In Cockpit, this appears as a new tab for KPI Overview, showing a list of individual instances and target/current/average duration, plus an indicator of overdue status of the instance and any contained tasks; there is also a decorator bubble on the top right of the task on the process model to show the number of overdue instances on the aggregate model, or overdue status as a check mark or exclamation on individual models. The Cockpit modifications were done by creating a plug-in to display KPI statistics, which queries and calculates on the fly – a potential performance problem that might be improved through pre-aggregation of statistics. He also demonstrated how to modify this basic KPI model to include an expected duration as well as maximum duration. A good start, although I think there’s a lot more that’s needed here.

Thorsen Lindhauer, a Camunda core BPM developer, discussed how to contribute to the Camunda open source community, both at camunda.org (engine and desktop modeler, same as the commercial product) and bpmn.io (JS tools). Possible contributions include answering questions on forums; logging error reports; documenting ideas for new functionality; and working on code. Code contributions typically start by having a forum discussion about planned new functionality, then a decision is made on whether it will be core code (higher quality standards since it will become part of the commercial product, and will eventually be maintained by Camunda) versus a community extension; this is followed by ongoing development, merge and release cycles. Camunda is very supportive of community contributions, even if they don’t become part of the core product: community involvement is critical to the health of any open source project.

The last presentation of the community day was Daniel Meyer discussing the product roadmap. The next release, 7.6, will be on November 30 – they have a strict twice-yearly release cycle. This release includes updates to DMN, CMMN, BPMN, rolling updates, Cockpit features, and UI/UX in web apps; I have captured a few notes here but see the linked roadmap for a more complete and accurate description and the online documentation as it is rolled out.

DMN:

Simpler decision table editing with drop-down lists of comparison/range operators instead of having to remember FEEL or Juel syntax
Ability to add list of selection values (advanced mode still exists for full flexibility)
Decisions with literal expressions
DMN engine performance 4-6x faster
Support for decision requirements diagrams/graphs (DRD/DRG) that can link decision tables; visualization in Modeler and Cockpit are not there yet but the structures are supported – in my experience, this is typical of Camunda, which builds and releases the engine capabilities early then follows with the visualization, allowing for a quicker start for executable diagrams

CMMN:

Modeler now completely models CMMN including technical attributes such as listeners
Cockpit (visualization still incomplete although we saw a brief view) will allow linking models of same or different types
Engine feature and functionality improvements

Rolling updates allow Camunda process engine to be updated without shutdown: guaranteed backwards compatibility of database schema to allow database to be updated first, then roll updates of engines by taking each offline individually and allowing load balancer to reroute sessions.
BPMN:

BPMN conditional event supported
Improved modeling including labels, collapsing/expanding subprocesses to switch between view types, and field injections in property panel.

Cockpit:

More flexible/granular human task monitoring
New welcome page with links to apps (Cockpit, Tasklist, Admin), user profile, and frequent links
Batch operations (cancel, suspend, etc.) based on batch action capability built for instance migration
CMMN and DMN DRD visualization

Daniel discussed some other minor improvements based on customer feedback, plus plans for 2017, including a web modeler for collaborative BPMN, CMMN and DMN modeling via a SaaS offering and a future on-premise version. They finished the day with a poll and community feedback to establish priorities for future versions.

I stayed on for the second day, which is actually a separate conference: BPMCon for Camunda’s enterprise (commercial) customers. Rather, I stayed on for Neil Ward-Dutton’s keynote, then ducked out for most of the rest of day, which was in German. Neil’s keynote included results from workshops that he has done with executives on digital transformation, and how BPM can be used to create the bridges between the diverse parts of a digital business (internal to external, automated to people-centric), while tracking and coordinating the work that flows between the different areas.

Disclaimer: Camunda paid my travel expenses to attend both conference days. I was not compensated in any way for attending or for writing this post, and the opinions here are my own.

Camunda Community Day: community contributions

Two years ago, I attended Camunda’s open source community day, and gave the opening keynote at their enterprise user conference the following day. I really enjoyed my experience at the open source day, and jumped at the chance to attend again this year – and to visit Berlin.

The first day was the community day, where users of Camunda’s open source software version (primarily developers) talk about what they’re doing with it, plus some of the contributions that the community is making to the project and updates from Camunda on new features on the horizon. To break this up a bit – since I’m already a week after the conference and want to get something out there – I’ll cover the community sessions in this post, then the Camunda technical sessions and a bit about the enterprise conference in a later post.

The first presentation was by Oliver Hock of Videa Project Services, demonstrating robot control using a LEGO Mindstorms robot to solve a Rubix cube. He showed how they used BPMN to define movements and decision tables to determine the move logic, then automated the solution using Camunda BPM. Although you may never want to build a robot to solve a Rubix cube, there are a lot of other devices out there that, like the Mindstorms robot, are controlled via Java APIs; Hock’s design showed how these Java-enabled devices can make use of higher-level modeling constructs such as BPMN and decision tables.

Next up was Jan Galinski of Holisticon to show the Spring Boot community code extension – an example of how the community of Camunda open source users give back to the open source project for everyone’s benefit. Spring Boot is a microservices framework allowing for fast deployment of web applications with a minimal amount of overhead; the Spring Boot starter extension to Camunda allows for using Camunda without a Java application server to essentially provide Camunda apps as microservices. The extension, consisting of about 5,000 lines of code, has been developed over two years with 10 contributors, including both community and Camunda contributors. Galinski showed a live coding demo of replacing JBoss server with Spring Boot starter in a Camunda application to show how this works; he has also written a post on the Camunda community site on the 1.3.0 version of Camunda BPM Spring Boot for more technical details. Although granualar process apps such as this are easier from a devops perspective in terms of deployment and scalability, the challenge is that there is no single point of entry for an end user to look at a worklist (for example). We saw some methods for dealing with this, where a workload service collects information from individual process services with the help of the Camunda BPM Reactor plugin and aggregates them; a federated task list is under development to bring together tasks from multiple process servers into a single list, with a simple completion form. Galinski walked through the general architecture for this, and noted that they are working on making this an official extension. Update: Jan Galinski pointed out in the comments that it was Simon Zambrovski (also of Holisticon) who did the portion on the presentation on cloud, universal tasklist and event processing — I missed the transition and his name in my hasty note-taking.

Jarl Friis of the Danish tax authority (SKAT) presented their use of the Camunda decision engine: they are using only decision services, and not the BPM capabilities, which likely makes them unusual as a Camunda customer. There are a couple of applications for them: first is to raise data quality in financial reporting to the IRS (for FATCA requirements), where they receive data from Danish financial institutions and have to process it into a specific XML format to send to the IRS. Although many of the data cleansing and transformation rules are in the XML schema definitions, some are not amenable to that format and are being defined in DMN decision tables instead. As this has rolled out, they see that decision tables give them an easier way to respond to annual rule changes, although their business people are not yet trained to make changes to the decision tables. That has resulted in developers having to make the decision table changes and test the results, which is one of the challenges that they have had to deal with: some of the developer test frameworks replicated the original decision table logic in code, which effectively tested the decision table implementation rather than the business logic. That test framework, of course, no longer worked when the decision table was changed, and Friis’ message to the audience was that organizations have to deal with challenges of ownership and responsibility for rules as well as rules testing.

Niall Deehan of Camunda gave a great presentation on on modeling anti-patterns: snaking models that are often used to fit models onto a single sheet of paper (instead, use model with happy path down the centre from left to right); inappropriate use of BPMN versus CMMN (e.g., voting scenarios); inappropriate use of BPMN versus process engine or Cockpit capabilities (e.g., service call with error exceptions for null pointer, bad response, service down); too many listeners on tasks (masks problems and pushes process logic into code, based on concept that analysts’ model should not be changed). He discussed some best practices for consistency: define the symbol set to be used by your analysts and lock down the modeler to remove elements that you don’t want people using; create and maintain your own best practices documentation; use model templates for commonly used activities; and proper training. I would love to see his presentation captured for replay: it was engaging and informative.

The last community presentation was Martin Schimak of plexiti, showing three community extensions used for automating testing of BPMN and CMMN models. Assert checks and sets the status of tasks in order to drive a process instance through a test scenario. Process Test Coverage visualizes test process paths and checks process model coverage ratio, as covered by individual test methods and entire test classes (e.g., using mockito). Assert Scenario is for writing robust test suites for process models; this was not covered in Schimak’s demo due to time contraints, but you can read more about it on his blog.

Before we started on the Camunda technical presentations, the community award was presented by Camunda as a reward for contributions on extensions: this went to Jan Galinski from Holisticon. It’s really encouraging to see the level of engagement between Camunda and their open source community: Camunda obviously realizes that the community is an important contributor to the success of the enterprise version of the software and the company altogether, and treats them as trusted partners.

Disclaimer: Camunda paid my travel expenses to attend both conference days. I was not compensated in any way for attending or for writing this post, and the opinions here are my own.

BPM Cyber Monday: Camunda 7.2 Adds Tasklist And CMMN

I caught up with Jakob Freund and Daniel Meyer of camunda last week in advance of their 7.2 release; with 1,700 person-days of work invested in this April-November release cycle, this includes a new tasklist application, an initial implementation of the Case Management Model and Notation (CMMN) standard, developer accelerators particularly for non-Java developers, and performance and stability improvements. You can hear more about the new release and see a demo in their webinar on Wednesday this week, and read their blog post about it. [Update: you can see the webinar replay here and the slides here, no registration required.]

It’s been interesting to track the progress of camunda and Activiti after camunda forked from the Activiti project in early 2013, since they are targeting slightly different markets but still offer competitive solutions in the open source BPM space. Last week, I wrote about Activiti’s recent release of their BPM Suite, which includes end-user task list and forms interfaces and tools for for “citizen developers” (read: non-hard-core-Java developers); we are seeing some similar themes in the new camunda release, although camunda sticks closer to a true open source model by releasing pretty much all of the code as part of the open source project, while Activiti is making everything except the core engine part of their commercial product.

The biggest news of this release is the tasklist app, since this marks a completely new foray by camunda into end-user interfaces that remain part of the open source project. Developed in conjunction with their customer DAB bank, it’s a single-page JavaScript application that is usable out of the box but intended to be developer-friendly for customization. The 3-panel UI (inspired by MS Outlook, although I thought that might not be considered very inspiring by some) shows filters, the task list resulting from the selected filter, and the details of the selected task. Any of the three columns can be collapsed or expanded by the user: you might want to maximize the active task form while completing a task, for example, or collapse the form to see just the filters and task list for quickly sorting or triaging work.

Filters can be created using expressions based on drop-down lists of instance properties, and can be shared for use by other users or groups. Filters can also specify process variables so that those variables are visible in the task list and can be used for searching and sorting, which makes it easier to locate a specific task without having to click on each task to see the details. If permissions allow, other user’s tasks can be included in the task list. During our demo, they pointed out the ability to use keyboard controls to navigate through the task list, something that was suggested by the users: having seen many keyboard-centric users slowed down by having to use a mouse for controlling their screens, this was not a surprise to me, but I think that many software developers don’t think about the needs of the old-school keyboarding users.

They have had to improve their APIs to expose more information and functionality to support the tasklist app, which helps developers even if they are not using the tasklist. For the forms interface of an active task, they created the camunda JavaScript Forms SDK to simplify the connection to their REST APIs, allowing HTML form controls to be bound directly to process variables so that the SDK handles the interactions with the process engine as well as type conversions. This can be used for the forms interface even if you replace the tasklist app with your own HTML5 UI, and the SDK optionally integrates with AngularJS for additional features such as form validation. There are some nice page/process-flow control features for tasks: the option to use a Save button as well as Complete, which saves the task in place (with any instance data updates) but does not move it through the process flow; and the ability for a user completing a task to directly access the following task for this instance if it is assigned to them.

The task detail pane of the UI contains more than just the task form: it also has a history tab showing events, due dates and comments; a diagram tab highlighting the current task within the BPMN diagram; and a description tab that I didn’t see, but I assume can contain task instructions.

The other new major user-facing functionality is support for CMMN; 2015 is definitely going to be the year when we see the BPM vendors pile on here, and camunda is out in front with this. Like many other BPM vendors, camunda’s BPMN implementation does not support ad hoc activities – arguably, support for events and ad hoc activities provides most of what is required for case management – so they are using the Trisotech CMMN modeler instead of their own modeler, but executing on the same core camunda engine which exposes standard engine features such as REST APIs and scripting. Cases can be instantiated through the API directly by a web form or other event, or can be instantiated using a call from a structured process. In turn, cases can instantiate structured processes through a call from an activity. This covers all of the use cases along the structured/unstructured spectrum: completely pre-defined processes, pre-defined with ad hoc exceptions, ad hoc with pre-defined fragments, and completely ad hoc. Calls from BPMN that instantiate a CMMN case can be synchronous (i.e., wait for completion), or asynchronous.

Case tasks, once instantiated, will appear in the tasklist along with those from BPMN processes; however, note that ad hoc tasks will require some sort of custom UI to allow a user to instantiate them if they are not triggered automatically by events or other tasks. They have nothing on the out of the box tasklist app to do this, although I can envision that they might extend this in the future to allow a case owner or participant to see and trigger ad hoc case tasks.

CMMN is still pretty new, although the concepts of case management have been around for a long time; camunda has some customers testing and providing feedback on their CMMN implementation, and they are expecting requirements and capabilities to emerge as they get more practical experience with it. By providing an open source engine that supports CMMN, they also hope to contribute to the CMMN standard and its use cases in general since others can use their engine to test the standard.

Moving on to some of the developer and non-functional features in the new release, they are positioning their developer-friendly methods against “death by property panel” in zero-code BPM suites. The subject of many conversations amongst BPM practitioners, fully model-driven development BPM suites may not require Java coding but often are suitably complex that they require developer skills to use them; this can frustrate developers who are saddled with inefficient (to them) environments for creating applications. However, not every developer is a hard-core Java coder, and many organizations have access to a middle ground of people with development skills but who want to use lighter-weight languages, often regional IT resources embedded within business units. Where Activiti’s new citizen developer tools leans more to the less-technical, model-driven developer, camunda focuses on developers using scripting languages such as JavaScript, Groovy, Jython and JRuby by allowing these languages to be used in place of the expression language or custom Java code within their implementation of BPMN models, including listeners, script tasks and sequence flow conditions. Wrappers for templates, e.g., XSLT, allow these to be added for composing or mapping data. There’s a new API for implementing connectors, with SOAP and REST connectors included out of the box with 7.2. Finally, Java objects are now serialized in XML/JSON rather than byte streams, allowing other languages – and BPMN processes, via APIs – to introspect and consume them directly without converting to Java objects, and making it possible to easily combine Java with other languages within an implementation.

On the execution performance side, camunda likes to distinguish their product within the open source BPM field as being for high-load straight-through processing rather than just manual activities, where they define “high-load” as 10 process instances per second: they’ve been steadily improving execution engine performance since the split and are even working on implementing with alternative storage/computing strategies such as in-memory grids (I jokingly asked if we would see camunda on HANA any time soon). In tests on an earlier 7.x release, they were already 10-30x (that’s times, not percent) faster than Red Hat’s jBPM; 7.2 improves the caching and load balancing, and enhances the asynchronous history logging to allow the level of data logged to be configured by process class and activity. This helps provides the level of scalability that they need for their highest-volume customers, such as telcos, that may be executing more than 1,000 process instances per second.

The camunda open source community edition is available for download and they will be pushing out updates to their Enterprise subscription customers. Check out their upcoming webinar, or sign up for the webinar and watch the recording later.

Disclosure: camunda has been my customer during 2014 for services including a webinar, white paper and the keynote at their user conference. However, I have not been compensated in any way for researching and writing this review.

What’s Next In camunda – Wrapping Up Community Day

We finished the camunda community day with an update from camunda on features coming in 7.2 next month, and the future roadmap. camunda releases the community edition in advance of the commercial edition; this is the way that open source should work, but some commercial open source vendors switch that around so that the community version lags by as much as a full version.

The highlights of the 7.2 release are as follows:

CMMN-based case management engine, which includes the core activities (stages, human tasks, process tasks, case tasks, milestones and sentries), the base case instance and plan item lifecycle, and a CMMN model API and REST API on a common process engine. They demonstrated a basic case manager UI that can manage cases and the related tasks; I assume that this is really just a demo of what can be done rather than intended as production code. They also don’t have case modeling in their modeler yet, so it’s early times.
A variety of functions for speeding development: connectors (currently REST and SOAP), dataformats, templating and scripting (calling external scripts, currently Groovy or Javascript but with others to come)
New tasklist, updating the tasklist UI that they released just before announcing camunda as an open source project. It allows filters to be defined, including specifying who can see the results of a filter in addition to the search criteria; that filter then appears as a tab on the task list, in the color defined by the filter author. The sort order can’t currently be defined as part of the filter, but can be set on the general tasklist interface. This adds a third (left) column to the tasklist UI, which also shows the list of tasks and the form for the selected task. Still work to be done, but the new filters capability is a big step up, providing a conceptually similar (but much different graphically) functionality to the Brazos portal filters.

There were a list of other smaller enhancements and fixes, from platform support to performance improvements to new functions.

We also saw some work in progress from the labs. First of all, an update on bpmn.io, which I saw at bpmNEXT earlier this year: a BPMN viewer and web modeler.. The viewer allows embedding of a BPMN diagram into a web page, including adding annotations, overlays and markers on the diagram, via a Javascript API. Check out a live demo here, demonstrating a BPMN diff function based on two similar process diagrams. From the viewer, you can export the model to a file. You can also create BPMN diagrams from scratch or import from a file, either directly on their site or embedded in another page. The modeler is still bit basic, and doesn’t handle containers (pools, lanes, subprocesses) very well yet, but that’s all coming; keep up with new functionality on the bpmn.io blog.

Another lab project is the camunda BPM workbench, a debugging tool that allows inspection of the runtime state of processes alongside the process model, allowing breakpoints to be set in the process model (rather than in code). A console interface allows for interrogation and updating of the process variables as the developer steps through the process. The process model is displayed using the bpmn.io viewer.

At the end of all the roadmap sessions, the audience had a chance to say what was most important for them in terms of what will be implemented when; there were questions about case management, centralized model repositories, bulk runtime operations and other features.

A great half-day; this is the first time that I’ve attended an open source code community day, and it’s quite a different environment from a typical vendor conference. We’re about to enter the beer-drinking portion of the day so I will sign off for today; I’m giving the keynote at the main camunda user conference tomorrow morning, and not sure how much blogging that I’ll do during after that.

Disclaimer: camunda paid my travel expenses to be here today and tomorrow, and is providing a speaking fee for tomorrow’s keynote. I was not compensated for blogging, and the opinions here (and in my keynote) are my own.

camunda Community Day technical presentations

The second customer speaker at camunda’s community day was Peter Hachenberger from 1&1 Internet, describing how they use Signavio and camunda BPM to create their Process Platform, which is in turn used by their clients’ developers for building and executing automated processes. His presentation was primarily about the details of their technical implementation of the platform; they have built some fairly comprehensive tools for monitoring and managing executing processes, many of which are facilitated by changes that they made to the core process engine, including retry behavior, process ID generator, multiple business keys, an asynchronous process starter API, an extended REST API and a few new commands. Since camunda BPM is open source, any customer such as 1&1 can take a copy of the code and make changes to it, optionally returning them to the community if they are valuable to others. There’s a bit of danger in this, in that if you make changes to core functionality (such as the engine) rather than create an extension or plug-in, and those changes do not end up back in the community version, you’re not only on your own for future development on those components but may not be able to upgrade to future versions.

We had a number of short (10 minute) presentations from community members to discuss extensions that they are working on:

Grails plugin to add camunda functionality to Grails applications
OSGi module extension for greater flexibility and configurability at runtime, including sharing process engines as services
Elasticsearch extension to write camunda BPM history data to an elasticsearch cluster to allow full-text searching, enabling more comprehensive analytics
camunda mocking extensions for process testing with mockito
Cockpit plugin to add interactive graphs and some statistical calculations (e.g., aggregation, regression, min/max) for process monitoring directly on the camunda history database

Some of these extension projects were done by camunda employees, but great to see the external community contributions as well.

Australia Post at camunda Community Day

I am giving the keynote at camunda’s BPMcon conference tomorrow, and since I arrived in Berlin a couple of days early, camunda invited me to attend their community day today, which is the open source community meeting. Nice to see such a great turnout — 70 or 80 people? — and I definitely wasn’t the one to travel the furthest to get here, since today’s opening presentation is from Rob Parker of Australia Post. Australia has a lot of the same issues as Canada when it comes to nation-wide services such as post, since we both have a lot of geography and not a lot of population: this means that a lot of services have to be delivered at a fiscal loss to sparsely-populated areas, with specific rules about what percentage of the population has to be within a certain distance of a postal outlet.

Post offices in particular are hard-hit by digital disruption; Australia Post has seen their letter delivery service decline by 1 billion articles (and the related revenue), even though the number of addresses to cover has increased. However, they have seen their parcel delivery business increase, even though this is a competitive business with courier companies. They’re also offering a number of other products, such as electronic bill payment, digital mail delivery and even passport interviews, which has driven them to create a more integrated multi-channel/multi-product architecture to be able to quickly bring new products to market. They’re using camunda BPM for their order management processses, both for customer orders and service fulfillment orders. Customer order processes support the various customer channels, then drive out one or more service order processes to fulfill a customer order.

They decided to use BPM in order to externalize processes from applications, making for more agile development and better reusability. They picked camunda because they wanted “just enough technology”: that is, they wanted to add process management to their existing Java application development environment, not rewrite all of their apps in a proprietary, monolithic BPMS app dev environment. camunda BPM is used to implement the multiple service order processes that might be kicked off by any given customer order, with their overall architecture handling the communication between the two order management layers: the customer order layer as a consumer for the service order layer producer.

Parker went into a lot of detail of how they have implemented this architecture, putting their BPM usage into the context of their overall technical architecture, and walked through the general process model for their service order that instantiates a dispatcher process for each customer order, which in turn instantiates a subprocess for each line item in the order. They really want to implement all of this in camunda, but are still using TIBCO for the dispatching process while they work out some of the sticky bits such as how subprocess cancelations are propagated to the parent process. They are also having some challenges with handling process versions, considering that they run 7×24: they need a mapping table that takes these temporal anomalies into consideration, so that the process version in use may be tied to the order date for longer-running order processes. They also created a business dashboard by modifying Cockpit, the camunda IT operations dashboard, to remove all of the “dangerous” operations while exposing the work in progress, and adding some additional functions such as searching by a business key.

Parker ended up with their outcomes, some expected, some less so: basically, BPMN 2.0 is really working for them both for business-IT collaboration and model-driven development; this level of business-IT alignment means that error handling can be shared, with business handling business errors, and IT handling IT errors. They found that developers became productive very quickly since they were just adding some tools to their existing familiar Java application development environment, although some had to be gently reminded to use the BPM capabilities instead of writing code.

It was great to see the reactions and interactions of the camunda team during the presentation: Australia Post is a “do-it-themselves” open source user of camunda, and as Parker discussed some of the shortcomings, they were obviously taking some notes for future work. The presentation finished with him being presented as an award for the non-camunda person who contributed most to the community forum discussions, suggesting that you get out of open source what you put into it.

Developer-Friendly BPM

I gave a webinar today sponsored by camunda on developer-friendly BPM, discussing the myth of zero-code BPM. I covered the different paradigms of BPM development, that is, fully model-driven versus process models and code, with some pointers for how to evaluate the different approaches within your organization – it’s not a simple one-size-fits-all decision for most large companies with complex application development needs.

I also wrote a white paper on the topic, it will be posted on the camunda site soon, as well as a link to a replay of the webinar.

Update: the webinar replay is here and the white paper is here. Registration on the camunda site required for both.

In the meantime, you can see my slides here:

Developer-Friendly BPM from Sandy Kemsley

Jakob Freund of camunda co-presented with me, his slides are here:

camunda for developer-friendly BPM from camunda services GmbH