Accelerating Process Modeling with Process Mining

Back in 2008, I started attending the annual academic research BPM conference, which was in Milan that year. I’m not an academic, but this wasn’t just an excuse for a week in Europe: the presentations I saw there generated so many ideas about the direction that the industry would/should take. Coincidentally, 2008 was also the first year that I saw process mining offered as a product: I had a demo with Keith Swenson of Fujitsu showing me their process discovery product/service in June, then saw Anne Rozinat’s presentation at the academic conference in September (she was still at Eindhoven University then, but went on to create Fluxicon and their process mining tool).

Over the years, I met a lot of people at this conference who accepted me as a bit of a curiosity; I brought the conference some amount of publicity through my blog posts, and pushed a lot of software vendors to start showing up to see the wild and wonderful ideas on display. They even invited me to give a keynote in 2011 on the changing nature of work. Two of the people who I met along the way, Marlon Dumas of University of Tartu and Marcello La Rosa of University of Melbourne, went on to form their own process mining company, Apromore.

I’ve recently written a white paper for Apromore to help demystify the use of process mining alongside more traditional process modeling techniques by business analysts. From the introduction:

Process modeling and process mining are complementary, not competitive, techniques: a business analyst needs both in their toolkit. Process mining provides exact models of the system-based portions of processes, while manual modeling and analysis captures human activities, documents informal procedures, and identifies the many ways that people “work around” systems.

Head on over to their site to read the full paper (registration required).

Process and content: the chocolate and peanut butter of business

I’m definitely a process person, but my start in the business was through document-driven imaging and workflow systems. It’s important to keep in mind, no matter where you lie on the spectrum of interest between process and content, that they are often intertwined: unstructured content may be a driver for process, or be the product of a process. Processes sometimes exist only to manage the content, and sometimes content only exists as supporting documentation for a process. A few years ago, I wrote about several of the process/content use cases that I see in practice for the Alfresco blog.

One thing I didn’t cover at that time is the use of processes (and rules) to govern access to content: although a good content management system will let the right person see the right content, not all unstructured content is stored in a content management system at all, much less a good one. Even if content is in a content management system, it may not be appropriate to just let everyone root around in there to find whatever documents that they might want to see. Access to content is often contextual, that is, when someone is acting in a certain role and performing a certain task, they should see specific content. In another context, they might see different content. This is even more important when you open up your processes and content to external participants, including customers and business partners.

I’ve had the chance to talk about some of these ideas in more detail in a couple of places. First, my most recent guest post on the Trisotech blog is called In financial services, process rules content, and looks at how this can work in financial applications such as insurance underwriting:

There are a lot of laggards [which have] a somewhat disorganized collection of content related to and created by processes, stored on multiple internal systems, with little or no internal access control, and no external access. In fact, I would say that in every insurance and financial operation that I’ve visited as a consultant, I’ve seen some variation of this lack of content governance, and the very real impacts on operational performance as well as privacy concerns. This is definitely a situation where process can come to the rescue for getting control over access to unstructured content.

Secondly, I’m presenting a webinar and writing a short paper for ASG Technologies on content governance in customer-facing processes. The webinar will be on May 5th, and the white paper available as a follow-on shortly after that. You can register here to attend. Hope to see you there!

Camunda Platform 7.15: now low-code (-ish)

I had a quick briefing with Daniel Meyer, CTO of Camunda, about today’s release. With this new version 7.15, they are rebranding from Camunda BPM to Camunda Platform (although most customers just refer to the product as “Camunda” since they really bundle everything in one package). This follows the lead of other vendors who have distanced themselves from the BPM (business process management) moniker, in part because what the platforms do is more than just process management, and in part because BPM is starting to be considered an outdated term. We’ve seen the analysts struggle with naming the space, or even defining it in the same way, with terms like “digital process automation”, “hyperautomation” and “digitalization” being bandied about.

An interesting pivot for Camunda in this release is their new support for low-code developers — which they distinguish as having a more technical background than citizen developers — after years of primarily serving the needs of professional technical (“pro-code”) developers. The environment for pro-code developers won’t change, but now it will be possible for more collaboration between low-code and pro-code developers within the platform with a number of new features:

  • Create a catalog of reusable workers (integrations) and RPA bots that can be integrated into process models using templates. This allows pro-code developers to create the reusable components, while low-code developers consume those components by adding them to process models for execution. RPA integration is driving some amount of this need for collaboration, since low-code developers are usually the ones on the front-end of RPA initiatives in terms of determining and training bot functionality, but previously may have had more difficult integrating those into process orchestrations. Camunda is extending their RPA Bridge to add Automation Anywhere integration to their existing UIPath integration, which gives them coverage of a significant portion of the RPA market. I covered a bit of their RPA Bridge architecture and their overall view on RPA in one of my posts from their October 2020 CamundaCon. I expect that we will soon see Blue Prism integration to round out the main commercial RPA products, and possibly an open source alternative to appeal to their community customers.
  • DMN support, including DRD and decision tables, in their Cawemo collaborative modeler. This is a good way to get the citizen developers and business analysts involved in modeling decisions as well as processes.
  • A form builder. Now, I’m pretty sure I’ve heard Jakob Freund claim that they would never do this, but there it is: a graphical form designer for creating a rudimentary UI without writing code. This is just a preliminary release, only supporting text input fields, so isn’t going to win any UI design awards. However, it’s available in the open source and commercial versions as well as accessible as a library in bpmn.io, and will allow a low-code developer to do end-to-end development: create process and decision models, and create reusable “starter” UIs for attaching to start events and user activities. When this form builder gets a bit more robust in the next version, it may be a decent operational prototyping tool, and possibly even make it into production for some simple situations.

They’ve also added some nice enhancements to Optimize, their monitoring and analytics tool, and have bundled it into the core commercial product. Optimize was first released mid-2017 and is now used by about half of their customers. Basically, it pumps the operational data exhaust out of the BPM engine database and into an elastic search environment; with the advent of Optimize 3.0 last year, they could also collect tracking events from other (non-Camunda) systems into the same environment, allowing end-to-end processes to be tracked across multiple systems. The new version of Optimize, now part of Camunda Platform 7.15, adds some new visualizations and filtering for problem identification and tracking.

Overall, there’s some important things in this release, although it might appear to be just a collection of capabilities that many of the all-in-one low-code platforms have had all along. It’s not really in Camunda’s DNA to become a proprietary all-in-one application development platform like Appian or IBM BPM, or even make low-code a primary target, since they have a robust customer base of technical developers. However, these new capabilities create an important bridge between low-code developers who have a better understanding of the business needs, and pro-code developers with the technical chops to create robust systems. It also provides a base for Camunda customers who want to build their own low-code environment for internal application development: a reasonably common scenario in large companies that just can’t fit their development needs into a proprietary application development platform.

ProcessMaker’s ProcessCon 2021 – it’s a wrap!

Today, I attended (and spoke at) ProcessMaker’s first user conference, ProcessCon 2021. This was virtual, of course, with some pre-recorded presentations and some — like mine — live. There was a reasonable amount of live Q&A, keeping things interesting.

We started with CEO Brian Reale’s opening keynote, then I was up with a presentation on Process Automation for Business Survival; you can see my slides below, and I’ll add a link to the presentation replays when I get it.

I took a bit of a break, then tuned back to see Alan Bollinger, ProcessMaker’s Director of Product & Engineering, talk about the next 12 months of their roadmap, with the upgrade of their ProcessMaker 4.0 to today’s release of v4.1. This adds more than 35 enhancements to the previous version, and cleans up a lot of open issues. I’m not a ProcessMaker aficionado so not sure all of what’s completely new versus enhancements, but there’s some cool new stuff with BPMN signals that allow for choreographed (rather than orchestrated) processes, new things with data object handling, and some new conversational forms capabilities for driving a chatbot-style interface.

Alan Bollinger presents new capabilities for user signals: trigger a process when a user is created

Beyond this newly-released version, they are adding user signals (BPMN signals that can be thrown with any changes to user profile data, e.g., to trigger a process when a new user is created), SCIM for cross-domain identity management, organization rules, parallel/serial multi-instance activities, and script APIs to allow direct REST access to any scripts without using BPMN.

There were some interesting customer case studies, and we finished the half-day conference with an open Q&A with a number of the speakers rejoining, including me. A lot of the questions were product-specific ones for Bollinger on the upcoming releases, but we had a good chat on the relative merits of choreography and orchestration, plus about conversational interfaces.

Keynote at ProcessMaker’s ProcessCon: Process Automation for Business Survival

This Thursday, ProcessMaker is having their first-ever user conference, and I’m giving a keynote! I’m going to look back at our year of living disruptively, and give my view of why process automation is no longer a luxury, but a necessity for businesses to survive and thrive.

The conference runs about 3.5 hours starting at 10am Eastern, and it will kick off with a keynote by founder and CEO Brian Reale on the future of hyperautomation and low-code before I take the virtual stage. The event is free and open to everyone, you will likely find some of the talks valuable even if you’re not a ProcessMaker customer. Also, there’s a Slack workspace to hang out and have discussions before, during and after the event.

Opening up open source development with Bonitasoft 2021.1

I had a briefing with Bonitasoft CEO Miguel Valdes earlier this week to hear more about their 2021.1 release, announced today. Note that they have changed their version numbers to align with the year and release number relative to that year, something I’ve seen starting to happen with a number of vendors.

The most important part of this release, in my opinion, is their shift in what’s open source versus commercial. Like most open source vendors, Bonitasoft is built on an open source core engine and has a number of other open source capabilities, but creates proprietary commercial components that they license to paying customers in a commercial version of the system. In many cases, the purely open source version is great for technical developers who are using their own development tools, e.g., for data modeling or UI development, but it’s a non-starter for business developers since you can’t build and deploy a complete application using just the open source platform. Bonitasoft is shaking up this concept by taking everything to do with development — page/form/layout UI design, data object models, application descriptors, process diagrams, organization models, system connectors, and Git integration — and adding it to the open source version. In other words, instead of having one version of their Studio development environment for open source and another for commercial, everything is unified, including everything in the open source version necessary to develop and deploy process automation applications. Having a unified development environment also makes it easy to move a Bonitasoft application from the open source to the commercial version (and the reverse) without project redevelopment, allowing customers to start with one version and shift to the other as project requirements change.

This is a big deal for semi-technical business developers, who can now use Bonita Studio to create complete applications on the same underlying platform as technical developers. Bonitasoft has removed anything that requires coding from Studio, recognizing that business developers don’t want to write code, and technical developers don’t use the visual development environment anyway. [As a Java developer at one of my enterprise clients once said when presented with a visual development environment, “yeah, it looks nice, but I’ll never use it”.] That doesn’t mean that these two types of developers don’t collaborate, however: technical developers are critical for the development of connectors, for example, which allow an application to connect to an external system. Once connectors are created to link to, for example, a legacy system, the business developers can consume those connectors within their applications. Bonitasoft provides connector archetypes that allow technical developers to create their own connectors, but what is missing is a way to facilitate collaboration between business and technical developers for specifying the functionality of a connector. For example, allowing the business developer to create a placeholder in an application/process and specify the desired behavior, which would then be passed on to the technical developer.

Miguel took me through a demo of Bonita Studio using only the open source version, showing their new starting point of MyProcessAutomation that includes sections for Organization, Business Data Model Applications, Diagrams, and Pages/Forms/Layouts. There are also separate sections for Custom Widgets and Environments: the latter is to be able to define separate environments for development, testing, production, etc., and was previously only in the commercial edition. It’s a very unified look and feel, and seems to integrate well between the components: for example, the expression editor allows direct access to business object variables that will be understandable to business developers, and they are looking at ways to simplify this further.

They are moving their UI away from Angular to Web Components, and are even using their own UI tools to create their new user and administrator portals. The previous Bonita Portal is now being replaced by two applications: one is a user portal that provides case and task list functionality, while the other is for administrators to monitor and repair any process instance problems. These two applications can be freely modified by customers to personalize and extend them within their own environment.

There are definitely things remaining in their commercial edition, with a focus on security (single sign-on), scalability (clustering, elasticity), monitoring, and automated continuous deployment. There are a few maintenance tools that are being moved from the open source to commercial version, and maintenance releases (bug fixes between releases) will be limited to commercial customers. They also have a new subscription model that helps with self-managed elastic deployments (e.g., Amazon Cloud); provides update and migration services for on-premise customers; and includes platform audits for best practices, performance tuning and code review. Along with this are some new packaged professional services offering: Fast Track for first implementation, on-premise to Bonita Cloud upgrade, and on-premise upgrades for customers not on the new subscription model.

The last thing that Miguel mentioned was an upcoming open source project that they are launching related to … wait, that might not have been for public consumption yet. Suffice to say that Bonitasoft is disrupting the open source process automation space to be more inclusive of non-technical developers, and we’ll be seeing more from them along these lines in the near future.

SAP acquiring Signavio into Business Process Intelligence unit

I first met Signavio CEO Gero Decker in 2008, when he was a researcher at Hasso Platner Institut and emailed me about promoting their BPMN poster — a push to have BPMN (then version 1.1) recognized as a standard for process modeling. I attended the academic BPM conference in Milan that year but Gero wasn’t able to attend, although his name was on a couple of that year’s modeling-related demo sessions and papers related to Oryx, an open source process modeling project. By the 2009 conference in Ulm we finally met face-to-face, where he told me about what he was working on, process modeling ideas that would eventually evolve into Signavio. By the 2010 BPM conference in Hoboken, he was showing me a Signavio demo, and we ended up running into each other at many other BPM events over the years, as well as having many online briefings as they released new products. The years of hard work that he and his team have put into Signavio have paid off this week with the announcement of Signavio’s impending acquisition by SAP (Signavio press release, SAP press release). There have been rumors floating around for a couple of days, and this morning I had the chance for a quick chat with Gero in advance of the official announcement.

The combination of business process intelligence from SAP and Signavio creates a leading end-to-end business process transformation suite to help our customers achieve the requirements needed to gain a competitive edge.

Luka Mucic, CFO of SAP

SAP is launching RISE with SAP today, with the Signavio acquisition a part of the announcement. RISE with SAP is billed as “business transformation as a service”, providing business process redesign (including Signavio), technical migration (which appears to be a push to get reluctant customers onto their current platform), and building an intelligent enterprise (which is mostly a cloud infrastructure message).

This is a full company acquisition, including all Signavio employees (numbering about 500). Gero and the only other co-founder still at Signavio, CTO Willi Tscheschner, will continue in their roles to drive forward the product vision and implementation, becoming part of SAP’s relatively new Business Process Intelligence unit, which is directly under the executive board. Since that unit previously contained about 100 people, the Signavio acquisition will swell those ranks considerably, and Gero will co-lead the unit with the existing GM, Rouven Morato. A long-time SAP employee, Morato can no doubt help navigate the sometimes murky organizational waters that might otherwise trip up a newcomer. Morato was also a significant force in SAP’s own internal transformation through analytics and process intelligence, moving them from the dinosaur of old to a (relatively) more nimble and responsive company, hence understands the importance of products like Signavio’s in transforming large organizations.

Existing Signavio customers probably won’t see much difference right now. Over time, capabilities from SAP will become integrated into the process intelligence suite, such as deeper integration to introspect and analyze SAP S/4 processes. Eventually product names and SKUs will change, but as long as Gero is involved, you can expect the same laser focus on linking customer experience and actions back to processes. The potential customer base for Signavio will broaden considerably, especially as they start to offer dashboards that collect information on processes that include, but are not limited to, the SAP suite. In the past, SAP has been very focused on providing “best practice” processes within their suite; however, if there’s anything that this past year of pandemic-driven disruption has taught us, those best practices aren’t always best for every organization, and processes always include things outside of SAP. Having a broader view of end-to-end processes will help organizations in their digital transformations.

Obviously, this is going to have an impact on SAP’s current partnership with Celonis, since the SAP Process Mining by Celonis would be directly in competition with Signavio’s Process Intelligence. Of course, Signavio also has a long history with SAP, but their partnership has not been as tightly branded as the Celonis arrangement. Until now. Celonis arguably has a stronger process mining product than Signavio, especially with their launch into task mining, and have a long history of working with SAP customers on their process improvement. There’s always room for partners that provide different functionality even if somewhat in competition with an internal functionality, but Celonis will need to build a strong case for why a SAP customer should pick them over the Signavio-based, SAP-branded process intelligence offering.

Keep in mind that SAP hasn’t had a great track record of process products that aren’t part of their core suite: remember SAP NetWeaver BPM? Yeah, I didn’t think so. However, Signavio’s products are focused on modeling and analyzing processes, not automating them, so they might have a better chance of being positioned as discovering improvements to processes that are automated in the core suite, as well as giving SAP more visibility into how their customers’ businesses run outside of the SAP suite. There’s definitely great potential here, but also the risk of just becoming buried within SAP — time will tell.

Disclosure: Signavio has been a client of mine within the last year for creating a series of webinars. I was not compensated in any way for writing this post (or anything else on this blog, for that matter), and it represents my own opinions.

Making experience matter by building the right incentives into processes

Last month at the Bizagi virtual conference, I gave a keynote on aligning intelligent process automation with employee incentives and business goals. I decided to expand on those themes a bit for my monthly post on the Trisotech blog. Rather than the usual sort of performance metrics, I suggest the following:

The key to designing metrics and incentives is to figure out the problems that the workers are there to solve, which are often tied in some way to customer satisfaction, then use that to derive performance metrics and employee incentives.

There are a lot of challenges with figuring out how to measure and reward experience and innovative thinking: if it’s done wrong, then companies end up measuring how long you spent with a particular app open on your screen, or how many times you clicked on your keyboard.

We’re going through a lot of process disruption right now, and smart companies are using this opportunity to retool the way that they do things. They also need to be thinking about how their employee incentives are lined up with that redesign, and whether business goals are being served appropriately.

You can check out the whole post over at Trisotech’s blog.

Disclaimer: Trisotech is my client.

(Post image by my talented friend Alison Garwood-Jones).

Happy birthday to BPMN 2.0

Bruce Silver points out that it’s been 10 years since the finalization of BPMN 2.0, the standard notation that we use for modeling (and sometimes executing) business processes. The standard wasn’t published until some time after that, and there have been revisions over the years, but BPMN 2.0 was the start of a wave of standardization in the BPMS market since it included the notation (a serialization file format) that made model interchange between products possible. There’s always been some amount of controversy swirling around BPMN: some consider it too difficult for non-technical people to understand, some consider it too restrictive for technical implementations, and more. I believe that a subset of BPMN is a good tool for process modeling by business people, and that it’s a powerful process implementation tool for developers, but I agree that it’s not the only tool you should have in your toolbox.

Bruce’s post takes us back to some basic definitions of a process, and why BPMN is a better fit for modeling processes. He also covers some of the things that are not great about the standard:

The ability to understand the process behavior in detail simply by inspecting the diagram, unfortunately, was not top of mind in the BPMN 2.0 task force in 2010. They were solely focused on making the diagrams executable on an automation engine. But to most BPMN users today, precise description based on the diagram alone is much more important.

To help with the adoption of the standard, Bruce developed conventions for the use of the standard, publishing his BPMN Method & Style books and training. Some modeling vendors have even incorporated this into their product, so that you can validate your models against his conventions.

Regardless of whether you love or hate BPMN, it’s impossible to deny the impact that it has had on the BPMS market.

On the folly of becoming the “product expert”

This post by Charity Majors of Honeycomb popped up in my feed today, and really resonated relative to our somewhat in-bred world of process automation. She is talking about the need to move between software development teams in order to keep building skills, even if it means that you move from a “comfortable” position as the project expert to a newbie role:

There is a world of distance between being expert in this system and being an actual expert in your chosen craft. The second is seniority; the first is merely .. familiarity

I see this a lot with people becoming technical experts at a particular vendor product, when it’s really a matter of familiarity with the product rather than a superior skill at application development or even process automation technology. Being dedicated to a single product means that you think about solving problems in the context of that product, not about how process automation problems in general could be solved with a wider variety of technology. Dedication to a single product may make you a better technician but does not make you a senior engineer/architect.

Majors uses a great analogy of escalators: becoming an expert on one project (or product) is like riding one long escalator. When you get to the top, you either plateau out, or move laterally and start another escalator ride from its bottom up to the next level. Considering this with vendor products in our area, this would be like building expertise in IBM BPM for a couple of years, then moving to building Bizagi expertise for a couple of years, then moving to Camunda for a couple of years. At the end of this, you would have an incredibly broad knowledge of how to solve process automation projects on a variety of different platforms, which makes you much more capable of making the type of decisions at the senior architecture and design level.

This broader knowledge base also reduces risk: if one vendor product falls out of favor in the market, you can shift to others that are already in your portfolio. More importantly, because you already understand how a number of different products work, it’s easier to take on a completely new product. Even if that means starting at the bottom of another escalator.