Shifting (back) from buy to build for digital automation

One of the advantages of being in the software industry for a long time is that I can watch trends come and go. Some of them many times. Take the buy versus build argument: is it better for an organization to build a system (for its own use) using best-of-breed components, or buy a best-in-class monolithic system from a single vendor? As with all things software, the answer is “it depends”: it depends on how well the company’s needs are accommodated by a single-vendor solution, and how much the company’s needs are expected to change on an ongoing basis.

Almost every end-customer organization that I talk to now, either in my consulting practice or through industry contacts, is deploying an in-house digital automation platform that allows them to quickly assemble capabilities into new business applications. Since business applications tend to be process- and case-centric, organizations have often ended up with a BPMS (or what Gartner might call an iBPMS) as a single-vendor solution for the core of their digital automation platform, although ERP and CRM platforms such as SAP and Salesforce are also making a play in this space.

BPMS — once (more or less) single-purpose systems for modeling and managing processes — have, Borg-like, assimilated so many other technologies and capabilities that they have become the monolith. If you sign up for their process management capabilities, you may also get decision management, analytics, event handling, user experience, social media and many other capabilities in the same box. This is what allows BPMS vendors to market their products as complete digital automation platforms, requiring only a bit of wiring to connect up with line-of-business systems and data.

If there’s one constant in how organizations work, it’s that they will outgrow their systems as their business environment (constantly) changes. And that’s exactly the problem with any monolithic system: there will certain capabilities that no longer meet your changing needs, or a disruptive new vendor or product that could replace specific capabilities with something transformative. Without the ability to decouple the components of the monolith, you may be stuck using the unwanted capabilities; at the very least, you’ll still be paying maintenance and upgrades for the entire suite rather than just the parts that you’re using.

The result of all this is that I’m seeing organizations starting to build their digital automation platforms with much more granular components, and BPMS vendors offering their products in a granularity to match that. It’s this pattern that I’ll be talking about in my bpmNEXT keynote in Santa Barbara on April 16, “Best of Breed: Rolling Your Own Digital Automation Platform using BPMS and Microservices”. Hope to see you there.

A variety of opinions on what’s ahead for BPM in 2018

I was asked to contribute to 2018 prediction posts on a couple of different sites, along with various other industry pundits. Here’s a summary. Predictions published The Year Ahead for BPM – 2018 Predictions from Top Influencers, introduced by’s Nathaniel Palmer and featuring mostly people who work for vendors but mostly whose opinions I respect. Many of the vendors’ predictions align with their product direction, either through good planning or happy coincidence. Winking smile

A few ideas that stood out:

Blockchain may be almost ready for its close-up. Miguel Valdés Faura (Bonitasoft) and Setrag Khoshafian (Pega) both mentioned the potential for integrating blockchain with processes using DPA (digital process automation) platforms. I’ve been watching this space for a couple of years, waiting for the connections to be made between BPM and blockchain, and in addition to these mentions in the predictions article, Bernd Ruecker (Camunda) published a post yesterday with a practical use case and MWD Advisors published a report on IBM Blockchain Platform that mentions its integration with IBM BPM.

Automated decisioning, whether DMN-based or AI/ML, is going to improve process automation significantly but there’s still a lot of trepidation. Denis Gagné (Trisotech) said that decision auditability – a legal requirement in some countries – could favor DMN-based decision services over AI/ML, while Roger King (TIBCO) sees AI-based automation as the key to having RPA replace workers. Keith Swenson (Fujitsu) predicts that deep learning will be both the most important and most disappointing innovation in 2018, while Peter Fingar is bullish on intelligent (AI) agents integrated with BPM. James Taylor (Decision Management Solutions) seemed a bit disheartened that DM is being trivialized as a “feature” of BPM rather than an independent stateless service where it can have the greatest impact.

Microservices architectures are replacing monolithic BPM systems. Brian Reale (ProcessMaker) predicts that microservices will disrupt the BPM market this year, and Roger King gave a nod to dynamically-orchestrated process fragments although didn’t explicitly mention microservices. I’m seeing microservices approaches from a few of the BPM vendors, and I agree that this has a lot of potential to shift away from the monolithic (and proprietary) platforms; watch for an article that I wrote for Alfresco on BPM and microservices to be published shortly on their blog.

Low code is allowing business users (analysts, really) to participate in DevOps directly. Malcolm Ross (Appian) sees low code as a catalyst for developer diversity and the blurring of lines between business and IT. Phil Simpson (Red Hat) states that low code and citizen developers are the only way to meet the need for constantly-changing applications. My concern is that, much like how business analysts were going to develop their own BPM applications when model-driven development came around several years ago, this isn’t actually going to happen in such an optimistic fashion.

Customer journey matters. Gero Decker(Signavio) is seeing top-level value chains being replaced by customer journey maps. To me, customer journey mapping feels like a bit of old wine in new bottles, 10 years after outside-in process modeling and other customer-centric views, but whatever it takes to get some traction around modeling process to include the customer and optimize from their point of view. I’m speaking on this topic at next week’s OPEX Week conference.

BPM is no longer BPM. The term is being replaced by digital process automation, digital transformation and a number of others as the platforms expand beyond just process modeling and execution. Neil Ward-Dutton (MWD Advisors) envisions different types of tools vying for the place that BPM platforms occupy now within organizations, from RPA to model-driven application development tools. As I wrote in my section the article, “BPM is dead…long live BPM!”

Lots of great insights in there, check out the entire article on Skills

iNIo-1aL_400x400Zbigniew Misiak on BPM tips takes a slightly different predictions approach, asking what BPM-related skills and techniques will be most in demand in 2018, where to learn those skills, and what’s no longer relevant in BPM Skills in 2018 – Hot or Not.

Unlike the list, which was dominated by vendors, this one has mostly opinions from consultants with some practitioners thrown in. There’s a lot of generic “people need to keep up on the latest technology trends” (duh), but some specific advice stood out:

Process/business architecture to connect processes to value. This ties in with the customer journey mapping trends that we saw in the article; here, Roger Burlton (Process Renewal/BPTrends) stresses the importance of including the customer and other external stakeholders in the processes and value definition, and Sandeep Johal (PPB Advisory) reminds us that the focus of process management is (or should be) on improving customer experience via a variety of technologies. Ian Gotts (Q9 Elements) identifies business analysis and critical questioning as key skills, linking to broader business requirements such as GDPR, and Jim Sinur (Aragon) lists journey mapping.

BPMN, CMMN and DMN for standardized modeling. Alan Fish (FICO) sees formal modeling of processes and decisions as important, and Juergen Pitschke (Process Renewal) believes both BPMN and DMN are important, but some say that this is no longer required in low code process application development tools. BJ Biernatowski (Nordstrom) wonders if BPMN has a future, Roger Burlton thinks that process analysts only need to know how to use the core elements, and Sandeep Johal advocates getting rid of manual current-state modeling as automated process discovery, analysis and improvement takes over.

RPA bot training. Abhijit Kakhandiki (Automation Anywhere) suggests this somewhat depressing skill – train the robot to do your job! – but I agree that learning this would help a lot of people create helper functions for their tasks or even completely automate some tasks. Much in the same way that we used to create Excel macros… On a similar note, I recommended that people gathering requirements become proficient with the low code BPM platforms to at least create prototypes, if not the full applications, and Phil Simpson (Red Hat) recommends that less-technical BPM practitioners start to gain an understanding of things that are likely to significantly impact how applications are designed and deployed, such as RPA and microservices.

Interpersonal and soft skills in BPM and change management, in addition to technical skills. Adrian Reed (Blackmetric) listed influencing, stakeholder engagement and conflict resolution as important to making sure that the technology part of the projects fit into the business and people. I discussed some similar skills: the ability to translate need into action, and the need for developers to learn more about what the business does.

Again, lots more to read here, check out the original on

ITESOFT | W4 Secure Capture and Process Automation digital business platform

It’s been three years since I looked at ITESOFT | W4’s BPMN+ product, which was prior to W4’s acquisition by ITESOFT. At that time, I had just seen W4 for the first time at bpmNEXT 2014, and had this to say about it:

For the last demo of this session, Jean-Loup Comeliau of W4 on their BPMN+ product, which provides model-driven development using BPMN 2, UML 2, CMIS and other standards to generate web-based process applications without generating code: the engine interprets and executes the models directly. The BPMN modeling is pretty standard compared to other process modeling tools, but they also allow UML modeling of the data objects within the process model; I see this in more complete stack tools such as TIBCO’s, but this is less common from the smaller BPM vendors. Resources can be assigned to user tasks using various rules, and user interface forms are generated based on the activities and data models, and can be modified if required. The entire application is deployed as a web application. The data-centricity is key, since if the models change, the interface and application will automatically update to match. There is definitely a strong message here on the role of standards, and how we need more than just BPMN if we’re going to have fully model-driven application development.

A couple of weeks ago, I spoke with Laurent Hénault and François Bonnet (the latter whom I met when he demoed at bpmNEXT in 2015 and 2016) about what’s happened in their product since then. From their company beginnings over 30 years ago in document capture and workflow, they have expanded their platform capabilities and relabelled it as digital process automation since it goes beyond BPM technology, a trend I’m seeing with many other BPM vendors. It’s not clear how many of their 650+ customers are using many of the capabilities of the new platform versus just their traditional imaging and workflow functions, but they seem to be expanding on the original capabilities rather than replacing them, which will make transitioning customers easier.

31 ITESOFT W4 platform as part of enterprise architectureThe new platform, Secure Capture and Process Automation (SCPA), provides capabilities for capture, business automation (process, content and decisions), analytics and collaborative modeling, and adds some nice extras in the area of document recognition, fraud detection and computer-aided process design. Using the three technology pillars of omni-channel capture, process automation, and document fraud detection, they offer several solutions including eContract for paperless customer purchase contracts, including automatic fraud detection on documents uploaded by the customer; and the cloud-based Streamline for Invoices for automated invoice processing.

Their eContract solution provides online forms with e-signature, document capture, creation of an eIDAS-compliant contract and other services required to complete a complex purchase contract bundled into a single digital case. The example shown was an online used car purchase with the car loan offered as part of the contract process: by bundling all components of the contract and the loan into a single online transaction, they were able to double the purchase close rate. Their document fraud detection comes into play here, using graphometric handwriting analysis and content verification to detect if a document uploaded by a potential customer has been falsified or modified. Many different types of documents can be analyzed for potential fraud based on content: government ID, tax forms, pay slips, bank information, and public utility invoices may contain information in multiple formats (e.g., plain text plus encoded barcode); other documents such as medical records often contain publicly-available information such as the practitioner’s registration ID. They have a paper available for more information on combatting incoming document fraud.

07 ITESOFT W4 Verifiable documentsTheir invoice processing solution also relies heavily on understanding certain types of documents: 650,000 different supplier invoice types are recognized, and they maintain a shared supplier database in their cloud capture environment to allow these formats to be added and modified for use by all of their invoice processing customers. There’s also a learning environment to capture new invoice types as they occur. Keep in mind that the heavy lifting in invoice processing is all around interpreting the vendor invoice: once you have that sorted out, the rest of the process of interacting with the A/P system is straightforward, and the payment of most invoices that relate to a purchase order can be fully automated. Streamline for Invoices won the Accounts Payable/Invoicing product of the year at the 2017 Document Manager Awards.

After a discussion of their solutions and some case studies, we dug into a more technical demo. A few highlights:

  • 09 ITESOFT W4 Web Modeler - concurrent updates of modelThe Web Modeler provides a fully BPMN-compliant collaborative process modeling environment, with synchronous model changes and (persistent) discussion thread between users. This is a standalone business analyst tool, and the model must be exported as a BPMN file for import to the engine for execution, so there’s no round-tripping. A cool feature is the ability to scroll back through the history of changes to the model by dragging a timeline slider: each changed snapshot is shown with the specific author.
  • Once the business analyst’s process model has been imported into the BPMN+ Composer tool, the full application can be designed: data model, full process model, low code forms-based user experience, and custom code (if required). This allows a more complex BPMN model to be integrated into a low code application – something that isn’t allowed by many of the low code platforms that provide only simple linear flows – as well as developer code for “beyond the norm” integration such as external portals.
  • Supervisor dashboards provide human task monitoring, including task assignment rules and skills matrix that can be changed in real time, and performance statistics.

The applications developed with their tools generally fall into the case management category, although they are document/data based rather than CMMN. Like many BPM vendors, they are finding that there is not the same level of customer demand for CMMN as there was for BPMN, and data-driven case management paradigms are often more understandable to business people.

They’ve OEM’d some of the components (the capture OCR, which is from ABBYY, and the web modeler from another French company) but put them together into a seamless offering. The platform is built on a standard Java stack; some of the components can be scaled independently and containerized (using Microsoft Azure), allowing customers to choose which data should exist on which private and public cloud infrastructure.

ITESOFT | W4 SCPA 2017-12 briefing

28 ITESOFT W4 timeline demoThey also showed some of the features that they demoed at the 2017 bpmNEXT (which I unfortunately missed): process guidance and correction that goes beyond just BPMN validation to attempt to add data elements, missing tasks, missing pathways and more; a GANTT-type timeline model of a process (which I’ve seen in BPLogix for years, but is sadly absent in many products) to show expected completion times and bottlenecks, and the same visualization directly in a live instance that auto-updates as tasks are completed within the instance. I’m not sure if these features are fully available in the commercial product, but they show some interesting views on providing automated assistance to process modeling.

What’s in a name? BPM and DPA

The term “business process management” (BPM) has always been a bit problematic because it means two things: the operations management practice of discovering, modeling and improving business processes, which may have no technology involved whatsoever; and the suite of technologies associated with automating processes. I’ve often heard – and sometimes participated in – arguments on the distinction between BPM-the-discipline and BPM-the-technology. Many people use “BPMS” (BPM system or suite) to define the technology while reserving “BPM” for the discipline, but that’s not sufficiently universal to avoid confusion.

Gartner iBPMS in 2011To compound the confusion, the components of a BPMS have grown from completely process-focused modeling and execution to more complete application development suites that may include decision management, analytics, content management and much more. Gartner relabelled this market “iBPMS” starting around 2011 when they realized that BPM suites were doing much more than just BPM:

The intelligent business process management suite (iBPMS) market is the natural evolution of the earlier BPMS market, adding more capabilities for greater intelligence within business processes. Capabilities such as validation (process simulation, including “what if”) and verification (logical compliance), optimization, and the ability to gain insight into process performance have been included in many BPMS offerings for several years. iBPMSs have added enhanced support for human collaboration such as integration with social media, mobile-enabled process tasks, streaming analytics and real-time decision management.

The term iBPMS makes it sound like what we were doing before wasn’t intelligent, which clearly is not the case, but it also made it obvious that we needed a different name to describe these technologies that we’re using to automate our business functions.

Since then, we’ve moved through a number of different names and acronyms in an attempt to describe these systems: for the more case-oriented (with little or no predefined processes), we have “case management” (confused with the non-technical term used in social sciences and healthcare) which is sometimes abbreviated as CM (confused with the abbreviation for content management, which is also abbreviated as ECM but has now be rebranded as content services) plus the variations of advanced or adaptive case management (ACM), and dynamic case management (DCM). Although there are differences between case management and BPM, there are also a lot of similarities and the distinction in products is sometimes a bit fuzzy. However, using the term “process” causes a certain amount of angst amongst the case managementerati.

This year, Forrester started using the term “digital process automation” (DPA), which is pretty much what Gartner is calling iBPMS. Forrester’s use of DPA seems to have been slightly preceded by the term “digital business automation”. Although “digital” and “automation” are a bit redundant in this context – we’re not going to do analog mechanical automation of most businesses – I think that the use of “business” rather than “process” is a much better fit. However, due to Forrester’s recent DPA wave report, vendors are leaping onto the DPA bandwagon, so we might be stuck there for a while.

From their report in February 2017, “Traditional BPM Gives Way To Digital Process Automation”, Forester describes why this shift is necessary without actually describing the differences between [i]BPM[S] and DPA; instead, this seems to be coming about because organizations took what should have been model-driven development (aka low-code) BPMS and used it in waterfall development environments, thereby turning what should have been agile into legacy. In other words, they seem to be hoping that changing the name of the class of tools will change how organizations use the tools. Call me a cynic, but I’m not completely hopeful about that.

I’m not arguing that the current low code, process/case-centric platforms that combine a full suite of business automation tools aren’t a step forward from yesterday’s BPM platforms in terms of enabling automation as a part of digital transformation. But what is going to change within customer organizations to prevent them from undermining the inherent rapid application development capabilities by enforcing antiquated software development lifecycle methods?

Bonus reading: check back on my review of a Gartner presentation from 2006 on the future of BPM, which looked forward as far as 2017! They were correct that the primary value of BPM moved from productivity to visibility to innovation, and I correctly predicted that their predictions would happen much faster than they expected.