Activiti BPM Suite – Sweet!

There are definitely changes afoot in the open source BPM market, with both Alfresco’s Activiti and camunda releasing out-of-the-box end-user interfaces and model-driven development tools to augment their usual [Java] developer-friendly approach. In both cases, they are targeting “citizen developers”: people who have technical skills and do some amount of development, but in languages lighter weight than Java. There are a lot of people who fall into this category, including those (like me) who used to be hard-core developers but fell out of practice, and those who have little formal training in software development but have some other form of scientific or technical background.

Prior to this year, Activiti BPM was not available as a standalone commercial product from Alfresco, only bundled with Alfresco or as the community open source edition; as I discussed last year, their main push was to position Activiti as the human-centric workflow within their ECM platform. However, Activiti sports a solid BPMN engine that can be used for more than just document routing and lifecycle management, and in May Alfresco released a commercially-supported Alfresco Activiti product, although focused on the human-centric BPM market. This provides them with opportunities to monetize the existing Activiti community, as well as evolving the BPM platform independently of their ECM platform, such as providing cloud and hybrid services; however, it may have some impact on their partners who were relying on support revenue for the community version.

The open source community engine remains the core of the commercial product – in fact, the enterprise release of the engine lags behind the community release, as it should – but the commercial offering adds all of the UI tools for design, administration and end-user interface, plus cluster configuration for the execution engine.

Activiti Administrator cluster monitoringThe Activiti Administrator is an on-premise web application for managing clusters, deploying process models from local packages or the Activiti Editor, and technical monitoring and administration of in-flight processes. There’s a nice setup wizard for new clusters – the open source version requires manual configuration of each node – and allows nodes within the cluster to be auto-discovered and monitored. The monitoring of process instances allows drilling into processes to see variables, the in-flight process model, and more. Not a business monitoring tool, but seems like a solid technical monitoring tool for on-premise Activiti Enterprise servers.

The Activiti Editor is a web-based BPMN process modeling environment that is a reimplementation of other open-source tools, refactored with JavaScript libraries for better performance. The palette can be configured based on the user profile in order to restrict the environment, which would typically be used to limit the number of BPMN objects available for modeling in order to reduce complexity for certain business users to create simple models; a nice feature for companies that want to OEM this into a larger environment. Models can be shared for comments (in a history stream format), versioned, then accessed from the Eclipse plug-in to create more technical executable models. Although I saw this as a standalone web app back in April, it is now integrated as the Visual Editor portion of Kickstart within the Activiti Suite.

Activiti SuiteThe Activiti Suite is a web application that brings together several applications into a single portal:

  • Kickstart is their citizen development environment, providing a simple step editor that generates BPMN 2.0 – which can then be refined further using the full BPMN Visual Editor or imported into the Eclipse-based Activiti Designer – plus a reusable forms library and the ability to bundles processes into a single process application for publishing within the Suite. In the SaaS version, it will integrate with cloud services including Google Drive, Alfresco, Salesforce, Dropbox and Box.
  • Tasks is the end-user interface for starting, tracking and participating in processes. It provides an inbox and other task lists, and provides for task collaboration by allowing a task recipient to add others who can then view and comment on the task. Written in Angular JS.
  • Profile Management to , for user profile and administration
  • Analytics, for process statistics and reports.

The Suite is not fully responsive and doesn’t have a mobile version, although apparently there are mobile solutions on the way. Since BP3 is an Activiti partner, some of the Brazos tooling is available already, and I suspect that more mobile support may be on the way from BP3 or Alfresco directly.

They have also partnered with Fluxicon to integrate process mining, allowing for introspection of the Activiti BPM history logs; I think that this is still a bit ahead of the market for most process analysts but will make it easy when they are ready to start doing process discovery for bottlenecks and outliers.

I played around with the cloud version, and it was pretty easy to use (I even found a few bugs Smile ) and it would be usable by someone with some process modeling and lightweight development skills to build apps. The Step Editor provides a non-BPMN flowcharting style that includes a limited number of functions, but certainly enough to build functional human-centric apps: implicit process instance data definition via graphical forms design; step types for human, email, “choice” (gateway), sub-process and publishing to Alfresco Cloud; a large variety of form field types; and timeouts on human tasks (although timers based on business days, rather than calendar days, are not there yet). The BPMN Editor has a pretty complete palette of BPMN objects if you want to do a more technical model that includes service tasks and a large variety of events.

Although initially launched in a public cloud version, everything is also available on premise as of the end of November. They have pricing for departmental (single-server up to four cores with a limit on active processes) and enterprise (eight cores over any number of servers, with additional core licensing available) configurations, and subscription licensing for the on-premise versions of Kickstart and Administrator. The cloud version is all subscription pricing. It seems that the target is really for hybrid BPM usage, with processes living on premise or in the cloud depending on the access and security requirements. Also, with the focus on integration with content and human-centric processes, they are well-positioned to make a play in the content-centric case management space.

Instead of just being an accelerator for adding process management to Java development projects, we’re now seeing open source BPM tools like Activiti being positioned as accelerators for lighter-weight development of situational applications. This is going to open up an entire new market for them: an opportunity, but also some serious new competition.

Activiti 2014 

What’s New With SAP Operational Process Intelligence

Just finishing up some notes from my trip to SAP TechEd && d-code last week with the latest on their Operational Process Intelligence product, which can pull events and data from multiple systems – including SAP’s ERP and other core enterprise systems as well as SAP BPM – and provides real-time analytics via their HANA in-memory database. I attended a session on this, then had an individual briefing later to round things out.

Big processes are becoming a thing, and if you have big processes (that is, processes that span multiple systems, and consume/emit big data and high volume from a variety of sources), you need to have operational intelligence integrated into those processes. SAP is addressing this with their SAP Operational Process Intelligence, or what they see as a GPS for your business: a holistic view of where you are relative to your goals, the obstacles in your path, and the best way to reach your goals. It’s not just about what has happened already (traditional business intelligence), but what is happening right now (real-time analytics), what is going to happen (predictive analytics) and the ability to adjust the business process to accommodate the changing environment (sense and respond). Furthermore, it includes data and events from multiple systems, hence needs to provide scope beyond any one system’s analytics; narrow scope has been a major shortcoming of BPMS-based analytics in the past.

In a breakout session, Thomas Volmering and Harsh Jegadeesan gave an update and demo on the latest in their OPInt product. There are some new visualization features since I last saw it, plus the ability to do more with guided tasks including kicking off other processes, and trigger alerts based on KPIs. Their demo is based on a real logistics hub operation, which combines a wide variety of people, processes and systems, with the added complexity of physical goods movement.

Although rules have always been a part of their product suite, BRM is being highlighted as a more active participant in detecting conditions, then making predictions and recommendations, leveraging the ability to run rules directly in HANA: putting real-time guardrails around a business process or scenario. They also use rules to instantiate processes in BPM, such as for exception handling. This closer integration of rules is new since I last saw OPInt back at SAPPHIRE, and clearly elevates this from an analytics application to an operational intelligence platform that can sense and respond to events. Since SAP BPM has been able to use HANA as a database platform for at least a year, I assume that we will eventually see some BPM functionality (besides simple queuing) pushed down into HANA, as they have done with BRM, allowing for more predictive behavior and analytics-dependent functions such as work management to be built into BPM processes. As it is, hosting BPM on HANA allows the real-time data to be integrated directly into any other analytics, including OPInt.

OPInt provides ad hoc task management using a modern collaborative UI to define actions, tasks and participants; this is providing the primary “case management” capability now, although it’s really a somewhat simpler collaborative task management. With HANA behind the scenes, however, there is the opportunity for SAP to take this further down the road towards full case management, although the separation of this from their BPM platform may not prove to be a good thing for all of the hybrid structured/unstructured processes out there.

The creation of the underlying models looks similar to what I’ve been seeing from them for a while: the business scenario is defined as a graphical flow model (or imported from a process in Business Suite), organized into phases and milestones that will frame the visualization, and connected to the data sources; but now the rules can be identified directly on the process elements. The dashboard is automatically created, although it can be customized. In a new view (currently still in the lab), you will also be able to see the underlying process model with overlaid analytics, e.g., cost data; this seems like a perfect opportunity for a process mining/discovery visualization, although that’s more of a tool for an analyst than whoever might be monitoring a process in real-time.

bpmNEXT 2014 Thursday Session 2: Decisions And Flexibility

In the second half of the morning, we started with James Taylor of Decision Management Solutions showing how to use decision modeling for simpler, smarter, more agile processes. He showed what a process model looks like in the absence of externalized decisions and rules: it’s a mess of gateways and branches that basically creates a decision tree in BPMN. A cleaner solution is to externalize the decisions so that they are called as a business rules activity from the process model, but the usual challenge is that the decision logic is opaque from the viewpoint of the process modeler. James demonstrated how the DecisionsFirst modeler can be used to model decisions using the Decision Model and Notation standard, then link a read-only view of that to a process model (which he created in Signavio) so that the process modeler can see the logic behind the decision as if it were a callable subprocess. He stepped through the notation within a decision called from a loan origination process, then took us into the full DecisionsFirst modeler to add another decision to the diagram. The interesting thing about decision modeling, which is exploited in the tool, is that it is based on firmer notions of reusability of data sources, decisions and other objects than we see in process models: although reusability can definitely exist in process models, the modeling tools often don’t support it well. DecisionsFirst isn’t a rules/decision engine itself: it’s a modeling environment where decisions are assembled from the rules and decisions in other environments, including external engines, spreadsheet-based decision tables, or knowledge sources describing the decision. It also allows linking to the processes from which it is invoked, objectives and organizational context; since this is a collaborative authoring environment, it can also include comments from other designers.

François Chevresson-Aubain and Aurélien Pupier of Bonitasoft were up next to show how to build flexibility into deployed processes through a few simple but powerful features. First, adding collaboration tasks at runtime, so that a user in a pre-defined step who needs to include other users at that point can do so even if collaboration wasn’t built in at that point. Second, process model parameters can be changed (by an administrator) at runtime, which will impact all running processes based on that model: the situation demonstrated was to change an external service connector when the service call failed, then replay the tasks that failed on that service call. Both of these features are intended to address dynamic environments where the situation at runtime may be different from that at design time, and how to adjust both manual and automated tasks to accommodate those differences.

We finished the morning with Robert Shapiro of Process Analytica on improving resource utilization and productivity using his Optima workbench. Optima is a tool for a serious analyst – likely with some amount of statistical or data science background – to import a process model and runtime data, set optimization parameters (e.g., reduce resource idleness without unduly impacting cycle time), simulate the process, analyze the results, and determine how to best allocate resources in order to optimize relative to the parameters. Although a complex environment, it provides a lot of visualization of the analytics and optimization; Robert actually encourages “eyeballing” the charts and playing around with parameters to fine-tune the process, although he has a great deal more experience at that than the average user. There are a number of analytical tools that can be applied to the data, such as critical path modeling, and financial parameters to optimize revenues and costs. It can also do quite a bit of process mining based on event log inputs in XES format, including deriving a BPMN process model and data correlation based on the event logs; this type of detailed offline analysis could be applied with the data captured and visualized through an intelligent business operations dashboard for advanced process optimization.

We have one more short session after lunch, then best in show voting before bpmNEXT wraps up for another year.

bpmNEXT Wrapup: The Good, The Bad And The Best In Show

The first bpmNEXT conference has finished, and it was a great experience for me. I’m still on the west coast, enjoying a half-day in San Francisco before flying home, and having a bit of time to reflect on what I liked — and the few things I didn’t like — about this week’s event.

First and foremost, this was primarily a meeting of peers to discuss BPM innovation in an open fashion. It was not targeted at customers, so there was little of the peacock-like preening and posturing that you get when vendors parade in front of them. It was also not attended by the major analyst firms, so ditto. Since the format required that presenters give a demo, there was a much heavier bias towards technical attendees, although many of them were in product management/marketing or company founder roles, so not code monkeys. For me, that’s the perfect group for networking: people who are technical and articulate, but want to talk about more than just the technology. The atmosphere was collegial and friendly, even (for the most part) between competitors, and I had the feeling that many of the presenters just wanted to show off their cool new stuff because they knew that this was the audience that would most appreciate it. I really think that Bruce and Nathaniel achieved their goal of making this “like DEMO for BPM”. For the vendors that didn’t attend, or who attended but didn’t participate because they didn’t want to show all their cool new stuff to their competitors: you are totally missing the point, and you missed a great opportunity. Ideas need a bit of exposure to the light in order to grow properly, and this is the place for that.

Second, cool and awesome demos! The “Best in Show” awards that we all voted on at the end (to Fluxicon, Whitestein and Fujitsu) were well-deserved, although many others were equally deserving. I loved the bleeding edge demos — Gero Decker of Signavio accidentally showed us how to do process modeling with head gestures — and the skunkworks projects that may never see the light of day but represent some different thinking — Keith Swenson of Fujitsu with his Cognoscenti demo really brought advanced case management to life. Anne Rozinat and Christian Gunther were in the first demo slot, which could have been tough, but they set the bar extremely high (they won first prize, after all) and wowed a lot of the North Americans there who had never had the chance to see their Disco process mining product, born out of their work at Eindhoven University. The demos that didn’t work as well were those that spent too much time on slides instead of the demo, those that were customer-facing rather than optimized for peer review, and those that tried to cover too much ground in a 20-minute demo. If I can give a word of advice to those vendors who have given me briefings in the past, treat this like that: no slides, no bullshit, no distractions.

Third, nice location, although there were some minuses as well as pluses here. Asilomar is beautiful, but we had no spare time to do more than take a quick walk to the beach on a break. I don’t propose lengthening the calendar or reducing the number of demos, but rather rearranging the first day: take a nice long break in the middle of the day after lunch for people to explore around, then go a bit later or even have some evening demos after dinner. Since pretty much everyone arrived by Tuesday dinner, there could have been demos on that evening, too. It’s a fairly remote location, people mostly didn’t do much after dinner anyway (except, for those of us from time zones further east, go to bed early), and evening demos could have been fun sessions with some libations involved — maybe a BPM buzzword drinking game? The remote location may have deterred some, it’s a 2-1/2 hour drive south of San Francisco, but it did mean that we mostly hung out with groups of other attendees, making for better networking.

Fourth, this was an unparalleled networking experience. As an extroverted introvert (really, just check my Myers-Briggs), I usually dislike networking events, but this felt less like an uncomfortable meet-and-greet and more like chatting with old friends. Which, for the most part, it was: I knew a lot of people there, both face-to-face and online. I had the chance to meet several people who I knew only online in the past but already felt like I knew, such as Anatoly Belaychuk and Ashish Bhagwat. About 1/3 of the 80 attendees were international (including three Canadians), meaning that there was a significant European contingent here, showing off some of the outstanding BPM innovation that is less often seen in North America, and possibly creating some trans-Atlantic relationships that might bloom into partnerships in the future.

Lastly, a few healthier, lower-carb snacks would not have gone amiss: I think that I ate my own weight in chocolate. Only the health benefits of the red wine offset all that. 🙂

bpmNEXT: Fluxicon, PeopleServ, Signavio, itp commerce

Full bpmNEXT program here. The format is 30 minutes per speaker: 20 minutes of demo, 10 minutes of Q&A.

Day 1, first session: it’s Model Morning!

Process Mining: Discovering Process Maps from Data, Anne Rozinat and Christian Gunther, Fluxicon – uncovering the secrets within your processes

Process mining, beginning with an analogy of analyzing wind and currents to optimize sailing routes. The process reality is always more complex than the ideal model. The model is helpful for understanding the overall flow, but you need to see what’s actually happening if you want to understand – and optimize – it in more detail. Demo of their Disco product, starting with a log file in a spreadsheet that includes a case ID, start and end timestamp, activity name, resource and role for each activity in a process (resource and role are not required, but can help with the analysis). Imported this into Disco, identify the fields, and Disco reconstructs the process as it was actually executed. The colors, thickness of arcs and numbers indicate the frequency of each path – great for identifying exceptions and non-compliance. Interactive filters can remove less frequent paths to show the main flow. Statistics views show aggregate information for the data set, such as case (process instance) duration and mean wait time. Filters can be added, e.g. to identify the cases with the longest duration, then switch back to the process diagram view to see the flow for that filtered set of cases to identify the bottlenecks and loopbacks. Performance view of process diagram shows the time for each path. Animation view feeds the actual log data through the process diagram to help with visualization of bottlenecks. Clicking on a path creates a filter for only those cases that follow that path — the example shown is to find the small number of cases that bypassed a mandatory compliance step — then view the actual log data to identify the cases and resources involved. Can also generate a performance view of the log data rather than a process view, that shows the path between roles, indicating handoffs in the process and allowing for identification of resource bottlenecks. They’re working with a simulation vendor to link the models that they generate with simulation capabilities, which would allow for what-if scenarios. A number of questions about the log file: obviously, if the data’s not there, you can’t analyze it, and you might have to massage the log file prior to importing to Disco.

Managing Process Roles and Relationships, Roy Altman, PeopleServ – putting people into context

People in today’s organizations may assume multiple roles and have complex relationships with others, with can make it difficult to manage an organizational view of the company as well as people’s responsibilities within processes. Most organizations have some sort of matrix structure when it comes to projects, for example, so that a person’s manager on a project is not the same as their reporting manager. Issues such as delegation can make things more complex. PeopleServ’s Connexxions uses a rules-based approach to determine the correct role for each resource in the context of each process; it creates a functional integration between people and a technical integration between systems. Within the tool, create a context, then add a person and department and create the relationship between them (e.g., manages). Add another person and create relationship between then, (e.g., peer). Add rules, which may include links to data from other systems such as HR systems (e.g., all of the people actively working in that department), LDAP and other sources of organizational information. Contexts can be nested, so one context can be added as a node in another context. The context can be traversed starting at a node (e.g., find a person’s manager in that context, or find all people reporting to a person): this is where a lot of the power comes in, since allows identification of someone’s role and relationships in each individual context, e.g., projects, teams and departments. These contexts/traversals can be accessed/consumed from applications in order to control access and establish contextual reporting structure. People throughout the organization could create and edit contexts as appropriate: HR may manage the overall organization chart, departmental and project managers may manage roles within their teams, and anyone may manage roles for more casual groups such as social teams.

Lowering the Barriers to BPMN, Gero Decker, Signavio – BPMN made easy, and cool

Signavio is used not just for BPMN process modeling for IT projects such as ERP, CRM and BPMS implementations; in fact, that’s only 30% of their business. 40% of customers use it purely for process analysis and improvement, and the remaining 30% used it as part of their certification, risk or compliance efforts. They focus on providing capabilities for both easy creation of an initial graphical process model, then more complex tools for validation, understanding and refinement of the model, in order to lower the barriers to good process modeling. For people less familiar with graphical modeling, they can use Quick Model to enter the activities in a spreadsheet-style format, and see the diagram generated as each activity is entered or edited. The columns in the activity entry table are What, Who, How, IT Systems, Input documents, and Output documents; only the What is required (which becomes the activity name), specifying the How causes swimlanes to appear. At any point, you can switch to the full graphical editor, which allows addition of gateways and other elements, and also supports voice input. The shape palette can be filtered by the BPMN 2.0 specification, e.g., core elements only, and there is a tool to validate against Bruce Silver’s BPMN Method and Style conventions, which displays the improper usages and suggests fixes. They can also apply best practices and other style guides, such as DODAF, and can show a list of the rules and which are violated by the model. The tool can maintain and compare different versions of a model, much like a code diff either for subsequent revisions on the same model, or variants that are branched off. Models can be exported in BPMN XML and other formats, and their tool is integrated and/or OEMed into several BPM automation tools as the front-end modeler. They have an experimental feature — not yet, or possibly ever, part of the core product — using a device that detects hand movements and gestures (or, as we saw accidentally, head movements) to edit the process models; although this would not work for fine-grained modeling, it would be cool when presenting and doing some minor edits on a model in front of a larger group. They’re also working on vocabulary analysis for refactoring/correcting element labels, which would ensure that standardized language is used within the models.

Automated Assessment of BPMN 2.0 Model Quality, Stephan Fischli and Antonio Palumbo, itp commerce – a quality engineer’s dream

itp commerce is a long-standing process modeling and analysis product that uses Microsoft Visio with their add-on as the front end graphical environment, which provides a transition for existing Visio users to create BPMN 2.0 standard process models. Bruce Silver uses this in his training (he also now uses Signavio, not sure if he’ll be maintaining both), since they were the first to provide validation of models against his style guide. They also provide a number of process model quality metrics — validation, conformance, complexity, consistency and vagueness — some of which are difficult or fuzzy or both. Looking at the model quality pop-up (from their Visio add-on), it shows overall model quality with a long list of checks to turn on/off, and preset filters for process analysis, process execution, diagram interchange, and process simulation to show the quality for those intended activities for the model. Models can also be assessed for maturity level, which is an indicator of an organization’s overall process maturity; obviously, models are only one aspect of organizational maturity, but looking at a maturity model such as OMG’s BPMM can yield requirements for process models at each maturity level, e.g., KPIs would be required in models at process maturity level 4. In their Repository Explorer tool, you can run a quality report on a process chain (a set of related processes), which generates a full spreadsheet of all of the individual metrics for each process, plus aggregate statistics on quality, conformance, maturity and much more.

Awesome first session at bpmNEXT. It’s only the mid-morning break on the first day, lots more to come.

What’s Next For BPM? bpmNEXT!

Sometime last year, I ran into Bruce Silver at a conference, and he told me about a new conference that he was planning together with Nathaniel Palmer, called bpmNEXT. As Bruce described it then, and as it says on the recently launched website, “Part TED, part DEMO, part Think Tank, bpmNEXT is not your usual BPM conference”. The concept is to see and discuss BPM innovation, not the same old stuff that we see at so many BPM vendor and analyst conferences every year, and to do it through demonstrations of new technology and ideas as well as presentations. I remember chatting with him at the time about how we needed to have something like the BPM Think Tanks back in the day when they were really about vendors, analysts and hard-core practitioners getting together to hash through ideas about how the industry needed to evolve. Then 2008 came, Think Tank tried to become a business-focused BPM conference with lots of case studies from customers, and it died a quick death – you can look back through my posts on three years of BPM Think Tank to see how it evolved.

We need a place where people involved in creating the next generation of BPM software can get together and collaborate, even if they’re competitors outside the conference. bpmNEXT has the potential to become that place.

bpmNEXT is now ready to go for March 19-21 at the Asilomar resort in Pacific Grove (Monterey), a 2-hour drive south San Francisco. The speaker list is impressive: these are all vendors, but the topics are focused on emerging capabilities. Process mining, analytics, simulation, internet of things, mobile, cloud, ACM and more; Bruce has a summary of the program in his latest blog post. Lots of people who I know, and many who I know virtually and look forward to meeting face-to-face.

You’ve already missed the extreme early bird pricing that they had last fall, but there is still a chance to sign up at a discount until February 19th – note that the price includes two nights lodging and meals. You can download a brochure here; I’m listed as a media sponsor, which means that I will be blogging from bpmNEXT, but I am only being comped the conference portion of the fee, not any travel and living expenses, and I’m definitely not being paid for my time.

I hope to see you there. If you’re in the Bay area and want to connect with me that week outside the conference days, let me know early enough that I can make extended travel arrangements.

BPM2012: Papers on Process Mining

I had a bit of blog fatigue earlier, but Keith Swenson blogged the session on process cloud concepts for case management that I attended but didn’t write about, and I’m back at it for the last set of papers for the day at BPM 2012, all with a focus on process mining.

Repairing Process Models to Reflect Reality

[link]

Dirk Fahland of Eindhoven University presented a paper on process repair, as opposed to process mining, with a focus on adjusting the original process model to maximize fitness, where fitness is measured by the ability to replace traces in the event log: if a model can replay all of the traces of actual process execution, then it is perfectly fit. Their methods compare the process model to the event log using a conformance checker in order to align the event log and the model, which can be accomplished with the methods of Adriansyah et al’s cost-based replayer to find the diagnostic information.

The result includes activities that are skipped, and activities that must be added. The activities to be added can be fed to an existing process discovery algorithm to create subprocesses that must be added to the existing process, and the activities that were skipped are either made optional or removed from the original process model.

Obviously, this is relevant in situations where the process model isn’t automated, that is, the event logs are from other systems, not directly executed from the process model; this is common when processes are implemented in ERP and other systems rather than in a BPMS, and process models are created manually in order to document the business processes and discover opportunities for optimization. However, as we implement more semi-structured and dynamic processes automated by a BPMS, the event logs of the BPMS itself will include many events that are not part of the original process model; this could be a useful technique for improving understanding of ad hoc processes. By understanding and modeling ad hoc processes that occur frequently, there is the potential to identify emergent subprocesses and add those to the original model in order to reduce time spent by workers creating the same common ad hoc processes over and over again.

There are other measurements of model quality besides fitness, including precision, generalization and simplicity; future research will be looking at these as well as improving the quality of alignment and repair.

Where Did I Misbehave? Diagnostic Information in Compliance Checking

[link to pdf paper]

Elham Ramezani of Eindhoven University presented a paper on compliance checking. Compliance checking covers the full BPM lifecycle: compliance verification during modeling, design and implementation; compliance monitoring during execution; and compliance auditing during evaluation. The challenge is that compliance requirements have to be decomposed and used to create compliance rules that can be formalized into a machine-understandable form, then compared to the event logs using a conformance checker. This is somewhat the opposite of the previous paper, which used conformance checking to find ways to modify the process model to fit reality; this looks at using conformance checking to ensure that compliance rules, represented by a particular process model, are being followed during execution.

Again, this is valuable for processes that are not automated using a BPMS or BRMS (since rules can be strictly enforced in that environment), but rather processes executing in other systems or manually: event logs from systems are compared to the process models that represent the compliance rules using a conformance checker, and the alignment calculated to identify non-compliant instances. There were some case studies with data from a medical clinic, detecting non-compliant actions such as performing an MRI and CT scan of the same organ, or registering a patient twice on one visit.

There was an audience question that was in my mind as well, which is why to express the compliance rules in Petri nets rather a declarative form; she pointed out that the best conformance checking available for aligning with event logs use operational models such as Petri nets, although they may consider adding declarative rules to this method in the future in addition to other planned extensions to the research. She also mentioned that they were exploring applicability to monitoring service level agreement compliance, which has a huge potential for business applications where SLA measurements are not built into the operational systems but must be detected from the event logs.

FNet: An Index for Advanced Business Process Querying

[link to pdf paper]

Zhiqiang Yan, also of Eindhoven University (are you seeing a theme here in process mining?), presented on querying within a large collection of process models based on certain criteria; much of the previous research has been on defining expressive query languages (such as BPMN-Q) that can be very slow to execute, but here they have focused on developing efficient techniques for executing the queries. They identify basic features, or small fragments, of process models, and advanced elements such as transitive or negative edges that form advanced features.

To perform a query, both the query and the target process models are decomposed into features, where the features are small and representative: specific sequences, join, splits and loops. Keywords for the nodes in the graphs are using in addition to the topology of the basic features. [There was a great deal of graph theory in the paper concerned with constructing directed graphs based on these features, but I think that I forgot all of my graph theory shortly after graduation.]

The results seem impressive: two orders of magnitude increase in speed over BPMN-Q. As organizations continue to develop large repositories of process models and hope to get some degree of reuse, process querying will become more important in practical applications.

Using MapReduce to scale events correlation discovery for business processes mining

[link]

The last paper of this session, and of the day, was presented by Hicham Reguieg of Blaise Pascal University in Clermont-Ferrand. One of the challenges in process mining and discovery is big data: the systems that are under consideration generate incredible amounts of log data, and it’s not something that you’re going to just open up in a spreadsheet and analyze manually. This paper looks at using MapReduce, a programming model for processing large data sets (usually by distributing processing across clusters of computers), applied to the specific step of event correlation discovery, which analyzes the event logs in order to find relationships between events that belong to the same business process.

Although he didn’t mention the specific MapReduce framework that they are using for their experiments, I know that there’s a Hadoop one – inevitable that we would start seeing some applicability for Hadoop in some of the big data process problems.

BPM2012: Wil van der Aalst BPM Research Retrospective Keynote

Day 2 of the conference tracks at BPM 2012 started with a keynote from Wil van der Aalst of Eindhoven University, describing ten years of BPM research on this 10th occasion of the International Conference on BPM. The conference started in Eindhoven in 2003, then moved to Potsdam in 2004, Nancy in 2005, Vienna in 2006, Brisbane in 2007, Milan in 2008 (my first time at the conference), Ulm in 2009, Hoboken in 2010, Clermont-Ferrand in 2011, then on to Tallinn this year. He showed a word cloud for each of the conference proceedings in the past, which was an interesting look into the hot topics at the time. The 2013 conference will be in Beijing – not sure if I’ll be attending since it’s a long trip – and I expect that we’ll hear where the 2014 conference will be before we leave Tallinn this week.

In his paper, he looked at the four main activities in BPM – modeling, analysis, enactment and management – and pointed out that much of the research focused on the first two, and we need more on the latter two. He also discussed a history of what we now know as BPM, from office automation to workflow to BPM, with contributions from many other areas from data modeling to operations management; having implemented workflow systems since the early 1990’s, this is a progression that I’m familiar with. He went through 20 BPM use cases that cover the entire BPM lifecycle, and mapped 289 research papers in the proceedings from the entire history of the BPM conferences against them:

  1. Design model
  2. Discover model from event data
  3. Select model from collection
  4. Merge models
  5. Compose model
  6. Design configurable model
  7. Merge models into configurable model
  8. Configure configurable model
  9. Refine model
  10. Enact model
  11. Log event data
  12. Monitor
  13. Adapt while running
  14. Analyze performance based on model
  15. Verify model
  16. Check conformance using event data
  17. Analyze performance using event data
  18. Repair model
  19. Extend model
  20. Improve model

He described each of these use cases briefly, and presented a notation to represent their characteristics; he also showed how the use cases can be chained into composites. The results of mapping the papers against the use cases was interesting: most papers were tagged with one or two of these use cases, although some addressed several use cases.

He noted three spikes in use cases: design model, enact model, and verify model; he found the first two completely expected, but that verifying models was a surprising focus. He also pointed out that having few papers addressing use case 20, improve model, is a definite weakness in the research areas.

He also analyzed the research papers according to six key concerns, a less granular measure than the use cases:

  1. Process modeling languages
  2. Process enactment infrastructures
  3. Process model analysis
  4. Process mining
  5. Process flexibility
  6. Process reuse

With these, he mapped the interest in these key concerns over the years, showing how interest in the different areas has waxed and waned over the years: a hype cycle for academic BPM topics.

He spent a bit of time on three specific challenges that should gain more focus research: process flexibility, process mining and process configuration; for example, considering the various types of process flexibility based on whether it is done at design time or runtime, and how it can be by specification, by deviation, by underspecification or by change.

One clear goal of his talk is to help make BPM research more relevant as it matures, in part through more evidence-based BPM research, to encourage vendors and practitioners to adopt the new ideas that are put forward in BPM. He makes some recommendations for research papers in the future:

  • Avoid introducing new languages without a clear purpose
  • Artifacts (software and data) need to be made available
  • Evaluate results based on a redefined criterion and compare with other approaches
  • Build on shared platforms rather than developing prototypes from scratch
  • Make the contribution of the paper clear, potentially by tagging papers with one of the 20 use cases listed above

As BPM research matures, it makes sense that the standards are higher for research topics in general and definitely for having a paper accepted for publication and presentation at a conference. Instead of just having a theory and prototype, there’s a need for more empirical evidence backing up the research. I expect that we’ll see an improvement in the overall quality and utility of BPM research in the coming years as the competition becomes more intense.

Process Intelligence White Paper

And here’s the white paper from this afternoon’s webinar (registration required) on Enabling Process Intelligence Through Process Mining & Analytics. Sponsored by Fujitsu, written by me, reviewed with many helpful comments by Keith Swenson and the rest of the Fujitsu team. When the on-demand webinar is available, I’ll post a link.

Enabling Process Intelligence Through Process Mining & Analytics

A bit short notice, but I’m doing a webinar this afternoon at 2pm (Eastern) on process intelligence through process mining and analytics, along with Keith Swenson of Fujitsu, and you can sign up here. Fujitsu is sponsoring the webinar, as well as the white paper that I have written to go into the topic in a bit more detail; if you sign up for the webinar, I think that they will send you a copy of the white paper.

Fujitsu has recently repackaged their Automated Process Discovery (process mining, which I have reviewed here previously) together with their analytics to create a sort of “intelligent process” suite: you use the mining part to find out what your processes are across multiple systems, then add the necessary instrumentation to those processes in order to feed into a consolidated analytics visualization. Whereas most process analytics are based just on the processes automated or orchestrated by a BPMS, Fujitsu is trying to expand that level of visibility into systems that aren’t connected to the BPMS. With my background in pattern recognition, I have a bit of interest in process mining, and have written about their process discovery tool previously as well as working my way through Wil van der Aalst’s recent book, Process Mining: Discovery, Conformance and Enhancement of Business Processes.

Hope that you can join us for the webinar today.