Category: process

Brandon Dean of BEA talked about how to use BPM for compliance and improved visibility into processes. I wrote a course on compliance and BPM recently, and I was interested in how they’re seeing this roll out amongst their customer base.

Regulatory compliance (e.g., SOX) and any sort of commercial compliance (e.g., SLAs) or organizational compliance (e.g., internal KPIs) have many of the same requirements: processes need to behave in a consistent fashion, and any exceptions have to be handled using a standard method. Measurements on how well that the process is meeting its stated compliance goals are critical to understand whether or not the underlying business process is compliant. This, of course, plays directly to the strengths of BPM: providing a platform for standardizing and, where possible, automating processes; integration of multiple systems; consistent exception handling; security on the process steps and a comprehensive audit trail on who did what, and when; and monitoring and reporting for visibility into the processes and proactive alerts when they start to wander out of compliance.

Dean covered on how to position BPM for compliance, starting with a great categorization of organizational types ranging from companies that already have compliant processes but just need a better audit trail, to those that are actively trying to find ways around compliance. He made a point that I also discussed in my compliance course: if you implement compliance on a regulation-by-regulation basis, it’s a lot more expensive and time-consuming. In fact, I used a quote from a Gartner report from 2004, in the middle of the SOX gold rush:

Enterprises that choose one-off solutions for each regulatory challenge that they face will spend 10 times more on compliance projects than their counterparts that take a proactive approach.

 He went through a number of case studies and how their compliance was facilitated by BPM:

• Dental insurance claims processing, which started out as a completely manual process that had no audit trail and didn’t enforce standard rates and practices. Using BPM, they not only had some processes decrease cycle time from 3 days to 8 minutes, but they were able to meet HIPAA compliance requirements.
• Trade processing, where the SLA was not being met and they were risking losing the ability to execute trades. BPM allowed them to set alerts on trades that arrived but didn’t complete for some reason, so that any manual intervention required could be performed in time to meet their SLA. This also allowed them to do follow-the-sun processing for more intelligent human resource allocation.
• Residential mortgage processing, which wasn’t able to track requests for special handling in loan origination, and was causing them to lose customers. Using BPM, documents were automatically rendezvoused with waiting processes, and the processes presented for work at the point when they were ready to be processed rather than having people track the missing documents manually. This also automated feedback to the brokers to submit the necessary documents to reduce the wait time. A major gain was in making sure that all the information was gathered in a timely manner, and not presented for processing until all the information was available.

Although I think that Dean’s definition of compliance is a bit stretched to include both customer SLAs and internal KPIs, his points are valid for developing many types of business cases for BPM.

Jesper Joergensen gave us BEA’s view of how BPM is changing business and the future of BPM. Actually, he starts out with Gartner’s view of expected market growth (hockey, anyone?) and how BPM is becoming more and more a part of organizations’ productivity improvement initiatives as it moves from opportunistic to pervasive adoption.

There’s a number of obstacles to BPM adoption, however, many of them cultural:

1. The ivory tower of process expertise, where a few process experts are doing the modelling. Easy-to-use modelling tools like ALBPM are helping to change that, but my opinion is that we need to have more pervasive technology to enable the shift, and that’s going to be web-based, like Appian‘s process designer or Lombardi’s Blueprint process discovery tool.
2. The ROI barrier: many current opportunistic BPM projects are low-hanging fruit in terms of ROI, but projects need to deliver faster and cheaper in order to implement processes with a lesser potential return.
3. Getting beyond just system-to-system orchestration and adding human-facing steps to the process. Stop thinking of those processes as “human-interrupted” and accept people as necessary actors in BPM-automated processes.
4. Managing complexity and scale.

Jesper went on to discuss a number of areas where practices and technologies need to evolve (note that this is not a statement of where ALBPM is going, just how he thinks that the BPM market needs to change):

• Web-based process modelling (which I obviously agree with).
• improved standards and interoperability, especially between a vendor’s own tools if they have multiple tools for discovery, modelling and design.
• Automated process discovery based on monitoring (which I saw recently in a vendor demonstration but can’t at this moment recall which one), where historical trends in manual decision-making are used to suggest automation of certain decision points.
• Better integration of BPM and BI, since many products currently have very separate BPM and analytics environments that don’t integrate well.
• Standardization and service-enablement of process data; I think that BPRI may help with some of the standardization, but it likely needs to be taken further in terms of how process instance data and be extracted from a process engine via RSS feeds or other integration mechanisms.
• Process decision support, e.g., making suggestions based on historical decisions, and potentially raising exceptions if the current decision doesn’t match the past trend.
• More open process flows to allow for dynamic changes to the process flow at runtime, even if they weren’t anticipated at design time.
• Social computing functionality, such as tagging.
• Better integration with SOA and ESB.
• Enterprise-scale process execution and management to allow for end-to-end cross-departmental processes rather than the more common departmental BPM implementations that we see today.

We’re in agreement on a lot of things on this list, and I’m looking forward to how some of these ideas might creep into the product in the future.

Last session of the morning was Mateo Almenta Recca of BEA and Kunal Shah of Citigroup talking about advanced process modelling — specifically process exceptions — in ALBPM. Exceptions can be either system exceptions, such as a service being unavailable, or business (user-defined) exceptions, such as an account being closed. System exceptions are typically handled through automated retries and/or transaction rollbacks, whereas business exceptions are modelled into the process by the process designer.

Exception handlers can be built into the process at either the individual activity, group (similar to a BPMN transaction) or process level. Exception handlers at an activity appear just as an alternative path out of that activity, although an exception is typically invoked by a timeout or other non-decision activity instead of an explicit decision at that point. Exception handlers at the group level are shown connected to the group wrapper boundary, as in BPMN transaction exceptions, and process exception handlers are visualized as disconnected from the process but on the same model.

All exception handlers can take one of three basic actions: abort the process and go to the end of the process, go back and retry the step that threw the exception, or skip the step that threw the exception and move on to the next step in the process. The back and skip functionality is always at the activity level: an exception at the group level that causes a “back” instruction from the exception handler would return to the specific activity to be retried, not the entire group; “skip” would skip the activity but continue on to any later activities in the same group. This was counter-intuitive for me and I asked specifically about that: I would have expected that a group would be treated more like a BPMN transaction wrapper, such that a retry or skip would apply to the entire group, not the specific activity. Exception handlers can be automatic or manual steps: an automatic exception handler might perform some related transaction rollback in another system before aborting a process, for example, whereas a manual exception handler might allow for data repair before retrying the failed step.

They then talked about compensation flows, which seems to match the BPMN meaning of a compensation, in that it reverses a completed activity or group of activities. This isn’t so easy as just rolling back any changed data values in the process instance, since there may have been external systems updated that now need to be rolled back to an earlier state, or non-transactional activities such as sending an email. Compensation flows are used when you can’t use automatic rollback because an activity executed successfully, but can also be called by exception handlers. Visually, these appear very similar on the process model to exception handlers, in that they can be attached at the activity, group or process level. Since groups can be nested in a process model, the compensation flow for a group will invoke the compensation flows for any groups nested within it as it rolls back the entire flow of the group.

They finished up with a short Citigroup case study on how they handle trade exceptions in their back-office processes. Although most financial trades are handled straight through with no manual intervention, they handle 2000 trade exceptions each day. From designing a number of similar transactional BPM implementations, I know that there’s huge financial risk if you don’t handle your exceptions in a timely manner: market fluctuations that occur after the trade is accepted and priced but before it’s completed are purely the risk of the financial institution, not the customer, so it’s key to get the exceptions resolved as quickly as possible. Citigroup has implemented this as process-level exception handlers that log the exceptions and pass them on for manual review. In most cases, the exception handling process is just a matter of some manual data repair and the trade is resubmitted to the automated process, although some trades are cancelled from within the exception handler.

I started my day in a session about what’s coming up in future versions of ALBPM; unfortunately, most of the information hasn’t been publicly released, so you’ll have to wait to read about it at a later date. BEA will be holding BPM steering group meetings in July, so if you’re an ALBPM customer and want to get involved in defining future versions of the product, this is your chance. I’d love to sit in on these, although I can’t imagine the size of the NDA that I’d have to sign first.

I’m back listening to Mariano Benitez with an architectural overview of ALBPM for administrators and operators; I think that he’s nervous now based on his reaction to my post yesterday. 🙂

We’re going to cover the components of the ALBPM solutions, the enterprise infrastructure, and the deployment alternatives. I’ll leave out a lot of the technical details since it would only be of interest if you were actually digging into ALBPM at this level in order to plan a deployment, in which case you’re probably in the room with me right now.

In short, an ALBPM project can be made up of many processes, where a process consists of the process flow, points of integration and the presentation layer. Also associated with a process are the roles used by the process (which map to enterprise security), service endpoints, and deployment methods. ALBPM allows you to define the organization as it pertains to the business processes: participants and groups, organizational units, and roles.

Taking a look at the ALBPM enterprise infrastructure, it’s (not surprising) three layers:

• There’s a number of data sources, including a directory repository (which maintains configuration information about the deployment as well as organizational models used both in process definition and for authentication), the engine back-end database for all information on work in progress, historical instance data archived from the work-in-progress database, a real-time BAM data store with one day’s worth of data aggregated for dashboard views, and an OLAP data store for more complete historical analytics. Either Oracle, MS SQL Server or DB2 can be used for the data sources, and although multiple execution engines don’t require a separate database for each, they do require at least a separate schema. For performance, however, I would assume that you’d tend to split any production engine data stores and the analytics data stores onto separate database servers for performance reasons.
• In the middle layer is the main process execution engine — the heart of the system — plus a few other services such as data warehousing to load the analytics data stores. There are a number of basic services provided by the engine that are used to execute running process instances; no big surprises here on an architectural level if you’ve seen the process engine of other BPM vendors, although every engine has its specific advantages.
• Layered above the engine are various web applications, such as the main Workspace UI application that can be used both for processing and monitoring work. Listening to highly-technical engineers talk about user functions is always pretty funny: Benitez refers to the action of processing work as “invoking instance operations”, so you can be sure that he’s not going to be writing any user documentation. To be fair, I used to talk like that when I wrote code, too.

We unfortunately had to rush through the deployment scenarios, but saw that it’s possible to deploy ALBPM either as a standalone BPM box or in a J2EE container (simple or clustered).