Column 2

I recently did a series of three webinars for TIBCO: process discovery, process modeling and process design.

If you just want to browse through the slides again, I’ve embedded them below.

Next Wednesday, I’m presenting a webinar on Process Optimization for the Real World, along with Rob Risany of Savvion (who are sponsoring the webinar). This is the same format as when Rob and I last did a webinar together: we both give a short presentation, then have a “fireside chat” discussion where we have a mostly unstructured conversation, which is a lot of fun for us and was pretty popular with the audience last time.

My part of the presentation is on how process optimization works — or doesn’t work — in reality, and some of the factors that impact it. In my experience, I find that most of the reasons that it doesn’t work are cultural, not technological, and can lie with both business and IT. Tune in next week to hear the whole thing.

We didn’t have time for live Q&A during the TIBCO-sponsored webinar that I gave on business process design last week, but here’s the questions that we received and my responses. There were some questions specific to TIBCO products that I’ve passed along to Emily Burns to answer; if you asked one of those, you should see a response by email from her shortly.

Q: How mature is BPDM and is there any documentation available from OMG?

A: BPDM (Business Process Definition Metamodel) has been under development since 2003 and the specification was approved earlier this year by OMG. According to their catalog page, the 1.0 version finalization is underway, although there’s not a link to the specification available. There is a preview version available on ModelDriven.org that includes the March 2007 “final submission” of the specification in PDF (click on the folder called “Specification” in the left sidebar), which is the one that was approved, and an HTML version if you want to browse around in it online.

Another measure of maturity is the market adoption of the standard, and that’s just starting to happen. The only BPM vendor of which I’m aware that has BPDM in their released product today is Lombardi; if there’s others of you out there, please add a comment to this post.

Q: Could you please explain more on what is meant by choreography in BPM?

A: This question was also asked as “differentiate process orchestration and process choreography”, and although we did cover that in the one question that I answered at the end of the webinar, I’ll include it here since both orchestration and choreography refer to the modelling of business processes and there’s a lot of confusion in the use of these terms.

Orchestration focuses on the behavior of a single participant, and is a hub-and-spoke model with a controller at a single location in the distributed system that enforces the progress of the process. An orchestration language is for processes that are controlled by one organization (although they may execute across multiple organizations), so it?s used to model what goes on inside the controlling organization plus the external linkages to the other participants. Choreography, on the other hand, is a peer-to-peer collaboration that doesn?t require a central controller. A choreography language is for collaboration between peer organizations, and is used to model the exchange of messages between the organizations but not their internal processes.

OMG refers to orchestration as “the modeling of directed, internal business processes” and choreography as “the specification of interactions between autonomous processes”.

I like Stefan Tilkov’s definition from a couple of years ago:

In orchestration, there?s someone ? the conductor ? who tells everybody in the orchestra what to do and makes sure they all play in sync. In choreography, every dancer follows a pre-defined plan ? everyone independently of the others.

Standards have tended to focus on either orchestration or choreography, not both; as more standards emerge that support both, the distinction between them will likely start to fade away.

Q: I’ve seen an Intalio product that has implemented BPEL4People in their product.

A: Not really a question, but yes, Intalio implemented an earlier version of BPEL4People (Ismael, jump in here if I have the details incorrect) based on the original 2005 white paper, which isn’t quite the same as the currently proposed specification. There’s more details on BPEL4People available from IBM and BEA, including the specification document from June 2007. According to the IBM reference, “the authors plan to submit the specifications to a recognized standards development organization in the near future”, by which they mean OASIS.

Q: Please explain the relationship between BPMN and UML.

A: UML is a modelling language originally used for modelling software as part of software requirements and design (check out the Wikipedia entry for more detail), adopted and standardized by OMG. One of UML’s 13 types of diagrams is the activity diagram, originally a variation of a state diagram, that is used for modelling logic and control flow within software (example here). Because it models flow, its usage was extended to model business process flows as well, and some BPM tools used it as the graphical notation for modeling business processes.

BPMN was developed purely as a business process modelling notation by BPMI (Business Process Management Initiative), and intended for use by business analysts. The first draft was released in 2002, and the 1.0 release was in 2004. The following year, in part due to the BPM Think Tank meetings, BPMI was absorbed into OMG and BPMN was designated as the standard to use for modelling business processes, effectively relegating UML back to software design and other more technical usages.

There is a 2004 white paper written by Stephen White of IBM, who has been a major contributor to the BPMN standard, comparing UML and BPMN; you can find it here (direct PDF link) or linked from the BPMI information page.

Q: Are there any standard design patterns (names) available/identified for BPM ?

A: There are 21 identified workflow patterns that White refers to in his white paper (see previous answer), with an expanded version of 43 patterns at the original source here. If you download the 2006 paper “Workflow Control-Flow Patterns: A Revised View” from the Documentation page on that site, you’ll find an assessment of various languages, standards and BPM products relative to the 43 patterns, including XPDL, BPMN, BPEL and UML. The product evaluations are a bit out of date since these things change quickly, but worth a glance.

These patterns range from simple (e.g., sequence) to fairly complex (e.g., interleaved routing), but all of them represent a small number of steps within a process map, not a complete design pattern for a process.

Q: Are there samples of the processes?

A: I assume that this was related to the section on design patterns. I have a lot of examples from my own experience, and have listed some for the first design pattern on initiating processes here:

• Human intervention: In a process for expense approval, the user requesting the approval initiates the process explicitly and fills in the information regarding the expense. Note that explicit process initiation may happen through the BPMS’ standard interface (most of which provide some sort of way to locate process templates and create an instance based on the template) or may be through a customized interface that masks the BPMS from the user.
• External event: A paper document containing instructions for one or more mutual fund transactions is scanned into a content repository, and the content repository triggers an event that initiates a process with that document attached.
• Invoked as a web service: A mainframe process that automatically adjudicates health insurance claims initiates a manual review process in BPM by calling a web service when the claim can’t be automatically adjudicated. Most BPMS’ allow a process to be exposed as a web service for this type of usage; you can consider this to be just a special case of external event triggering.

Since I don’t have any sort of synchronization between when the questions were asked and what slide was up at the time, let me know if this is addressing your particular question by adding a comment to this post, and I can expand on other areas as required.

Q: How do you suggest building resilience (error detection, error recovery, process restarts)?

A: Resilience at a low level has to be built into the BPMS, e.g., if an error occurs when executing a step or invoking a web service, the system needs to have some way to either alert the executing process or recover/retry the action. Assuming that the executing process is notified of a low-level error, you can then design a recovery path. If the error is not something recognized by the BPMS itself (e.g., a web service call executes properly but returns an error code), then your design will need to check for that error just the same as if you were writing code to do so: think of BPMN in this context as a visual coding language of sorts. Regardless of how the error is detected, the best way to handle the errors (if the BPMS that you’re using fully supports BPMN) is through the use of the error, cancel and compensation events, which include recovery and rollback functionality.

Q: Do users find it cumbersome to get work items from one system (BPM) and perform them in another (e.g. legacy system)? Do you have any recommendations for addressing this?

A: In most cases, users are working with at least two different “input methods” to drive their workflow: paper documents, email or spreadsheets may all be providing the necessary input to key information into a legacy transaction processing system. Moving their worklist from a manual method to a BPMS on their screen is usually not a big transition, in my experience. When I first started implementing human-facing workflow/BPM systems in the early 90’s, there were a lot of problems in adapting because the users were moving from dumb terminals to Windows workstations as part of the implementation, and were not adept at switching between windows in order to maximize their screen usage. Nowadays, I see users flipping between multiple windows on their screen, or even making use of dual screens that allows them to rearrange their most commonly-used windows in key locations, then just pop up others as they need them.

Even though this sort of side-by-side windowed environment is not a challenge for today’s users, I would still recommend some integration between the systems. Say, for example, that the work item in BPM already has a customer number captured as a data field; when that work item is opened by the user, you’d really like to look up the associated customer record on the legacy system instead of them having to re-enter the number on the legacy screens and search for it. The methods of doing this depend on the nature of the legacy system and its user interface: if it’s still a green screen (and I see lots of those around), then you’ll need to pass commands via the terminal emulator software to drive the screens as if a user were typing the information. This is cumbersome, and often doesn’t work all that well when an error occurs on the legacy system. If your legacy system has a nicer interface that supports some sort of messaging protocol or has an API, then you can do the search that way.

The other side of that is to capture information from the legacy system back to the work item in the BPMS, either for passing on to another system involved in an orchestration process (including auto-indexing attached documents from a content management system) or to make routing decisions within the process flow. For green screen systems, you’ll do this with “screen scraping” via the terminal emulator software, which extracts data from the screen at a specific pre-determined location and passes it back. With a smarter system interface, you’ll make an API call or pass a message.

Q: How can a company implement a simple BPM implementation when at a minimum most companies want to automate processes that access a database which requires some kind of integration?

A: Many BPM systems support database access pretty much out of the box, in the same sort of way as they support web services, so this may require only some configuration of the database connection parameters at a task and no real coding. For example, in TIBCO you add a service task to your process map, configure it with the server and database name, enter the name of the stored procedure that you want to call in that database, then specify the mapping between the stored procedure parameters and the work item data fields. No code, although you may need to create stored procedures in your database.

Q: Can you describe the appropriate level for modeling a process? What do you consider an atomic piece of work?

A: The appropriate level really depends on the purpose of the model. If you’re doing high-level strategy or business planning, then likely you’ll only model the high-level activities and won’t model data fields, roles, web service calls and other details. If you’re creating a model that will be used to execute the process, then you need to model each step in the process, whether automated or human-facing, and specify all the parameters required by the step.

The definition of an atomic piece of work is dependent on your application, but you’ll likely determine it by going back to whatever it is that initiates a process instance and thinking through how that is going to be processes through its life cycle. Using the examples that I discussed for process initiation earlier in this post, the atomic piece of work in the first case is a single expense approval request; in the second case, it’s all of the transactions documented on the attached document (although that might be broken down into individual transactions that each spawn their own subprocess if that’s how the work should be executed); in the third case, it’s the adjudication case rejected from the legacy process. This is a pretty critical piece of the design in your BPM implementation, since it impacts complexity of design, the ability to process in parallel, and even the statistics that can be gathered about a process. Specifically, in the second case, you’ll need to decide whether to split a single initiated process into multiple pieces of work; I’ve done it both ways depending on a number of other factors in the overall design.

Q: What difference there is between “your” BI description and BAM?

A: In my opinion, the larger field of business intelligence (BI) includes near-real-time dashboards such as those used for business activity monitoring (BAM). There are various opinions on this: some people consider BI to be only historical reporting; others (including analyst groups such as Forrester) consider BAM to be a part of BI. Other terms like CPM (corporate performance management), CEP (complex event processing) and EDM (enterprise decision management) overlap the BI space significantly, which I think is what has led to the recent relabelling of most technologies related to data collection, aggregation and analysis as business intelligence.

Q: Difference between Business Process Workflow and Business Process Management.

A: I don’t make a real distinction between workflow and BPM from an implementation standpoint, although the term workflow has typically been used in the past to denote human-facing processes. Microsoft, for one, now uses the term workflow to mean any sort of process flow: their site refers to “system workflow and human workflow”.

Many of the analysts define BPM as not just the tools used to automate processes, but also the management practice around business processes: Gartner defined BPM in 2005 as “a management practice that provides for governance of a business’s process environment toward the goal of improving agility and operational performance”.

 

Thanks to all of you who made the time to attend.