Integrating BPM and Enterprise Architecture

Michael zur Muehlen presented this morning on integrating BPM and enterprise architecture, based on work that he’s done with the US Department of Defense. Although they use the DoDAF architecture framework in particular, the concepts are applicable to other similar EA frameworks. Like the Zachman framework, DoDAF prescribes the perspectives that are required, but doesn’t specify the artifacts (models) required for each of those perspectives; this is particularly problematic in DoD EA initiatives where there are likely to be many contractors and subcontractors involved, all of whom may use different model types to represent the same EA perspective.

He talked briefly about what makes a good model: the information must be correct, relevant (and complete) and economical (with respect to level of detail), as well as clear, comparable (linked to reality) and systematic. From there, he moved on to their selection of BPMN as the dominant standard for process modeling, since it has better event handling than UML activity diagrams, better organizational modeling than IDEF0, and better cross-organizational modeling than simple flowcharts. However, many tools support only a subset of BPMN – particularly those intended for process execution rather than just process modeling – and some tools have non-standard enhancements to BPMN that inhibit interoperability. Another issue is that the BPMN specification is enormous, with over 100 elements, with some different constructs that mean the same thing, such as explicit versus implicit gateways.

They set out to design primitives for the use of BPMN: where they “outlawed” the use of certain symbols such as complex gateways, and developed best practices for BPMN usage. They also mapped the frequency of BPMN symbol usage from internal DoD models, those that Michael sees in his practice as a professor of BPM at Stevens Institute of Technology, as well as samples found on the web, and came up with a distribution of the BPMN elements by frequency of usage. This research led to the creation of the subsets that are now part of the BPMN standard, as well as usage guidelines for BPMN in terms of both primitives and patterns.

In addition to the BPMN subsets (e.g., the most commonly implemented Descriptive subclass), they developed naming conventions to use within models, driven by the vocabulary related to their domain content. This idea of separating the control of model structure from the vocabulary makes sense: the first is more targeted at an implementer, while the second is targeted at a domain/business expert; this in turn led to vocabulary-driven development, where the relationship between capabilities, activities, resources and performers (CARP analysis) is established as a starting point for the labels used in process models, data models (or ontologies/taxonomies), security models and more as the enterprise architecture artifacts are built out.

Having defined how to draw the right models and how to select the right words to put in the models, they looked at different levels of models to be used for different purposes: models focused on milestones, handoffs, decisions and procedures. These are not just more detailed versions of the same, but rather different views on the process. The milestones view is a high-level view of the major process phases; handoffs looks at transitions between lanes with all activities with a lane rolled up to single activity, primarily showing the happy path; decisions look at major decision points and exception/escalation paths; and procedures showing a full requirements-level view of the process, i.e., the greatest level of detail that a business analyst is likely to create before involving technical resources to add things such as service calls.

To finish up, he tied this back to the six measures of model quality and how this approach based on primitives conforms to these measures. They’ve achieved a number of benefits, including minimizing modeling errors, ensuring that models are clear and consistent, and ensuring that the models can be converted to an executable form. I’m seeing an increased interest with my clients and in the marketplace on how BPM and EA can work together, so this was a great example of how one large organization manages to do it.

Michael posted earlier this year on the DoDAF subset of BPMN (in response to a review that I wrote of a BPMN update presentation by Robert Shapiro). If we go back a couple of years before that, there was quite a dust-up in the BPMN community when Michael first published the usage distribution statistics – definitely worth following the links to see where all this came from.

Learning to Love BPMN 2.0

The last presentation of the IRM BPM London conference before the final panel, and Chris Bradley and Tim Franklin of IPL are presenting on BPMN 2.0. Bradley started with a brief history of BPMN from its 1.0 release in 2004 by BPMI to the present day 2.0 release, now under OMG. It was interesting to see their list of what BPMN does not do: state transitions, functional decomposition, organizational hierarchies and data modelling, which explains why some BPMS products are starting to build those functions into their integrated development environment to be defined along with the process models. [Note that although I normally use US spelling due to the geographic location of most of my audience, I’m using “modelling” here after Bradley point out that the US spelling, “modeling” should rhyme with “yodeling” 🙂 ]

Franklin took over to get into the details of the notation, particularly the differences between the 1.x and 2.0 versions and the new elements and diagram types in 2.0. I’m not going to review all of that; there’s a lot of existing material both on this blog and in other locations, including a webinar that Robert Shapiro gave earlier this year on BPMN 2.0.

Bradley took the stage again to discuss all the other things that have to happen after you get started on BPMN 2.0, particularly modelling data and aligning that with the process models, whether that’s being done in an integrated tool or two different modelling tools. I agree with him that it’s critical for process, data and organizational modelling efforts to be linked, although I think that’s more likely to have happen via MDM rather than by having a single integrated modelling tool.

His summary said it all: BPMN is simple (if you can read a flowchart, you can understand BPMN); BPMN is robust (can be used for both high-level design/architecture and detailed process model execution/implementation); and most importantly, BPMN and process models are only part of the big picture, and need to be linked to other information assets such as data and organizational models.

You may not have come out of this session actually loving BPMN 2.0, but at least you’ll respect it in the morning.

The Great BPMN Debate of 2010

I go off on vacation for a week, and a firestorm erupts around BPMN usage by business people. It’s taken me the weekend to comb through all the posts and comments; there’s a lot of reading here, and I recommend checking out the discussions in the comments on each of these posts as well as the posts themselves.

Where it all started

BPMN for Business Professionals: Burn Baby Burn: Jim Sinur blogs about how BPMN is too hard for business people, touching off a storm of comments on this post, and several posts from others on the same subject.

The responses

Process for the Enterprise » Blog Archive » Apparently BPMN is Too Hard: Scott Francis responds to Jim Sinur’s post, saying that if you’re already flowcharting processes, then BPMN (at least the basic set) really isn’t much different from that, and provides the benefit of standardization. He doesn’t want to let the business off the hook of learning a new skill when it’s really not that hard, and provides some benefit to the business (not just to IT).

Dave Thinking Aloud: BPMN only part of the solution: Dave French agrees with Scott’s post and offers a great quote: “It would be really scary if those responsible for the operation of multimillion dollar enterprises can’t take on the meaning of a set of symbols that can be put on a small wallchart”. My thoughts are pretty much aligned with Scott and Dave’s.

BPMN 2.0: no longer for Business Professionals | On Collaborative Planning: Keith Swenson points out that most of the enhancements to the new version of BPMN are for developers, not business people, turning it into a graphical programming language for processes. He questions whether vendors will move from BPMN 1.2 to 2.0 if their focus is on modeling rather than execution. I think that Keith is throwing the baby out with the bathwater here: although a lot of new constructs have been added that make BPMN more useful for developers, that doesn’t make the basic subset inappropriate for business people who are already mapping their business processes with flowcharts. In the absence of any specific direction, I most often see business people (and business analysts) use flowcharts to represent their business processes; introducing them to the simplest BPMN subset will at least put some standardization around those flowcharts so that there’s no confusion over the shapes used on the diagram, and to reduce some of the spaghetti around flowcharting events.

On IT-business alignment and related things » BPMN “not for the business”? Let’s lose the hype: Neil Ward-Dutton takes a similar middle ground to mine: “there’s significant evidence to suggest that a core subset of BPMN symbols are absolutely usable by business analysts with experience in high-level analysis and design and provide great results in terms of delivering a common language across multi-disciplinary teams”. BPMN is not useful to everyone. And business people are never going to learn more than a subset of BPMN. That’s not justification for stating that BPMN has no value for business.

Chris Adams Responds to: “BPMN for Business Professionals: Burn Baby Burn” by Jim Sinur (Gartner): Chris Adams relates a conversation with a CIO who had BPMN representations created for business processes that were too complex for the business to understand. Sounds like these weren’t modeled by the business, or at the right level.

The wrap-ups

Process for the Enterprise » Blog Archive » I See Business Professionals… Using BPMN: Scott Francis wraps up a lot of the discussions, finishing with the statement “Regardless of what the theory says, the practical reality is our customers’ businesses are using this stuff”. I find that this is true for me as well: my customers are using flowcharting extensively for modeling business processes, and with a small amount of guidance, they are using the simplest subset of BPMN. I know that’s right up there with the theory that it’s aerodynamically impossible for bumblebees to fly, and yet they persist in doing so.

BPMN Quotes of the week « Adam Deane: Adam Deane wraps up the week of BPMN blog posts with the best quotes drawn from several of them.

My take

BPMN | How to explain BPMN to Business Users | VOSibilities: The replay of a webinar that I did in June on BPMN and business users. I make the points that the subset is all that’s needed for business, and that there’s a difference between knowing enough BPMN to create models and knowing enough BPMN to understand models. Here’s just the slides:

In short: BPMN isn’t for all uses, but if your business people are already flowcharting their business processes, then teaching them a few BPMN symbols in order to standardize those flowcharts has benefits.

BPM Summer Camp: Business Users and BPMN

I presented today on the second part of Active Endpoints’ BPM Summer Camp, discussing just how much BPMN your business users and analysts need to know. Michael Rowley, CTO of Active Endpoints, gave a demo of BPMN using their system, including illustrating a number of the concepts that I introduced in my presentation.

You can view and download my slides here:

A few other references based on the questions at the end of the presentation:

We have the third and last part of BPM Summer Camp, “Five Things You Should Never, Ever Try in Process Development”, on July 22nd; head over to the landing page to sign up for that, as well as see a replay of the first two parts.

BPMN In The Real World Slideshow

Webinar done, we’re just on the final Q&A; I saw about 170 people logged in at one point, a pretty good turnout. The replay will be available on the VOSibilities blog or on iTunes.

Here’s my slides, with the “Process Model Hall Of Shame” removed:

As I mentioned in the presentation, there are a lot of great resources on the BPMN standard; my presentation was about how people are actually using it rather than the standard itself.

BPMN 2.0 Industry Update

It’s webinar day here at Column 2: this is my third in a row, this one an update on the BPMN 2.0 standard by Robert Shapiro, who participates in both the OMG BPMN 2.0 and WfMC XPDL 2.2 standards efforts. We’re already starting to see vendor support for BPMN 2.0, even though it’s not yet fully released, as well as books and training materials.

The concept of subclasses in process modeling has been included in this version, where there is a simple subset of eight elements used for process capture by non-technical process analysts/owners (start, end, sequence flow, task, subprocess, expanded subprocess, exclusive gateway, parallel gateway), then a larger subset for a “descriptive” persona, a larger-still subset for a “DODAF” persona, then the entire set of more than 100 elements:

BPMN 2.0 element subclasses

You can download the accompanying PowerPoint deck for a more complete view of subclasses and their corresponding personas. I can certainly understand why many of the event variations were pushed out of the simple subclass, but I’m not sure that I agree with excluding pools and lanes, since these are pretty commonly used constructs. Also not sure why the US DoD’s enterprise architecture standard is impacting what is supposed to be an international standard.

These subclasses are important for vendors of modeling tools, but also for those looking to use BPMN as a standard for representing processes: this gives a good idea of how to split up the standard by the type of reader (persona) so that you don’t overwhelm the less technical audiences with too much detail, but also provide the greater levels of details for complete process specification.

Shapiro went on to discuss what most consider to be the most important (and likely the most controversial) part of BPMN 2.0: diagram interchange; BPMN 2.0 does not include an XSD schema, and there is ongoing work to create an XSD that is aligned with the metamodel. For those of you who follow BPM standards, you’ll know that XPDL is currently the de facto standard for process model interchange, supported by many vendors; these efforts are continuing in a separate organization (BPMN is managed by OMG, XPDL by WfMC) so it’s good that Shapiro and others are there to bridge the efforts across the two standards. We’re now seeing the emergence of XPDL 2.2, which will support the interchange of BPMN 2.0 process models. XPDL may eventually disappear in the face of a comprehensive BPMN 2.0 diagram interchange standard, but that will take years to happen, and a lot can happen in that time. In the meantime, XPDL will likely be used as an alternative diagram interchange format for BPMN 2.0 diagrams, with vendor support required for both standards.

The Business Process Incubator site has been created by several of the companies participating in both BPMN and XPDL standards efforts as a source for information as well as a variety of standard-related tools such as Visio templates. Shapiro also predicts that many tool vendors will release web-based BPMN 2.0 modelers, as well as BPMN and XPDL converters.

If you’re interested in where BPM standards are headed, it’s worth listening to the entire webinar, especially the Q&A at the end; I imagine that it will be available at the registration link on the WfMC site that I posted in the first paragraph.

Robert Shapiro on BPMN 2.0

Robert Shapiro spoke on a webinar today about BPMN 2.0, including some of the history of how BPMN got to this point, changes and new features from the previous version and the challenges that those may create, the need for portability and conformance, and an update on XPDL 2.2. The webinar was hosted by the Workflow Management Coalition, where Shapiro chairs the conformance working group.

He started out with how WPDL started as an interchange format in the mid-90’s, then became XPDL 1.0 around 2001, around the time that the BPMN 1.0 standard was being kicked off. For those of you not up on your standards, XPDL is an interchange format (i.e., the file format) and BPMN prior to version 2.0 is a notation format (i.e., the visual representation); since BPMN didn’t include an interchange format, XPDL was updated to provide serialization of all BPMN elements.

BPM standards timeline

With BPMN 2.0, serialization is being added to the BPMN standard, as well as many other new components including formalization of execution semantics and the definition of choreography model. In particular, there are significant changes to conformance, swimlanes and pools, data objects, subprocesses, and events; Shapiro walked through each of these in detail. I like some of the changes to events, such as the distinction between boundary and regular intermediate events, as well as the concept of interrupting and non-interrupting events. This makes for a more complex set of events, but much more representative.

BPMN event types

Bruce Silver, who has been involved in the development of BPMN 2.0, wrote recently on what he thinks is missing from BPMN 2.0; definitely worth a read for some of what might be coming up in future versions (if Bruce has his way).

One key thing that is emerging, both as part of the standard and in practice, is portability conformance: one of the main reasons for these standards is to be able to move process models from one modeling tool to another without loss of information. This led to a discussion about BPEL, and how BPMN is not just for BPEL, or even just for executable processes. BPEL doesn’t fully support BPMN: there are things that you can model in BPMN that will be lost if you serialize to BPEL, since BPEL is intended as a web service orchestration language. For business analysts modeling processes – especially non-executable processes – a more complete serialization is critical.

In case you’re wondering about BPDM, which was originally intended to be the serialization format for BPMN, it appears to have become too much of an academic exercise and not enough about solving the practical serialization problem at hand. Even as serialization is built into BPMN 2.0 and beyond, XPDL will likely remain a key interchange format because of the existing base of XPDL support by a number of BPM and BPA vendors. Nonetheless, XPDL will need to work at remaining relevant to the BPM market in the world of BPEL and BPMN, although it is likely to remain as a supported standard for years to come even if the BPMN 2.0 serialization standard is picked up by a majority of the vendors.

The webinar has about 60 attendees on it, including the imaginatively named “asdf” (check the left side of your keyboard) and several acquaintances from the BPM standards and vendor communities. The registration page for the webinar is here, and I imagine that that will eventually link to the replay of the webinar. The slides will also be available on the WfMC site.

If you want to read more about BPMN 2.0, don’t go searching on the OMG site: for some reason, they don’t want to share draft versions of the specification except to paid OMG members. Here’s a direct link to the 0.9 draft version from November 2008, but I also recommend tracking Bruce Silver’s blog for insightful commentary on BPMN.

BPM Milan: Paul Harmon keynote

After a few brief introductions from the various conference organizers (in which we learned that next year’s conference is in Ulm, Germany), we had a keynote from Paul Harmon on the current state and future of BPM. It covered a lot of the past, too: from the origins of quality management and process improvement through every technique used in the past 100 years to the current methods and best practices. A reasonable summary of how we got to where we are.

His “future promise”, however, isn’t all that future: he talks about orchestrating ERP processes with a BPMS, something that’s already a well-understood functionality, if not widely implemented. He points out (and I agree) that many uses of BPMS today are not that innovative: they’re being used the same way as the workflow and EAI systems of 5 years ago, namely, as better programming tools to automate a process. He sees the value of today’s BPMS as helping managers to manage processes, both in terms of visibility and agility; of course, it’s hard to do that unless you have the first part in place, it’s just that a lot of companies spend too much effort on the first level of just automating the processes, and never get to the management part of BPM.

He discussed the importance of BPMN in moving BPMS into the hands of managers and business analysts, in that a basic — but still standards-compliant — BPMN diagram can be created without adornment by someone on the business side without having to consider many of the exception flows or technical implementation details: this “happy path” process will execute as it is, but won’t handle all situations. The exceptions and technical details can be added at a second modeling/design phase while still maintaining the core process as originally designed by the business person.

He also showed a different view of a business process: instead of modeling the internal processes, model the customer processes — what the customer goes through in order to achieve their goals — and align that with what goes on internally and what could be done to improve the customer experience. Since the focus is on the customer process and not the internal process, the need for change to internal process can become more evident: a variation on walking a mile in their shoes.

His definition of BPM is very broad, encompassing not just the core processes, but performance management, people, technology, facilities, management and suppliers/partners: an integration of quality, management and IT. Because of the broad involvement of people across an organization, it’s key to find a common language about process that spans IT and business management.

Although they’re not there yet, you can find a copy of his slides later this week by searching for BPM2008HarmonKeynote at BPtrends.com.

Another new BPMN book

Another new BPMN book, this one by Stephen White (arguably the inventor of BPMN) and Derek Miers: BPMN Modeling and Reference Guide. It won’t be released until September, with a public launch at the Gartner BPM summit in DC. From the product description:

This book is for both business users and process modeling practitioners alike. Part I provides an easily understood introduction to the key components of BPMN (put forward in a user-friendly fashion). Starting off with simple models, it progresses into more sophisticated patterns. Exercises help cement comprehension and understanding (with answers available online). Part II provides a detailed and authoritative reference on the precise semantics and capabilities of the standard.

I wrote earlier this week about the just-released BPMN book by Tom Debevoise and Rick Geneva; this is obviously the year that BPMN goes mainstream, or at least makes the attempt. White and Miers’ book, although a bit longer than Debevoise and Geneva’s, is also more than twice the price, and also doesn’t seem to offer an e-book option: hard to become a staple of every process-oriented person in an organization at a $40 price point.

I’ll be very interested to read Bruce Silver‘s review of these books. Unless, of course, he’s writing his own. 🙂

Microguide to BPMN

I noticed in one of Tom Debevoise’s posts last week that he recently co-authored the book The Microguide to Process Modeling in BPMN, and on closer examination, I see that his co-author is Rick Geneva of Intalio, with Ismael Ghalimi writing the foreword.

From the product description on Amazon:

With over fifty implementations listed, Business Process Modeling Notation (BPMN) is an increasingly successful Object Management Group (OMG) standard. Whether you are in government, manufacturing or retailing you can accurately depict your processes in BPMN! Yet, OMG BPMN specification 1.1 is abstract, lengthy and complicated. So, learning to use BPMN can be daunting. So you will need the strait forward [sic] information in this book. This guide gathers all the ideas, design, and problem solving of BPMN into one simple, focused book, and offers concrete true-life examples that explain BPMN’s approach to process modeling.

I haven’t had a chance to read it yet so can’t compare it to the many other sources of BPMN instruction out there, such as the recently-released BPMN, the Business Process Modeling Notation Pocket Handbook. Unfortunately, Debevoise and Geneva’s book doesn’t appear to be available as an e-book.