Category: process

business process management

Convergence of BPM and BI

If you’re interested in more about the Forrester report about BPM and BI that I mentioned a couple of weeks ago, but still don’t want to shell out the cash for it, you can find a more complete summary on here on BPM.com (registration required), written by a Forrester VP who was one of the original report authors.

The more that I look at compliance, which I’ve been doing a lot of lately, the more that I understand that BPM needs to feed its performance data into a larger BI infrastructure. And I can buy into what the Forrester report refers to as Process to Data, or P2D (as if we needed another TLA), which is the two-way synergy between BPM and BI: BPM feeds process performance data to BI, and BI invokes processes in BPM in order to gather information. However, I have to draw the line at their statement:

BI and BPM can no longer live without each other, and the time is right for these technologies to merge.

I don’t think so, any more than BPM will merge with any number of other technologies, although obviously there will be closer and closer integration ties in order to make all of this work smoothly. BPM and BI are strong, distinct markets served by a variety of strong vendors, and customers develop quite a bit of loyalty to their BPM and BI vendors because both of these technologies are pervasive in an organization’s IT infrastructure. If BPM vendor A decides to merge with BI vendor B and suggests to their customer that they should change all of their existing BI from vendor C to vendor B, there would be a great deal of rolling around on the floor and laughing. There’s a much stronger argument to be had for merging BPM and BR (business rules), but I don’t think that’s going to happen either, for much the same reasons: both technologies have strong, independent markets (that is, the technologies can exist successfully in organizations without each other) and there is little reason for a customer to want to buy their corporate-wide BR from their BPM vendor, or vice versa.

If the merging that Forrester suggets actually occurs, we’ll end up with monolithic Swiss-Army-knife-like vendor offerings from a few large players — how retro! — and a lot of unhappy customers. What most customers want is the best of both worlds: the best-of-breed technology, and the minimum amount of integration; hence the huge popularity of SOA. In the past, you couldn’t get both: you could buy best-of-breed from different vendors and take a lot of effort to integrate it, or you could buy a fully-integrated suite from a single vendor that included at least one sub-optimal component. Today, however, with the advent of integration standards and SOA, the integration effort for properly-constructed products from different vendors should be no more than that required for products from the same vendor.

My conclusion: as long as the vendors do what they’re supposed to in order to enable easy integration, there’s very few customer drivers for merging BI and BPM.

Agility and BPMS Architecture

Bruce Silver writes about the distinction between workflow architecture and service orchestration architecture, pointing out why it’s important to distinguish between these two process architectures and see how they fit together. If you’re having trouble figuring out the difference between what the former workflow and former EAI vendors are saying, this might help.

He also discusses what’s changed in BPMS to actually make it as agile as the vendors would have you believe that it’s been all along: easier application integration. As he puts in, in the bad old days, when the EAI part of workflow systems didn’t work very well:

If you went to a workflow conference or trade show, any vendor could show you how to build a workflow in 30 minutes using no programming, just graphical drag-and-drop design. So once you bought the technology, why did it take six to twelve months to design and deploy real-world workflows?

There was a huge market for systems integration companies to do all that tricky application integration stuff — I know, because I ran a 40-person version of that model up until 2000, with a crack team that did design, development, testing, documentation, training and installation, all targeted at trying to turn the dream that the workflow vendors sold into a reality. If I hadn’t already got out of that business, I’d certainly be looking to do so now, as the EAI part of BPM gets easier and easier, leaving much less of the complex systems integration that used to drive a huge part of the market.

What I find amazing is that some of the large systems integrators continue to stick their heads in the sand and refuse to buy into the new way of doing things. Even though capabilities exist for much simpler (and therefore faster and less expensive) integration of BPMS with other technologies, the SIs have too much of a vested interest in selling huge pieces of custom code that they’ve written to augment previous versions of various vendor products, and the associated 1000’s of hours of customization that they can sell to the unsuspecting customer. They actually win work based on the premise that if it takes that long and costs that much, it must be good.

Back to the Silver article that started this thought, I also find it amazing when vendors (and therefore, by extension, their clients) don’t make the distinction between the two types of process architectures, and try to sell their legacy technology (usually either workflow or EAI) as if it spanned the entire space. Since most former workflow vendors have hopped on the EAI bandwagon by buying in that capability early, I see the problem more often with the former EAI vendors trying to sell service orchestration as if it were workflow, with only the barest of long-running process and human-facing capabilities. I remember leading a discussion on exactly this topic with a client’s IT group who had selected an EAI product as their BPM corporate standard, and never being able to get through to them that there are really two major classes of process functionality and architectures, only one of which was addressed by the product that they selected. Silver’s article may have helped.

BPM course at FileNet user conference

If you want to spend two days learning how to “Make BPM Mean Business“, I’m leading a seminar as part of the pre-conference training at FileNet’s user conference in Las Vegas in November. I’ve tried to strip out all the technical stuff (which FileNet can teach you better than I can anyway), so this is really targeted at business managers, business analysts and process designers.

Sign up through FileNet if you want to do this in Vegas prior to the conference, but if you can’t make it to the conference, I own the course content and can do a private gig elsewhere, preferably somewhere warm and exotic as winter descends on Toronto. The first day (“BPM in Business” and “Modeling Your Processes”) is not specific to any vendor’s products and would be suitable even if you’re not a FileNet customer.

SOA and banking

I viewed a webinar today featuring a senior integration architect at Bank of America talking about how they use webMethods to facilitate their massive integration efforts, apparently with some degree of success. I liked the admission by both the customer and vendor that SOA is an architectural design approach of loosely-coupled components, not a particular technology or product, although products can obviously help you to build a service-oriented architecture faster, cheaper and in a more standardized manner.

This definitely put me in mind of a recent engagement where I spent a year as the acting application architect for one business silo within a financial institution, facing many of the same business and technology challenges as BofA. I also sat on the architectural review board, so had a view of the common challenges across the entire organization and the technology infrastructure being deployed to address them. Almost all large financial institutions got big through a combination of acquisition and growth, and both of those growth methods can lead to disparate systems in the back office — even within one silo — that don’t intercommunicate well (if at all), which in turn leads to a lot of internal inefficiencies and a less-than-optimal customer experience. Furthermore, most organizations are not set up to share across the business silos, so attempts to create common infrastructure such as those that support SOA can be tricky unless an architect is empowered to work across the silos.

As you move up the services stack, the focus changes from the base services and registry layers (where webMethods plays) to orchestration, where services are assembled into processes that relate to actual business processes (you knew that I’d get around to process eventually, right?): the task flow and BPM layers in the integration stack that I referenced previously, although the lower levels of that particular stack are MOM (“first generation SOA”) rather than XML-based web services (“second generation SOA”). As you bring these concepts together, and especially when you start looking at the technology stacks, it becomes obvious that you can’t have an effective BPM strategy without SOA. You can implement BPM without SOA, but it won’t have the ability to become the strategic orchestration layer that’s required to pull together disparate systems and provide new integration functionality in the future.

Processes “R” Us

I had several appointments and errands today, and I listened to podcasts as I walked around downtown Toronto. One of them was the Sound of Vision podcast from back in May wherein Ethan Johnson interviews me about BPM (starting at 21:00 in the ‘cast), and there’s one point where I get really passionate about the fact that everything is a process: my true evangelist colours shining through. I do have a very process-centric view of business, to the point where some work that I’ve been doing recently on compliance started out being about content and records management, and has shifted to have a very strong focus on process.

I also saw an article this afternoon by Terry Schurter of BPMG, and he states that BPM and a process-centric view are so popular because such a high percentage of BPMS implementations (compared to other enterprise software) deliver on their promise of ROI. His view is that taking a process-centric view — “the idea that businesses can be viewed as a series of processes, and that those processes can be identified and managed to improve quality, efficiency, and cost-effectiveness” — resonates with end-user organizations, vendors and analysts, and that BPM aligns with the natural business structure.

It seems that you can’t pick up a business or technology article these days without it containing some reference to process, which means that Terry and I are not alone in our views.

More open source BPM

An infrequently-updated but informative blog on open source BPM:

We share our discoveries in our search for open source workflow management tools. For tools that we find interesting, we download the code, try to execute the engine, or even get our reference process model working.

Also includes a list of open source BPM tools, including their evaluation notes where applicable.

Note that these are the only two sections of the site in English; for the rest, you’d better brush up on your Dutch (or use AltaVista’s Babel Fish translation utility, since Google’s translation doesn’t support Dutch).

BPM templates

I tuned in to a Global 360 webinar today for long enough to hear Nathanial Palmer from Delphi speak about process templates and their importance in BPM (you should be able to find a replay of the webinar here in a few days). He revealed some very telling numbers, soon to be officially released, from a recent survey of over 100 active BPM project participants:

  • 98% agreed that pre-defined templates accelerate BPM deployments. 73% answered definitely “yes”, while the other 25% said “maybe”, and only for simple or standardized processes. I’m curious to know what the 2% “no” contingent was thinking, since it’s hard to imagine anyone not seeming some potential value in a pre-defined solution template.
  • Although few people expect templates to be an application rather than a project jump-start, 70% expect them to be a fairly complete framework with screens, rules, integration adapters and the like. In other words, the respondants definitely expect the templates to be customizable, but they want to have a pretty high starting point.
  • 70% stated that they would be more likely to buy a software solution that had process templates specific to their industry, which seems obvious but is something that many vendors haven’t figured out yet.
  • 76% agreed that the templates should be documented in “business” language rather than being a tool for IT, and one of the key values stated for process templates was to align busines value with IT.

Not surprisingly, SOX compliance was at the top of the list of which processes should be templated, although the votes were pretty evenly spread over all of the business processes surveyed.

The value of vendor white papers

Every vendor writes white papers that try to appear vendor-neutral (in an effort to build their street cred), but which have subtle — or not-so-subtle — hidden agendas. That doesn’t mean that you shouldn’t read them, just don’t take them as the whole truth and nothing but the truth. The best approach is to read a variety of sources on a subject from both vendors’ white papers and “independent” analysts (who, of course, have their own agendas as well) and start looking at the common themes and messages across all of them. If a vendor doesn’t mention something that is considered important by several others, that could highlight a missing functionality or flaw in their system. However, when one vendor offers an opinion that is vastly different from all the others, then either they’re trying to justify a flaw, or they’ve really discovered a better way of doing things and are the first to flaunt it. Don’t discount the latter possibility out of hand, because a vendor is much more likely to be silent about their flaws rather than try to justify them, so if they’re talking about it in a white paper, then they may have something to say.

I read a lot of vendor white papers and analyst reports because it’s my job to stay on top of what’s happening in the BPM and related spaces, and it’s nice to run across one that I can recommend every once in a while. Last week, I saw this report from Metastorm published on TechRepublic (TechRepublic free registration required to view) about choosing a BPMS. It includes five very valid steps along the selection path:

  1. Consider the vendor’s core competence.
  2. Consider the vendor’s reputation and position in the market.
  3. Validate the essentials.
  4. Evaluate implementation & maintenance requirements.
  5. Ensure the solution can support your unique process needs.

and provides a checklist of activities as well as a description for each of these activities. In the first step, they take a direct shot at EAI vendors who have grown into the BPM space: a hidden agenda of sorts, since Metastorm built their product as BPM from the start, but one that I don’t necessarily disagree with:

Many of the solutions marketed for BPM today were not actually developed for BPM — for example, EAI may be great for data to data integration and automation of system flows, but because it was not developed to address human interaction, it is unlikely to be the ideal solution for human-intensive process management, where human involvement and decision making is key.

I’ve run into situations in the past where an EAI vendor has positioned themselves as BPM and made large (and inappropriate) sales because of it. At some point, however, an EAI vendor will presumabely have built sufficient BPM functionality and expertise to be allowed into the club. Furthermore, it’s fair to say that the line between EAI and BPM is sufficient vague that there will be many vendors who play in both arenas instead of being in one and interfacing to the other: check out my post last week on integration stacks as well as Gartner’s emerging view of what’s in a BPMS.

All in all, there’s very little in the Metastorm white paper that most other BPM vendors would disagree with. At only 2-1/2 pages of real content, it’s a somewhat superficial look at the vendor evaluation process, but a good starting point.

We needed the business for this?

Gartner recently issued a report entitled Business Process Management’s Success Hinges on Business-Led Initiatives, the abstract of which states “organizations that had the most-successful BPM initiatives spent more than 40 percent of the initial project time on process discovery.”

This is newsworthy? Hands up if you still thought that IT could implement BPM without a significant involvement by the business. Of course, there are a few IT departments that I know that definitely haven’t figured that out yet.

BPM standards, architecture and modeling

Thanks to David Ogren’s BPM Blog for a pointer to “What is Business Process Modeling?” Like one of the commenters on the article, I also disagree with Mike Havey’s use of the term “business process modeling” as being interchangeable with “business process management”, since modeling is typically seen as being the business-focused activity at the front end of BPM, not the entire range of activities. I also don’t make such a sharp distinction between BPM and EAI: they’re part of the same continuum of technologies, and it is common to include EAI as part of BPM functionality (or vice versa): certainly the former EAI vendors who have hopped on the BPM bandwagon would agree with me on this point. His vision of a BPM architecture is not well-grounded in the current reality of BPMS, starting with “the heart of the architecture is a runtime engine that executes processes whose source code is written in the XML-based BPEL language” (I wish) and continuing with “human participants view and execute pending manual work activities using a graphical worklist application” (not necessarily).

Other than those minor points, it’s a good look at how the current morass of overlapping and competing BPM standards fit together, and has a very clear view of how BPM components are used during the design, development, deployment and monitoring cycle. I’ve been blogging about BPM standards from the beginning, and am completely on side with anyone supporting standards over the continuation of proprietary fiefdoms of process modeling and execution.

Last, but not least, he shows some examples of and provides a reference to itp-commerce‘s Process Modeler for Microsoft Visio, which allows you to create BPMN process models and (in the professional edition) transfer them to BPEL for execution. itp-commerce’s site links to an article by Bruce Silver that reviews the product, but more importantly, talks about the use of BPMN in general, and the use of Visio as a business process modeling tool.

By the way, if you’re interested in some of the theory behind BPM, Mike Havey’s written an article on Pi Calculus and Petri nets.