business process management
I’ll be attending the Gartner BPM Summit next week in Nashville, and I just noticed on Sanjay Kalra’s blog that he will be there as well, and is looking to meet other bloggers.
Anyone else interested in meeting up, maybe Monday night after the solution showcase reception finishes at 7:45?
A great post by James Taylor on business and IT collaboration as it applies to business rules:
This might lead one to suppose that business users should just be given tools that let them right the rules themselves in some unconstrained way. Like the healthcare folks I was talking with, I don’t think so. The reality is that the rules must be deployed in a real system with performance and scalability needs. The rules must execute against data stored and managed in other information systems. Business users are not as used to the discipline of QA and testing that most organizations would want to see before new rules are rolled out.
He goes on to discuss how to create the right environment for business and IT to collaborate on business rules management.
Now, just take his post and replace the word “rules” with “process”, and you’ve made my case as well.
A couple of weeks ago, I posted a letter from a reader who asked the question “Is Automation BPM?” Peter Dawson and David Chassels both chimed in with their comments, and I want to add my thoughts. Excepts from the original letter:
I have a hard time referring to it as BPM when a person simply automates a business process. To me that is workflow. Granted, we have powerful stand-alone workflow engines now that allow a workflow to route work between more people and systems than ever before, but it is still workflow.
If you have a workflow process that is less focused on automating work and more focused on gathering data about a process for later analysis, then you have entered the realm of BPM.
I disagree. I started in the old workflow world: for the past 18 years, I’ve been focussed on processes that usually have a large human-facing component. I started with traditional workflow products that only provided person-to-person routing of scanned documents, which is about as old fashioned as it gets in the workflow world. Since I worked for most of that time providing services to the customers of these systems, I did a lot of automation and integration work from scratch because that functionality just didn’t exist in the workflow systems. Want an automated function in your 1993 IBM ImagePlus system to escalate items that have been sitting in a queue too long? No problem, we’d write a daemon that would cycle through the work in a queue and push specific items to an exception queue. Want to integrate your 1991 Keyfile system with an Oracle database application? Give us a few weeks, we’d write a front-end application that integrated the two systems at the presentation layer by making API calls to each system. Want to have some process reports from your 1994 FileNet Visual WorkFlo system? Okay, we’d write a batch process to interrogate the database history logs each night, extract the process-related data and print some reports.
The point is, the systems as provided by the vendors had no automation and no integration: they were concerned only with the flow of work from one queue to another. Work. Flow. Workflow. Everything else — integration, automation, process monitoring and governance — was custom built. I could, in fact, make the argument that what we did as services firms was to turn vendors’ workflow products into business process management systems.
From a semantic standpoint, the term “workflow” has been absorbed into most definitions of “BPM” (except, perhaps, those definitions created by ESB vendors who prefer an integration-focussed definition). Even the old-style workflow that I describe above — just routing from one person to another — is a part of BPM, even though a system that only provides that level of functionality would not compete very well in the broad BPM marketplace. The fact is, however, that almost every product in the BPM space has a set of functions that doesn’t quite overlap with anyone others, and doesn’t overlap at all with some other products that are also called BPM. Gartner published a report on BPMS selection criteria a few months back that listed 10 major areas of functionality, but there are many products in the BPM space that don’t have one or more of those functions covered. The problem, as I’ve written about before, is that the term is too broad, and was a bit of an artificial creation on the part of the analysts, who took workflow, EAI, and anything else remotely related to process and called it all BPM.
To get back to my rebuttal, then, I believe that as soon as you add integration and automation capabilities to human-facing workflow, you’ve got BPM (as opposed to just being able to call it BPM because workflow is now defined as a subset of BPM). From the other side of the house, the same can be said about adding human-facing functionality to EAI to create BPM.
Two worthwhile SOA webinars from ebizQ in the past two days. Yesterday, it was Developing an SOA Ecosystem with Steve Craggs from the Integration Consortium. Craggs, who also runs a U.K.-based consulting company, Saint Consulting, spoke for a solid 40 minutes, starting with a quick but thorough definition of SOA, moving on to the concept and components of an SOA ecosystem as it should exist in your organization, and finishing up with some tips for success for building an ecosystem that can support 100’s of services. And, I loved two of his quotes near the end: “SOA is not just about technology — it’s a transformation of the business” and “SOA ecosystems ensure sustainable advantage”, very business-oriented views of SOA benefits that are often lost in the more-frequent technical discussions about SOA.
He talked about how SOA has changed from a focus purely on web services (the little blue bit in the diagram below) to a focus on all the other things that need to exist in order to make SOA successful:
My only argument with his taxonomy of the ecosystem is that he puts orchestration as part of the mediation services, and BPM floating somewhere out there as a consumer of the SOA ecosystem, almost as part of the application layer. However, I consider orchestration to be part of the larger definition of BPM, hence believe that BPM belongs as part of the SOA ecosystem itself and in fact includes most of the mediation services that he lists.
My favourite bit was where he referred to web services as “an answer looking for a problem”, and went on to list why just using web services over HTTP is problematic in building an SOA, and why it led to the development of the SOA ecosystem concept. He also gave some compelling reasons why the “best of breed” approach is still the best route in assembling your SOA ecosystem today, and the value of an enterprise service bus and the service registry. None of this is rocket science — in fact, his bit on the importance of correct granularity for services is really just an SOA twist on the age-old programmers’ dilemma of granularity — but it is a nicely packaged talk, delivered in a way that’s understandable to an SOA newbie. Definitely worth a listen.
Today, Frank Kenney from Gartner gave a short but impassioned talk on Policy-Driven SOA, followed by a fairly informative session from Sean Fitts, chief architect at Amberpoint, the webinar sponsor.
Under the heading “SOA does not come for free”, Kenney talked about everything that’s required to get SOA in place: infrastructure, methodology, services creation, testing, etc., but states that it’s worth the price — an argument that I seem to be making to customers over and over again.
He feels that you need to have a certain type of infrastructure that allows you to govern what’s going on in SOA. It’s a different view than Cragg’s SOA ecosystem shown above — kind of a slice across the lower half of Cragg’s diagram, or maybe a third dimension.
Kenney talks about services, processes and policies as assets that can be used for competitive differentiation, which really lines up with Cragg’s statement about ensuring sustainable advantage. What they’re both saying is that the old cowboy web services methods aren’t enough any more: if you really expect to reap the potential benefits of SOA, you need to start putting some discipline in place.
He finished up by stating that governance is a business issue, and that no one vendor can sell governance, but they can help you to achieve governance.
Replays are available at the links. I’d love to see these available as downloadable video for my iPod, by the way.
Registration for the AIIM Expo 2006, to be held May 16-18 in Philadelphia, is now available online. I haven’t attended AIIM in a few years, but I’m thinking about attending the exhibits. So far, however, the list of BPM vendors and BI vendors look pretty sparse — I think that most of these vendors tend to hit industry-related shows rather than general (and somewhat outdated) shows like AIIM, or hold public webinars. The advantage of attending a trade show, in theory, is that you see the new/little vendors/products that might not otherwise hit your radar screen, but I’m not sure that many of them are coming to AIIM.
And, if I want a really good product demo, there’s better ways to do that than at a trade show: I’ve had two private demos directly from vendors via Webex in the past two weeks, with another coming up soon. Not that I don’t want to visit Philly, but Webex is definitely the way to go for demos.
Gartner has released their Magic Quadrant for business process analysis, and the first big news is that they’ve updated the style of the graphics: rounded corners, one-sided arrows and –wait for it — orange dots!
Seriously, though, this is a good report and comparision of the primary tools used for modelling and analyzing your business processes, whether or not you’re going to automate some of those processes using BPM. However, it’s the linkage between BPA and BPM that is really driving the BPA marketplace, including the whole round-tripping process that allows BAM results to be fed back into the BPA tool for further analysis and optimization.
Many of the BPM vendors have BPA included in their product; in fact, Gartner makes a note that they dropped FileNet from this report since their BPA is so deeply embedded that it’s not sold as a separate product, but they’ve added other BPM vendors such as Fuego (now BEA) and Savvion, although they languish in the lower left corner. In other words, you’d use their BPA if you were using their BPM, but you’re not going to buy it as a standalone BPA tool, as I’ve discussed previously. For that, you look to the big guns in the top right of the quadrant, such as IDS Scheer and Proforma. Although IDS Scheer focusses purely on process modelling, Proforma, Telelogic (formerly Popkin) and a few others go far beyond that to full enterprise architecture modelling, of which process modelling is an important but small part. I haven’t written much about EA lately, but it’s definitely a topic very near and dear to my heart.
I find it interesting that Gartner has chosen the vendors for this MQ in such a way that they are only in the “leaders” or “niche players” quadrants: not a one in the “visionaries” or “challengers” quadrants. They give an explanation for why this happened in the full report, but I feel that the comparison chart is less useful for tracking future trends without the visionaries and challengers. Personally, I would have put Microsoft (Visio in the BPA product under review) a bit more to the left into the challengers’ quadrant, since it’s not clear to me that their vision for making Visio a full BPA offering is particularly complete.
Last year, I posted about BPTrends‘ report on enterprise architecture, process modelling and simulation tools, which includes many of those in Gartner’s upper right quadrant. The vendors paid to be part of the BPTrends report, so it’s not exactly indepedent analysis, but it includes some good background material on the market (and it’s free).
James Taylor‘s been at the Gartner Business Intelligence Summit this week. On Monday, he posted some great thoughts on process, rules, BI and agility:
You can use business rules to automate decisions in business processes and then use analytics to optimize these decisions and hence the processes…
You must be able to change a process that you are monitoring when your monitoring tells you that something is wrong. Real-time measurement should not be combined with systems that take weeks or months to change.
Although there are caveats to that last sentence — for example, some real-time measurement is intended to allow the human elements in a process to change rather than the system, such as work re-allocation — I’d still like to have it tattooed on my forehead for every client to read. Making measurements with the intention of enabling agility is useless in many of the BPM installations today, not because the underlying BPMS isn’t agile, but because the customer chooses (or is coerced) to undertake a huge degree of customization that effectively pours concrete over the system.
Then later that day, he posts more on how BPM, BRE and analytics go together like chocolate and peanut butter (that’s my characterization, but I’m sure James would agree) — that seems to be a popular theme at the summit. He also posts about the Tuesday and Wednesday sessions, although less BPM-related than the Monday sessions.
Maybe because I come from the BPM side of the house, I don’t really see why the big fuss to rename parts of the BI space: BI seems to be an outdated term now, referring only to reporting on historical information from a data warehouse or operational data store. Other terms like CPM (corporate performance management), BAM (business activity monitoring), CEP (complex event processing) and EDM (enterprise decision management, which also involves BRE) have sprung up to cover the near-real-time space that I still think of as BI — after all, there’s much of the data aggregation, analytics and other common technology at the core. Many of these newer terms are touted as “[something more fabulous] BI”, such as James’ reference to EDM as “deployable BI”, but it feels a bit like the emperor’s new clothes. Maybe they’re all just BI 2.0.
I received the following opinion by email from a Column 2 reader who makes a distinction between automation/workflow and BPM. I invite discussion, and I’ll be adding in my $0.02.
I have a hard time referring to it as BPM when a person simply automates a business process. To me that is workflow. Granted, we have powerful stand-alone workflow engines now that allow a workflow to route work between more people and systems than ever before, but it is still workflow. Just because the workflow engines now call themselves BPM does not mean that a person is doing BPM when they use them.
When a website or an article about BPM touts the benefits of BPM as being the ability to speed up or automate a business process, reduce the time work is in transit, pinpoint where work is real-time, gather all relevant data on one screen for the worker, increase accountability or allow for the automation of exceptions, I feel that that author just does not ‘get it’. Getting the right thing to the right person (or system) at the right time is workflow; its benefits have long been established (i.e. http://e-workflow.org).
Workflow is powerful and generally has very high payback. One of the weaknesses of workflow is that the process you just automated may not be the best or most efficient process. That is where BPM comes in. If you have a workflow process that is less focused on automating work and more focused on gathering data about a process for later analysis, then you have entered the realm of BPM. If there are places in the BPM process that can be automated while still keeping the focus on gathering the metrics, that is and added bonus. In my mind that constitutes the incorporation of workflow capabilities and benefits into your BPM effort.
I envision a BPM flow initially being only human steps where a person reports back that they have done certain things. In the beginning it would just be additional overhead. As more information is gathered and analyzed this human process is rearranged quickly to remove bottlenecks and inefficiencies. Over time automation of redundant or time consuming steps are added. When automation is added we must be very careful not to make the flow inflexible since the primary goal is to gather data and be flexible, not to automate. If a point of automation is identified but it will add inflexibility, then maybe a tactical workflow application is created and the BPM flow just consumes it as a service. You get the automation but keep the BPM flow as flexible as possible.
Workflow is workflow and BPM is BPM. The trend is to use the workflow engines to do BPM, but they are still distinct entities and are implemented in distinctly different ways. I think this distinction is important because you are not going to get all the benefits promised by the BPM crowed if you spend all your time automating. Automation is good, but BPM + Strategic Automation is better.
David Ogren continues his inside look at being acquired by BEA, while taking a swipe at Bruce Silver, Sanjay Kalra, myself and undoubtedly others for being so bold as to compare this to the TIBCO-Staffware acquisition. Charmingly loyal.
I’ve just finished listening to an ebizQ webinar How to Take BPM to Another Level in Your Organization, featuring Jim Sinur from Gartner. webMethods is the sponsor, so Susan Ganeshan, their VP of Product Development, is also on the line.
Sinur made a quick plug for the upcoming overpriced Gartner BPM summit, but then dug into some interesting material. Much of it was a duplicate of what I had seen presented by Michael Melenovsky in Toronto a few weeks ago — such as the comparison between a functionally-driven and process-driven enterprise, and the set of practices required around BPM — but it was good to see it again since I still haven’t received a set of slides from the Toronto seminar so had to rely on my rough notes. I liked the slide on the roles and responsibilities of business and IT, especially the centre column showing the shared responsibilities:
- Process deployment
- Process execution and performance
- Business and process rules analysis and management
- Operational procedures including version level control
- Creation of process, rules, and events repository
- Detailed process design
- Training and education
- Event analysis and management
This lines up with what I’m seeing with my customers, as IT relinquishes more control to the business, and the business starts to step up to the challenge in order to ensure that what gets implemented is actually what they need. I believe that the business still needs to be more proactive in this area: in the large financial institutions that I mainly work with, many parts of the business have become completely passive with respect to new technology, and just accept (while complaining) whatever is given them. I’m not in favour of giving over design control to the business, but they definitely need to be involved in this list of activities if the BPM project is going to hit the ground running.
I also liked the slide on what’s happening with rules and BPM in the context of policy-managed applications:
This is a very enterprise architecture-like view of the process, where you see the business policies at the top, the resultant rules immediately below that, then the linkages from the rules to the data and services at the bottom, which are in turn plugged into specific steps of the process. Making these linkages is what ultimately provides business agility, since it allows you to see what parts of the technology will be impacted by a change in a policy, or vice versa. There really needs to be more enterprise architecture modelling going on as part of most organizations, but particularly as it related to process orchestration and BPM in general.
At the 22 minute mark, Sinur returned to another 2 minutes of shameless plugging for the conference. Considering that we were just warming up for the “real” vendor to start plugging their product, the commercial content of this webinar was a bit high.
Although webMethods supposedly entered the BPM market back in 2001 on their acquisition of IntelliFrame, I’m not seeing them on any BPM radar screens — they’re really considered more of an integration suite. Ganeshan really should have practiced reading her script before she read it on the air, too, although she was much better at the Q&A. Except for the remark about how they don’t do BPMN because their customers really like webMethods’ “richer” proprietary interface instead (as if they have a choice).
At the end, the moderator showed the results of the survey question that she had posed near the beginning of the webinar. No big surprises here, although interesting to note that all of the drivers (except for document compliance) are becoming equally important to people. The trend that I’m seeing is that the goal of improving operational efficiency (shown on this survey as “reduce operational costs” and “reduce process errors”) are being taken for granted: everyone expects that will be the result of implementing BPM, so it’s not considered the main business driver. Instead, process visibility and process orchestration are moving to the forefront, which in turn drive the agility that allows an organization to bring its products and services to market faster.
ebizQ is using some new webinar software lately (or maybe a new version) which has some nice new features. It allows you to download the presentation at any time (other webinar providers could learn a HUGE lesson from that), zoom on the slides, and other features from the previous version, but now also shows a list of upcoming webinars in the sidebar and allows you to pop up a list of the attendees (first name and company only). Unfortunately, I had some major connectivity problems that resulted in a five minute gap right at the beginning of the Q&A, so I’ll have to go back to the replay and check it out. Fortunately, they publish the replays very quickly after the event, using the original URL, another good lesson for other webinar providers.
