Enterprise 2.0: an update

A checkpoint on Web 2.0 in the enterprise circa mid-2007Dion Hinchcliffe wrote a great post a couple of weeks ago giving a checkpoint on Web 2.0 in the enterprise. I was going to just tag it in my del.icio.us links, which would have had it come up in my daily links post, but I wanted to take the time to comment on it since I think that he has a great way of capturing the essential information about the subject. Also, I’ve been waiting for Part 2, which was promised in advance of his Web 2.0 for the Enterprise webinar on July 31st, but I haven’t seen it yet.

His graphic, which you can see in small form on the right (click through to his Flickr page for the larger version, or see it in situ in the linked article above), divides Web 2.0 in the enterprise into four quadrants based on two factors: social versus technical, and internally versus externally-facing. In this first part, he walks through each of the aspects shown on the right, or internally-facing, part.

My thoughts on some of his notes, especially as they relate to BPM:

  • Wikis are now something that I discuss with every end-customer organization that I visit, even though I’m almost always there to do BPM-related work, not talk to them about Enterprise 2.0. I’m seeing uses in a number of line-of-business areas, such as sharing vendor pricing information across different company locations, and collaborating on updates to operational procedures manuals. Dion points out their importance for internal collaboration as well as a platform for user-built web pages, which ties in with his later comments about collective intelligence.
  • Collaboration 2.0 is like a wiki with a bit more structure, and an important class of applications/functionality that lies between unstructured collaboration and the highly-structured end of the spectrum that we see in BPM. As this space becomes more defined, I expect to see BPM vendors expand in this direction.
  • His comments on emergent structure are completely bang on: “The underlying concept of emergent structure is that we guess far too much up front about the features of the software we need or the way the data should be organized.” This speaks to a number of problems in current enterprise software development practices, most importantly that of the massive over-design of software that tends to take too long to deploy, ends up not being what the users need anyway, and being custom-coded hence not very agile. I’m a strong proponent of the “give the users the out of the box tools and see what they come up with” school of customization when it comes to BPM products (as well as many other software products) since the tools provided by the vendors are, for the most part, fairly usable by a trained user or business analyst.
  • Data aggregation is critical for reuse, but visibility/availability is really the key issue here. In my Enterprise 2.0 conference coverage, I mentioned a few times that I see RSS (or rather, the ability to subscribe to events and data) as a key functionality that’s going to sweep through enterprises. He covers syndication as a separate point later, but these are really two sides of the same issue.
  • Enterprise mashups are also going to be critical — and make use of that aggregated data via feeds — as the next generation of end-user computing platforms. Instead of Excel and Access, semi-technical users within business departments are going to be using mashup platforms to build the simple applications that they can never get IT to build for them.
  • Rich user experiences are already throughout organizations for end-user functionality, but that usage needs to continue to expand to include tools such as (in the BPM space) process modellers/designers so that they can be run anywhere by anyone, given the appropriate user authentication. FileNet has long had a Java applet for their process designer; the download is a bit clunky and it looks quite outdated now, but it still doesn’t require the software to be explicitly installed/upgraded at each workstation. Other more modern (Ajax-based) process designers are appearing from BPM vendors such as Appian and Cordys, and this trend is not going to reverse.

He covers a number of other aspects that I haven’t mentioned here; definitely worth a read. I look forward to Part 2 of this article.

Update: Part 2 of Dion’s article published here.

Process optimization webinar

Next Wednesday, I’m presenting a webinar on Process Optimization for the Real World, along with Rob Risany of Savvion (who are sponsoring the webinar). This is the same format as when Rob and I last did a webinar together: we both give a short presentation, then have a “fireside chat” discussion where we have a mostly unstructured conversation, which is a lot of fun for us and was pretty popular with the audience last time.

My part of the presentation is on how process optimization works — or doesn’t work — in reality, and some of the factors that impact it. In my experience, I find that most of the reasons that it doesn’t work are cultural, not technological, and can lie with both business and IT. Tune in next week to hear the whole thing.

BPM and SOA webinar

I’m listening in on a webinar with the rather grandiose title of “BPM and SOA: State of the Nation”, sponsored by webMethods. Paul Harmon of BPTrends kicked it off, and talked for some time about some general BPM and SOA definitions before diving into the results of a survey that they did earlier this year.

He had a great slide on how the maturity models for BPM and SOA intersect, with SOA MM level 5 (requires that companies know and track process measures) overlapping BPMM level 4 (processes are measured and managed systematically).

I found some of the survey questions odd; one of them offered four mutually exclusive responses:

  • You should not approach BPM without SOA
  • SOA cannot, ultimately, be successful without a BPM overlay
  • BPM is more successful and drives more benefits when deployed in an SOA environment
  • SOA is more successful and has more business relevance when tied to BPM

Not surprisingly, 74% of the responses were split between the last two; however, I think that most people would have selected both of those if they were able. That’s the whole point, right? BPM makes SOA more relevant, and SOA makes BPM easier to deploy. Interestingly, 19% chose the second option, that is, that SOA can’t succeed without BPM; I’d certainly agree that BPM makes it a lot easier to justify your SOA program, although I think that there’s other (non-BPM) business applications that also consume services and therefore help to justify it, so I wouldn’t make such a definitive statement.

He finished up with some good summary points:

  • BPMS and SOA are coming together. If you don’t know about either, you should learn about both.
  • If you are considering a BPMS tool, you’ll want to know how well it supports SOA.
  • Processes and services to automate BPMS and SOA will need to be carefully managed. You’ll especially want to know how any BPMS product supports SOA governance.

You can download the entire survey at BPTrends.com, and I’m sure that the webinar will be available for replay although it’s not at the original URL that I linked to yesterday; likely webMethods will have a link on their site.

Ex-BPMG contractor makes good

Jim Baird, who had organized the Toronto BPMG chapter meetings that I attended previously, also was a contractor to BPMG for delivering courses. With the recent demise of BPMG, Jim has stepped up to the plate and offered to provide courses for free to those who got stiffed by BPMG. From his recent comment on my original post about BPMG’s closure:

Hi folks. I would like to confirm what you have seen in this blog and thank Kevin and Sandy for bringing it to my attention. In the next couple of weeks we will be announcing the formation of the Business Process Transformation Group which will be available at www.BPTG.org. A new course schedule will be posted and anyone who has paid for training at BPMG will be offered the course that they have paid for at no charge. Please keep in mind that I was as surprised as everyone else with the demise of BPMG as I was contracted to train for them and was not a shareholder of the organization. Although we did not receive any of those funds that were paid to BPMG we find that the situation is deplorable and feel that this is the honourable thing to do. If you have any further questions please do not hesitate to contact me at Jim.Baird@BPM3inc.com.

Jim also contacted me to see about organizing a non-partisan BPM interest group meeting in Toronto, tentatively scheduled for September 21st. Stay tuned for more details.

BPM webinars this week

A couple of webinars to keep you amused this week:

Tomorrow, BPM and SOA: State of the Nation sponsored by webMethods (now part of Software AG), featuring Paul Harmon of BPTrends discussing their recent survey on BPM and SOA.

On Thursday, My BPM Success Story: Boundless Network sponsored by Lombardi, which features the VP of Development of Boundless Network talking about how they went from start to “success” (does that mean production?) in under 60 days.

I’ll drop in to one or both of these if I don’t end up travelling.

Process Design webinar Q&A

We didn’t have time for live Q&A during the TIBCO-sponsored webinar that I gave on business process design last week, but here’s the questions that we received and my responses. There were some questions specific to TIBCO products that I’ve passed along to Emily Burns to answer; if you asked one of those, you should see a response by email from her shortly.

Q: How mature is BPDM and is there any documentation available from OMG?

A: BPDM (Business Process Definition Metamodel) has been under development since 2003 and the specification was approved earlier this year by OMG. According to their catalog page, the 1.0 version finalization is underway, although there’s not a link to the specification available. There is a preview version available on ModelDriven.org that includes the March 2007 “final submission” of the specification in PDF (click on the folder called “Specification” in the left sidebar), which is the one that was approved, and an HTML version if you want to browse around in it online.

Another measure of maturity is the market adoption of the standard, and that’s just starting to happen. The only BPM vendor of which I’m aware that has BPDM in their released product today is Lombardi; if there’s others of you out there, please add a comment to this post.

Q: Could you please explain more on what is meant by choreography in BPM?

A: This question was also asked as “differentiate process orchestration and process choreography”, and although we did cover that in the one question that I answered at the end of the webinar, I’ll include it here since both orchestration and choreography refer to the modelling of business processes and there’s a lot of confusion in the use of these terms.

Orchestration focuses on the behavior of a single participant, and is a hub-and-spoke model with a controller at a single location in the distributed system that enforces the progress of the process. An orchestration language is for processes that are controlled by one organization (although they may execute across multiple organizations), so it?s used to model what goes on inside the controlling organization plus the external linkages to the other participants. Choreography, on the other hand, is a peer-to-peer collaboration that doesn?t require a central controller. A choreography language is for collaboration between peer organizations, and is used to model the exchange of messages between the organizations but not their internal processes.

OMG refers to orchestration as “the modeling of directed, internal business processes” and choreography as “the specification of interactions between autonomous processes”.

I like Stefan Tilkov’s definition from a couple of years ago:

In orchestration, there?s someone ? the conductor ? who tells everybody in the orchestra what to do and makes sure they all play in sync. In choreography, every dancer follows a pre-defined plan ? everyone independently of the others.

Standards have tended to focus on either orchestration or choreography, not both; as more standards emerge that support both, the distinction between them will likely start to fade away.

Q: I’ve seen an Intalio product that has implemented BPEL4People in their product.

A: Not really a question, but yes, Intalio implemented an earlier version of BPEL4People (Ismael, jump in here if I have the details incorrect) based on the original 2005 white paper, which isn’t quite the same as the currently proposed specification. There’s more details on BPEL4People available from IBM and BEA, including the specification document from June 2007. According to the IBM reference, “the authors plan to submit the specifications to a recognized standards development organization in the near future”, by which they mean OASIS.

Q: Please explain the relationship between BPMN and UML.

A: UML is a modelling language originally used for modelling software as part of software requirements and design (check out the Wikipedia entry for more detail), adopted and standardized by OMG. One of UML’s 13 types of diagrams is the activity diagram, originally a variation of a state diagram, that is used for modelling logic and control flow within software (example here). Because it models flow, its usage was extended to model business process flows as well, and some BPM tools used it as the graphical notation for modeling business processes.

BPMN was developed purely as a business process modelling notation by BPMI (Business Process Management Initiative), and intended for use by business analysts. The first draft was released in 2002, and the 1.0 release was in 2004. The following year, in part due to the BPM Think Tank meetings, BPMI was absorbed into OMG and BPMN was designated as the standard to use for modelling business processes, effectively relegating UML back to software design and other more technical usages.

There is a 2004 white paper written by Stephen White of IBM, who has been a major contributor to the BPMN standard, comparing UML and BPMN; you can find it here (direct PDF link) or linked from the BPMI information page.

Q: Are there any standard design patterns (names) available/identified for BPM ?

A: There are 21 identified workflow patterns that White refers to in his white paper (see previous answer), with an expanded version of 43 patterns at the original source here. If you download the 2006 paper “Workflow Control-Flow Patterns: A Revised View” from the Documentation page on that site, you’ll find an assessment of various languages, standards and BPM products relative to the 43 patterns, including XPDL, BPMN, BPEL and UML. The product evaluations are a bit out of date since these things change quickly, but worth a glance.

These patterns range from simple (e.g., sequence) to fairly complex (e.g., interleaved routing), but all of them represent a small number of steps within a process map, not a complete design pattern for a process.

Q: Are there samples of the processes?

A: I assume that this was related to the section on design patterns. I have a lot of examples from my own experience, and have listed some for the first design pattern on initiating processes here:

  • Human intervention: In a process for expense approval, the user requesting the approval initiates the process explicitly and fills in the information regarding the expense. Note that explicit process initiation may happen through the BPMS’ standard interface (most of which provide some sort of way to locate process templates and create an instance based on the template) or may be through a customized interface that masks the BPMS from the user.
  • External event: A paper document containing instructions for one or more mutual fund transactions is scanned into a content repository, and the content repository triggers an event that initiates a process with that document attached.
  • Invoked as a web service: A mainframe process that automatically adjudicates health insurance claims initiates a manual review process in BPM by calling a web service when the claim can’t be automatically adjudicated. Most BPMS’ allow a process to be exposed as a web service for this type of usage; you can consider this to be just a special case of external event triggering.

Since I don’t have any sort of synchronization between when the questions were asked and what slide was up at the time, let me know if this is addressing your particular question by adding a comment to this post, and I can expand on other areas as required.

Q: How do you suggest building resilience (error detection, error recovery, process restarts)?

A: Resilience at a low level has to be built into the BPMS, e.g., if an error occurs when executing a step or invoking a web service, the system needs to have some way to either alert the executing process or recover/retry the action. Assuming that the executing process is notified of a low-level error, you can then design a recovery path. If the error is not something recognized by the BPMS itself (e.g., a web service call executes properly but returns an error code), then your design will need to check for that error just the same as if you were writing code to do so: think of BPMN in this context as a visual coding language of sorts. Regardless of how the error is detected, the best way to handle the errors (if the BPMS that you’re using fully supports BPMN) is through the use of the error, cancel and compensation events, which include recovery and rollback functionality.

Q: Do users find it cumbersome to get work items from one system (BPM) and perform them in another (e.g. legacy system)? Do you have any recommendations for addressing this?

A: In most cases, users are working with at least two different “input methods” to drive their workflow: paper documents, email or spreadsheets may all be providing the necessary input to key information into a legacy transaction processing system. Moving their worklist from a manual method to a BPMS on their screen is usually not a big transition, in my experience. When I first started implementing human-facing workflow/BPM systems in the early 90’s, there were a lot of problems in adapting because the users were moving from dumb terminals to Windows workstations as part of the implementation, and were not adept at switching between windows in order to maximize their screen usage. Nowadays, I see users flipping between multiple windows on their screen, or even making use of dual screens that allows them to rearrange their most commonly-used windows in key locations, then just pop up others as they need them.

Even though this sort of side-by-side windowed environment is not a challenge for today’s users, I would still recommend some integration between the systems. Say, for example, that the work item in BPM already has a customer number captured as a data field; when that work item is opened by the user, you’d really like to look up the associated customer record on the legacy system instead of them having to re-enter the number on the legacy screens and search for it. The methods of doing this depend on the nature of the legacy system and its user interface: if it’s still a green screen (and I see lots of those around), then you’ll need to pass commands via the terminal emulator software to drive the screens as if a user were typing the information. This is cumbersome, and often doesn’t work all that well when an error occurs on the legacy system. If your legacy system has a nicer interface that supports some sort of messaging protocol or has an API, then you can do the search that way.

The other side of that is to capture information from the legacy system back to the work item in the BPMS, either for passing on to another system involved in an orchestration process (including auto-indexing attached documents from a content management system) or to make routing decisions within the process flow. For green screen systems, you’ll do this with “screen scraping” via the terminal emulator software, which extracts data from the screen at a specific pre-determined location and passes it back. With a smarter system interface, you’ll make an API call or pass a message.

Q: How can a company implement a simple BPM implementation when at a minimum most companies want to automate processes that access a database which requires some kind of integration?

A: Many BPM systems support database access pretty much out of the box, in the same sort of way as they support web services, so this may require only some configuration of the database connection parameters at a task and no real coding. For example, in TIBCO you add a service task to your process map, configure it with the server and database name, enter the name of the stored procedure that you want to call in that database, then specify the mapping between the stored procedure parameters and the work item data fields. No code, although you may need to create stored procedures in your database.

Q: Can you describe the appropriate level for modeling a process? What do you consider an atomic piece of work?

A: The appropriate level really depends on the purpose of the model. If you’re doing high-level strategy or business planning, then likely you’ll only model the high-level activities and won’t model data fields, roles, web service calls and other details. If you’re creating a model that will be used to execute the process, then you need to model each step in the process, whether automated or human-facing, and specify all the parameters required by the step.

The definition of an atomic piece of work is dependent on your application, but you’ll likely determine it by going back to whatever it is that initiates a process instance and thinking through how that is going to be processes through its life cycle. Using the examples that I discussed for process initiation earlier in this post, the atomic piece of work in the first case is a single expense approval request; in the second case, it’s all of the transactions documented on the attached document (although that might be broken down into individual transactions that each spawn their own subprocess if that’s how the work should be executed); in the third case, it’s the adjudication case rejected from the legacy process. This is a pretty critical piece of the design in your BPM implementation, since it impacts complexity of design, the ability to process in parallel, and even the statistics that can be gathered about a process. Specifically, in the second case, you’ll need to decide whether to split a single initiated process into multiple pieces of work; I’ve done it both ways depending on a number of other factors in the overall design.

Q: What difference there is between “your” BI description and BAM?

A: In my opinion, the larger field of business intelligence (BI) includes near-real-time dashboards such as those used for business activity monitoring (BAM). There are various opinions on this: some people consider BI to be only historical reporting; others (including analyst groups such as Forrester) consider BAM to be a part of BI. Other terms like CPM (corporate performance management), CEP (complex event processing) and EDM (enterprise decision management) overlap the BI space significantly, which I think is what has led to the recent relabelling of most technologies related to data collection, aggregation and analysis as business intelligence.

Q: Difference between Business Process Workflow and Business Process Management.

A: I don’t make a real distinction between workflow and BPM from an implementation standpoint, although the term workflow has typically been used in the past to denote human-facing processes. Microsoft, for one, now uses the term workflow to mean any sort of process flow: their site refers to “system workflow and human workflow”.

Many of the analysts define BPM as not just the tools used to automate processes, but also the management practice around business processes: Gartner defined BPM in 2005 as “a management practice that provides for governance of a business’s process environment toward the goal of improving agility and operational performance”.

 

Thanks to all of you who made the time to attend.

Business Process Design webinar today

At noon Eastern time, I’ll be giving the last in the 3-part series of webinars sponsored by TIBCO, Business Process Design:

In this session Sandy will dive into the nitty gritty of process design, including:

  • BPM design patterns
  • Human-facing steps: structured versus collaborative
  • Data synchronization requirements
  • Process automation: invoking services and being invoked

The tactics discussed in this webinar will ensure that the models you create are more than just pretty pictures — that they are, in fact, viable, workable models that you can use to automate your business processes and realize significant ROI.

You can register here to attend.

The first two, on process discovery and modeling respectively, are available for replay although I can’t find the links right now.

BPM Think Tank Day 3: Enterprise 2.0/BPM Mashups Roundtable

I facilitated one of the last roundtables of on the conference, about Enterprise 2.0 and BPM mashups.

Mashups (considered a part of Enterprise 2.0) are lightweight integration of web-based services and data, often in ways that the service providers never intended them to be used; personally, I think that as mashup techniques get easier, mashups will become the technology of choice for what’s referred to as “end-user computing”, that is, all the stuff that is created within business units (typically now using Excel or Access) because it’s either too small for IT to take on as a project or they can’t turn it around in a timely manner. I see software-as-a-service BPM and other services as having an impact on the ability to do mashups, since these platforms are often designed with a bit more openness in mind.

I’ve looked a lot in the past at Enterprise 2.0 and BPM, and the features that are (or should be) creeping into BPM under the influence of Enterprise 2.0: RSS, tagging, SaaS, mashups, collaboration, and all sorts of user-created content in general. There’s a lot of challenges around this, many of the cultural, since Enterprise 2.0 decentralizes control of IT assets and requires a certain level of user participation.

We spent most of the session talking about BPM mashups, not Enterprise 2.0 in general. At one level, a BPMS can be considered to be a mashup platform, given the right business services available for assembly.

BPM mashups can take several forms:

  • Lightweight assemblies of subprocesses and services
  • User-facing information at a step in the process, e.g., Google maps mashed up with BPM data and presented to a user in a form in order to complete a task
  • BPM as a component within a portal, possibly assembled by a user

Issues around mashup adoption include IT not trusting something that is user-created, and business analysts not understanding the concept of mashups as well as not yet having easy enough tools to do mashups. There are also issues around discoverability of services (as I discussed the previous week in a Mashup Camp session) and the use of internal versus external services, where both types require some sort of SLA to be included in any sort of production mashup.

By lowering the barrier to entry, mashups can play an important role as application prototypes, or emergent applications that IT wouldn’t have thought to build for the business; IT can learn from what the business creates for itself in order to create more structured applications and processes.This is similar to the concept of how a folksonomy is used to gradually become a taxonomy: allow the users to do it themselves, then observe and detect the patterns. My favourite phrase that someone used at this point was to “intelligently stumble upon the future” and the whole idea of unintended consequences of mashups, although there was some discussion as to whether is was closer to serendipity or Frankenstein. Along this line, we talked about how to keep bad things from happening in mashups, and agreed that the services and data to be mashed up had to be controlled in some way (by IT) so that, for example, someone couldn’t do an unindexed full text search on a multi-million record database.

Without a doubt, mashups enable agility in application development, and BPM stands to benefit from enabling all types of BPM mashups.

There was some discussion around whether business users/analysts were asking for this, and whether they really wanted a full mashup capability, or just some parameterized configuration. I think that they don’t even know what’s possible through mashups, and if they did, they’d want it.

BPM Think Tank Day 3: Colin Teubner

Colin Teubner of Forrester gave us a lunchtime presentation, hence my notes were on paper and it’s taken a bit of time to transcribe them. However, I’m on a roll to get all my Think Tank coverage wrapped up today so that I can take 4 days off for the holiday weekend.

Colin’s talk was on issues, challenges and trends in BPM, and the general opinion around my lunch table is that it was a bit lightweight, although a reasonable summary of the current state of affairs. I certainly don’t envy him the task of speaking over the clanking of cutlery and buzz of other conversations as people eat their lunch.

He sees that a maturing of tools and practitioners is pulling more tool types into BPM, particularly a convergence of BPM and BI, and a convergence of content management, collaboration and human-centric BPM. Seeing as how we’ve only just managed to pry content management and human-centric BPM apart, I’m not sure the latter is good news. As he pointed out, BPM is more than modelling and automation, although a lot of projects (and products) get stuck there and don’t do the monitor/manage/optimize parts very well.

He returned to the discussion on BI and BPM that came out of the previous day’s roundtable that he led:

  • BI on a process
  • BI triggering a process
  • BI affecting a process (e.g., event)
  • BI inside a process decision
  • BI inside a human task assignment (inform rather than automate decision)
  • BI to help humans with process work
  • BI to predict the future of process work

BI is positioned as making data actionable. Data-driven BI is bereft of process, and focussed on reports and presentation. Process-centric BI (mainly from BPM vendors) has awareness of BPM and the processes; there may be a tie-in with BPRI although there’s no standard linkage between process models and runtime data that could be consumed by a 3rd party BI product. No BI vendors are doing real BPM-aware BI yet.

He then discussed collaboration and information, showing that BPM is typically only used for the structured part of processes. Interestingly, he just redivided the BPM marketplace into ad hoc/collaborative, production and integration workflow, which is where we were 7 years ago before this all got lumped together as BPM. The future of BPM is a 360-degree view of business processes; the main barrier to that now is that there’s no collaboration in BPM products and no process management in collaboration products. Some BPM vendors are starting to pull in collaborative functions, such as discussion threads, process wikis, email notifications, embedded analytics, dynamic task support, and portal integration.

A few wrapup points on what all this means:

  • BPM vendors must partner to integrate with other functions, such as content management
  • Standards are essential to driving the integration partnerships
  • End users need to think about process, collaboration and ECM together, not as separate issues