BPM and Application Composition Webinar This Week

I’m presenting a webinar tomorrow together with Sanjay Shah of Skelta – makers of one of the few Microsoft-centric BPM suites available – on Tuesday at noon Eastern time. The topic is BPM and application composition, an area that I’ve been following closely since I asked the question five years ago: who in the BPM space will jump on the enterprise mashup bandwagon first? Since then, I’ve attended some of the first Mashup Camps (1, 2 and 4) and watched the emerging space of composite applications collide with the world of BPM and SOA, to the point where both Gartner and Forrester consider this important, if not core, functionality in a BPM suite.

I’ll be talking about the current state of composite application development/assembly as it exists in BPM environments, the benefits you can expect, and where I see it going. You can register to attend the webinar here; there will be a white paper published following the webinar.

HandySoft Process Intelligence and User Experience

Wow, has it really been a month since I last blogged? A couple of weeks vacation, general year-end busyness and a few non-work side projects have kept me quiet, but it’s time to get back at it. I have a few partially-finished product briefings sitting around, and thought it best to get them out before the vendors come out with their next versions and completely obsolesce these posts. 🙂

I had a chat with Garth Knudson of HandySoft in late November about the latest version of their BizFlow product, specifically around the new reporting capabilities and their WebMaker RIA development environment. Although these don’t show off the core BPM capabilities in their product suite (which I reviewed in late 2009), these are two well-integrated tools that allow for easy building of reports and applications within a BizFlow BPM environment. I always enjoy talking with Garth because he says good things about his competitors’ products, which means that not only does he have good manners, but he takes enough care to learn something about the competition rather than just tarring them all with the same brush.

We first looked at their user-driven reporting – available from the My AdHoc Reports option on the BizFlow menu – which is driven by OEM versions of the Jaspersoft open source BI server components; by next year, they’ll have the entire Jaspersoft suite integrated for more complete process analytics capabilities. Although you can already monitor the current processes from the core BizFlow capability, the ad hoc reporting add-on allows users (or more likely, business analysts) to define their own reports, which can then be run on demand or on a schedule.

HandySoft BizFlow Advanced Reporting - select data domainIf you’ve seen Jaspersoft (or most other ad hoc reporting tools) at work, there isn’t much new here: you can select the data domain from the list of data marts set up by an administrator, then select the type of report/graph, the fields, filtering criteria and layout. It’s a bit too techie for the average user to actually create a new report definition, since it provides a little much close contact with the database, such as displaying the actual SQL field names instead of aliases, but once the definition is created, it’s easy enough to run from the BizFlow interface. Regular report runs can be scheduled to output to a specific folder in a specific format (PDF, Excel, etc.), based on the underlying Jaspersoft functionality.

The key integration points with BizFlow BPM, then, are the ability of an administrator to include process instance data in the data marts as well as any other corporate data, allowing for composite reporting across sources; and access to the report definitions in the My AdHoc Reports tab.

The second part of the demo was on their WebMaker application development environment. Most BPM suites these days have some sort of RIA development tool, allowing you to build user forms, screens, portals and dashboards without using a third-party tool. This is driven in part by the former lack of good tools for doing this, and in part by the major analyst reports that state that a BPMS has to have some sort of application development built in to it. Personally, I’m torn on that: most BPMS vendors are not necessarily experts at creating application development tools, and making the BPMS capabilities available for consumption by more generic application development environments through standard component wrappers fits better with a best-of-breed approach that I tend to favor. However, many organizations that buy a BPMS don’t have modern application development tools at all, so the inclusion of at least an adequate one is usually a help.

HandySoft BizFlow WebMaker - specify field visibiltyHandySoft’s WebMaker is loosely coupled with BizFlow, so it can be used for any web application development, not just BPM-related applications. It does integrate natively with BizFlow, but can also connect with any web service or JDBC-compliant database (as you would expect) and uses the Model-View-Controller (MVC) paradigm. For a process-based application, you define the process map first, then create a new WebMaker project, define a page (form), and connect the page to the process definition. Once that’s done, you can then drag the process variables directly onto the form to create the user interface objects. There’s a full array of on-form objects available, including AJAX partial pages, maps, charts, etc., as well as the usual data entry fields, drop-downs and buttons. Since the process parameters are all available to the form, the form can change its appearance and behavior depending on the process variables, for example, to allow a partial page group to be enabled or disabled based on the specific step in the process or the value of the process instances variables at that step. This allows a single form to be used for multiple steps in a process that require a similar but not identical look and feel, such as a data entry screen and a QA screen; alternatively, multiple forms can be defined and assigned to different steps in the same process.

To be clear, WebMaker is not a tool for non-technical people: although a trained business analyst could probably get through the initial screen designs, there is far too much technical detail exposed if you want to do anything except very vanilla static forms; the fact that you can easily expose the MVC execution stack is a clue that this is really a developer tool. It is, however, well-integrated with BizFlow BPM, allowing the process instance variables to be used in WebMaker, and the WebMaker forms to be assigned to each activity using the Process Studio.

HandySoft is one of the small players in the BPMS market, and has focused on ad hoc and dynamic processes from the start. Now that all of the BPMS vendors have jumped into the dynamic BPM fray, it will be interesting to see if these new BizFlow tools round out their suite sufficiently to compete with the bigger players.

Customizing the IBM Case Manager UI

Dave Perman and Lauren Mayes had the unenviable position of presenting at the end of the day, and at the same time as the expo reception was starting (a.k.a. “open bar”), but I wanted to round out my view of the new Case Manager product by looking at how the user interfaces are built. This is all about the Mashup Center and the Case Manager widgets; I’ve played around with the ECM widgets in the past, which provide an easy way to build a composite application that includes FileNet ECM capabilities.

Perman walked through the Case Manager Builder briefly to show how everything hangs together – or at least, the parts that are integrated into the Builder environment, which are the content and process parts, but not rules or analytics – then described the mashup environment. The composite application development (mashup) environment is pretty standard functionality in BPM and ACM these days, but Case Manager comes with a pre-configured set of pages that make it easy to build case application UIs. A business analyst can easily customize the standard Case Manager pages, selecting which widgets are included and their placement on the page, including external (non-Case Manager) widgets.

The designer can also override the standard case view pages either for all users or for specific roles; this requires creating the page in the mashup environment and registering it for use in Case Manager, then using the Case Manager Builder to assign that page to the specific actions associated with a case. In other words, the UI design is not integrated into the Case Builder environment, although the end result is linked within that environment.

Mayes then went through the process of building and integrating 3rd party widgets; there’s a lot of material on the IBM website now on how to build widgets, and this was just a high-level view of that process and the architecture of integrating between the Mashup Center and the ACM widgets, themes and ECM services on the application server. This uses lightweight REST services that return JSON, hence easier to deal with in the browser, including CMIS REST services for content access, PE REST services for process access, and some custom case-specific REST services. Since there are widgets for Sametime presence and chat functionality, they link through to a Sametime proxy server on the application server. For you FileNet developer geeks, know that you also have to have an instance of Workplace XT running on the application server as well. I’m not going to repeat all the gory details, but basically once you have your custom widget built, you can deploy it so that it appears on the Mashup Center palette, and can be used like any other pre-existing widget. There’s also a command widget that retrieves all the case information so that it’s not loaded multiple times by all of the other widgets; it’s also a controller for moving between list and detail pages.

This is a bit more information that I was counting on absorbing this late in the day, and I ducked out early when the IBM partner started presented about what they’ve done with custom widgets.

That’s it for today; tomorrow will be a short day since I fly home mid-day, but I’ll likely be at one or two sessions in the morning.

TIBCO ActiveMatrix BPM In Depth

Back in May at TIBCO’s TUCON conference, I had a chance for a briefing on their new ActiveMatrix BPM, although not a full demo. Since then, however, I’ve had about three hours of demo sessions with Roger King to see more of what they’re doing with ActiveMatrix (which I will refer to as AMX) BPM and Silver BPM, the cloud version. King has been at the core of TIBCO’s BPM products for a long time: he was at Staffware long before their acquisition by TIBCO, and now heads up BPM product strategy and product management, so is in a position to not only give a thorough demo, but to provide context for where they’ve come from and where they’re going in BPM. He also does a mean Austin Powers imitation, which is kind of funny when you consider that he’s a Brit imitating a Canadian who is imitating a Brit, all for the amusement of a Canadian. 🙂

AMX BPM Architecture

ActiveMatrix BPM ArchitectureThe AMX BPM platform has five components – Workspace, Work Manager, Event Collector, Process Manager, and Openspace – wired together using public services, which allows anyone to use those same services to replace or augment the components. In fact, the BPM platform and AMX BPM applications are all just AMX composite applications.

There is a common administration interface for all AMX products, and when we’re in the AMX Admin interface, the common nature of the platform becomes more evident: you can see the AMX platform, AMX BPM, sample applications and organizational models that we’ll be looking at in the demo. Selecting the AMX BPM application, you’ll see the underlying five components as well as all public services available.

The AMX service governance and management tools share common components with AMX SOA, but are not included in AMX BPM.

What this new architecture means is that Business Studio can be used to create services using combinations of components, services and other object types, which allows processes to be embedded as part of any application: process is not a separate thing used to orchestrate applications, but part of the application itself.

Organizational Models

Business Studio, as the design/modeling environment for composite model-driven process applications, now supports five model types: process, page flow, organizational, data and form. Organizational models are considered a key starting point: an organizational model becomes a functional overlay on your corporate LDAP, and can include multiple organizations such as trading partners in order to define organizational scope during runtime. Metadata is defined on a per-organization level, which can be used to partition visible information if people from multiple organizations participate in a process at runtime.

Busines Studio organization model privileges assigned to positionOrganizational models allow you to define Positions, which represent roles that can be used for work assignment in a process, e.g., all CSRs or all Life CSRs in the insurance claims scenario that’s provided as their sample organizational model. A position can include capabilities such as language skills, and privileges such as ability to authorize claims, scoped to an organization and assigned at an instance level to a process model. You can also define groups, e.g., Managers, and assign capabilities and privileges with groups, as well as access to system actions. Resources in the organization model can be mapped to LDAP attributes at runtime using queries. Later, during process modeling, positions are used to assign participants to a step in a process model, and to perform capability-based routing.

On the end-user side, the model appears in the Organization Browser within the ActiveMatrix Workspace if the user has permissions to see this: an admin, for example, might be able to see all of the organization models for multiple organizations, see the participants that are part of any position, and view or create the queries that bind the resources from the underlying LDAP. A regular user may see only their own organization’s structure, without the ability to modify the participants in a position or any other resources.

Business Object (Data) Models

Business Studio define business object modelTIBCO isn’t the first BPM vendor to jump on the process data model bandwagon, and they won’t be the last: the link between MDM and process instance data needs to be firmly established so that you don’t end up with data definitions within your BPMS that don’t match up with the other data sources in your organization.

In Business Studio, you can define a business object model – a sort of simplified UML data model – that allows structured data models to be shared within or across packages and applications The business object model can have subclasses as well as data relationships, and supports a hierarchy of data models. Even better, you can import it from UML, XSD, WSDL or directly from a database, and can export it to XSD or WSDL.

A business object model is then attached via an external reference to a process model to become part of the process instance data. Access control (in/out/bidirectional) can be set for that business object at each step in the process, as with any other instance parameter, in order to control whether the process step can read and write to the instance data defined by the business object. If you create scripts (which are Javascript in AMX BPM) to manipulate the instance data, the script editor will look up attributes and type directly from the business object model, which means that it’s fully-accessible instance data, not a blob that you need to decode manually.

Page Flow Models

One optimization that BPM vendors are starting to learn from application development environment products is the idea of screen flow: a stateless mini process model that runs in the web tier to handle a series of screens/pages for a single user in a monolithic operation. Think of it as the underpinnings for a wizard-type interface, where the user is stepped through multiple pages of information and data entry in order to complete a single task in a process. This is similar to the capabilities emerging in web application platforms such as NetSuite’s SuiteFlow, Salesforce’s Visual Process Manager and the process modeling within OutSystems, except in this case, the page flow is what happens at a single step in a larger process model. It contains a lot of the functionality of a process model, including gateways, conditional logic, data access, service calls, etc., but when the page flow completes, it returns to process engine to be sent on to the next step in the orchestration.

Business Studio page flow editorPage flows are another important efficiency improvement in BPM systems architecture; otherwise, you end up doing a lot of dequeuing and enqueuing of work items to and from the process engine, which has a lot of overhead. Some systems have had ways to kludge this for many years by binding together several sequential steps in the process model, such that the work item was dequeued at the beginning of the first step in the group, then not enqueued again until the end of the last step. In the case of TIBCO AMX BPM, the page flow looks almost like a subprocess call, with the monolithic step shown in the process map, then the steps in the page flow shown in a limited version of what looks like the process modeler. The alternative to using something like a page flow in any BPM environment is to build that capability into the user interface for that process step, which tends to make the UI very complex, and embeds far too much of the business logic (including routing logic) into the UI rather than in the process and data flow models.

An interesting use case that we discussed was to create a page flow for the start node of a process and expose that as a business service: since the start node page flow can include steps, forms and data access/validation before instantiating the (stateful) process, it provides an environment for initial data gathering at the start of a process without having to use some other web form to gather the data necessary to kick off a process instance. This can then be published as a business service to call from other appslications and environments, wherever you want the process to be invoked.

User Interface Form Models

User interface forms can be assigned to each step in a process model, and will be auto-generated for the specific task based on the process instance data, which could be a business object model. The default form that is generated – and will be rendered in General Interface to preview the user interface at that step – isn’t that pretty, but it’s fairly straightforward to then pretty it up in the forms editor, adding master-detail structures and different control types, and associating objects with CSS classes for styling.

Although you may not end up using these auto-generated forms without some amount of customization, they remain invaluable for rapid prototyping of processes.

It’s also possible (although only feasible in simple cases) to specify no form URL attached to a process or page flow step, which will generate a form at runtime based on the process instance data exposed at that step.

Process Patterns in Process Models

We didn’t spend much time on the Business Studio process modeler, since there is a lot of common functionality with most other process modelers (although Roger will cringe to hear me say that), but just focused on a few new features.

One of these features is a set of predefined process patterns, similar in functionality to what you might do with business rules, but more visually associated with the process model. We looked at two of these: chaining, where subsequent human tasks for same item are worked by same person if they are capable, but can be assigned to others if the first participant can’t perform all of the tasks; and separation of duties, where subsequent tasks (e.g., work and QA) must be done by different people.

At the time that I saw the demo, chained events were visualized as embedded subprocesses, and patterns couldn’t be combined; these need to be fixed in order to make the patterns more visually accurate as well as more functional.

ActiveMatrix Workspace

AXM BPM new claims work items in OFFERED stateThe ActiveMatrix Workspace is the web-based end-user environment, with views of the work available to the current user as well as views of the organizational model as discussed above. Taking a look at the work views, however, is where we see some of the first major changes to the underlying process engine: there are no more physical queues; rather, what appear as logical queues (or work lists) to users are created with dynamic queries. In the old days – and in the old architecture of iProcess and some of its competitors – segregating work items into explicit queues was necessary for performance reasons; today, faster hardware and in-memory software allows this to be done dynamically. When I think of all the times that I’ve have to work around queuing mechanisms in BPM systems, both for work assignment and reporting purposes, this is huge since it provides much easier slicing and dicing of the work items. For example, a standard work view for a supervisor would be “Supervised Work”, which shows all work items that are both available but not yet assigned to a user, plus those that have been allocated to specific users: in effect, a view of the unassigned and assigned work in progress, regardless of its logical queue location. Now, you could argue that queues in most BPMS are usually implemented as database constructs anyway, which is mostly true; however, access to work except by queue is often not very efficient in these queue-oriented systems.

In addition to the Work Views and Organization Browser, the Workspace includes the following user views:

  • Business Services, which is a user view of process-based composite applications, so that user doesn’t think about kicking off a process, but about doing a business function. In the sample insurance scenario that we were looking at, this included functions such as “First Notice of Loss Notification”.
  • Process Views, which shows a view of the executing processes.
  • Event Views, which shows an administrator-like view of login/logout and other system events, plus events related to process instances such as access and update. This is tied closely to the Process Views, which links back to the Event Views in order to show instance events.

We walked through the invocation of a business service in the Business Services view, which essentially kicked off a process instance. However, the first two steps of the “process” were actually screens in a page flow: the simple data entry screens were auto-generated, and the process wasn’t actually instantiated until after the Submit button was clicked on the second screen. We followed the process through the remainder of the steps, which showed off most of the capabilities that we had seen in Business Studio during design:

  • Looking at the Work View, the new claims show up in an OFFERED state (meaning that they aren’t yet claimed for work by a participant) in all CSRs’ work lists. Each user’s view of the work list depends on their group membership, so it completely dynamic.
  • Data is entered by the CSR for the work item to record the claim; the UI in this case was generated from the business object, including multi-entry field types such as multiple claimants for one claim.
  • A language filter based on capability is used to assign work for the Customer Interview step; this step also used a 2-screen page flow to gather information while on the phone with the customer.
  • The work item is processed through the Assess Claim step, and that work item is excluded from same user for the Pay Claim task using the Separation of Duties process pattern.

TIBCO Silver and Silver BPM

TIBCO’s big splash at the TUCON conference this year was around Silver, their cloud environment based on Amazon EC2, and specifically Silver BPM, their cloud-based BPM offering. This is not a cloud-based process discovery or modeling tool; this is the actual processes executing in the cloud.

Setting Silver BPM apart from some of the other cloud BPM environments is the shared design tool, Business Studio, that can connect simultaneously to both on-premise AMX environments and Silver environments. The same models, applications and other services built in Business Studio can be deployed to either environment without having to export and import them into another design tool. This immediately suggests an obvious use case of Silver for development and testing of process applications, even if they are going to be deployed on premise, since each member of the development and test teams could have their own virtual AMX servers, if desired. Although there is no guarantee of the ActiveMatrix and Silver BPM platforms providing exactly the same functionality at any point in time, I would guess that TIBCO will have to keep them fairly closely aligned in order to support this dev/test use case. IBM uses exactly this strategy for marketing their Smart Business Dev/Test environments (although not for any of their BPM products), and the potential cost reductions for these non-production environments is starting to resonate with enterprise clients, not just smaller businesses. That’s not to say that you can’t use Silver BPM for production environments, just that I am imagining that many larger enterprises won’t yet trust the cloud for core production capabilities.

Although EC2 is underneath Silver, the customer only needs to set up their Amazon EC2 account, enable the EC2, S3 and RDS services, and pay their Amazon bill at the end of the month: all other provisioning and administration is done through the Silver Center console. Silver customers are on a one-year or three-year subscription license with TIBCO (as usual, I forgot to ask about pricing) that is an additional cost above the Amazon services.

Silver BPM create a new instance - select softwareFrom within Silver Center, you can create a new Silver BPM instance using the following steps:

  1. Name the instance, then select the product group (e.g., Silver BPM) and software version.
  2. Enter the name and passwords for the database administrator, user and sample user accounts.
  3. Specify the underlying EC2 properties: size/memory configuration, geographic zone, some security parameters and whether to use an elastic (fixed) IP address.

This will start a new EC2 instance that combines a basic Linux image with the selected Silver BPM software image, which takes about 20 minutes to provision and boot the virtual server.

Once up and running, the Silver Administrator and Silver Workspace look amazingly similar to their on-premise ActiveMatrix counterparts: if I didn’t look at the top banner or the URL, I would not have been able to distinguish them. Even from within Business Studio, once you set up the connection parameters, a Silver BPM server looks pretty much like an AMX server, and you can connect to multiple of each type from Business Studio. Business Studio Silver server connectionThen, you deploy models to any of the servers by dragging and dropping them onto the server.

There are some obvious holes in the current Silver BPM offerings. First, they have cloud-based execution but no cloud-based discovery or modeling: Business Studio is an Eclipse-based desktop application, with models stored in a local workspace and shared primarily through the use of a Subversion repository (more commonly used by developers for source code control). This is not process discovery/modeling for the masses, and doesn’t contain the collaborative features such as interactive sharing of process models that we see emerging in many other tools.

Second, there is no multi-tenancy, but only a 1:1 relationship between Silver BPM and EC2: an EC2 instance can contain only one Silver BPM instance, and a Silver BPM instance cannot span multiple EC2 instances. In the future, they will allow for 1:many and many:1, but for now, this is really more like dedicated cloud resources rather than multi-tenancy hosting.

Third, there is an LDAP server created within the Silver BPM instance, but you need to manually upload your enterprise LDAP and use appropriate tools in order to keep it in sync. You can, however, use your corporate LDAP if you use Amazon Virtual Private Cloud.

TIBCO’s not the first to provide process execution in the cloud – Appian, Fujitsu, Intalio, Cordys and others have come before them – but having a big BPM name like TIBCO enter the cloud BPM market helps to further validate the use of the cloud for process execution. In fact, I’ve had a discussion with one of the more established cloud BPM vendors since my briefing with TIBCO, and I advised them not to think of TIBCO only as a competitor, but as a collaborator helping to expand this market. I don’t imagine that we’ll see many Silver customers before next year’s TUCON, but TIBCO will be out there building the cloud mindshare.


We finished our marathon demo sessions with a look at AMX openspace, TIBCO’s gadget-based container application for building composite user interfaces. In short: a mashup builder.

The standard AMX/Silver Workspace is fine for some users, while others need a highly-customized UI created by developers for advanced functionality; there is, however, a third category that prefer to roll their own user experience. I first blogged about the potential for enterprise mashups in early 2006, and attended the first MashupCamp in 2006 (and the second one in 2007), so it’s no surprise that I’ve been impatiently tapping my foot waiting for many of the larger BPM/enterprise software vendors to catch up with this trend. True, the customers aren’t there yet, but they’re not going to get there on their own, they need to see mashup/portal builders in action to recognize the value that these tools can bring to experienced users and administrators.

AMX openspace with process monitoring gadgetsThe basics: openspace supports a number of different gadget standards, including Shindig, OpenSocial, iGoogle and GWT, and uses a publish-subscribe model for inter-gadget communication. Gadgets are arranged on tabs/pages within the openspace application, and can be added from the predefined palette of gadgets, from the gadget store hosted on the server, or by specifying the URL and any required parameters for external gadgets such as Google Maps. Each gadget can be themed independently, although that just seems like a recipe for a graphic design nightmare. Once created, openspace pages can be published for other users to use.

This ties back into AMX BPM with the availability of BPM gadgets: process participation gadgets such as Business Services and ToDo List, basic monitoring and reporting gadgets for end users and supervisors, and Spotfire gadgets for more advanced analytics and visualization. You have to be licensed for all the underlying software such as AMX BPM and Spotfire; this just provides a new way to visualize and interact with those tools.


If you read my previous post on AMX BPM, you already know that I was impressed with what I saw. There are a number of BPM products launched around the same time as Staffware’s product (which became TIBCO’s iProcess) in the early 2000’s, and although most of these need a complete architectural overhaul to bring them up to date, this is the first that I’ve seen of a vendor biting the bullet: AMX BPM is not an iProcess upgrade; it’s a completely new next-generation BPM product. Although they gain immensely in product capabilities, they are leaving behind their existing iProcess customers, although I’m sure that they’ll offer some sort of migration assistance. There are unlikely to be many enhancements to iProcess in the future, but there are no plans for end of life, and it is expected that many current customers will keep their legacy applications on iProcess while looking to AMX BPM for new applications and functionality.

With AMX and Silver, TIBCO have leapfrogged many of their newer competitors in terms of architecture and functionality. Their challenges lie in shifting their existing customers’ new development onto AMX or Silver, and convincing potential customers that they’re no longer part of the BPM old guard.

TIBCO ActiveMatrix BPM - 2010

Metastorm M3 Demonstration

I had a briefing on Metastorm’s M3 collaborative modeling and Smart Business Workspace two weeks ago, and last week we had a follow-up demo. This is the start of a push towards a full BPM suite in the cloud, providing collaborative process modeling and the end user runtime hosted on Azure, but Microsoft still needs to add some planned functionality to Azure in order to allow Metastorm to move the BPM engine there as well. When that happens, however, the Azure Fabric Connector will allow the BPM engine to connect to on-premise systems and data sources, regardless of whether the Azure instance is on-premise or hosted elsewhere.

We first walked through M3, which provides self-registration for a modeling account. This isn’t just process modeling, however; based on their Provision acquisition, there are 11 different types of models available: Workflow, Organization, Goal, Location, System, Capability, Activity, Deliverable, Project, Requirement and Rule. Although I have seen multiple model types in some of the other collaborative modeling tools – such as strategy and capability maps in IBM’s BPM BlueWorks, this goes beyond that in scope, and has a more robust backing of the ProVision metamodel, allowing the models to be exported from M3 and imported into the full version of ProVision. It’s also possible to create associations between different model types: for example, linking an activity in a workflow model with a measurement or location. Models can be exported in ProVision’s CIF (Common Interchange Format) only, although there are tools to transform a process model in CIF to XPDL or BPEL.

We also viewed a sharing session, which is a synchronous collaboration of two or more people that allows for interactive whiteboarding and chat. Although users in an interactive whiteboarding environment will more likely use telephone as their primary communications tool rather than chat, the chat is useful because it is logged as part of the session history, so can be used to record decisions and notes. A shared session can be played back using a VCR-like control to see how a model evolved over the session.

M3 provides extensive help for modelers, including best practices and strategies for modeling, and will continue to be augmented with feedback from the online Metastorm community. There’s not a direct link to that community, which would be useful; it seems like some of the best practice sections in the help have just been copied from the community site, not directly linked.

Metastorm M3 - 2010

The second part of the demo was on Smart Business Workspace, Metastorm’s Silverlight-based composite application development (mashup) environment. Except for the fact that it’s based on Silverlight (which may not be considered an advantage in some circles), there’s not much different here than most other mashup environments except for the inclusion of their own BPM and model widgets. There’s a role-based starting point for the workspace, and pages can be fully personalized if the user has the appropriate permissions. Widgets are dragged on from a predefined palette, and can be dynamically sized and the general page layout changed. Administrators and page designers can lock down specific pages and widgets for a more controlled environment. Depending on the type of widget, there is publish/subscribe wiring between the widgets to allow for standard use cases such as list-detail or map display of data. Branding and general appearance of the workspace can be styled with CSS and .Net resource files.

Smart Business Workspace allows you to add any Silverlight widget, but does not support other widely-used widget standards. Although you can add any webpage as a “personal widget”, these are really more like unwired portlets than true widgets; you’ll have to use the widget designer to turn something into a first class widget.

Metastorm Smart Business Workspace - 2010

Metastorm New Releases: Collaborative Modeling with M3 and Smart Business Workspace Application Builder

Although we didn’t have a chance for a demo, I had a quick briefing with Greg Carter, Metastorm’s CTO, on the announcements that they made today.

M3 Collaborative Modeling

We discussed M3, their cloud-based collaborative process modeling tool. This is one of the first BPA/BPM offerings that I’ve seen on the Microsoft Windows Azure platform, allowing for a variety of hosting options: private hosted, private on-premise or public cloud, all accessed via a browser. Unlike many of the cloud process modeling tools available that offer fairly simple workflow and interaction models, Metastorm has taken advantage of the rich modeling capabilities from the Provision acquisition and are offering multiple model types, objects and viewpoints for more complete business models. Their rich object-based models allow more information to be attached to objects in addition to more model types and the ability to associate objects with multiple model types: for example, a goal model will include the definition of measurements, and those same measurements can be associated with a related process model.

Their press release lists the model types available:

Modeling capabilities for 11 different model types: goal, organization, capability, system, process/workflow, activity, rule, project, requirement, location and deliverable; providing 23 modeling objects and viewpoints beyond just a process model to deliver more comprehensive, holistic business improvements that factor in a full range of enterprise assets

Metastorm M3 Sharing SessionM3 also includes a number of collaboration features: you can share models (with different levels of access control) with other users, and have an interactive modeling session between multiple users. The entire session is available for playback, including any chat conversations that occurred during the session.

The big question is how this will fit into their existing portfolio of modeling tools: ProVision EA for full enterprise architecture modeling; Provision BPA, a subset of EA that includes 12 models used for process modeling; and the process designer within Metastorm BPM. These are all desktop applications without explicit collaboration capabilities, providing a greater range of modeling and analysis functionality than M3: ProVision has more model types, plus analytics and simulation, and the BPM process designer includes service and integration management.

Metastorm’s answer to this is that the three products (if you consider ProVision EA and BPA as a single product) are targeted at three different personas: M3 is for the casual, untrained user that needs to do some lightweight modeling and process discovery; ProVision is for the trained analyst or architect to do more comprehensive analysis and optimization of the models; and BPM is for the developer to hook up all the technical underpinnings to the model in order to make it executable. That’s a reasonable split of capabilities, although there are a couple of issues with that: model portability/round-tripping, and vastly different user experiences. They are approaching the model portability problem by moving towards directly sharing models in a common repository; this will require that BPM be modified to use the ProVision metamodel (M3 already uses the same metamodel), then bring them together into a shared repository. Once that is done, a model could be started in M3, then worked on in ProVision and/or BPM directly. As for the multiple user interfaces, Carter said that they would likely deprecate one of the modelers in the future; I would expect to see the BPM process designer replaced by additional functionality in ProVision, for example, although he didn’t say that.

The Azure platform provides a multi-tenant environment that can be run in the public cloud, or as a private cloud either hosted or on-premise. Process models created in the public cloud version can be shared with an on-premise version of ProVision; in the future, this will be done using Azure’s app fabric to make it more seamless. Microsoft is also implementing features in Azure that allow for data location control, so that specific data objects can be defined as being held in a specific geographic region.

The public cloud version of Metastorm M3 is free to existing Metastorm customers in 2010, and subscription pricing will be introduced in 2011. The M3 product page includes a “Register for an Account” link at the bottom if you want to try it out.

The market place for collaborative process modelers is very busy right now, with long-time players like IBM (Lombardi) Blueprint and a raft of more recent entrants, but I wouldn’t call the market mature at this time: there is definitely room for new players, especially if they can offer a richer modeling experience such as Metastorm is claiming to provide.

Smart Business Workspace

We also discussed Smart Business Workspace (SBW), Metastorm’s entry into the rich internet application/mashup development area. Based on Microsoft Silverlight, it provides an environment for integrating Metastorm applications and any other applications into a common user environment. It’s not just for creating standard mashup pages, however: driven by metadata, it can change the environment based on the user’s role, skills and other information.

Metastorm SBW Multiple Layouts

Metastorm publishes the specifications for widgets that can be used in SBW, and provides a widget designer, but doesn’t interface with JSR168, Google widgets or other existing standards. So although you can, in theory, integrate any application or data, it’s likely going to take a bit of work.

Although it seems like M3 and SBW are completely different products, they’re actually quite closely related: M3 is based on SBW, so all the hosting and collaboration features discussed for M3 can be generalized to any SBW application: public and private cloud, plus chat and shared whiteboard sessions.

This is Metastorm’s start to creating a workspace in the cloud. Although SBW based in the public cloud can’t yet make a connection to on-premise Metastorm BPM, that will be available in a future release. Also, there are plans to move Metastorm BPM into the cloud using Azure, making it easy for business process outsourcers to offer process as a service.

Using Business Space for Human Workflows

Back to the breakouts for the rest of the afternoon, I attended a presentation and demo by Michael Friess of IBM’s BBlingen R&D lab on using Business Space to build user interfaces for human-centric processes.

Business Space is what I would call a mashup environment, although I think that IBM is avoiding that term because it just isn’t taken seriously in business; in other words, a portal-like composition application development environment where pre-built widgets from disparate sources can quickly be assembled into an application, with a great deal more interaction between the widgets than you would find in a simple portal. Business Space is, in fact, built on the Lotus Mashup Center infrastructure; I think they just prettied it up and gave it a more corporate-sounding name, since it bears a resemblance to the Lotus Mashup Center version that I played with a while back with the FileNet ECM widgets. It’s browser-based and is fairly clean-looking, with easy placement, resizing and configuration of widgets.

Friess considered both “traditional” (predefined structured) and dynamic human BPM, where the dynamic side includes collaboration, allowing the user to organize their own environment, and adaptive case management. Structured BPM typically has fixed user interfaces that have a specific mode of task assignment (get next, personal task list, team task list, or team-based allocation). Business Space, on the other hand, provides a semi-structured framework for BPM user interfaces where the BPM widgets can be assembled under the toolbar-like links to other spaces and pages; the widgets use REST interfaces to back-end IBM services such as WPS, Business Compass, Business Monitor, Business Fabric and ESB, as well as any other services available internally or externally via REST. Templates can be used to create pages with standard functionality, such as a vanilla BPM interface, which can then be customized to suit the specific application.

Each widget can be configured for the content (which tasks and properties are visible and editable to the user), the actions available to the user, and the display modes such as list or table view. Even if a specific user isn’t allowed to choose the widgets that appear on the page, they typically will have the ability to customize the view somewhat through built-in (server-side) filtering and sorting.

Once widgets are placed on a page and configured, they are wired together in order to create interactions between the widgets: for example, a task list widget will be wired to a task details widget so that the item selected in the list will be displayed in the details view.

There are a number of BPM widgets available, including task list, task details, escalations list, human workflow diagram (from the process model, which will change to indicate any new collaboration tasks) and even free-form forms; these in turn allow any sort of BPM functionality such as spawning a collaboration task. Care must be taken in constructing the queries that underlay the list-type widgets, although that would be true in any user interface development that presents a list to a user; the only specific consideration here is that the mashup may not be constructed by an developer, but rather by a business analyst, which may require a developer to predefine some views or queries for use by the widgets.

If you’ve seen any mashup environment, this is all going to look pretty familiar, but I consider that a good thing: the ability to build composite applications like this is critical in many situations where full application development can’t be justified, especially for prototype and situational applications, but also to replace the end user computing applications that your business analysts have previously built in Excel or Access. Unfortunately, I think that some professional services types feel that mashup environments and widgets are toys rather than real application development tools; that’s an unfortunate misconception, since these can be every bit as functional and scalable as writing custom Java code, and a lot more agile. You’re probably not going to use mashups and widgets for every user interface in BPM, but it should be a part of your application development toolkit.

WebSphere BPM Product Portfolio Technical Update

The keynotes sessions this morning were typical “big conference”: too much loud music, comedians and irrelevant speakers for my taste, although the brief addresses by Steve Mills and Craig Hayman as well as this morning’s press release showed that process is definitely high on IBM’s mind. The breakout session that I attended following that, however, contained more of the specifics about what’s happening with IBM WebSphere BPM. This is a portfolio of products – in some cases, not yet really integrated – including Process Server and Lombardi.

Some of the new features:

  • A whole bunch of infrastructure stuff such as clustering for simple/POC environments
  • WS CloudBurst Appliance supports Process Server Hypervisor Edition for fast, repeatable deployments
  • Database configuration tools to help simplify creation and configuration of databases, rather than requiring back and forth with a DBA as was required with previous version
  • Business Space has some enhancements, and is being positioned as the “Web 2.0 interface into BPM” (a message that they should probably pass on to GBS)
  • A number of new and updated widgets for Business Space and Lotus Mashups
  • UI integration between Business Space and WS Portal
  • Webform Server removes the need for a client form viewer on each desktop in order to interact with Lotus Forms – this is huge in cases where forms are used as a UI for BPM participant tasks
  • Version migration tools
  • BPMN 2.0 support, using different levels/subclasses of the language in different tools
  • Enhancements to WS Business Modeler (including the BPMN 2.0 support), including team support, and new constructs including case and compensation
  • Parallel routing tasks in WPS (amazing that they existed this long without that, but an artifact of the BPEL base)
  • Improved monitoring support in WS Business Monitor for ad hoc human tasks.
  • Work baskets for human workflow in WPS, allowing for runtime reallocation of tasks – I’m definitely interested in more details on this
  • The ability to add business categories to tasks in WPS to allow for easier searching and sorting of human tasks; these can be assigned at design time or runtime
  • Instance migration to move long-running process instances to a new process schema
  • A lot of technical implementation enhancements, such as new WESB primitives and improvements to the developer environment, that likely meant a lot to the WebSphere experts in the room (which I’m not)
  • Allowing Business Monitor to better monitor BPEL processes
  • Industry accelerators (previously known as industry content packs) that include capability models, process flows, service interfaces, business vocabulary, data models, dashboards and solution templates – note that these are across seven different products, not some sort of all-in-one solution
  • WAS and BPM performance enhancements enabling scalability
  • WS Lombardi Edition: not sure what’s really new here except for the bluewashing

I’m still fighting with the attendee site to get a copy of the presentation, so I’m sure that I’ve missed things here, but I have some roundtable and one-on-one sessions later today and tomorrow that should clarify things further. Looking at the breakout sessions for the rest of the day, I’m definitely going to have to clone myself in order to attend everything that looks interesting.

In terms of the WPS enhancements, many of the things that we saw in this session seem to be starting to bring WebSphere BPM level with other full BPM suites: it’s definitely expanding beyond being just a BPEL-based orchestration tool to include full support for human tasks and long-running processes. The question lurking in my mind, of course, is what happens to FileNet P8 BPM and WS Lombardi (formerly TeamWorks) as mainstream BPM engines if WPS can do it all in the future? Given that my recommendation at the time of the FileNet acquisition was to rip out BPM and move it over to the WebSphere portfolio, and the spirited response that I had recently to a post about customers not wanting 3 BPMSs, I definitely believe that more BPM product consolidation is required in this portfolio.

BPM, Collaboration and Social Networking

Although social software and BPM is an underlying theme in a lot of the presentations that I give, today at the Business Rules Forum is the first time that I’ve been able to focus exclusively on that topic in a presentation for more than 3 years. Here’s the slides, and a list of the references that I used:


There are many other references in this field; feel free to add your favorites in the comments section.

IBM FileNet P8 BPM V4.5

I’ve had a couple of briefings on the 4.5 release of IBM/FileNet P8 BPM, which was released in November but is likely just starting to hit customer sites so I thought that it would be good timing for a review post. As a point of disclosure, I worked for FileNet in 2000-1 and have worked with their BPM software extensively in two of my own companies including my current consulting practice, but I don’t do any work for IBM, only for their customers. That means that I am probably more familiar with their system than with any other BPMS, but they are not compensating me in any way for this post (they don’t even cover analyst/bloggers expenses to attend their user conferences, so I don’t) nor do they have any editorial control, which means that I will likely manage to say something to annoy IBM management here, as I usually do.

I’ve blogged in the past about the IBM-FileNet acquisition, specifically my comments at the time that the acquisition was announced, at an analyst briefing just after that, then a follow-up last June comparing it to the Oracle-BEA acquisition: in brief, I noted the transition in product positioning from that of a full BPMS product to document-centric BPMS so as not to compete with WebSphere Process Server. I still think that both IBM and its customers would have been better served by ripping BPM out of the P8 product line and adding it to WebSphere to round out the human-facing capabilities, producing a single BPMS product at IBM. Instead, if a customer wants both human-centric functionality and services orchestration, IBM will be in there selling them two products – each with their own modeling, execution and monitoring environments – rather than one, which is going to be a bit of a hard sell in this economy. They’re working to bring some of that together, but fundamentally it’s still two products to do what many other vendors do with one. There are a few point of integration now — the WebSphere modeler can export FileNet-compliant XPDL, and the WebSphere monitoring tools can monitor the FileNet process engine – and they’ll be doing a bit more cosmetic integration to make it more palatable, but there’s no plan for a unified execution engine. Strangely, the recent Gartner report on BPMS doesn’t both to distinguish them: it bases its analysis on the combination of WebSphere Dynamic Process Edition and FileNet Active Content Edition, which is a bit bogus (in my opinion).

That being said, the current positioning of FileNet P8 BPM is around “agile ECM”, with active content being a key differentiator. Active content, in the FileNet world, is the ability to capture content events (such as creation and versioning) and trigger activities in response, either launching new process instances in BPM, or making external calls. If you’re proficient with the FileNet BPM design tools, that means that you can create a new process, link it via a workflow subscription to the events occurring on a class of content, and have that process automatically trigger when that event occurs for a document in that class. In my world of back-office transaction processing, where there is still a lot of paper, this could be the creation of a process instance in response to a new scanned document being added to the content repository, all without writing a line of code.

IBM FileNet P8 BPM 4.5There’s more to their agile message than active content, however: IBM is also bundling in a new set of BPM widgets and the IBM (Lotus) Mashup Center to allow for the much easier creation of user interfaces. This has always been a problem in the past: although the Process Designer will auto-generate a user interface for each step that allows for view and update of the parameters exposed at that step, it’s not very pretty. The options were to use the FileNet e-forms product – which required some technical fiddling to integrate – or create custom interfaces using some other development tools. Although the widgets don’t provide a fully-customizable forms interface, they do provide a way to put together configurable user screens that work well for prototyping and for some lighter-weight/tactical production applications.

I liked what I saw with the widgets, despite the limitations, since I think that it’s a move in the right direction. They use the iWidget specification, which is an open standard created by IBM and used natively in the Mashup Center, and there’s also a wrapper to turn an iWidget widget into a JSR-168 compliant portlet, with the cross-widget wiring exposed, for use in other environments such as the WebSphere portal product. The BPM widgets are built using the new REST services that wrap around the process engine Java API; you can also call the REST services directly from other application development environments. Although the widgets are referred to as “ECM widgets” in the IBM documentation, they all (with the exception of a document viewer widget) provide BPM functionality. There’s a lot more that I saw about the widgets; I might do a separate post just on that for those who are evaluating this product.

Some partners are also creating widgets for the mashup framework; I can see this as a key way for partners to add value through providing interoperable components rather than monolithic applications, and I would hope to see some of these emerging for free as companies try out this new technology.

There’s no requirement for all-or-nothing with the mashups, either: each step in the process can invoke a different UI from a different source, so that one step might have a custom application, another an e-form and another a mashup. As far as the process is concerned, that’s just what is invoked at the step to manage the user interaction, not an integral part of the process.

One issue is that WebSphere Business Space will replace Mashup Center as the mashup environment included with P8 BPM, although it’s not clear what degree of functional overlap there is, or when to use one versus the other. The Mashup Center appears to be positioned as being for prototyping and tactical situational applications, whereas Business Space is more of an enterprise portal, but it’s not clear that you couldn’t build an enterprise-strength application using the Mashup Center (unless you’re afraid that IT will laugh at you for using the words “mashup” and “enterprise” in the same sentence). Business Space supports the ECM widgets, but would require a few “minor functional changes” (IBM’s words) to get things working.

FileNet BPM Process DesignerOn the process modeling side, the Process Designer now has two modes: diagram mode for business analysts, and design mode for technical analysts, with user access rights determining which that a specific user can access. In diagram mode, the user draws the process map, adds the description and instructions at each step, and a description for each route between steps. Design mode is the full “classic” view, with all parameters visible, where a developer can take the description entered by the business analyst and map that into parameters, rules, assignments, deadlines and web services calls. However, the Designer still is not BPMN compliant: if you want BPMN, you can do it in Visio with a BPMN template that they provide, then import the results into the Designer, but it’s a one-way trip. They do plan to leverage some of what’s been done with BPMN in the WebSphere process modeler to bring that into the P8 BPM designer, but there’s nothing concrete to talk about yet.

There’s also some new user roles functionality built in to the designer (and runtime, obviously) that is based on the Business Process Framework, an add-on product to BPM used for creating case management processes. I suspect that we’ll see more of the useful bits of BPF integrated into the core BPM product in the coming releases, to the point where it won’t exist as a separate product, although no one at IBM said that.

Simulation is now web-based and integrated within the process designer, rather than being a separate application: one of the tabs in the design view of a process is Simulation, which allows durations for steps and weights (%) for routes to be entered. Configuration and administration is also now done within the process designer rather than in a separate configuration console.

For business rules, ILOG (a recent IBM acquisition) is being integrated into the WebSphere suite; since it provides a web services interface, it can easily be called at a step in a BPM process for adding business rules more complex than can be handled by the built-in expression engine in BPM.

The BAM product integrated into the P8 BPM product line is also now IBM: originally it was Celequest, which was acquired by Cognos, which was in turn acquired by IBM; the branding on the last set of product slides that I saw is “Cognos Now”.

IBM is starting to push Lotus Forms with BPM, although it is not yet integrated to the same degree as FileNet eForms, which can replace the user interface at a step in a process. I can’t believe that IBM will maintain two e-forms products in the long run, but they can’t really cut off FileNet eForms until they complete that integration.

Overall, FileNet’s legacy of content and process together has grown into fully-featured document-centric BPM capability. Unfortunately, they positioned themselves as pure-play BPMS just long enough to get some customers on that bandwagon, leaving those customers with some uncomfortable migration decisions in their future.