process – Page 72

Lombardi analyst call

We had a quick update yesterday from Rod Favaron and Phil Gilbert at Lombardi on their third analyst call. This was mostly an H108 update on revenue growth, new customers and new partners/geographies, none of which I usually spend a lot of time on, but I’ll summarize:

85% license revenue increase and overall 50% growth compared to H107, still primarily in the US but with increasing markets in Europe and Asia Pacific
Teamworks deployments in 27 countries
Increasing revenue per customer, indicating broader deployment within a customer
More than 30 new enterprise BPM customers
25% headcount growth, with about 20 positions still open (in case you’re looking)
New partners in South America, Eastern Europe and Asia Pac

We also had a review of product and service updates, most of which I discussed in my coverage of the Lombardi user conference last month:

Teamworks 6 shipped, with major updates in integration, presentation layer and administration capabilities
Upcoming Teamworks 7 release, focused on BPM programs rather than individual deployments
Blueprint summer release completed, including Visio importing, MS-Word output, and linking between processes
Delivering Process Packages (service offerings announced on the last analyst call), with the process analysis offering being particularly popular

Phil teased us with talk about a BPM training and certification program that they’ll be announcing later this year, but I guess I’ll have to wait until October for that.

The Q&A, not surprising given the “review of the old stuff” rather than “exciting announcements about new stuff”, had little to do with the content of the preceding presentation, and more to do with the general BPM market and Lombardi’s view of their competitors.

IBM to acquire ILOG

IBM and ILOG announced today that IBM will be acquiring ILOG for €10/share, or about $US340 million in total.

IBM’s goal is to integrate ILOG’s business rules technology into their existing BPM and SOA offerings:

When completed, the acquisition of ILOG will strengthen IBM’s BPM and SOA position by providing customers a full set of rule management tools for complete information and application lifecycle management across a comprehensive platform including IBM’s leading WebSphere application development and management platform.

The funny part is that the IBM press release take two paragraphs to explain what BPM is, and how business rules are used in the context of BPM, indicating just how niche these technologies still are in the broader business scope.

This may not be good news for ILOG’s other BPM partners; one less independent BRMS company means less choice when it comes to putting your processes and rules together.

Savvion’s Super-charged Partner Program

At the Gartner BPM summit in February, I met up with Dr. Ketabchi, the founder and CEO of Savvion. Shortly before that, Savvion had axed most of their marketing department, and I was eager for the bigger picture since I knew that a lot of those people were very good at their jobs. As he explained at the time and again in a recent briefing, the reason for this shift in team was a shift in direction, under wraps until now, towards much more of a solutions focus. They’ve done a pretty significant internal reorganization, and completely reset their strategy and goals.

Ketabchi believes that the potential for BPM is much larger than is now being achieved: many vendors have been focused only on the core BPM services of model/design, execute and monitor. The extended BPM space surrounding that includes additional tools such as process libraries and business rules — similar to the Gartner definition of a comprehensive BPM suite — and a still-larger ring of vertical applications built on that extended platform. The value to the customer with generic components is limited; domain knowledge is required to increase that value through the vertical applications.

To serve these needs, Savvion is extending their BPM platform both horizontally and vertically. First of all, version 7.5 (shipping in August) will include the following enhancements:

Business process library
Multi-channel routing engine, allowing for the inclusion of multiple interaction channels including voice, fax and internet
Model-monitor-improve cycle
Content management, either through a direct bundling of Alfresco (open source), or through links to Documentum or IBM-FileNet
Business rules, with their own rules engine, allowing processes to invoke rules but also (CEP-like), rules can initiate processes
Multi-tenanted to allow for on-demand BPM

The new release will include the Business Process Center, where all process assets are managed: models, operational procedures, references and everything else needed to build and execute a business process.

There’s also some interesting project management-like visualizations, for bringing together the concepts of project portfolio management and BPM. This tabular view provides an alternative representation of the process that make it easier to pinpoint critical paths and areas requiring optimization.

Vertically, they’ll extend with on-demand and on-premise business applications:

Accounts payable
Telecom order processing
Clinical trial management
Mobile asset management
Customer on-boarding

This vertical expansion is more challenging than horizontal expansion, since Savvion doesn’t have the necessary domain knowledge, so that’s where their new Business Solutions Alliance Program comes in: they’re partnering with resellers, customers and other companies that do have domain knowledge in a sort of crowdsourcing development model. Savvion is building the application platforms, then vertical applications are built on top of the frameworks; since they applications share a common framework, they will be better integrated. But Savvion provides more than a technology framework: they’ll provide sales and marketing, support and a host of other services, while the partners develop the thin, uppermost application layer using their domain expertise, and provide an entry point into the vertical market through their existing customer contacts. This allows the Savvion-centered ecosystem to target a larger portion of the vertical market, since vertical applications can be developed more easily, while still providing a fairly consistent and well-integrated set of applications built on a common framework.

They visualize this application development ecosystem as a mindmap, with the foundation (first level from the root) being developed by Savvion, and the “leaf” applications being built by partners. The goal is to make the leaf-level development as much of a cookie-cutter process as possible, and Savvion is expecting — possibly optimistically — to bundle and integrate the applications within a branch into a cohesive suite, even if built by different partners. Since the new core engine will be multi-tenanted, these applications can be hosted either by Savvion or their partners.

When a vertical solution ecosystem works well, it’s a win-win all around: the vendor makes more sales of the underlying platform, the partners leverage their unique knowledge to build and sell reusable apps rather than just selling services, and the customers can buy mostly off-the-shelf solutions at a lower price point than custom solutions. It’s an interesting model for a partner program: leverage domain knowledge from partners, whether professional services firms or customers, without them having to be crack application development shops or have the capabilities to market and sell the solutions. It’s also a pretty big gamble for Savvion, who are now betting the whole farm on this vertical partner-assisted model instead of the more common horizontal model that we see in the BPMS marketplace.

Appian funding

Appian, a self-funded company until now, is taking on $10M in venture from Novak Biddle, who primarily fund IT startups. Appian plans to use the funding in four areas:

On-demand: enhance their Appian Anywhere on-demand BPM platform, including the application marketplace.
Channel and vertical applications: work with the partner ecosystem to build vertical applications in areas where the partners specialize.
Sales and services: expand geographically, particularly in EMEA and APAC.
Marketing: invest in targeted brand marketing and improve technology partnerships.

I took part last week in an analyst call with Matt Calkins, Appian’s CEO, and Samir Gulati, VP Marketing, about the announcement. They’ve generated $140M in revenue over the 9 years that they’ve been in business, so the big question is why go for VC now?

Calkins sees this as a way for them to move up the food chain, not just competing with vendors their own size (he mentioned Lombardi and Savvion specifically), but taking on the BPM behemoths like IBM and Oracle. The marketing investment is a big part of this, since you can’t play with the big kids unless you look the part. They’ve doubled their marketing budget and added 15 people to their professional services team, and Calkins stated that “we now have the budget to be the company that we wanted to be”. When I asked if this was the first step on the road to an IPO, he confirmed that they were moving towards that goal.

The VCs that they spoke with were impressed by what they’ve done, coming a bit late to the BPM market but being opportunistic and building a leadership position while still generating significant revenue. $10M doesn’t sound like a lot these days, especially given the ambitious scope of their plans, but they have a reputation for doing good things on a shoestring and hopefully the influx of cash won’t turn their heads. Remember, this is the conservative east coast, not the excesses of the Valley.

In response to a question about the difficulty in moving from selling software licenses to selling on-demand software, Calkins responded that they’re creating a separate Appian Anywhere sales force and developing a sales model that will compensate them based on projected 3-year revenues, which makes it easier to compare the on-demand and enterprise sales teams. They plan to shift to a SaaS “metabolism” for Appian Anywhere, allowing for frequent code releases in the on-demand platform, while only occasionally freezing that code base for the Enterprise edition. This keeps the two versions in sync, but provides a functionality advantage to the Appian Anywhere customers, who will see the new features earlier.

Business Rules Management and Business Process Management: Turning Policies into Action

Recently, I wrote a white paper for Corticon Technologies on the synergy between BPM and BRM. From the introduction:

The mantra of today’s business environment is “build for change”, driving many process improvement initiatives. Businesses must realize, however, that the decisions within the business processes are at least as critical in the search for agility, since the decisions change more frequently than the processes. Combining business rules management and business process management provides that agility by allowing the decisions, and their underlying rules, to be changed independently from the processes, often in real-time by business managers.

This white paper examines the intersection of business rules management and business process management: what they are, how they interact, and why this is important to the agility, accuracy, cost and compliance of your business processes.

You can find the white paper on their site here (available as a link from their Solutions / Business Process Management page), no registration required.

Do BPM vendors eat their own dogfood?

I really dislike that expression, but it’s commonly recognized to mean that a company is using its own products to run its own business. I believe that a lot of BPM vendors do use their own products in some way, but how much? Are they just playing around with expense approvals, or have they drunk the process Kool-Aid and embedded BPM within their critical business processes?

One company that does appear to be taking their own sales pitch to heart is Appian, who just published a case study about themselves (you can find it linked at the bottom of their About Us page, but it requires registration – tsk, tsk). They use it for departmental and enterprise-wide applications, and have 40-50,000 process instances running at any given time across Finance, HR, Sales, Support, Marketing, Product Management, IT and Employee Development. Furthermore, all of the business applications are developed by business people, not IT, and can be changed in flight by business people using the rules capability within Appian Enterprise.

By implementing BPM internally, which included moving some functionality out of Peoplesoft and into Appian, they’re saving about $500k in hard costs each year, primarily through reduced licensing and support costs for packaged applications. I’m sure that there’s also soft cost savings in terms of improved efficiency and productivity, although they haven’t stated those.

I’m curious to hear from other BPM vendors about their own internal case studies — add your comments with your experiences.

Oracle BEA Strategy Briefing

Not only did Oracle schedule this briefing on Canada Day, the biggest holiday in Canada, but they forced me to download the Real Player plug-in in order to participate. The good part, however, is that it was full streaming audio and video alongside the slides.

Charles Phillips, Oracle President, kicked off with a welcome and some background on Oracle, including their focus on database, middleware and applications, and how middleware is the fastest-growing of these three product pillars. He described how Oracle Fusion middleware is used both by their own applications as well as ISVs and customers implementing their own SOA initiatives.

He outlined their rationale for acquiring BEA: complementary products and architecture, internal expertise, strategic markets such as Asia, and the partner and channel ecosystem. He stated that they will continue to support BEA products under the existing support lifetimes, with no forced migration policies to move off of BEA platforms. They now consider themselves #1 in the middleware market in terms of both size and technology leadership, and Phillips gave a gentle slam to IBM for over-inflating their middleware market size by including everything but the kitchen sink in what they consider to be middleware.

The BEA developer and architect online communities will be merged into the Oracle Technology Network: Dev2Dev will be merged into the Oracle Java Developer community, and Arch2Arch will be broadened to the Oracle community.

Retaining all the BEA development centers, they now have 4,500 middleware developers; most BEA sales, consulting and support staff were also retained and integrated into the the Fusion middleware teams.

Next up was Thomas Kurian, SVP of Product Development for Fusion Middleware and BEA product directions, with a more detailed view of the Oracle middleware products and strategy. Their basic philosophy for middleware is that it’s a unified suite rather than a collection of disjoint products, it’s modular from a purchasing and deployment standpoint, and it’s standards-based and open. He started to talk about applications enabled by their products, unifying SOA, process management, business intelligence, content management and Enterprise 2.0.

They’ve categorized middleware products into 3 categories on their product roadmap (which I have reproduced here directly from Kurian’s slide:

Strategic products

BEA products being adopted immediately with limited re-design into Oracle Fusion middleware
No corresponding Oracle products exist in majority of cases
Corresponding Oracle products converge with BEA products with rapid integration over 12-18 months

Continue and converge products

BEA products being incrementally re-designed to integrate with Oracle Fusion middleware
Gradual integration with existing Oracle Fusion middleware technology to broaden features with automated upgrades
Continue development and maintenance for at least 9 years

Maintenance products

BEA had end-of-life’d due to limited adoption prior to Oracle M&A
Continued maintenance with appropriate fixes for 5 years

For the “continue and converge” category, that is, of course, a bit different than “no forced migration”, but this is to be expected. My issue is with the overlap between the “strategic” category, which can include a convergence of an Oracle and a BEA product, and the “continue and converge” category, which includes products that will be converged into another product: when is a converged product considered “strategic” rather than “continue and converge”, or is this just the spin they’re putting on things so as to not freak out BEA customers who have put huge investments into a BEA product that is going to be converged into an existing Oracle product?

He went on to discuss how each individual Oracle and BEA product would be handled under this categorization. I’ve skipped the parts on development tools, transaction processing, identity management, systems management and service delivery, and gone right to their plans for the Service-Oriented Architecture products:

Strategic:

Oracle Data Integrator for data integration and batch ETL
Oracle Service Bus, which unifies AquaLogic Service Bus and Oracle Enterprise Service Bus
Oracle BPEL Process Manager for service orchestration and composite application infrastructure
Oracle Complex Event Processor for in-memory event computation, integrated with WebLogic Event Server
Oracle Business Activity Monitoring for dashboards to monitor business events and business process KPIs

Continue and converge:

BEA WL-Integration will be converged with the Oracle BPEL Process Manager

Maintenance:

BEA Cyclone
BEA RFID Server

Note that the Oracle Service Bus is in the “strategic” category, but is a convergence of AL-SB and Oracle ESB, which means that customers of one of those two products (or maybe both) are not going to be happy.

Kurian stated that Oracle sees four types of business processes — system-centric, human-centric, document-centric and decision-centric (which match the Forrester divisions) — but believes that a single product/engine that can handle all of these is the way to go, since few processes fall purely into one of these four categories. They support BPEL for service orchestration and BPMN for modeling, and their plan is to converge a single platform that supports both BPEL and BPMN (I assume that he means both service orchestration and human-facing workflow). Given that, here’s their strategy for Business Process Management products:

Strategic:

Oracle BPA Designer for process modeling and simulation
BEA AL-BPM Designer for iterative process modeling
Oracle BPM, which will be the convergence of BEA AquaLogic BPM and Oracle BPEL Process Manager in a single runtime engine
Oracle Document Capture & Imaging for document capture, imaging and document workflow with ERP integration [emphasis mine]
Oracle Business Rules as a declarative rules engine
Oracle Business Activity Monitoring [same as in SOA section]
Oracle WebCenter as a process portal interface to visualize composite processes

Similar to the ESB categorization, I find the classification of the converged Oracle BPM product (BEA AL-BPM and Oracle BPEL PM) as “strategic” to be at odds with his original definition: it should be in the “continue & converge” category since the products are being converged. This convergence is not, however, unexpected: having two separate BPM platforms would just be asking for trouble. In fact, I would say that having two process modelers is also a recipe for trouble: they should look at how to converge the Oracle BPA Designer and the BEA AL-BPM Designer

In the portals and Enterprise 2.0 product area, Kurian was a bit more up-front about how WebLogic Portal and AquaLogic UI are going to be merged into the corresponding Oracle products:

Strategic:

Oracle Universal Content Management for content management repository, security, publishing, imaging, records and archival
Oracle WebCenter Framework for portal development and Enterprise 2.0 services
Oracle WebCenter Spaces & Suite as a packaged self-service portal environment with social computing services
BEA Ensemble for lightweight REST-based portal assembly
BEA Pathways for social interaction analytics

Continue and converge:

BEA WebLogic Portal will be integrated into the WebCenter framework
BEA AquaLogic User Interaction (AL-UI) will be integrated into WebCenter Spaces & Suite

Maintenance:

BEA Commerce Services
BEA Collabra

In SOA governance:

Strategic:

BEA AquaLogic Enterprise Repository to capture, share and manage the change of SOA artifacts throughout their lifecycle
Oracle Service Registry for UDDI
Oracle Web Services Manager for security and QOS policy management on services
EM Service Level Management Pack as a management console for service level response time and availability
EM SOA Management Pack as a management console for monitoring, tracing and change managing SOA

Maintenance:

BEA AquaLogic Services Manager

Kurian discussed the implications of this product strategy on Oracle Applications customers: much of this will be transparent to Oracle Applications, since many of these products form the framework on which the applications are built, but are isolated so that customizations don’t touch them. For those changes that will impact the applications, they’ll be introduced gradually. Of course, some Oracle Apps are already certified with BEA products that are now designated as strategic Oracle products.

Oracle has also simplified their middleware pricing and packaging, with products structured into 12 suites:

He summed up with their key messages:

They have a clear, well-defined, integrated product strategy
They are protecting and enhancing existing customer investments
They are broadening Oracle and BEA investment in middleware
There is a broad range of choice for customer

The entire briefing will be available soon for replay on Oracle’s website if you’re interested in seeing the full hour and 45 minutes. There’s more information about the middleware products here, and you can sign up to attend an Oracle BEA welcome event in your city.

Fujitsu Interstage BPM Version 10

Fujitsu is releasing version 10 of their Interstage BPM, and I had a chance for an in-depth demo a few weeks ago in advance of today’s announcement. On the design side, their new version of Studio now allows business analysts and IT to work together, and includes forms development. In terms of end-user functionality, there’s some improvements to workflow to enable collaboration, and new dashboard functionality. Most exciting (I think) is full support for multi-tenanting in order to allow for shared services and SaaS.

Key new features in Studio V10:

Full application development rather than just process modeling in Studio: create AJAX forms (rich user interfaces) to be attached at a point in the process, BAM designer, and simulation.
BPMN compliance, including annotation capabilities for inline documentation, plus some extensions to the notation in order to show things such as the presence of a form attached at a step. I explicitly asked which BPMN objects are not supported, and got a sort of fuzzy answer, that is, that they support all the BPMN objects required for their modeling paradigm. There are two other BPMN views besides the baseline standard: one showing the role at each step — which assumes that you’re not using swimlanes for roles — and one version with coloured swimlanes.
WYSIWYG forms designer allows widgets to be dragged and dropped from the palette, including complex widgets such as Google maps, breadcrumbs, and file uploaders: this is almost a mashup builder rather than just basic forms. There’s also BPM-specific widgets to allow you to easily add controls both for a work item’s metadata as well as functionality. You can add validation rules to fields.
Once a form is created in the forms designer, it can be dragged onto a workflow step to assign the form to that step.
The analytics designer is a separate perspective in Studio that allows creation of alert rules that will fire based on conditions in a workflow, then take an action: open an alert window to a user, generate a chart, or trigger another action. Charts are defined as independent objects, then a presentation dashboard can be built including a selection of charts, including role-based security.
There’s a built-in decision tables functionality that separates rules from processes, so that process-related rules can be changed independently and will affect work in progress. I still don’t like this approach as much as an external rules system, but this is much better than using just a simple expression engine with “rules” embedded at points in the process. Interstage can, of course, also support third-party rules engines plugged in for more comprehensive rules capabilities.
Simulation based on predefined values or historical values, plus the specification of resource costs. No tools for automatically identifying areas requiring improvement, although they hinted that this was in the works.

End-user interface updates:

The user interface is now created primarily using the forms created in the new form designer. In the sample that I saw in the demo, there was a multi-page form to represent a process instance, where the first page was text fields, and second page was a Google Map with the property location related to the process instance tagged on it: a nice example of the actual use of a Google Maps mashup in a business application. The form provides access to the instance metadata and any attachments, then allows the user to fill out additional form data and complete the task.
Depending on the user’s privileges, there’s an administrator’s view of the process, and a browser-based (Flash) process designer to allow either complete process modeling or (more likely) to modify a process in flight. Changes made to a single process in this environment can be migrated to all process in flight.

Interstage BPM enforces stringent J2EE compliance, such that any process model, rule, etc. built automatically become a web service that can be exposed.

Application partitioning capabilities to allow for SaaS: support for multiple applications on a single BPM instance with fully partitioned data, security and administration so that it appears as a private BPM instance, and utilization can be tracked by application. This is great for large enterprises that want to run virtual independent BPM environments for different divisions or applications, as well as companies that might want to use Interstage as a foundation for a SaaS BPM offering.

Oracle-BEA versus IBM-FileNet: the Borg versus death by a thousand cuts

Almost two years ago, I reported on the IBM acquisition of FileNet, wherein I quoted their plan to “integrate IBM’s BPM and SOA technologies with the FileNet platform”. I interpreted this to mean that FileNet BPM could finally get separated from its document-centric chains, and become the product that it should have been years ago. Just as Jessica Rabbit said “I’m not bad, I’m just drawn that way”, the FileNet BPM wasn’t (isn’t) document-centric, it’s just marketed that way. As the former director of e-business evangelism for FileNet in 2000-1 when they were launching this generation of the BPM product, I had some idea of what I was talking about — I saw that 40% of the BPM installations in some countries did not involve documents at all, and that this was due to the local sales and marketing messages and techniques rather than any inherently different BPM requirements between countries. So several years after I left FileNet, when the acquisition occurred and I saw that initial press release, I imagined that the best possible thing would be if the BPM product were to be separated out and made part of the IBM WebSphere suite, in order to flesh out the badly-needed human-facing workflow side of things over there. I realized that would mean some major surgery on the product, but a stronger unified BPM suite would emerge from that.

A few days later, an analyst call with IBM set me straight on that: the GM of the IBM Information Management unit that would be absorbing FileNet referred to FileNet’s BPM as “content-centric”. Uh-oh. I knew that couldn’t mean anything good for the product. I still have a lot of friends inside IBM-FileNet, and at first, it seemed like they were being allowed some degree of autonomy. Then, they gradually started being pulled into the “IBM way”, and a lot of people started getting unhappy, and many eventually left. There were few explicit purges, except for some redundant administration, but that doesn’t mean that there weren’t losses.

Recently, IBM announced its grand plan for BPM; Bruce Silver covered it here, describing how they plan to bring together process and content:

[T]he steps that involve content operations (e.g. adding/revising/securing documents) are done by a FileNet P8 process, which is invoked via WSDL as a subprocess in the end-to-end WebSphere process.

In other words, after more than a year, my premonition about the FileNet BPM product was realized, and it became — from a marketing standpoint — a shadow of its former self. I’m sure that customers using FileNet BPM for end-to-end processes (which IBM thinks are more suitable for WebSphere Process Server) cringed as they see the future of their installed product atrophy. I can only imagine the impact on the (still fairly segregated) sales team: one day, they’re selling FileNet BPM as a full BPM solution, the next day, it’s a document-centric subprocess handler, to be called from WPS. In much the same way that other BPEL vendors handle human-facing tasks as second-class citizen, calling a subsystem using WSDL to manage them, IBM is demoting FileNet BPM to the same realm. Although I predicted this right from the first analyst call, it doesn’t give me much satisfaction when I think about what it could have been.

The current monster acquisition in this space is Oracle and BEA, and it’s interesting to see the contrast in acquisition styles. Unlike the death-from-a-thousand-cuts method inflicted by IBM on FileNet, Oracle is more like the Borg: BEA is being assimilated, and fast. The acquisition — from the first rejected offer last October to the final accepted offer this January — was completed in late April, less than two weeks before the BEA Participate user conference. At the conference, I found that many of my BEA acquaintances were not staying on with Oracle; many others have announced that they’re leaving since then as well.

The Register reported yesterday on how Oracle is dismantling the AquaLogic product line: the AquaLogic web products (portals and Web 2.0) and, it appears, the WebLogic products will go one direction (hypothesized by the Register to be the Stellent team), with some rationalization of AquaLogic and WebLogic finally occurring; AquaLogic BPM will be moved under the Fusion middleware team. I haven’t dealt much with the WebLogic side of BEA, but when I dared to suggest that AquaLogic Pages could be used as a lightweight replacement for WebLogic Portal, I was “corrected” on the official party line. Given my confusion over this, I have to assume that some part of the market has also been confused over why BEA offers two different portal products. If it takes Oracle’s firm hand to finally merge them, that’s a good thing.

Oracle is doing exactly what I thought IBM should have done with FileNet: split up the product where it made sense and merge those pieces into similar teams that already exist, rather than try to maintain an intact semi-autonomous unit and brand, then gradually re-jig the product, potentially misleading customers about the final outcome.

As a member of the Enterprise Irregulars, I’ve been invited to a dinner hosted by Oracle next week at the Enterprise 2.0 conference; it will be interesting to see if BEA is so completely assimilated that the name isn’t even uttered.

Fujitsu Process Discovery

A few weeks ago, I had the chance to see Fujitsu’s new process discovery product/service in action. Unlike the usual sort of process discovery, which involves business analysts running around and documenting what people are doing, this automates the discovery of business processes by examining logs of existing applications.

The problem with the manual process discovery — based on observation, interview and operations/procedures manuals — is that it’s very labor-intensive, and can produce inaccurate results. The inaccuracies can be due to the level of experience of the business analysts, as well as what Michael zur Muehlen refers to as “process confabulation”: when you ask someone about what they do and they don’t have a good answer, they’re just as likely to make something up as they are to admit that they don’t know. I’ve seen this a lot, and tend to base most of what I do in process discovery on observation rather than interviews so that I can see the actual process; my ears perk up when I hear “well, we’re supposed to do it this way, but this is what we actually do”.

In a lot of cases, information about the business processes that are actually executed is embodied within existing enterprise applications, and saved to databases and log files. These could be supply chain, ERP, databases, legacy transactional systems or any other system that records events in some fashion. It is possible to install measurement tools on these systems to track what’s happening, or modify these systems to emit events, but Fujitsu’s new process discovery product/service extracts events from existing database and logs without modifications to existing systems. The real magic, however, comes in the software that Fujitsu has created to visualize those events in the context of a business process, and separate the “happy path” from the exceptions. Once that’s done, it’s possible to look at the ways to reduce exceptions, since the events that cause those exceptions are well understood.

I use the term “product/service” since all this manipulation and analysis requires some amount of training, and Fujitsu currently offers it as a consulting service, although there are standard software portions involved as part of that service. They have three levels of service, ranging from a basic extraction and visualization, to a more comprehensive analysis, to a customized offering. They’ve been using this with customers in Japan for some time — hence the Japanese case studies — and have recently launched their North American trial. Furthermore, it’s not specific to Fujitsu Interstage customers: this is really an independent effort of looking at your existing systems and optimizing your business processes. If that led to you buying Interstage, they’d be thrilled, but it’s not a prerequisite.

Looking at the new toys is the fun part of my job, so Keith Swenson took me for a test drive of the visualization tool. We were looking at a map of the events extracted from database logs for about 5,000 process instances, which were read into the process viewer using a standard CSV import. The straight-through path is detected and shown — in this case, it came from 632 process instances — but a slider control allowed us to change the threshold and add in more of the instances, up to 100% (which showed a pretty complex mess of a process).

That’s a pretty cool visualization, since the exception paths appear dynamically as the slider moves, but you really want to pick a simple baseline version of the process using the slider control — not the trimmed-down straight-through path, but one that includes the most common routes — then select one or more exception paths to overlay on it. Exception paths can be sorted and filtered for selection, using criteria including frequency, deviation, repetition, backtracks, and specific transitions.

Visually, the straight-through path shows as a thick black line, the alternative routes as gray lines, and the selected exception paths are in red. Although the number beside the black lines indicate the number of process instances that traveled that path, we figured out that the number beside the red (exception) lines indicated the step in the process sequence, since that’s more relevant when you’re tracking exceptions. Other information about transition times can also be shown, indicating bottlenecks in the process.

The tool is intended to be used interactively by a knowledgeable analyst in order to guide the views that will highlight the trouble spots: sort of like using simulation with real data. This isn’t an automated tool that takes data in one end and spits out the answers at the other; it’s meant to drive discussion and analysis, not replace it.

Keith showed me the results of three other case studies, and a couple of interesting effects of the data visualization and analysis that would make a quality manager’s heart sing:

In a hard drive manufacturing plant, the assembly portion of the process was simple, but the pre-process and test lines were much more complex. Eliminating the exception routes to improve the process actually had the effect of improving the quality of the drives being manufactured on the line.
In the second case, the processes of two different branch offices were compared, and it was identified that in one, half of the orders that their customers placed were for items that were not in stock, whereas the other office shipped most of their orders from their existing stock. There was no judgement about which was better, but it’s useful to be able to identify that one is using a just-in-time approach to reduce inventory costs, whereas the other is focused on reducing time to ship.
The third case showed some time analysis of processes to see the degree of deviation in the transition time between two steps. This is a classic statistical analysis, where a larger deviation (usually) indicates a higher potential for process improvement.

Typical benefits that their customers have seen so far are to identify and optimize inefficient processes, identify exceptions and infrequent paths, visualize the gap between the expected and actual processes, identify location-specific differences, and locate process bottlenecks.

This doesn’t, of course, consider all of the purely manual processes that are not captured in system logs, but would greatly reduce the amount of work required to map out the processes that do touch these systems in some way. Furthermore, it happens with little or no time investment from business people and analysts, hence has less impact on the customer than other discovery and analysis techniques. The reality is that this would work great in combination with some skilled manual discovery; maybe Lombardi should use this as part of their process discovery service offerings 🙂