Opening the second conference day, James Taylor and Neil Raden gave a keynote about competing on decisions. First up was James, who started with a definition of what a decision is (and isn’t), speaking particularly about operation decisions that we often see in the context of automated business processes. He made a good point that your customers react to your business decisions as if they were deliberate and personal to them, when often they’re not; James’ premise is that you should be making these deliberate and personal, providing the level of micro-targeting that’s appropriate to your business (without getting too creepy about it), but that there’s a mismatch between what customers want and what most organizations provide.
Decisions have to be built into processes and systems that manage your business, so although business may drive change, IT gets to manage it. James used the term “orthogonal” when talking about the crossover between process and rules; I used this same expression in a discussion with him yesterday in discussing how processes and decisions should not be dependent upon each other: if a decision and a process are interdependent, then you’re likely dealing with a process decision that should be embedded within the process, rather than a business decision.
A decision-centric organization is focused on the effectiveness of its decisions rather than aggregated, after-the-fact metrics; decision-making is seen as a specific competency, and resources are dedicated to making those decisions better.
Enterprise decision management, as James and Neil now define it, is an approach for managing and approving the decisions that drive your business:
- Making the decisions explicit
- Tracking the effectiveness of the decisions in order to improve them
- Learning from the past to increase the precision of the decisions
- Defining and managing these decisions for consistency
- Ensuring that they can be changed as needed for maximum agility
- Knowing how fast the decisions must be made in order to match the speed of the business context
- Minimizing the cost of decisions
Using an airline pilot analogy, he discussed how business executives need a number of decision-related tools to do their job effectively:
- Simulators (what-if analysis), to learn what impact an action might have
- Auto-pilot, so that their business can (sometimes) work effectively without them
- Heads-up display, so they can see what’s happening now, what’s coming up, and the available options
- Controls, simple to use but able to control complex outcomes
- Time, to be able to take a more strategic look at their business
Continuing on the pilot analogy, he pointed out that the term dashboard is used in business to really mean an instrument cluster: display, but no control. A true dashboard must include not just a display of what’s happening, but controls that can impact what’s happening in the business. I saw a great example of that last week at the Ultimus conference: their dashboard includes a type of interactive dial that can be used to temporarily change thresholds that control the process.
James turned the floor over to Neil, who dug further into the agility imperative: rethinking BI for processes. He sees that today’s BI tools are insufficient for monitoring and analyzing business processes, because of the agile and interconnected nature of these processes. This comes through in the results of a survey that they did about how often people are using related tools: the average hours per week that a marketing analyst spends using their BI tool was 1.2, versus 17.4 for Excel, 4.2 for Access and 6.2 for other data administration tools. I see Excel everywhere in most businesses, whereas BI tools are typically only used by specialists, so this result does not come as a big surprise.
The analytical needs of processes are inherently complex, requiring an understanding of the resources involved and process instance data, as well as the actual process flow. Processes are complex causal systems: much more than just that simple BPMN diagram that you see. A business process may span multiple automated (monitored) processes, and may be created or modified frequently. Stakeholders require different views of those processes; simple tactical needs can be served by BAM-type dashboards, but strategic needs — particularly predictive analysis — are not well-served by this technology. This is beyond BI: it’s process intelligence, where there must be understanding of other factors affecting a process, not just measuring the aggregated outcomes. He sees process intelligence as a distinct product type, not the same as BI; unfortunately, the market is being served (or not really served) by traditional query-based approaches against a relatively static data model, or what Neil refers to as a “tortured OLAP cube-based approach”.
What process intelligence really needs is the ability to analyze the timing of the traffic flow within a process model in order to provide more accurate flow predictions, while allowing for more agile process views that are generated automatically from the BPMN process models. The analytics of process intelligence are based on the process logs, not pre-determined KPIs.
Neil ended up by tying this back to decisions: basically, you can’t make good decisions if you don’t understand how your processes work in the first place.
Interesting that James and Neil deal with two very important aspects of business processes: James covers decisions, and Neil covers analytics. I’ve done presentations in the past on the crossover between BPM, BRM and BI; but they’ve dug into these concepts in much more detail. If you haven’t read their book, Smart Enough Systems, there’s a lot of great material in there on this same theme; if you’re here at the forum, you can pick up a copy at their table at the expo this afternoon.
“What process intelligence really needs is the ability to analyze the timing of the traffic flow within a process model in order to provide more accurate flow predictions, while allowing for more agile process views that are generated automatically from the BPMN process models. ”
Sounds like a CEP-based BAM application. However, I would argue that insight is not sufficient – for “intelligent processes” (surely better than “process intelligence”) you also need dynamic, context-sensitive business processes.
Your “argument” that insight is not sufficient seems pointless. I doubt Neil was advancing such a point. I suspect he was saying, rather, that insight gleaned from process intelligence is necessary to optimizing intelligent processes. (And put that hammer away; you seem to be seeing CEP nails everywhere.)
🙂
Unfortunately I missed this talk so I can only comment on Sandy’s comments. Sorry you misinterpreted my comment as an attack on Neil as this patently not the case.
John, perhaps you could explain how complex events are NOT involved in monitoring process events. I have use cases that indicate they do.
Cheers (and yes, event “nails” are indeed everywhere)
Paul,
I recall that you and many others were involved in a lively discussion about What is CEP? a few weeks ago.
http://www.thecepblog.com/2008/08/27/cep-is-not-bpm-bam-bpm-brms-or-soa/
I believe Neil’s talk was not about CEP, but about contrasting Process Intelligence to BI for BPM analytics.
Hi John – Neil’s talk included the comment about process intelligence. Not sure why you are referring to Tim Bass’ blog as AFAIK this is not about process intelligence. (Unless you are really Tim!). Instead, check out:
http://tibcoblogs.com/cep/2008/10/07/bpmi-tt-08-cep-augmenting-bpms-for-agility/
Lets see if I can persuade you with some basics:
You say “I suspect he was saying, rather, that insight gleaned from process intelligence is necessary to optimizing intelligent processes”.
Sandy reports: “What process intelligence really needs is the ability to analyze the timing of the traffic flow within a process model in order to provide more accurate flow predictions, while allowing for more agile process views that are generated automatically from the BPMN process models.”
So lets map this out.
1. How do you get “insight”? You monitor the processes. What do you monitor in the processes? Normally the process start/end/transition events.
2. What do you do with these events? Do you just display them on a dashboard? Or do you correlate them? Neil wants to check their occurances across time.
3. If you correlate the events, eg to see if a particular process is taking longer to perform over time and across process instances, then you are processing complex (ie abstract) events.
4. If you detect an particular anomoly in these event patterns you trigger a reactive decision. This would be a form of a business rule.
5. The action from the rule hopefully resolves the problem in an intelligent fashion. This is process intelligence.
Of course, I was actually assuming the above was “common knowledge”, and I apologize for the assumption. And of course the point I was actually making was that one of the process-event decisions might be to actually change the process to avoid the errant process – in other words event-driven dynamic processes – aka intelligent processes.
I hope this helps. Let me know if you have any more questions about the applicability of CEP to Neil’s vision.
Cheers
Geesh, Paul, would you put the hammer away?
No – its here to stay!
OK you two, calm down.
First of all, I want to reiterate what I said on IntelligentEnterprise in a comment to Sandy’s cross-post, that the concepts and some of the content of my presentation resulted from collaboration with John Patton, CEO of SightSoftware. We’ve been kicking this around for a while.
But I gave the presentation myself, so I take responsibility for it and I’ll try to address Paul’s questions.
The taxonomy of BI, Operational BI, Operational Intelligence, BAM, Process Intelligence and CEP is pretty leaky, so these kinds of arguments are tend to be pretty emotional, precisely because the stakes are so low.
I was trying, in a scant 30 minutes, to illustrate one topic, really: BI doesn’t work for analyzing processes. It loses data, it doesn’t understand time STEPS very well (as opposed to period and snapshots, which is does quite well), it rarely is applied to atomic instance data except to aggregate it.
I’m not sure Process Intelligence must of necessity involve rules. Like BI, it may be used only after-the-fact for analysis, not for remdiation. I’m not suggesting that is the preferred approach, but we’ll have to see. It may not be the process itself that is errant, but rather the exogenous influences that turn it on its head.
So I don’t really know what the difference is between CEP and Process Intelligence and I don’t think it matters. I was not, however, describing CEP in the presentation (I don’t think).
-Neil Raden
Dear Folks,
I can’t believe that Paul Vincent made such a comment, LOL. Then again, I can believe it, LOL 🙂
Anyway, John, thanks for posting a link to our spirited discussions.
IMHO, CEP, as a concept, is on a downward spiral, slowly being killed by the constant hype, spin and repositioning by software vendors who are looking for any hook they can hang their hat on to make a sale.
The Enemy of CEP is CEP Vendors
http://www.thecepblog.com/2008/11/01/the-enemy-of-cep-is-cep-vendors/
Yours faithfully, Tim
Hi Neil –
“BI doesn’t work for analyzing processes” – agree, its temporal, and BI is traditionally on static data.
“I’m not sure Process Intelligence must of necessity involve rules” – true: you can introspect / “query” real-time process data with other technologies.
“So I don’t really know what the difference is between CEP and Process Intelligence and I don’t think it matters. I was not, however, describing CEP in the presentation (I don’t think).” – Well, I was trying to draw an inference – the aggregation of real time process data (events) into process information is “event aggregation” is “CEP”… but it was just an observation. I surprised it drew such ire from “John Patton”.
Hi Tim – Which comment are you referring to? 🙂
Cheers
Follow-up. Ah it looks like I owe Tim an apology – I think “John Patton” is in fact a real person from a “Process Intelligence” vendor. His tool might even do CEP to provide “process intelligence”. Oh well. Apologies to anyone hoping for a useful discussion here. 🙁
Cheers
I think that you also owe John Patton an apology, not the least for putting his name in quotes in your last comment. 🙂
Hi Sandy – fair enough – apologies to John for putting his name in quotes. And questioning whether his name was being used by someone else.
I would still love to know whether my original assertion is considered “correct” or “wrong” by the “process intelligence” specialists. No one seems to have refuted the points in comment 5. But then, maybe I’m the only one to find this interesting! 😉
And apologies to Sandy for taking up space in the Column2 comments – I’ll start up the assertion / comments / use cases in the TIBCO blog to avoid any further, er, miscommunications.
Cheers
No problem with having the conversation here, glad that I could write something that would inspire all of this!
Paul,
First, let me point out that in Post #5 you “apologized” for assuming that your points were common knowledge. Then in Post #13 you proclaim your interest in a response. Why, if it is common knowledge, would you expect a response? (BTW, I believe Tim O’Reilly is the originator of the “common knowledge” form of blog insult.)
Second, putting aside the implied insult, I think your post didn’t receive a response because you are saying “Isn’t Process Intelligence a form of CEP?” which is another way of asking, “What is CEP?” Process Intelligence can be distinguished from BI by some key technology differences (e.g., an object model vs. a data model; an event store vs. relational or OLAP store.) So the comparison is really BI vs. PI. Which was the point of my initial posts all along.
So I think you overreach by trying to draw PI into the CEP world. But here’s my quick take on CEP. CEP and BRM form EDM. CEP is a distinct technology defined by the simultaneous presence of (1) a pattern language and (2) stream (non-persisted) data.
OK, so against my better judgment I’ve commented on CEP.
So while PI, BI, CEP, BRM and are all part of the BPM ecosystem, they are distinct technologies.
To this note, I don’t really see BAM being a distinct technology. It is simply the dashboarding of data, accomplished in any BI tool. I believe this is why there are very few pure play BAM vendors left. Those that are left, like Systar, are really KPI management applications. BAM was never a distinct technology, only dashboarding for BPM.
“First, let me point out that in Post #5 you “apologized” for assuming that your points were common knowledge. Then in Post #13 you proclaim your interest in a response. Why, if it is common knowledge, would you expect a response? ”
Good qu. Perhaps I was wrong in post 5, based on the sample of 2 responses (yourself and Neil). So if its common in the CEP community, for example, it looks like it isn’t in the Process Intelligence community (and assuming your identity is the same John Patton who commented in one of the Smart Enough Systems blogs, and therefore from the latter community).
[I’ll leave the discussion on insults for someone else to comment on, as obviously I’m biased]
“I think your post didn’t receive a response because you are saying “Isn’t Process Intelligence a form of CEP?” which is another way of asking, “What is CEP?””
No, and no. CEP is about abstract events, so if process intelligence abstracts events, should it not be connected with the idea of CEP? And if anyone wants to check out definitions, see http://www.ep-ts.com and its glossary.
“Process Intelligence can be distinguished from BI by some key technology differences (e.g., an object model vs. a data model; an event store vs. relational or OLAP store.) So the comparison is really BI vs. PI. Which was the point of my initial posts all along.”
Interesting: object model, event store… very much is a part of the CEP technology space. Too.
“So I think you overreach by trying to draw PI into the CEP world. But here’s my quick take on CEP. CEP and BRM form EDM. CEP is a distinct technology defined by the simultaneous presence of (1) a pattern language and (2) stream (non-persisted) data.”
OK, so this tells me the “PI community”, if you are representative of that group, has not yet understood CEP. Because you are describing to me event stream processing. But thats OK, because it cancels your implied inference that PI is nothing to do with CEP.
“OK, so against my better judgment I’ve commented on CEP.”
No, its an interesting comment, and no more invalid than my statement on Process Intelligence.
“So while PI, BI, CEP, BRM and are all part of the BPM ecosystem, they are distinct technologies.”
Funny, I see the opposite: processes being implemented as rules, rule management for CEP, etc etc.
“To this note, I don’t really see BAM being a distinct technology. It is simply the dashboarding of data, accomplished in any BI tool. I believe this is why there are very few pure play BAM vendors left. Those that are left, like Systar, are really KPI management applications. BAM was never a distinct technology, only dashboarding for BPM.”
Interesting again, as I see the difference as real-time / event driven dashboards (push model) versus historic views that are not updated (except by a pull model aka refresh).
Finally, thank you for your comments. I don’t agree, but at least we know where you stand on this!
Cheers
Paul,
I didn’t know there was a Process Intelligence community, perhaps I should join it.
I don’t know why you and John have to make such pointed comments at each other. I think this is pretty simple. CEP doesn’t have the word “intelligence” in it, so I suggest that PI is something like BI, only it is designed to understand process activity somewhat after the fact. If that understanding is moved up to real time, then maybe it is very similar to CEP. They may be converging in effect, but they derive from different disciplines. PI is analytical, CEP is actionable. PI may be actionable, CEP may be analytical.
So why do we have to argue about this?
-NR
twitter nraden
I see the above arguments being somewhat challenged by the lack of clear definition of terms such as BAM, BI, and Process Intelligence.
You may be better served by describing the problem that you have and the proposed solution for that.
For example, BAM in the context of BPM can be construed in different ways.
One way to look at BAM for BPM: I would like to monitor the execution of process steps managed by my BPM application. In this example, you are only monitoring activity inside the BPM tool itself. Many BPM vendors offer this form of business process monitoring, and call it BAM.
Another way to look at BAM for BPM: I would like to monitor KPIs for my overall business process. For example, I would like to make sure that all sales activity initiated this morning before 10am completes by 3pm today. Monitoring this activity may include visibility into the BPM application’s execution, the underlying IT infrastructure supporting that process, external indicators such as weather in the region, etc. If it is 12pm now, I would like to know if any events inside or outside of my BPM platform, related to the sales activity at hand, would impact me meeting my 3pm deadline. A pure-play vendor like Systar considers BAM defined more along these lines, as opposed to the view in the first scenario.
Getting back to my initial point, you may want to consider better defining what and how you want to monitor the business activity, and seek a solution that meets those criteria. Unfortunately for now, the overlap and confusion in industry definitions of BI, BAM, PI, etc. may just perpetuate your disagreements.
Derek,
Good point. If I wasn’t clear, here is a distinction between BAM in a BPM context and PI in the same context – as you said, BAM monitors. PI analyzes. I guess the definitional conundrum occurs when PI approaches real time. But it is still essentially analytical, not a monitor. What happened, why did it happen, what it is related to, what are the causal factors, what should be done? This is where it bleeds into the definition of CEP.
-NR
Interesting discussion.
I remember discussing with one of the industry’s luminaries not so long ago – and he was joking (well not really) that the best way to make a CEP vendor cringe was to ask ‘what do you think is an event’, and then ask ‘what do you mean by an event happens’. Semantic mud, arms thrown up in the air, red faces.
But more to the point of this exchange, I agree with one core argument pushed by Tim Bass (although I could access his article to know whether I agree with the rest) – and this discussion is a clear illustration; CEP suffers from vendors trying to position it to be everything and anything. It does remind me of the early days of BREs when they all were presented as being able to do anything and everything (yes, I am guilty). Well, not any more: clear distinctions are commonly agreed upon between BRs, BPs, etc. A lot us contributed to that clarification becoming crisper, and the industry is better of thanks to that.
Efforts to refine, rules and process semantics are on the way – I think it’s an excellent idea to think about those semantics in concert, but I am fairly certain that what these efforts will lead to is a common understanding of what the different areas of responsibilities are.
Bear with the opinion of a BRMS-turned-EDM guy.
I would love this to be really really simple, So simple that even I will not want to acknowledge that this is that simple after I write it – but that is what you need to do to simplify: turn it into too simplistic and build from there.
– Events are all around detecting what decision needs to be taken in a complex temporal context (essentially event correlation, converting monitored ‘ambient’ events into relevant business events that require reaction)
– Decisions (and I will use that term instead of rules) are all about deciding what needs to be done as a reaction to those business events
– Processes are all about executing the decision taken
Very simple, very simple.
CEP addresses the first, EDM (with BRMS as the first child) the second, BP the third. Yes, it’s reductive with respect to what the CEP vendors would like (sounds a lot like event correlation as in what used to be done 15 years ago) – but, hey, simplifying always starts with cutting too much to add more later. If need be.
Event Management, Decision Management, Process Management.
The fact that platform vendors are integrating the 3 does not mean they are the same. Or that they should be.
What I just outlined is centered operational aspects – as in the operational / improvement aspects distinction presented by Carole-Ann in her talks about the EDM vision at ORF and BRF.
The improvement aspects cover how to leverage the information gathered by the operational system – as well as expertise – in order to improve the decisions. It starts by gathering the information and understanding it. All aspects above are to be covered: Am I capturing the right events? What if I decided to capture something slightly different: does that make my business events more relevant? Why did I capture this event at that point in time? What other events are captured within a given contextual distance from this one? Ditto for the decisions (the rules). Ditto for the execution (the processes).
Why not just call these things "Event Analytics", "Decision Analytics", "Process Analytics"?
Yes, they are connected. But different.
I know, simplistic. Too simplistic.
But deep down in my simple mind, the logorrhea of acronyms that seems to accompany any discussion on CEP / BPM / BRMS (sorry) is indicative of too much confusion.
When we talk to enterprise application architects – they have much simpler descriptions for what they need than what we give them in return. How many presentations have I seen that end up in the customer asking "so you mean this is just "? We should keep the complexities to us, and present very clear and focused concepts and features to them.
My 2 cents (asymptotically reaching 0 these days).
– Carlos, retreating into his shell.
Heh – I think Carlos meant to say:
“that the best way to make a CEP vendor cringe was to ask ‘what do you think is NOT an event’” 🙂
Speaking from a vendor that provides various levels of event processing (human, complex-automated, service-oriented), the classification scheme of event-decision-process makes lots of sense.
Of course, we are talking technologies not vendor market areas. So consider a BPM system generating events indicating the completion of activities. These can be complex events (abstracted), and you can view them, make decisions about them, and have a process to resolve problems in them. This is, at least in some sense, “process intelligence”. And if you want, you could also run analytics on that process event-data…
Cheers