sandy

Shallow vs. Deep Knowledge

The EDS Fellows’ Next Big Thing blog today discusses how business applications continue moving towards less custom coding and more off-the-shelf reusable vendor components, and the impact that has on an integrator’s knowledge of the vendor components. Interesting that some of the best minds at this large SI are pointing out that their portion of any particular job is likely to continue to shrink, something that I wrote about last week, although they don’t discuss how EDS or other large SIs are going to fill the in gaps in their past business model of “build everything”.

The point of their post is, however, how can someone working on a business application have sufficient knowledge in order to understand the strengths and weaknesses of a given vendor component when they haven’t seen the source code? They go on to provide a scientific method for gaining a deeper knowledge of a component without access to the source code, but their entire argument is based on an old-style mainframe integration (which, to be fair, was/is EDS’ sweet spot) where it was fairly common to have access to vendors’ source code.

I have to say, welcome to the real world: I’ve been doing integration for over 15 years, have a very deep knowledge of a few vendors’ products, plus a shallower knowledge of a bunch of other products, and I’ve never seen a line of vendor source code. Personally, I can’t think of very many cases where access to the source code would have improved the end result; as any good software QA team can tell you, you don’t need to see the code in order to determine the behaviour and boundaries of a component.

Furthermore, their scientific method doesn’t include a vital component: vendor relationships. If you’re building a significant business on a specific vendor’s products, you have to establish and maintain a relationship with them so as to have relatively easy access to their internal technical resources, the people further behind the customer support front line. Having done this with a couple of vendors in the past (and then being accused of being in bed with them for my efforts), I know that this is a key contributor to gaining the requisite deep knowledge for a successful integration.

Visio for BPMN

Another article by Bruce Silver on the value of using Visio for process modelling. His argument is that it’s already a de facto standard, with two-thirds of the world’s existing process models done in Visio, so why not find ways to just make it ready to play in the big leagues?

One way is to allow import of Visio models into enterprise architecture modelling tools such as Popkin, where they would then be “finished” by developers, but I don’t like that method because it takes control of the modelling — includuing ongoing maintenance for the models — out of the hands of non-technical process designers.

The other is to use add-ons to Visio that allow a process designer to create BPMN-compliant process models in place, then generate BPEL or XPDL. ITP-commerce, for one, has a tool to do this, and I think it could be a better solution in many cases, since it allows the process designers to use a familiar tool and easily work from their existing library of models.

Bits and pieces

I’m heads-down on a project this week so not much time for catching up on the news and blogging. However, interesting things keep happening whether I’m watching or not…

RUNA WFE 1.0.1, an open-source workflow based on JBOSS-JBPM was released. More details here, including a link to an online demo. Open source BPM is going to be a market force in some sectors, so best to be aware of what’s happening there.
Greg Wdowiak published an interesting post on the role of integration brokers within an integration stack. In particular, he discusses what you should expect to get from the integration broker portion of the stack, and where some of the vendors are lacking. If you’re new to EAI, you can read his excellent background post on bus versus broker models. In particular, he talks about how organizations move from a broker to a bus model as their integration needs become larger and more complex.
BEA buying Plumtree has been all over the tech news, with lots of interesting analysis. MWD blog thinks that the purchase may not be about what BEA says that it’s about, but more about moving away from complete Java-centricity and into a more neutral technology territory by supporting .Net. The Butler Group sees this as a better fit than some of the previous portal buy-outs, although an earlier post ponders the fate of the Plumtree-Fuego OEM agreement in light of BEA’s existing BPM strategy.
On the enterprise architecture front, The first issue of the Journal of Enterprise Architecture was published. Via Nick Mudge’s blog.

Lastly, if you’re free today at noon Eastern, there’s a webinar roundtable on Winning at BPM discussing IBM‘s WebSphere process integration products.

Enjoy.

BPM soft skills

I saw this Gartner article a while back about a “new” BPM definition, and was reminded of it when it hit BPM.com again this week. He turns the focus from the hard technical skills required for BPM to the soft social skills required: much as I have done in my own career over the past 15 years or so: putting the “business” and “management” back in BPM, which has too long been focused on the mechanical “process” part. It’s not by accident that the small systems integration company that I owned in the late 90’s had the slogan “We Make Technology Mean Business”, and that I’m now developing a course called “Making BPM Mean Business“.

In that same vein, Gartner has recreated their definition of BPM to move the focus away from the tools and the mechanics of BPM:

BPM is a management practice that provides for governance of a business’s process environment toward the goal of improving agility and operational performance. BPM is a structured approach employing methods, policies, metrics, management practices and software tools to manage and continuously optimize an organization’s activities and processes.

Right on!

Convergence of BPM and BI

If you’re interested in more about the Forrester report about BPM and BI that I mentioned a couple of weeks ago, but still don’t want to shell out the cash for it, you can find a more complete summary on here on BPM.com (registration required), written by a Forrester VP who was one of the original report authors.

The more that I look at compliance, which I’ve been doing a lot of lately, the more that I understand that BPM needs to feed its performance data into a larger BI infrastructure. And I can buy into what the Forrester report refers to as Process to Data, or P2D (as if we needed another TLA), which is the two-way synergy between BPM and BI: BPM feeds process performance data to BI, and BI invokes processes in BPM in order to gather information. However, I have to draw the line at their statement:

BI and BPM can no longer live without each other, and the time is right for these technologies to merge.

I don’t think so, any more than BPM will merge with any number of other technologies, although obviously there will be closer and closer integration ties in order to make all of this work smoothly. BPM and BI are strong, distinct markets served by a variety of strong vendors, and customers develop quite a bit of loyalty to their BPM and BI vendors because both of these technologies are pervasive in an organization’s IT infrastructure. If BPM vendor A decides to merge with BI vendor B and suggests to their customer that they should change all of their existing BI from vendor C to vendor B, there would be a great deal of rolling around on the floor and laughing. There’s a much stronger argument to be had for merging BPM and BR (business rules), but I don’t think that’s going to happen either, for much the same reasons: both technologies have strong, independent markets (that is, the technologies can exist successfully in organizations without each other) and there is little reason for a customer to want to buy their corporate-wide BR from their BPM vendor, or vice versa.

If the merging that Forrester suggets actually occurs, we’ll end up with monolithic Swiss-Army-knife-like vendor offerings from a few large players — how retro! — and a lot of unhappy customers. What most customers want is the best of both worlds: the best-of-breed technology, and the minimum amount of integration; hence the huge popularity of SOA. In the past, you couldn’t get both: you could buy best-of-breed from different vendors and take a lot of effort to integrate it, or you could buy a fully-integrated suite from a single vendor that included at least one sub-optimal component. Today, however, with the advent of integration standards and SOA, the integration effort for properly-constructed products from different vendors should be no more than that required for products from the same vendor.

My conclusion: as long as the vendors do what they’re supposed to in order to enable easy integration, there’s very few customer drivers for merging BI and BPM.

Agility and BPMS Architecture

Bruce Silver writes about the distinction between workflow architecture and service orchestration architecture, pointing out why it’s important to distinguish between these two process architectures and see how they fit together. If you’re having trouble figuring out the difference between what the former workflow and former EAI vendors are saying, this might help.

He also discusses what’s changed in BPMS to actually make it as agile as the vendors would have you believe that it’s been all along: easier application integration. As he puts in, in the bad old days, when the EAI part of workflow systems didn’t work very well:

If you went to a workflow conference or trade show, any vendor could show you how to build a workflow in 30 minutes using no programming, just graphical drag-and-drop design. So once you bought the technology, why did it take six to twelve months to design and deploy real-world workflows?

There was a huge market for systems integration companies to do all that tricky application integration stuff — I know, because I ran a 40-person version of that model up until 2000, with a crack team that did design, development, testing, documentation, training and installation, all targeted at trying to turn the dream that the workflow vendors sold into a reality. If I hadn’t already got out of that business, I’d certainly be looking to do so now, as the EAI part of BPM gets easier and easier, leaving much less of the complex systems integration that used to drive a huge part of the market.

What I find amazing is that some of the large systems integrators continue to stick their heads in the sand and refuse to buy into the new way of doing things. Even though capabilities exist for much simpler (and therefore faster and less expensive) integration of BPMS with other technologies, the SIs have too much of a vested interest in selling huge pieces of custom code that they’ve written to augment previous versions of various vendor products, and the associated 1000’s of hours of customization that they can sell to the unsuspecting customer. They actually win work based on the premise that if it takes that long and costs that much, it must be good.

Back to the Silver article that started this thought, I also find it amazing when vendors (and therefore, by extension, their clients) don’t make the distinction between the two types of process architectures, and try to sell their legacy technology (usually either workflow or EAI) as if it spanned the entire space. Since most former workflow vendors have hopped on the EAI bandwagon by buying in that capability early, I see the problem more often with the former EAI vendors trying to sell service orchestration as if it were workflow, with only the barest of long-running process and human-facing capabilities. I remember leading a discussion on exactly this topic with a client’s IT group who had selected an EAI product as their BPM corporate standard, and never being able to get through to them that there are really two major classes of process functionality and architectures, only one of which was addressed by the product that they selected. Silver’s article may have helped.

BPM course at FileNet user conference

If you want to spend two days learning how to “Make BPM Mean Business“, I’m leading a seminar as part of the pre-conference training at FileNet’s user conference in Las Vegas in November. I’ve tried to strip out all the technical stuff (which FileNet can teach you better than I can anyway), so this is really targeted at business managers, business analysts and process designers.

Sign up through FileNet if you want to do this in Vegas prior to the conference, but if you can’t make it to the conference, I own the course content and can do a private gig elsewhere, preferably somewhere warm and exotic as winter descends on Toronto. The first day (“BPM in Business” and “Modeling Your Processes”) is not specific to any vendor’s products and would be suitable even if you’re not a FileNet customer.

SOA and banking

I viewed a webinar today featuring a senior integration architect at Bank of America talking about how they use webMethods to facilitate their massive integration efforts, apparently with some degree of success. I liked the admission by both the customer and vendor that SOA is an architectural design approach of loosely-coupled components, not a particular technology or product, although products can obviously help you to build a service-oriented architecture faster, cheaper and in a more standardized manner.

This definitely put me in mind of a recent engagement where I spent a year as the acting application architect for one business silo within a financial institution, facing many of the same business and technology challenges as BofA. I also sat on the architectural review board, so had a view of the common challenges across the entire organization and the technology infrastructure being deployed to address them. Almost all large financial institutions got big through a combination of acquisition and growth, and both of those growth methods can lead to disparate systems in the back office — even within one silo — that don’t intercommunicate well (if at all), which in turn leads to a lot of internal inefficiencies and a less-than-optimal customer experience. Furthermore, most organizations are not set up to share across the business silos, so attempts to create common infrastructure such as those that support SOA can be tricky unless an architect is empowered to work across the silos.

As you move up the services stack, the focus changes from the base services and registry layers (where webMethods plays) to orchestration, where services are assembled into processes that relate to actual business processes (you knew that I’d get around to process eventually, right?): the task flow and BPM layers in the integration stack that I referenced previously, although the lower levels of that particular stack are MOM (“first generation SOA”) rather than XML-based web services (“second generation SOA”). As you bring these concepts together, and especially when you start looking at the technology stacks, it becomes obvious that you can’t have an effective BPM strategy without SOA. You can implement BPM without SOA, but it won’t have the ability to become the strategic orchestration layer that’s required to pull together disparate systems and provide new integration functionality in the future.

Is anyone thinking about the users?

Everyone once in a while (okay, maybe more often than that), I’ll see some piece of crappy user interface design and have a private little rant about it. Since I’ve been designing UI since back before it was called “user experience”, and am a heavy user of a large number of systems with different UI’s, I have some idea of what works and what doesn’t work.

Often, this happens when I’m in a retail environment and the store employee is fighting through a variety of screens to achieve what should be a common and easily-accessible task: I really have the sense that the software designers didn’t bother to consider usability because they knew that the users would be trained on the software, that is, it’s not web-based consumer software that can be abandoned in favour of a different communications channel, it’s a part of the person’s job and they must learn to use it.

Occasionally, this does happen with web-based software that I use as a consumer, such as banking websites. When I used to travel almost full-time for business, I got in the habit of doing everything online: if a supplier (bank, phone, whatever) couldn’t give me a way to check and pay my account online, then I went elsewhere. That philosophy has done well for me over the years, and I still stick by it. I bank with one of the large Canadian banks, and I was commenting yesterday on how there are some really stupid, albeit minor, design flaws in their “account download” screens that make me do a few extra clicks every day when I download my account information. They might think that a few clicks don’t mean much, but I used to design UI for transaction processing staff at financial institutions, and we squeezed every keystroke out in order to maximize the efficiency of that particular factor. Why can’t web UI designers consider that some (many?) of the users are concerned about efficiency, want to use the least possible keystrokes — and even less mouse movements — in order to do “chores” such as online banking. I just want to get in, download the transactions and get out as quickly as possible, I have no desire to linger on their site and check out some fancy UI widget.

Although I’m not picking on this particular bank, because every bank that I have dealt with has similar (or worse) problems, I have a few other bones to pick with their systems people. I applied for a registered investment account online with them recently, but because I was transferring in assets from another institution, I took the completed application form to the bank instead of mailing it in, so that I could get the account number right away and initiate the transfer. Although the application was retrievable by me online with its unique ID, the person at the bank who assisted me had to key in all the information over again because there is no way for him to pull up the already-completed application form from their own systems and complete the account opening. Total waste of my time (if I had know that, I wouldn’t have spent the time online completing the app in the first place, then had to wait while he re-keyed it) and total waste of the bank’s employee’s time, which costs them money and customer goodwill.

The latest in the continuing saga came this morning, in response to my request that they discontinue my monthly brokerage statements by snail mail since I’ve been downloading the PDF versions from their website for more than three years:

Regrettably, due to our current internal platform, we do not offer the option to discontinue paper-based statements. However, this feature is on our agenda for consideration for future system enhancements.

Waste of paper, waste of energy printing the statements, damage to the environment from the trucks used to ship all this paper around, waste of my time opening and shredding the statement, and again, loss of customer goodwill. All this from a bank that rakes in a couple of billion in profits every year.

What is all adds up to is IT departments that are not focussed on their customers’ needs, whether these are internal or external customers. With so many people using the systems created by these IT departments, how can they continue to justify the philosophy that the users will just put up with bad software? Most IT departments don’t even think about the fact that the business side of the organization is their customer, as well as potentially external customers, and if they don’t service those customers adequately then they may find themselves outsourced out of existence.

Processes “R” Us

I had several appointments and errands today, and I listened to podcasts as I walked around downtown Toronto. One of them was the Sound of Vision podcast from back in May wherein Ethan Johnson interviews me about BPM (starting at 21:00 in the ‘cast), and there’s one point where I get really passionate about the fact that everything is a process: my true evangelist colours shining through. I do have a very process-centric view of business, to the point where some work that I’ve been doing recently on compliance started out being about content and records management, and has shifted to have a very strong focus on process.

I also saw an article this afternoon by Terry Schurter of BPMG, and he states that BPM and a process-centric view are so popular because such a high percentage of BPMS implementations (compared to other enterprise software) deliver on their promise of ROI. His view is that taking a process-centric view — “the idea that businesses can be viewed as a series of processes, and that those processes can be identified and managed to improve quality, efficiency, and cost-effectiveness” — resonates with end-user organizations, vendors and analysts, and that BPM aligns with the natural business structure.

It seems that you can’t pick up a business or technology article these days without it containing some reference to process, which means that Terry and I are not alone in our views.