Category: services

(micro-)service oriented architecture

ESB-CON IV

There’s a half-day ESB/SOA online conference coming up on October 18th, with speakers from BEA, IBM, Oracle and Progress. I’m sure that there will be a lot of vendor marketing messages included, but it might be worthwhile if you’re just starting to look in this area and want to get an earful from some of the key vendors.

Forrester Day 1: Robert Wiseman, Sabre

The last session before lunch was by Robert Wiseman, CTO of Sabre Holdings — interesting that the two customer keynotes this morning were both by travel-related companies. They have the same problems as many other organizations, in that they have to deal with large numbers of transactions and compete on pricing, but they’re doing it at a much higher volume and with a much greater abandonment rate than most other companies: think about how many times that you use Travelocity (one of their companies) to search for prices versus how often you actually book the travel.

Wiseman talked about how it’s necessary to focus on the differentiators: their business is differentiated by the quality and quantity of content that they provide to their customers, and the level of efficiency that enables the customers to locate and consume the content that’s right for them. Everything else, as he says, is just plumbing. In that plumbing, they push towards a “cookie cutter architecture”, where technology is commoditized which in turn makes for easier build and test cycles, and improves time to market as well as TCO.

They design for technology obsolescence by staying as vendor agnostic as possible, and abstracting the technology layers.

They also design for failure: since they run 20,000 transactions per second, 24×7, all technology will eventually fail, and the fail-overs must be obvious to the operational support staff but transparent to users. The goal, in the case of failure, is to continue limping along in some fashion rather than a total meltdown.

They design for flexibility, using XML-based APIs and web services against some rather ancient mainframe technology; however, they’re now redesigning some of their web services to be less granular (so as to not invoke 30 million web service calls per day) based on tracking consumer behaviour on the website. This is an interesting concept for designing the right granularity of web services: implement your best guess, tending to be more granular, then watch the behaviour and rebuild them into less granular services.

As you might imagine given their transaction volumes, they’re very focussed on performance testing, and designing for the purposes of facilitating performance testing by standardizing hardware (e.g., blade servers) and software (e.g., DBMS) so that it’s possible to test a full range of situations. He stressed how important it is to consider performance testing at design time, not just as an afterthought.

In the Q&A, he talked more about their standardization and reuse efforts: pretty thorough at the infrastructure and even the middleware layers, but not so easy at the application layers. They’ve standardized on Linux, which an audience question referred to as “open source shareware”, which was a bit funny — they’re using a supported version of Linux, so it’s not like they’re pirating software from the internet or something. He also made an interesting comment about how the best strategy for staying vendor independent when you have a single source is to have a really good exit strategy for that vendor’s products: don’t stay with it just because it’s too hard to migrate away from it.

Forrester Day 1: Sandy Carter, IBM

I saw Sandy Carter speak a week ago at the Gartner BPM conference, and as expected, there’s some amount of overlap from what I heard there in the first section on what’s driving business today and the nature of globally-integrated enterprises. Less talk about BPM (since this isn’t a BPM conference) and much more on SOA and how it contributes to these types of enterprises.

She covers off four distinct styles or paths of SOA:

  • Foundational, which is focussed on proven, high-ROI projects; typically there’s only 5% of functions as services, and less than 10% reuse of these services.
  • Extend end-to-end, with optimization and innovation across the entire value chain using BPM, with up to 40% of functions expressed as services, and up to 20% of services reused.
  • Transform, where the business model is being transformed and IT provides a strategic advantage; up to 50% of functions are expressed as services, and up to 80% reusable.
  • Adapt dynamically, where the technology becomes invisible and predictive business functions drive process innovation; more than 80% of functions are expressed as services, and more than 50% are reusable.

She went through each of these in detail, with examples, and talked about how and why each of the approaches are typically selected from both the business and technology standpoint.

There was a booklet distributed to each of us when the session started written by Carter, entitled “The New Language of Business: SOA & Web 2.0”, and she used it as a launch point to discuss enterprise mashups and how they can be used to extend the use of SOA. There’s been lots written about mashups as the global SOA; like BPM, mashups are a primary consumer of services that help to increase reusability and therefore cost-justify SOA in the first place. She showed us some nice services-based mashups that IBM has built for their own internal use; as usual, I wish that IBM would move the internal stuff out to the real world a lot sooner since it seems that they’re doing some really interesting stuff internally that takes years to reach the market.

She then introduced Mohammed Farooz, CTO of the state of Texas, to talk about their Health and Human Services department and the transformational work that they’ve done. They’ve moved from a single channel client interaction model (where each program had its own client interaction) to a multi-channel client interaction where all programs were available to citizens through a common interface, whether they were doing web self-service or calling in to the call center. They accomplished this with an agile business framework (business modeling, capability management, performance management and governance) on top of a flexible SOA stack (Rational, WebSphere and a few other IBM bits).

Carter wrapped up with some comments on end-to-end process integrity — interaction, transactional and information — and how IBM is focussing on this. She showed the same video about Second Life “BPM Flight Simulator” as we saw last week. When they talk about simulating business processes within a simulated environment, I still just have to shake my head, although apparently there’s a greater retention rate for people using this for training versus traditional methods.

Forrester Day 1: Connie Moore Keynote

Connie Moore gave the opening keynote on Design for People, Build for Change: Transforming the Nature of Work. Her focus is on how business and IT have to work together in order to achieve this, but she likes the term “blended business-IT” rather than “business-IT alignment” because she wants them to be seen as a single entity rather than two separate bodies that need to be aligned in some way. I’ve heard Moore speaking at other conferences and on webinars previously, usually on the topic of BPM, and it’s significant that Forrester puts a BPM analyst in the keynote position at this forum: it really drives home that the key focus here is on process.

She posed three questions about this sort of transformation: why now, what underpins this trend, and how will it unfold.

In the “why now” category, she discussed the evolution in design that’s underway in all sorts of consumer products, and asked us to envision what would happen if the leaders in consumer product design (e.g., Apple) came into organizations and set to redesigning enterprise systems. Interesting thought, and something that I’ve written about when discussing Enterprise 2.0, which brings consumer Web 2.0 functionality into enterprise applications. The new generation of workers, dubbed “millenials” by Moore, have grown up with completely different experiences and therefore have completely different expectations about what systems will look like as they enter the workforce, particularly around social networking. Added to all this is the evolution of process management as a discipline, and the dissolution of monolithic business applications into composite applications that use BPM, SOA, business rules, collaboration and other technologies, either on-premise or as SaaS.

As for what underpins this trend, Moore discussed the dichotomy between the detailed transactional type of work (which she characterizes as left-brained) and the big picture type of work (right-brained) that have to be supported simultaneously by our systems. She lays out four key principles for designing for people, and gave a detailed example of each (including a really interesting Second Life example for the 4th point):

  1. Business processes adapt to changing business conditions.
  2. Applications evolve continuously while preserving process integrity.
  3. Processes, tasks and the associated information always maintain context.
  4. Systems are unitary, information-rich and reflect the social needs of the business.

The first two of these are about build for change, and the last two are about design for people

This is all unfolding with the big vendors making some large investments in BPM-related technologies as well as newer things like Web 2.0 and mashups. Cisco’s TelePresence got a huge plug here (I’m guessing that they’re a big client of Forrester 🙂 ), including a clip from 24 that used it. This new focus will require some new skills as well: business analysts need to become process designers, and developers need to become (application) assemblers: this is how design for people and build for change come together. This is completely aligned with what I plan to discuss tomorrow in terms of putting the design of processes in the hands of the business and creating agile processes.

Moore finished up with how to get started on all this, from the viewpoint of IT management, business managers, process designers, application developers, enterprise architects, and the CIO.

TIBCO webinar: Why and how to move ahead with BPM and SOA

I’ve missed a few of the TIBCO webinar series this summer, such as the one on BPMN, and wanted to tune into this one to look at the modelling-to-execution process as TIBCO presents it.

They’re using a different webinar provider this time — the webinars that I did with them were all done with Webex, and we had one live failure that was a bit inconvenient although not disastrous. Ironically, the one slide that was incorrectly rendered in today’s presentation was related to visibility:

Emily Burns starts out with some starting slides on why people are implementing BPM. Reason #1: peer pressure from all your competitors that you read about in the Forrester and Gartner reports. 🙂  She quoted some interesting recent Forrester research that showed that a little over half of the respondents have an SOA strategy, and about 2/3 of the respondents that have some level of BPM consider it critical for their SOA strategy.

Jason McMahon of AmeriCredit was up next to talk about how they used TIBCO in their automotive loan servicing: 25,000 new loans each month, and over a million active loans. They had to streamline their processes and provide greater visibility into the processes, as well as ensure that they’re meeting regulatory requirements as they expand into new geographic regions. They had a fairly fast deployment cycle: the project kicked off in Q106, design started in Q2, development in Q3, and they were rolling it out in January 07. Version 2 is almost ready to go live. They saw a number of benefits:

  • Optimized business process by doing the “deal-killer” due diligence checks up front and reducing hand-offs
  • Customized processing of contracts by state/country (AmeriCredit operates in Canada, despite their name) and type of loan
  • Business process visibility, including better resource management, and automated alerts and reporting
  • Reduced training time since the rules are embedded in the process
  • Outsourcing readiness so that they can add in an outsourcing vendor for contract data entry (which is available now from at least one provider but not in use at AmeriCredit) and data validation/verification processes

McMahon sees a strong need for BPM-driven SOA design, where BPM is the consumer of services and therefore defines a big part of the requirements for services. This includes discovering services by looking at the current EAI-type integration points in existing processes, as well as during the design of future processes. Outside the actual process, there’s also a number of services that are required to service the processes even though they’re not called directly from the BPMS, such as process triggering events, user/role management, and open/close/purge of cases.

He talked in detail about what AmeriCredit has done with web services, including wrapping business rules/logic in web services so that they can be called from both a non-BPM web interface and the BPMS — exactly the reason that I believe in the separation of BPM and BR. He stepped through a number of best practices for defining services, such as establishing the appropriate level of granularity and ensuring encapsulation of some systems, then continues with an example of looking at a process with its existing integration points and combining the appropriate ones into specific web services.

Since they started with BPM and drove towards SOA, they ended up with a very BPM-focussed SOA, much of it defined by the BPM integration points but also with consideration of other applications that might share the same services.

McMahon’s last words of wisdom: using the BPM tool is simple for redesigning the processes; the hard part is having business users buy in to what the future-state processes should be, and you need a good business analyst to be the mediator here.

They use TIBCO’s iProcess for BPM and BusinessWorks for SOA, in case you haven’t guessed that already. Other implementation points: they built the entire UI in .Net, and extract the process data to a data warehouse for BI rather than hitting the BPMS directly.

A recording of the presentation will be available over the next few days, watch the TIBCO site for details or maybe someone from TIBCO will post the link here as a comment (hint, hint). You can check out the slides here, but you’ll really want to hear the audio as well, especially the Q&A.

BPM and SOA webinar

I’m listening in on a webinar with the rather grandiose title of “BPM and SOA: State of the Nation”, sponsored by webMethods. Paul Harmon of BPTrends kicked it off, and talked for some time about some general BPM and SOA definitions before diving into the results of a survey that they did earlier this year.

He had a great slide on how the maturity models for BPM and SOA intersect, with SOA MM level 5 (requires that companies know and track process measures) overlapping BPMM level 4 (processes are measured and managed systematically).

I found some of the survey questions odd; one of them offered four mutually exclusive responses:

  • You should not approach BPM without SOA
  • SOA cannot, ultimately, be successful without a BPM overlay
  • BPM is more successful and drives more benefits when deployed in an SOA environment
  • SOA is more successful and has more business relevance when tied to BPM

Not surprisingly, 74% of the responses were split between the last two; however, I think that most people would have selected both of those if they were able. That’s the whole point, right? BPM makes SOA more relevant, and SOA makes BPM easier to deploy. Interestingly, 19% chose the second option, that is, that SOA can’t succeed without BPM; I’d certainly agree that BPM makes it a lot easier to justify your SOA program, although I think that there’s other (non-BPM) business applications that also consume services and therefore help to justify it, so I wouldn’t make such a definitive statement.

He finished up with some good summary points:

  • BPMS and SOA are coming together. If you don’t know about either, you should learn about both.
  • If you are considering a BPMS tool, you’ll want to know how well it supports SOA.
  • Processes and services to automate BPMS and SOA will need to be carefully managed. You’ll especially want to know how any BPMS product supports SOA governance.

You can download the entire survey at BPTrends.com, and I’m sure that the webinar will be available for replay although it’s not at the original URL that I linked to yesterday; likely webMethods will have a link on their site.

BPM webinars this week

A couple of webinars to keep you amused this week:

Tomorrow, BPM and SOA: State of the Nation sponsored by webMethods (now part of Software AG), featuring Paul Harmon of BPTrends discussing their recent survey on BPM and SOA.

On Thursday, My BPM Success Story: Boundless Network sponsored by Lombardi, which features the VP of Development of Boundless Network talking about how they went from start to “success” (does that mean production?) in under 60 days.

I’ll drop in to one or both of these if I don’t end up travelling.

Take the Ajax challenge

I had a chance to talk to Kevin Hakman of TIBCO late last week about their Ajax Challenge (Kevin co-founded General Interface, which was acquired by TIBCO a couple of years back), the goal of which is to build the world’s largest mashup. You have to use General Interface to build it, but more interestingly, you have to use PageBus, a JavaScript client-side message bus that TIBCO just contributed to the OpenAjax Alliance as open source to become part of the OpenAjax Hub 1.0 installation.

The contest is splashy (win an oversized TV! win a video iPod!) but PageBus is the real news here: it provides a message bus for mashups in an attempt to eliminate the spaghetti mess of point-to-point integrations that we’re already starting to see emerge. In the enterprise world, this is why ESBs have become an essential part of any sizable application integration effort: without a message bus, you’re creating a unique integration between each pair of applications. Okay when you have two applications, but not when you have 10. [To be fair, usually you don’t have every application interact with every other application in a complex integration: each one may only interact with a couple of others, but that just shifts the pain point, it doesn’t eliminate it.]

Getting back to mashups and the OpenAjax Hub, the PageBus exposes the basic functions of messaging — publish, subscribe, unsubscribe — all in less than 5k of JavaScript, so that multiple Ajax components on an HTML page can share data using these standard methods. This allows the development of a mashup to be more easily split up between multiple developers since each can focus on their specific component and not on the interface between components; it will also allow for easier “no programing required” assembly of components within a PageBus-enabled mashup framework.

This is a pretty important step in mashup-land: I’m starting to see a lot of things referred to as mashups that are actually portals, where the components don’t intercommunicate, but the fundamental benefit of mashups is that they are an integration, not just components that happen to coexist on the same page.

TIBCO is apparently already using this in their BPM product for things such as task list publication, which means (I think) that you could create a mashup between your iProcess task list and some other component or data source — a real BPM mashup. Although many vendors are starting to provide RSS feeds of task lists/inboxes (I hope that my past year of nagging about this has had some contribution to those efforts), this is the first truly mashup-enabled BPM environment of which I’m aware.

The full OpenAjax Hub specification is about 4-6 weeks away from release, but the project is already on SourceForge. TIBCO will continue to develop the source and contribute to the open source efforts in the future; their press release about PageBus is here.

BPM Think Tank Day 3: BPM & SOA panel

We’re starting to wind down a bit, and many of the east coast people have taken off already to avoid the red-eye flight home so the audience is getting a bit sparse. Those of us with presentations this afternoon, however, are still here.

First after lunch is a panel on BPM & SOA, and how they complement each other, with Tony  Baer of onStrategies and Brenda Michelson of the SOA Consortium. This is more of the mini-presentation format rather than a true panel, but I promised Brenda that I wouldn’t blame the presenters for that. 🙂

Tony started out with the “BPM is from Venus, SOA is from Mars” phrase, which we’ve all been bandying about for a while, although he really meant ‘business is from Venus and IT is from Mars). Considering, however, that Venus is the goddess of love (collaboration in its most basic form, perhaps?) and Mars is the god of war (technology shoot-outs and other battle language), that may not be far from the truth.

He addressed the culture issues: both business and IT talk about business processes, but business tends to take a top-down approach versus IT’s bottom-up approach, and business is using BPM to rationalize the business whereas IT is using SOA as the next great way to integrate applications. He sees a process orchestration battleground between BPM and SOA about where to do integration in a process. He also pointed out that BPEL is still at the “checklist” level (that is, it’s on the RFP checklist but not actually used) for most BPM applications, an opinion that I stated here a couple of weeks ago.

Brenda was up next talking about business-driven SOA and the SOA Consortium, and looked at the correlation between an Economist survey of late last year with her personal findings in touring around talking to CIOs and CTOs: the top thing that they state is critical for both revenue generation and cost cutting is the creation of services. One CTO saw BPM, SOA, Lean and Six Sigma all as the same basic thing, namely business strategy and structure, and they need to work together without artificial divisions between them in order to enable a platform for business agility.

Before SOA, business and IT strategy weren’t well aligned and were often developed independently, and the business process became an output of an IT solution rather than driven directly by business requirements. Business and IT need to collaborate on both strategy and architecture, which in turn drives out portfolio planning and delivery of the business solutions. She pointed out that also “enterprise architecture” is currently mostly technology architecture with a bit of business architecture on the side (if you’re lucky), in the future it will become more balanced with equal contributions from business and technical architecture.

Part of what the SOA Consortium is doing is providing guidelines for how the too-technical technology architects can become more valuable enterprise architects, and to break the artificial divide between business and IT. Part of this, I think, is similar to something I posted a year and a half ago, where enterprise architecture is not an IT function, but something that is in a strategic position between business and IT.

We’re off to do the last of the roundtables now, where I’ll be leading one on Enterprise 2.0 and BPM mashups. My notes will be on paper, and I’ll summarize them over here sometime on that overnight flight home tonight.

BPM Think Tank Day 3: Randy Heffner

Day 3 opened with a keynote from Randy Heffner of Forrester on BPM in the world of digital business architecture.

He spoke about the old model of enterprise applications, which was that of functional silos with point-to-point integration between them, and how that’s changing to a process-centric model: not just using BPM to connect up the functional silos, but breaking down the functional silos so that the technology becomes a better reflection of the essential business processes. He envisions a number of portals for different worker roles — sales process portal, executive portal, fulfillment portal, etc. — with a layer of business-oriented services that support those portals by accessing virtualized enterprise data sources (which, of course, may still come from those old enterprise applications).

He seems focussed on SOA and business services rather than BPM and the orchestration of those services; he models the enterprise as portals consuming the services where presumably BPM is implicit in the portal in some way rather than discussing BPM directly. That becomes a bit more clear in a layered diagram of the new “programming” model, with various user interaction channels at the top, then a layer of interaction services, then human-centric process flow, then business services, then integration-centric process flow, then the underlying systems and data sources. Along the side spanning the layers are both business metadata (across interaction services and human-centric process flow) and technical metadata (across business services and integration-centric process flow), and business measurement and optimization across all the layers.

In his slide on the future of key technology platforms, there’s this big fuzzy bit in the middle called “business metadata core / business design platform”, which he admits is poorly defined and states that BPM provides a start to some of that functionality. The surrounding technologies, including the SOA sweet spot of business services, are otherwise pretty well defined in his view.

He finishes with a list of core competencies for the future of IT:

  • Deep integration of business and technology savvy: cross-functional focus on business design
  • Architecture visioning and strategy: vision + implementation = street-level strategy; multilevel investment strategy (strategic, soft dollar, hard dollar)
  • Portfolio management: road map for your business to position and justify investments
  • Project-level architecture governance: incremental build-out against architecture strategy

Forrester seems to have a distinct division between SOA and BPM in its analysts: they either know one domain or the other, but don’t seem all that comfortable talking about the other side. Heffner is definitely an SOA guy.