Although I attended some sessions related to BPM and operational process intelligence, last week’s trip to SAP TechEd && d-code 2014 gave me a bit more breathing room to look at the bigger picture — and aspirations — of SAP and their business technology offerings.
I started coming to SAPPHIRE and TechEd when SAP released a BPM product, which means that my area of interest was a tiny part of their primary focus on ERP, financials and related software solutions; most of the attendees (including the analysts and bloggers) at that time were more concerned with licensing models for their Business Suite software than new technology platforms. Fast forwarding, SAP is retooling their core software applications using HANA as an in-memory platform (cloud or on-premise) and SAP UI5/Fiori for user experience, but there’s something much bigger than that afoot: SAP is making a significant development platform play using those same technologies that are working so well for their own application refactoring. In other words, you can consider SAP’s software applications groups to be software developers who use SAP platforms and tools, but those tools are also available to external developers who are building applications completely unrelated to SAP applications.
They have some strong components: in-memory database, analytics, cloud, UI frameworks; they are also starting to push down more functionality into HANA such as some rudimentary rules and process functionality that can be leveraged by a development team that doesn’t want to add a full-fledged BRM or BPM system.
This is definitely a shift for SAP over the past few years, and one that likely most of their customers are unaware; the question becomes whether their application development tools are sufficiently compelling for independent software development shops to take a look.
Disclaimer: SAP paid my travel expenses to be at TechEd last week. I was not compensated for my time in any way, including writing, and the opinions here are my own.
Just finishing up some notes from my trip to SAP TechEd && d-code last week with the latest on their Operational Process Intelligence product, which can pull events and data from multiple systems – including SAP’s ERP and other core enterprise systems as well as SAP BPM – and provides real-time analytics via their HANA in-memory database. I attended a session on this, then had an individual briefing later to round things out.
Big processes are becoming a thing, and if you have big processes (that is, processes that span multiple systems, and consume/emit big data and high volume from a variety of sources), you need to have operational intelligence integrated into those processes. SAP is addressing this with their SAP Operational Process Intelligence, or what they see as a GPS for your business: a holistic view of where you are relative to your goals, the obstacles in your path, and the best way to reach your goals. It’s not just about what has happened already (traditional business intelligence), but what is happening right now (real-time analytics), what is going to happen (predictive analytics) and the ability to adjust the business process to accommodate the changing environment (sense and respond). Furthermore, it includes data and events from multiple systems, hence needs to provide scope beyond any one system’s analytics; narrow scope has been a major shortcoming of BPMS-based analytics in the past.
In a breakout session, Thomas Volmering and Harsh Jegadeesan gave an update and demo on the latest in their OPInt product. There are some new visualization features since I last saw it, plus the ability to do more with guided tasks including kicking off other processes, and trigger alerts based on KPIs. Their demo is based on a real logistics hub operation, which combines a wide variety of people, processes and systems, with the added complexity of physical goods movement.
Although rules have always been a part of their product suite, BRM is being highlighted as a more active participant in detecting conditions, then making predictions and recommendations, leveraging the ability to run rules directly in HANA: putting real-time guardrails around a business process or scenario. They also use rules to instantiate processes in BPM, such as for exception handling. This closer integration of rules is new since I last saw OPInt back at SAPPHIRE, and clearly elevates this from an analytics application to an operational intelligence platform that can sense and respond to events. Since SAP BPM has been able to use HANA as a database platform for at least a year, I assume that we will eventually see some BPM functionality (besides simple queuing) pushed down into HANA, as they have done with BRM, allowing for more predictive behavior and analytics-dependent functions such as work management to be built into BPM processes. As it is, hosting BPM on HANA allows the real-time data to be integrated directly into any other analytics, including OPInt.
OPInt provides ad hoc task management using a modern collaborative UI to define actions, tasks and participants; this is providing the primary “case management” capability now, although it’s really a somewhat simpler collaborative task management. With HANA behind the scenes, however, there is the opportunity for SAP to take this further down the road towards full case management, although the separation of this from their BPM platform may not prove to be a good thing for all of the hybrid structured/unstructured processes out there.
The creation of the underlying models looks similar to what I’ve been seeing from them for a while: the business scenario is defined as a graphical flow model (or imported from a process in Business Suite), organized into phases and milestones that will frame the visualization, and connected to the data sources; but now the rules can be identified directly on the process elements. The dashboard is automatically created, although it can be customized. In a new view (currently still in the lab), you will also be able to see the underlying process model with overlaid analytics, e.g., cost data; this seems like a perfect opportunity for a process mining/discovery visualization, although that’s more of a tool for an analyst than whoever might be monitoring a process in real-time.
I spent yesterday getting to Las Vegas for SAP TechEd && d-code and missed last night’s keynote with Steve Lucas, but up this morning to watch Björn Goerke — head of SAP Product & Innovation Technology — give the morning keynote on putting new technology into action. With the increasing rate of digital disruption, it’s imperative to embrace new ways of doing business, or risk becoming obsolete; this requires taking advantage of big data and real-time analytics as well as modern platforms. SAP’s current catch phrase is “Run Simple”, based in part on the idea of “one truth”, that is, one place for all your data so that you have a real-time view of your business rather than relying on separate sources for operations and analytics. You can’t run — and respond — at the speed that business requires if your analytics are based on yesterday’s transactions.
SAP HANA — their in-memory data store — allows for real-time analytics directly on operational transaction data, events, IoT machine data, social media data and more, all in a single data store. With the release of SAP HANA SPS09, they are adding support for dynamic tiering, streaming, enterprise information management, graphing, Hadoop user-defined functions, and multi-tenancy; these improve the management capabilities as well as the functionality. SAP deploys all of their business software solutions on HANA (although some more traditional databases are still supported in some products) with the goal to providing the basis for the “one truth” within business data.
Goerke was joined on stage by a representative from Alliander, an energy distribution company based in the Netherlands, and he demonstrated a HANA-based analytical dashboard based on geographic data that reduces the time required for geospatial queries — such as filtering by pipelines that are within a certain distance from buildings — from hours using more traditional database technology, to seconds with HANA. Geospatial data is one of the areas where in-memory data and analytics can really make a difference in terms of performance; I did a lot of my early-career software development on geospatial data, and there are some tough problems here that are not easily addressed by more traditional tools.
Another part of the simplicity message is “one experience” via the SAPUI5-based Fiori, providing for a more unified experienced between desktop and mobile, including management and distribution of mobile apps. They’ve added offline capabilities for their mobile apps – a capability widely ignored or dismissed as “unimportant” by developers who live and work only in areas blanketed in 4G and WiFi coverage, but critical in many real-world applications. Goerke demonstrated using some of the application development services — with some “help” from Ian Kimbell — to define an API, use it to create a mobile app, deploy it to a company app store, then install and run it: not something that most executives do live on stage at a keynote.
SAP now has a number of partnerships with hardware and infrastructure vendors to optimize their gear for SAP and especially for HANA: last week we saw an announcement about SAP running on the IBM cloud, and today we heard about how sgi is taking their well-known computational hardware capabilities and applying them to running transactional platforms such as SAP. SAP has also partnered with small software development shops to deliver the innovations in HANA-based applications needed to drive this forward. Applications developed on HANA can run on premise or in SAP’s managed cloud (and now IBM’s managed cloud), where they manage HANA and the SAP applications including Business Suite and Business Warehouse. Through a number of strategic acquisitions, SAP has much more than just your ERP and financials, however: they offer solutions for HR management, procurement, e-commerce, customer engagement and more. They also offer a rich set of development tools and application services for software development unrelated to SAP applications, allowing for applications built and deployed on HANA with modern mobile user interfaces and collaboration. In keeping with Goerke’s Star Trek theme in the keynote, very Borg-like.
Lots more here than I could possibly capture; you can watch the keynotes and other presentations online at SAP TechEd online.
AIIM is holding a series of ECM-related seminars across North America, and since today’s is practically in my back yard, I decided to check it out. It’s a free seminar so heavily sponsored; most of the talks are from the sponsor vendors or conversations with them, but John Mancini kicked things off and moderated mini-panels with the sponsor speakers to tease out some of the common threads.
The morning started with John Mancini talking about disruptive consumer technologies — cloud, mobile, IoT — and how these are “breaking” our internal business processes by fragmenting the channels and information sources. The result is information chaos, where information about a client lives in multiple places and often can’t be properly aggregated and contextualized, while still remaining secure. Our legacy systems, designed to be secure, were put in place before the devices that are causing security leaks were even invented; those older systems can’t even envision all the ways that information can leak out of an organization. Furthermore, the more consumer technologies advance, the further behind our IT people seem, making it more likely that business users will just go outside/around IT for what they need. New technologies need to be put in the context of our information management practices, and those practices adjusted to include the disruptors, rather than just ignore them: consider how to minimize risk in this information chaos state; how to use information to engage and collaborate, rather than just shutting it away in a vault; how to automate processes that involve information that may not be stored in an ECM; and how to extract insights from this information.
A speaker from Fujitsu was up next, stating some interesting statistics on just how big the information chaos problem is:
50% of business documents are still on paper; most businesses have many of their processes still reliant on paper.
Departmental CM systems have proliferated: 75% of organizations with a CM system have more than one, and 25% have more than four. SharePoint is like a virus among them, with an estimated 50% of organizations worldwide using SharePoint ostensibly for collaboration, but usually for ad hoc content management.
Legacy CM systems are themselves are a hidden source of costs, inefficiency and risk.
In other words, we have a lot of problems to tackle still: large organizations tend to have a lot of non-integrated content management systems; smaller organizations tend to have none at all.
We finished the first morning segment with an introduction from the event sponsors at small booths around the room:
An obvious omission (to me, anyway) was IBM/FileNet — not sure why they are not here as a sponsor considering that they have a sizable local contingent.
The rest of the morning was taken up with two sets of short vendor presentations, each followed by a Q&A session moderated by John Mancini: first Epson, K2 and EMC; then KnowledgeLake, HP Autonomy, Kodak alaris and OpenText. There were audience questions about information security and risk, collaboration/case management, ECM benefits and change management, auto-classification, SharePoint proliferation, cloud storage, managing content retention and disposal, and many other topics; lots of good discussions from the panelists. I was amazed (or maybe just sadly accepting) at the number of questions dealing with paper capture and disposal; I’ve been working in scanning/workflow/ECM/BPM since the late 80’s, and apparently there are still a lot of people and processes resistant to getting rid of paper. As a small business owner, I run a paperless office, and have spent a big chunk of my career helping much larger enterprises go paperless as part of streamlining their processes, so I know that this is not only possible, but has a lot of benefits. As one of the vendors pointed out, just do something, rather than sitting frozen, surrounded by ever-increasing piles of paper.
I skipped out at lunchtime and missed the closing keynote since it was the only bit remaining after the lunch break, although it looked like a lot of the customer attendees stayed around for the closing and the prize draws afterwards, plus to spend time chatting with the vendors.
We finished the camunda community day with an update from camunda on features coming in 7.2 next month, and the future roadmap. camunda releases the community edition in advance of the commercial edition; this is the way that open source should work, but some commercial open source vendors switch that around so that the community version lags by as much as a full version.
The highlights of the 7.2 release are as follows:
CMMN-based case management engine, which includes the core activities (stages, human tasks, process tasks, case tasks, milestones and sentries), the base case instance and plan item lifecycle, and a CMMN model API and REST API on a common process engine. They demonstrated a basic case manager UI that can manage cases and the related tasks; I assume that this is really just a demo of what can be done rather than intended as production code. They also don’t have case modeling in their modeler yet, so it’s early times.
New tasklist, updating the tasklist UI that they released just before announcing camunda as an open source project. It allows filters to be defined, including specifying who can see the results of a filter in addition to the search criteria; that filter then appears as a tab on the task list, in the color defined by the filter author. The sort order can’t currently be defined as part of the filter, but can be set on the general tasklist interface. This adds a third (left) column to the tasklist UI, which also shows the list of tasks and the form for the selected task. Still work to be done, but the new filters capability is a big step up, providing a conceptually similar (but much different graphically) functionality to the Brazos portal filters.
There were a list of other smaller enhancements and fixes, from platform support to performance improvements to new functions.
Another lab project is the camunda BPM workbench, a debugging tool that allows inspection of the runtime state of processes alongside the process model, allowing breakpoints to be set in the process model (rather than in code). A console interface allows for interrogation and updating of the process variables as the developer steps through the process. The process model is displayed using the bpmn.io viewer.
At the end of all the roadmap sessions, the audience had a chance to say what was most important for them in terms of what will be implemented when; there were questions about case management, centralized model repositories, bulk runtime operations and other features.
A great half-day; this is the first time that I’ve attended an open source code community day, and it’s quite a different environment from a typical vendor conference. We’re about to enter the beer-drinking portion of the day so I will sign off for today; I’m giving the keynote at the main camunda user conference tomorrow morning, and not sure how much blogging that I’ll do during after that.
Disclaimer: camunda paid my travel expenses to be here today and tomorrow, and is providing a speaking fee for tomorrow’s keynote. I was not compensated for blogging, and the opinions here (and in my keynote) are my own.
The second customer speaker at camunda’s community day was Peter Hachenberger from 1&1 Internet, describing how they use Signavio and camunda BPM to create their Process Platform, which is in turn used by their clients’ developers for building and executing automated processes. His presentation was primarily about the details of their technical implementation of the platform; they have built some fairly comprehensive tools for monitoring and managing executing processes, many of which are facilitated by changes that they made to the core process engine, including retry behavior, process ID generator, multiple business keys, an asynchronous process starter API, an extended REST API and a few new commands. Since camunda BPM is open source, any customer such as 1&1 can take a copy of the code and make changes to it, optionally returning them to the community if they are valuable to others. There’s a bit of danger in this, in that if you make changes to core functionality (such as the engine) rather than create an extension or plug-in, and those changes do not end up back in the community version, you’re not only on your own for future development on those components but may not be able to upgrade to future versions.
We had a number of short (10 minute) presentations from community members to discuss extensions that they are working on:
Grails plugin to add camunda functionality to Grails applications
OSGi module extension for greater flexibility and configurability at runtime, including sharing process engines as services
Elasticsearch extension to write camunda BPM history data to an elasticsearch cluster to allow full-text searching, enabling more comprehensive analytics
camunda mocking extensions for process testing with mockito
Cockpit plugin to add interactive graphs and some statistical calculations (e.g., aggregation, regression, min/max) for process monitoring directly on the camunda history database
Some of these extension projects were done by camunda employees, but great to see the external community contributions as well.
I am giving the keynote at camunda’s BPMcon conference tomorrow, and since I arrived in Berlin a couple of days early, camunda invited me to attend their community day today, which is the open source community meeting. Nice to see such a great turnout — 70 or 80 people? — and I definitely wasn’t the one to travel the furthest to get here, since today’s opening presentation is from Rob Parker of Australia Post. Australia has a lot of the same issues as Canada when it comes to nation-wide services such as post, since we both have a lot of geography and not a lot of population: this means that a lot of services have to be delivered at a fiscal loss to sparsely-populated areas, with specific rules about what percentage of the population has to be within a certain distance of a postal outlet.
Post offices in particular are hard-hit by digital disruption; Australia Post has seen their letter delivery service decline by 1 billion articles (and the related revenue), even though the number of addresses to cover has increased. However, they have seen their parcel delivery business increase, even though this is a competitive business with courier companies. They’re also offering a number of other products, such as electronic bill payment, digital mail delivery and even passport interviews, which has driven them to create a more integrated multi-channel/multi-product architecture to be able to quickly bring new products to market. They’re using camunda BPM for their order management processses, both for customer orders and service fulfillment orders. Customer order processes support the various customer channels, then drive out one or more service order processes to fulfill a customer order.
They decided to use BPM in order to externalize processes from applications, making for more agile development and better reusability. They picked camunda because they wanted “just enough technology”: that is, they wanted to add process management to their existing Java application development environment, not rewrite all of their apps in a proprietary, monolithic BPMS app dev environment. camunda BPM is used to implement the multiple service order processes that might be kicked off by any given customer order, with their overall architecture handling the communication between the two order management layers: the customer order layer as a consumer for the service order layer producer.
Parker went into a lot of detail of how they have implemented this architecture, putting their BPM usage into the context of their overall technical architecture, and walked through the general process model for their service order that instantiates a dispatcher process for each customer order, which in turn instantiates a subprocess for each line item in the order. They really want to implement all of this in camunda, but are still using TIBCO for the dispatching process while they work out some of the sticky bits such as how subprocess cancelations are propagated to the parent process. They are also having some challenges with handling process versions, considering that they run 7×24: they need a mapping table that takes these temporal anomalies into consideration, so that the process version in use may be tied to the order date for longer-running order processes. They also created a business dashboard by modifying Cockpit, the camunda IT operations dashboard, to remove all of the “dangerous” operations while exposing the work in progress, and adding some additional functions such as searching by a business key.
Parker ended up with their outcomes, some expected, some less so: basically, BPMN 2.0 is really working for them both for business-IT collaboration and model-driven development; this level of business-IT alignment means that error handling can be shared, with business handling business errors, and IT handling IT errors. They found that developers became productive very quickly since they were just adding some tools to their existing familiar Java application development environment, although some had to be gently reminded to use the BPM capabilities instead of writing code.
It was great to see the reactions and interactions of the camunda team during the presentation: Australia Post is a “do-it-themselves” open source user of camunda, and as Parker discussed some of the shortcomings, they were obviously taking some notes for future work. The presentation finished with him being presented as an award for the non-camunda person who contributed most to the community forum discussions, suggesting that you get out of open source what you put into it.
James Taylor of Decision Management Solutions and I are doing some research into the use and integration of BPM (business process management) and DM (decision management) technology into mobile applications, as background for co-authoring a white paper. For this study we are focused on the mobile applications that you use to interact with businesses (as a consumer), government (as a constituent) or your employer (as an employee), NOT games or personal productivity tools.
As part of this research, we are doing a short survey on the use, development and support of mobile applications; please help us out by taking the survey:
The first few questions are about your own use of mobile applications. If you are involved with the development of mobile applications, there are a few additional questions. It should take no more than 5-10 minutes to complete.
Last week BP3 announced the latest addition to their Brazos line of UI tooling for IBM BPM: Brazos Portal. Scott Francis gave me a briefing a few days before the announcement, and he had Ivan Kornienko, their director of UI/UX, along to give a demo.
As a recap, BP3 is a boutique-sized IBM BPM services and ISV partner, made of up a large contingent of ex-Lombardians (Lombardiites?) who know the IBM BPM product as well or better than any other partner out there. In addition to implementation services, they offer a free UI toolkit that allows you to easily build responsible HTML interfaces for IBM BPM applications that can support both mobile and desktop browsers. They sell the services and support around the Brazos UI Toolkit, but give it away to encourage adoption. This seems to be working: as of the end of May, they had more than 400 registered developers who had downloaded the toolkit and about 40 production deployments.
However, like that tendency to start redecorating your whole house once those new curtains make you realize how outdated everything else is, customers using applications built with the Brazos UI Toolkit wanted a more responsive and modern portal interface than the out-of-the-box IBM portal. After all, the portal is where many BPM users start and organize their day, only launching the applications once they’ve identified a specific item that they want to work on. According to BP3, the IBM BPM portal, in addition to not being optimized for modern browsers or mobile devices, has some performance issues due to server-side searching, sorting and filtering; it’s also difficult to customize and can be brittle once customized.
BP3’s answer to this is the Brazos Portal: responsive and mobile-ready, customizable, and with local search, sort, filter and task display for optimal performance. Although Brazos Portal is being positioned as a mobile portal – I suspect that BP3 doesn’t want to explicitly position against IBM’s own desktop portal – it appeared to be completely usable as a desktop portal as well, with the added benefit of a unified experience across all platforms. However, the two portals can coexist peacefully on the same process data, allowing for only certain users or devices to be on Brazos while the others are on IBM.
I’m getting to the part about the ring, really.
We started the demo with a sleek-looking but fairly standard view of an unfiltered user task list, showing a few metadata columns (task name, class, due date), with displays at the top showing aggregate information for work in progress, complete and due/overdue tasks. Clicking on an item in the task list opens the task application screen for processing the task (a Coach, in the IBM BPM terminology); in the demo, this opened an application written using the Brazos UI Toolkit although I’m not sure that’s a requirement [Scott, feel free to chime in here]. Clicking on a column in the task list sorts by that column, as expected; this is fast since it is done locally rather than requiring a re-query of the BPM server as (I think) the IBM portal does. It’s all in monochromatic greys and black, except for thin bars of color on the left of each item in the task list.
It’s also responsive: we were viewing it in a desktop browser, and when the browser was resized to the size of a tablet or mobile phone, the portal responded appropriately.
Then there’s the ring.
At the top right of the portal is a circular graphical element, with the BP3 logo in the center, that I initially just gloss over as window dressing. However, in the same block, there are some small controls to the right of it, a count of the cases and tasks in the current list, and a search field. This ring interface, or “Task Drive”, is actually a metadata filter control, positioned at the top right so that it’s right under where your right thumb would be on a touch-enabled screen, and the search field combines with the filtering selected on the ring. The appearance of the Task Drive changes as you interact with the portal to show you information about what is currently displayed, and how it breaks down by metadata. In our unfiltered list, for example, we had 44 cases and tasks, but hovering over one of the sectors in the ring showed us the corresponding metadata field that would be filtered and the resulting number of cases and tasks. Selecting a segment of the ring filtered the list to just those in the class “Client On-Boarding”, which had 14 cases and tasks. There are actually concentric rings in the Task Drive control, giving us a visual indicator of the potential filters; we had initially filtered based on the work class, and could then filter further by task name. We could also filter via a free-form search on any task metadata by entering text in the search field beside the ring.
Things start to get colorful with groups, which are pre-defined filters: you define a view of up to three columns with filter criteria, using any metadata including custom fields defined on a work class as well as standard fields like due date and priority. This creates a new group in the task list, which is assigned a unique color, showing those tasks that meet the filtering criteria. Those that don’t meet the filter criteria are still in the black and grey “Unfiltered” list. Presumably, items can appear in more than one group if they meet the criteria, so the counts of the items in each group may be more than the overall count; only those that do not meet any of the groups’ criteria remain in the Unfiltered list.
Once you define groups, the Task Drive becomes equally colorful: the ring is now in sectors by color, with concentric rings indicating the metadata filtering/drilldowns for each group. We looked at an example that had four groups defined: High Priority Claims, Client On-Boarding, Other Claims, and New Coverage Branch, appearing as orange, red, green and purple (actually more like lavender). Each group’s list could be collapsed into a single title bar showing only the group name and item count, or expanded to show all items within it. Hovering over the green ring segment showed us that that corresponded to Other Claims, and had 6 cases and tasks. Clicking on the purple segment drilled into the New Coverage Branch group directly, then hovering over segments of the now-purple ring showed us the breakdown of items by task name.
The graph displays in the upper left of the portal actually contain live controls as well: the center one is a date control that cycles between today, days, weeks and months for filtering the other content in this navigation header including the Task Drive. Checking the “show complete” control to the right of the Task Drive also makes the Complete graph at the top left interactive.
After the briefing with a live demo, I played around with the online demo version; one interesting feature (bug?) is that on my touch-screen Lenovo Yoga Windows 8 laptop, I could only get the ring to respond to the touch screen, not to the mouse control, although everything else worked fine with either.
The ring – sorry, Task Drive – is an extremely clever and beautiful piece of UI. However, I’m concerned about its usability by more traditional users, since it may not be as intuitive as the designers hope. Having the ability to pre-define groups for users (which is planned for the paid versions) will help sort out some of the confusion for users, but many may ignore the ring as if it were a graphical element, and just scroll through the lists in the groups rather than using the metadata filtering. BP3 reported that they had had very good reception of this as a filtering control, but their test group is likely skewed towards mobile users and the inherent selection bias; if they expect this to be used on the desktop as well, or by more traditional remote workers whose paper clipboard is replaced with a tablet, then they need to understand the learning curve behind that control.
Brazos Portal supports IBM BPM 8.0.1 and forward. It does not yet expose the new case management features in the 8.5 release, but that will probably show up in a future release. They have a fairly ambitious list of potential future features, and will likely develop these according to demand, so if you try out Brazos Portal and find something missing, be sure to give your feedback to BP3.
Brazos Portal is available in both free and paid versions, with the free version having launched last week. The free version has no active support, but BP3 will likely help out if they can in order to spur adoption, and would probably be happy to sell you some services to get you up and running. The premium version will include standard support and SLAs, plus features for pre-defined/shared user groups, custom branding and API incorporation; the enterprise version will include a support agreement customized to the organization, plus custom integration and cross-platform federation to allow multiple BPM servers to be integrated into a single portal.