Sal Vella on Technologies for a Smarter Planet at CASCON2011

I attended the keynote at IBM’s CASCON conference in Toronto today, where Judy Huber, who directs the IBM Canada software lab, kicked off the session by reminding us that IBM software development has been happening in Canada since 1967 and continues to grow, and of the importance of collaboration between the research and industry communities. She introduced Joanna Ng, who is the head of research at the lab, to congratulate the winners of the most influential paper from CASCON 2001 (that date is not a typo, it’s a 10-year thing): Svetlana Kiritchenko and Stan Matwin for “Classification with Co-Training” (on email classification).

The main speaker was Sal Vella, VP of architecture and technology within the IBM software group, talking about technologies to build solutions for a smarter planet. Fresh from the IOD conference two weeks ago, I was all primed for this; there was a great booth at IOD that highlighted “smarter technology” with some interesting case studies. IBM’s smarter planet initiative is about technologies that allow us to do things that we were never able to do before, much of which is based on the immeasurable volume of data constantly produced by people, devices and systems. Consider electricity meters, like the one that you have in your home: it used to be that these were read once per month (if you were lucky) by a human, and the results entered into a billing system. Now, smart meters are read every 15 minutes to allow for time of use billing that rewards people for shifting their electricity usage away from peak periods. Analytics are being used in ways that they were never used before, and he discussed the popular Moneyball case of building a sports team based on player statistics. He also spoke about an even better use of analytics to create “solutions for a partying planet”: a drinks supplier predicting sports games outcomes to ensure that the pubs frequented by the fans of the teams most like to win had enough alcohol on hand to cover the ensuing parties. Now that’s technology used for the greater good. Winking smile

There are a lot of examples of big data and analytics that were previously unmanageable that are now becoming reasonable targets, most of which could be considered event-based: device instrumentation, weather data, social media, credit card transactions, crime statistics, traffic data and more. There are also some interesting problems in determining identity and relationships: figuring out who people really are even when they use different versions of their name, and who they are connected to in a variety of different ways that might indicate potential for fraud or other misrepresentation. Scary and big-brotherish to some, but undeniably providing organizations (including governments) with deeper insights into their customers and constituents. If those who complain about governments using this sort of technology “against” them would learn how to use it themselves, the tables might be turned as we gain insight into how well government is providing services to us.

We heard briefly from Charles Gauthier, acting director at the institute for information technology at National Research Council (NRC) Canada. NRC helped to create the CASCON conference 21 years ago, and continue to sponsor it; they support research in a number of areas that overlap with CAS and the other researchers and exhibitors presenting here.

The program chairs, Marin Litoiu of York University and Eleni Stroulia of University of Alberta presented awards for the two outstanding papers from the 22 papers at the conference:

  • “Enhancing Applications Robustness in Cloud Data Centres” by Madalin Mihailescu, Andres Rodriguez and Cristiana Amza of University of Toronto, and Dmitrijs Palcikovs, Gabriel Iszlai, Andrew Trossman and Joanna Ng of IBM Canada
  • “Parallel Data Cubes on Multi-Core Processors with Multiple Disks” for best student paper, by Hamidreza Zaboli and Frank Dehne of Carlton University

We finished with a presentation by Stan Matwin of University of Ottawa, co-author of the most influential paper presentation on email classification from the CASCON of 10 years past (his co-author is due to give birth on Wednesday, so decided not to attend). It was an interesting look at how the issue of email classification has continued to grow in the past 10 years; systems have become smarter since then, and we have automated spam filtering as well as systems for suggesting actions to take (or even taking actions without human input) on a specific message. The email classification that they discussed in their paper was based on classification systems where multiple training sets were used in concert to provide an overall classification for email messages. For example, two messages might both use the word “meeting” and a specific time in the subject line, but one might include a conference room reference in the body while the other references the local pub. Now, I often have business meetings in the pub, but I understand that many people do not, so I can see the value of such a co-training method. In 2001, they came to the conclusion that co-training can be useful, but is quite sensitive to its parameters and the learning algorithms used. Email classification has progressed since then: Bayesian (and other) classifiers have improved drastically, data representation is richer (through the use of meta formats and domain-specific enrichment) to allow for easier classification. social network and other information can be correlated, and there are specific tailored solutions for some email classification applications such as legal discovery. Interesting to see this sort of perspective on a landmark paper in the field of email classification.

I’m not sticking around for any of the paper presentations, since the ones later today are a bit out of my area of interest, and I’m booked the rest of the week on other work. However, I have the proceedings so will have a chance to look over the papers.

NSERC BI Network at CASCON2011 (Part 2)

The second half of the workshop started with Renée Miller from University of Toronto digging into the deeper database levels of BI, and the evolving role of schema from a prescriptive role (time-invariant, used to ensure data consistency) to a descriptive role (describe/understand data, capture business knowledge). In the old world, a schema was meant to reduce redundancy (via Boyce-Codd normal form), whereas the new world schema is used to understand data, and the schema may evolve. There are a lot of reasons why data can be “dirty” – my other half, who does data warehouse/BI for a living, is often telling me about how web developers create their operational database models mostly by accident, then don’t constrain data values at the UI – but the fact remains that no matter how clean you try to make it, there are always going to be operational data stores with data that needs some sort of cleansing before effective BI. In some cases, rules can be used to maintain data consistency, especially where those rules are context-dependent. In cases where the constraints are inconsistent with the existing data (besides asking the question of how that came to be), you can either repair the data, or discover new constraints from the data and repair the constraints. Some human judgment may be involved in determining whether the data or the constraint requires repair, although statistical models can be used to understand when a constraint is likely invalid and requires repair based on data semantics. In large enterprise databases as well as web databases, this sort of schema management and discovery could be used to identify and leverage redundancy in data to discover metadata such as rules and constraints, which in turn could be used to modify the data in classic data repair scenarios, or modify the schema to adjust for a changing reality.

Sheila McIlraith from University of Toronto presented on a use-centric model of data for customizing and constraining processes. I spoke last week at Building Business Capability on some of the links between data and processes, and McIlraith characterized processes as a purposeful view of data: processes provide a view of the data, and impose policies on data relative to some metrics. Processes are also, as she pointed out, are a delivery vehicle for BI – from a BPM standpoint, this is a bit of a trivial publishing process – to ensure that the right data gets to the right stakeholder. The objective of her research is to develop business process modeling formalism that treats data and processes as first class citizens, and supports specification of abstract (ad hoc) business processes while allowing the specification of stakeholder policies, preferences and priorities. Sounds like data+process+rules to me. The approach is to specify processes as flexible templates, with policies as further constraints; although she represents this as allowing for customizable processes, it really just appears to be a few pre-defined variations on a process model with a strong reliance on rules (in linear temporal logic) for policy enforcement, not full dynamic process definition.

Lastly, we heard from Rock Leung from SAP’s academic research center and Stephan Jou from IBM CAS on industry challenges: SAP and IBM are industry partners to the NSERC Business Intelligence Network. They listed 10 industry challenges for BI, but focused on big data, mobility, consumable analytics, and geospatial and temporal analytics.

  • Big data: Issues focus on volume of data, variety of information and sources, and velocity of decision-making. Watson has raised expectations about what can be done with big data, but there are challenges on how to model, navigate, analyze and visualize it.
  • Consumable analytics: There is a need to increase usability and offering new interactions, making the analytics consumable by everyone – not just statistical wizards – on every type of device.
  • Mobility: Since users need to be connected anywhere, there is a need to design for smaller devices (and intermittent connectivity) so that information can be represented effectively, and seamless with representations on other devices. Both presenters said that there is nothing that their respective companies are doing where mobile device support is not at least a topic of conversation, if not already a reality.
  • Geospatial and temporal analytics: Geospatial data isn’t just about Google Maps mashups any more: location and time are being used as key constraints in any business analytics, especially when you want to join internal business information with external events.

They touched briefly on social in response to a question (it was on their list of 10, but not the short list), seeing it as a way to make decisions better.

For a workshop on business intelligence, I was surprised at how many of the presentations included aspects of business rules and business process, as well as the expected data and analytics. Maybe I shouldn’t have been surprised, since data, rules and process are tightly tied in most business environments. A fascinating morning, and I’m looking forward to the keynote and other presentations this afternoon.

NSERC BI Network at CASCON2011 (Part 1)

I only have one day to attend CASCON this year due to a busy schedule this week, so I am up in Markham (near the IBM Toronto software lab) to attend the NSERC Business Intelligence Network workshop this morning. CASCON is the conference run by IBM’s Centers for Advanced Studies throughout the world, including the Toronto lab (where CAS originated), as a place for IBM researchers, university researchers and industry to come together to discuss many different areas of technology. Sometimes, this includes BPM-related research, but this year the schedule is a bit light on that; however, the BI workshop promises to provide some good insights into the state of analytics research.

Eric Yu from University of Toronto started the workshop, discussing how BI can enable organizations to become more adaptive. Interestingly, after all the talk about enterprise architecture and business architecture at last week’s Building Business Capability conference, that is the focus of Yu’s presentation, namely, that BI can help enterprises to better adapt and align business architecture and IT architecture. He presented a concept for an adaptive enterprise architecture that is owned by business people, not IT, and geared at achieving measurable business success. He discussed modeling variability at different architectural layers, and the traceability between them, and how making BI an integral part of an organization – not just the IT infrastructure – can support EA adaptability. He finished by talking about maturity models, and how a closed loop deployment of BI technologies can help meet adaptive enterprise requirements. Core to this is the explicit representation of change processes and their relationship to operational processes, as well as linking strategic drivers to specific goals and metrics.

Frank Tompa from University of Waterloo followed with a discussion of mapping policies (from a business model, typically represented as high-level business rules) to constraints (in a data model) so that these can be enforced within applications. My mind immediately went to why you would be mapping these to a database model rather than a rules management system; his view seems to be that a DBMS is what monitors at a transactional level and ensures compliance with the business model (rules). His question: “how do make the task of database programming easier?” My question: “why aren’t you doing this with a BRMS instead of a DBMS?” Accepting his premise that this should be done by a database programmer, the approach is to start with object definitions, where an object is a row (tuple) defined by a view over a fixed database schema, and represents all of the data required for policy making. Secondly, consider the states that an object can assume by considering that an object x is in state S if its attributes satisfy S(x). An object can be in multiple states at once; the states seem to be more like functions than states, but whatever. Thirdly, the business model has to be converted to an enforcement model through a sort of process model that also includes database states; really more of a state diagram that maps business “states” to database states, with constraints on states and state transitions denoted explicitly. I can see some value in the state transition constraint models in terms of representing some forms of business rules and their temporal relationships, but his representation of a business process as a constraint diagram is not something that a business analyst is ever going to read, much less create. However, the role of the business person seems to be restricted to “policy designer” listing “states of interest”, and the goal of this research is to “form a bridge between the policy manager and the database”. Their future work includes extracting workflows from database transaction logs, which is, of course, something that is well underway in the BPM data mining community. I asked (explicitly to the presenter, not just snarkily here in my blog post) about the role of rules engines: he said that one of the problems was in vocabulary definition, which is often not done in organizations at the policy and rules level; by the time things get to the database, the vocabulary is sufficiently constrained that you can ensure that you’re getting what you need. He did say that if things could be defined in a rules engine using a standardized vocabulary, then some of the rules/constraints could be applied before things reached the database; there does seem to be room for both methods as long as the business rules vocabulary (which does exist) is not well-entrenched.

Jennifer Horkoff from University of Toronto was up next discussing strategic models for BI. Her research is about moving BI from a technology practice to a decision-making process that starts with strategic concerns, generates BI queries, interprets the results relative to the business goals and decide on necessary actions. She started with the OMG Business Motivation Model (BMM) for building governance models, and extended that to a Business Intelligence Model (BIM), or business schema. The key primitives include goals, situations (can model SWOT), indicators (quantitative measures), influences (relationships) and more. This model can be used at the high-level strategic level, or at a more tactical level that links more directly to activities. There is also the idea of a strategy, which is a collection of processes and quality constraints that fulfill a root-level goal. Reasoning that can be done with BIMs, such as whether a specific strategy can fulfill a specific goal, and influence diagrams with probabilities on each link used to help determine decisions. They are using BIM concepts to model a case study with Rouge Valley Health System to improve patient flow and reduce wait times; results from this will be seen in future research.

Each of these presentations could have filled a much bigger time slot, and I could only capture a flavor of their discussions. If you’re interested in more detail, you can contact the authors directly (links to each above) to get the underlying research papers; I’ve always found researchers to be thrilled that anyone outside the academic community is interested in what they’re doing, and are happy to share.

We’re just at the md-morning break, but this is getting long so I’ll post this and continue in a second post. Lots of interesting content, I’m looking forward to the second half.