TransitCamp: MyTTC

My friend Kieran, as a result of last year’s TransitCamp, got together with a couple of other transit/technology geeks to create MyTTC, a community-driven site for Toronto transit information that will launch in about two weeks, in time for the next TransitCamp event. They did this in response to the deficiencies of the official TTC website, which is famously bad for providing decent information about actually using the third most heavily used transit system in North America.

MyTTC is focused primarily on routes and schedules, and contains all of the route and schedule data that they’ve gleaned from the TTC site and other sources. Each route is mapped using embedded Google Maps, with individual stops marked and the arrival times expected for surface vehicles at each stop. There’s also transfer information for each stop and route, so that you can see which other routes connect to that route and stop, and other context-sensitive information about stops.

But it’s more than just a static information site; it has a lot of wiki-like aspects for user-generated content (although it’s built on Ruby and Merb rather than a wiki platform). Users can add stops to routes — the TTC routes/schedules only show main stops, not every stop — and the scheduled time for the newly-added stops will be interpolated from the existing data. Users can also add other context-sensitive information about stops, such as typical delays at certain times of day, points of interest, and places to wait out of the weather where you can see the bus/streetcar approaching. Users can also bookmark their favourite stops, which will appear on their personalized home page when they’re logged in.

One of the largest issues that they’ve had, and are likely to continue to have, is getting data from the TTC: many transit authorities are notoriously miserly with their data, in spite of the fact that it’s created mostly with public funding. For MyTTC, they used the route and schedule data that’s available on the TTC site, much of it captured manually (and painfully), but TTC hasn’t been willing to share any data in a more easily-ingestible form, or to share the data that will soon be generated by global positioning systems on surface vehicles. This, of course, is a political problem rather than a technical problem, and likely others will need to be involved to help resolve this.

MyTTC is attempting to create a platform where the data is fully open (although they are not, at this time, making the code open source): they will be providing full XML and JSON APIs, SQL dumps of the data, and GTFS (Google Transit Feed Spec). In fact, their contacts at Google have said that Google Transit will accept their data in place of TTC’s, likely because it’s more accessible and complete. They don’t plan to monetize the APIs, but would rather have other sites export the data and use it for their own purposes directly.

The platform itself is transit system-agnostic, and could be used for any transit system. There’s explicit support for the iPhone, and other mobile platforms such as Blackberry will be tested — in fact, I think that I just volunteered for that. There are plans for some SMS interfaces, such as being able to send an SMS message to the site with your current location and get back related information, and we discussed the use of Twitter including direct access (like the BART experiment last year) and Twitter-based applications (like CommuterFeed). They also want to add a trip planner to MyTTC before the launch.

The wiki page for this session, which includes these notes of mine plus other attendees’ contributions, is here.

TransitCamp!

Most of the unconference "camp" type events that I attend are technology-related, but today I’m attending TransitCamp in Toronto, originally conceived by a few of my TorCamp friends and now happening in other cities as well as becoming a vital part of the greater Toronto area transit planning process. What started out a year ago as an informal collection of people interested in transit has evolved into a much more mainstream channel for sharing the conversation about transit amongst the providers, consumers and any other interested parties.

One of the general managers of Metrolinx said a few words in the opening session; it’s great that they’re participating directly although this isn’t their event. This is the first in a series of events that will happen around the greater Toronto area, since transit is not just about downtown, but also about how all the regional transportation options tie together.

If you’re attending TransitCamp or want to follow along with the sessions, they’re all on the wiki (or will be, once all the notes are entered).

If you’re interested in transit in Toronto, join the community at the Metronauts site.

You can follow a collection of the conversations on the Onaswarm Metronauts swarm, or through Twitter on the #metronauts or #transitcamp hashtags.

Experience!Tech closing keynote: Tom Kelley, IDEO

For the closing keynote, we’re back to the live video feed from the IDC conference in Boston to hear Tom Kelley of IDEO. He wrote The Art of Innovation, and is following it up with The Ten Faces of Innovation, which is also the title of his presentation today (alarmingly, scheduled for 1.25 hours at the end of the day).

Not surprisingly, he’s talking about innovation. In theory, everyone is in favour of innovation, but in practice, it falls into Covey’s quadrant #2: important but not urgent. This causes innovation to be put off repeatedly, until someone else starts taking your business because they’re out-innovating you. So even though it tends to live in quadrant #2, it can take on an urgency if you fail to attend to it.

Some thoughts:

  • The type of innovation required in today’s markets is to passionately pursue new ways to serve your customers.
  • Standing still will result in the death of a brand: to quote the Red Queen, “If you want to get somewhere else, you must run at least twice as fast as that.”

Samsung is using design and innovation to outpace Sony: the brand value of Samsung surpassed Sony in 2004 for the first time ever, primarily because Sony thought that their superior brand value would carry them without maintaining a high level of innovation (or increasing their level of innovation): they’re still innovators, just not as innovative as Samsung, and barely enough to keep their brand value from dropping further.

In his first book, he focused on the tools of innovation; in the second book, he focuses on the people side, or the roles and corporate culture required for innovation. He sees several roles that contribute (which are his ten faces of innovation):

  • The learning roles:
    • The anthropologist
    • The experimenter
    • The cross-pollinator
  • The organizing roles:
    • The hurdler
    • The collaborator
    • The director
  • The building roles:
    • The experience architect
    • The set designer
    • The caregiver
    • The storyteller

He then looked at two of those roles: the anthropologist and the experience architect (he actually allowed the audience in Boston to decide between him talking about the experimenter and the experience architect by throwing a little squishy ball that was provided at their seat).

The anthropologist goes into the field to observe people’s experiences with products in order to drive innovation of those products through empathic design. They have the ability to see things that have previously gone unobserved, and have a lot of “a-ha” moments of discovering innovative ideas and unsolved problems. Anthropologists are necessary because when you’re immersed in an environment, you can’t see either the problems or potential solutions. Kelley had a lot of great anecdotes on how anthropologists have improved product design.

The experience architect is focused on creating an amazing experience for each individual consumer of a product: not just creating a great product, but marketing it as such so that people will chose it over the competitor, even paying more or going out of their way to get it. Think Starbucks coffee or Westin’s Heavenly Bed, both of which have built a brand with a very loyal following. Companies are building in features that no one uses, but the key is to build something simple, looking at the entire customer experience to identify the best places to make their experience extraordinary.

I really enjoyed Kelley’s talk: very engaging and lots of great anecdotes about design, making the 1.25 hours speed by. Having been to a lot of conferences where the closing keynote is completely unrelated with anything to do with the conference content, it was great to hear a keynote that’s relevant as well as entertaining.

Experience!Tech Social Media panel

I’m in the panel on social media featuring Jay Goldman of Radiant Core, Jeremy Wright of B5 Media and Tomi Poutanen, former head of Yahoo Social Search, moderated by Jesse Hirsh.

Jeremy: social networks are broken. We have a huge amount of data that we can pivot around people who we know, but we should really be pivoting around what interests us.

Jay: there is a perception of social network fatigue amongst those of us who are early adopters, but the mainstream hasn’t even adopted them yet.

Tomi: “I haven’t deleted my Facebook account…yet”.

Twitter channel: if you have Facebook fatigue, you’re using it wrong.

Tomi: social networks tend to attract like-minded people when they’re starting up, but can grow to a size where they’re too general to have a cohesive community.

Jay: Pownce is like that summer blockbuster movie that you’re looking forward to, but it sucks when you finally try it.

Jesse: should social networks be purely for entertainment, or can they be used in the enterprise?

Jay: different social networks can solve different problems for different people. He goes to LinkedIn to connect to people professionally, but doesn’t hang out there.

Tomi: some social networking apps provide recommendations for purchases, which solves a problem for the consumer.

Jesse: how to balance the need for community with the vision to reach a broad audience?

Jeremy: our community wants to be social.

Tomi: Yahoo Answers failed because there was too much push put into it before the market/community was there that could really use it as it was intended. Facebook Beacon failed because they didn’t handle privacy properly. A more valuable linkage between Facebook and online retailers would be, for example, to see recommendations for a book made by my Facebook friends while on Amazon.

Jesse: is Facebook so far ahead that OpenSocial will never take off?

Jay: the same was said about MySpace when Facebook started, yet Facebook (in spite of still having less users that MySpace) is the social network that we talk about [at least in grownup company].

Jeremy: OpenSocial is like Java for social networks, in the “write once, run anywhere” sense. [Jay commented that this is the part of Java that is least successful since it doesn’t work in many cases]

Jesse: to what extent should privacy be central to any social networking strategy?

Jay: most people don’t know that there’s a privacy problem until someone tells them that there’s a problem; the reaction to Beacon was driven by those people who are closer to the tin foil hat end of the spectrum. However, eventually privacy does become an issue and must be respected at the risk of legal action or abandonment of the community.

Jay: LinkedIn is playing catchup with Facebook, although it offers the “degree of separation” measure that’s useful for business people. A lot of the types of connections that he used to have on LinkedIn now occur on Facebook. [I’ve seen some recent writings that indicate the business people are moving back to using LinkedIn for professional networking, and using Facebook for personal networking; this is much the way that I use the two social networks.]

Tomi: combining social networks into one isn’t necessarily the answer: do you want the same social network for everything, or be able to use different social networks to segment your communities? At Yahoo, they couldn’t agree how to merge social networks.

Jay: dataportability.org is trying to solve this problem. This is not a new battle. [In fact, this is really no different conceptually than the problem of islands of information within enterprises, where customer and other data is replicated many times over in multiple systems.]

Experience!Tech 08 Web 2.0 Panel

The afternoon began with two concurrent sessions — Enterprise 2.0 and Web 2.0 — and the Enterprise 2.0 session didn’t seem too enterprise-y, so I’m in the Web 2.0 session, featuring Jeff Fedor, formerly of Covarity, Leila Boujnane of Idée and David Crow of Microsoft, moderated by Stuart MacDonald of TripHarbour. After the introductions, Stuart wandered back to his laptop at the lectern and announced that he was watching the Twitter channel both for comments and questions.

This session is bouncing back and forth between the participants and subject matter a lot, so I’ll just list some of the interesting disconnected snippets of conversation, with my comments [like this]:

Leila: Web 2.0 has resulted in a great proliferation of data, much of it still in silos, driving the need for federated search (including image search such as provided by Idée).

David: the importance of the social graph (exposing your relationships online) in driving the new level of connectivity amongst people on the internet, but this capability is not being used within enterprises. Particular concern with how your stupid antics on Facebook will impact your ability to be hired.

Jeff: we all maintain different contexts of online life — work, home, etc. — but there is no good way to synchronize this. [This is driving all of the lifestreaming apps that we’re seeing today, like FreindFeed and OnaSwarm]

David: is Web 2.0 a set of technologies, a set of social behaviours, a conference run by O’Reilly? A company like Amazon, which started as an online bookseller has become a huge provider of cloud computing platform, moving from being a web-based business to a web services provider.

Leila: we still need to have business/transactions at the heart of whatever we’re doing in Web 2.0, regardless of how inexpensive cloud computing is.

Stuart: no significant business is created with mashups, it’s done in the old-fashioned way.

Jeff: mashups are for demos, not for building real business solutions.

[My opinion is that mashups, for the most part, are for prototyping and as the new generation of end-user computing, the latter of which is definitely real business solutions.]

Leila: scaling is irrelevant but important: you have to build it into your architecture but usually aren’t worried about it when you first get your product out the door.

Jeff: need to understand the business pain points before building a solution, or you’ll end up re-doing large parts of what you did.

Jeff: the bar has gone up in requirements for capital; it’s not good enough to have an idea, you must at least have a POC and some business validation.

David: the amount of capital required to start is smaller, since less investment in infrastructure due to cloud computing both for development and scalable production

David: there is value to reputation, both for individuals and businesses, but you don’t have to translate the social economy into the “real” economy

Leila: it’s not about just delivering something that does the job, the user experience can completely change how people use your product.

David: there are local events happening (e.g., TorCamp) that can directly impact the larger business experience. There is a stratification of the community and we need to bring the groups together.

[Question from the audience on how to bring together the stratified groups, particularly to include people who don’t live/work in the downtown core.]

David: we need to diversify and include more people in the *Camp groups and other social/business venues. The only requirement to belong is to be interesting/interested.

Leila: Canada is an amazing place to build something because there are few barriers, but we don’t mentor enough to help foster growth and share experiences. We have a responsibility to mentor others to help the business community grow.

David: we need to toughen up our entrepreneurs so that they can handle hearing “no” and can absorb criticisms appropriately.

David: the community is the framework; it’s not what you take out of the community, it’s what you put in.

Experience!Tech 08 morning sessions

I’m at Experience!Tech 2008 today at the MaRS Centre in Toronto — nice to be attending a conference in my hometown for a change. This morning’s sessions are being beamed to us live from the IDC Directions conference in Boston, so although we have the timeliness of seeing the speakers present, it’s not quite the same as seeing them in person. This afternoon will be in-person panels, which should be great.

Unfortunately, it appears that I missed the only good presentation of the morning, given by Grover Righter of iMobileInternet, but apparently the slides are online and a number of my peeps were Twittering about it. The rest of the morning’s presentations, all provided by IDC executives, sound like guys who are either scrambling to figure out what Web 2.0 is, think that they’re teaching Web 2.0 101 to some of the suits in the audience, or inadvertently loaded a 2006 slide deck. Honestly, if I hear one more middle-aged guy talk about how his kids shop/watch TV/live their lives on the internet (implying, of course, that he still has his executive assistant print out his email for him), I will not be responsible for my actions. Obviously, I’m living in Middle Age 2.0, because I’m sitting in the audience Twittering with the people who are sitting directly beside me, and creating this blog post.

We’re in the presentation by Scott Lundstrom of IDC that is causing a great deal of barely-suppressed laughter in the TorCamp group around me, as he very carefully explains Web 2.0 to us:

  • Blogging as a business (said very slowly, to make sure that we all understand that you can actually make money at this)
  • Schadenfreude as a business model (a view, in my opinion, held by those who are terrified of the transparency that social networking brings; in fact, he refers to “the bloggers” in a very “us versus them” way, as in “you don’t want to become a victim of the bloggers” — oh, wait, am I victimizing him with this post? 🙂 )

Last up for the morning is Frank Gens of IDC looking at enterprise IT in the “post-disruption” marketplace, listing what business execs want from their CIOs. The top five:

  • Need to speed up
  • Improve information access
  • More innovative ideas from IT
  • Improve IT support
  • Support modern tools (e.g. Web 2.0) — this was not previously surveyed, but made the top 5 in its first year
  • More high-value business services

He then moves on to what CIOs want from their suppliers:

  • Very competitive pricing
  • Support for industry standards — this has moved up significantly in the rankings
  • Understand my industry/business
  • Clear business case for offerings
  • Strong community of “solutions” partners

IDC is really big on surveys, so not surprising that the presentation is mostly made up of their survey results (no real innovative ideas here) plus a few case studies (such as P&G, who have done some very innovative things in R&D and business intelligence). Unfortunately, the conclusions that he draws are out of date and not revolutionary thinking at all: “services will be delivered over the web”, “Google will buy Salesforce.com or some other player in the SaaS space”, “the mobile web is important”, “big names will drive the market”, “disruption is the new status quo”.

“Eureka 2.0”? Oy vey.

I think that Scott Brooks summed it up best.

Blogging conferences

It seems that some conferences still aren’t plugged into the blogosphere as a PR engine, including some from surprising quarters. I applied for a press pass to next week’s O’Reilly’s ETech back in January via the press link on their site, and after a couple of weeks received the following reply from a Maureen Jennings, their conference publicist:

Press credentials for our conferences are limited and intended for journalists from established technical publications with significant readership. Therefore, I’m unable to issue you media credentials for the conference. Thanks for your interest in ETech.

Ouch! I understand that they have a limited number of press passes, but I’ve been writing this technical blog (that would be a “publication”, Maureen) for three years, and see a readership of around 1,200 unique visitors per day on my own site, plus I’m syndicated on Intelligent Enterprise and the FASTforward blog. I know, hardly Scoble numbers, but it’s not insignificant.

Regular readers also know that I have a long track record of prolific live-blogging coverage of more than 20 technology conferences dating back to 2005; at the recent FASTforward conference, I wrote over 10,000 words about the conference in two days.

I responded to Maureen:

Just to clarify, do you mean that you don’t consider a technically-focused blog about of 3 years to be an “established technical publication”?

After a pause of several days, she came back with:

As I’m sure you understand, we have a limited number of passes that we can issue for each conference. So we sometimes have to make hard calls, based on getting the news the widest possible readership. Sorry about that, and thanks for your understanding.

Yes, I understand that there are limited press passes, and that I might not get one. What I don’t understand is why your initial email to me would suggest that my blog is not an established technical publication, and doesn’t have significant readership. Someone needs to attend a remedial class in blogger relations.

FASTforward: Using Search to Achieve a Complete Customer View

Final session of the conference, and I’m in the financial services breakout track to hear Lee Atkinson, SVP at iDNA, and Marc Hebert, CMO of Virtusa, discuss a real-world example of using enterprise search to build a true 360-degree customer view. They started with some background on the problems with enterprise applications and how dynamic business applications (Forrester’s term for composite applications; he’s quoting from a Forrester report) are starting to take on some of the functionality. Enterprise search applications provide a layer on top of the enterprise applications, files and databases to create search solutions such as corporate search or intelligence solutions. They see enterprise search as a key enabler of dynamic business applications in order to provide both speed and agility of those applications.

The case study that they presented is the implementation of enterprise search in a top ten global bank, providing fast retrieval of structured and unstructured data, and integration with business process management (not a term that I expected to hear at this conference!) to manage events being generated by the system. Their technology platform included enterprise search, BPM, SOA and enterprise portals, and they addressed the 360-degree customer view and a GRC (governance, risk and compliance) event management system. They didn’t use search because of its inherent properties — in fact, the users don’t even see this as a search application, and the applications look like standard structured data extracted from operational systems — they use it to extract information quickly from heterogeneous data sources. Prior to this, the bank had point solutions, multiple data warehouses, and no commonality between systems and databases to provide any sort of consolidated customer view. They needed to integrate data from multiple sources, and also meet their compliance regulations.

They started by providing a compliance and risk view, plus product and customer profiling, in 2005, then expanded to include event-driven business processes and exception management in 2006. In 2007, they brought in the single customer view and more advanced business processes and analysis. Of course, there is no such thing as a single view of a customer in a large organization: different business areas (compliance, operations) have different needs, and each user type’s customized view of a customer is actually a subset of the entire customer model, with other supplementary information pulled in and additional analysis added appropriate to the task at hand. The result is a number of different end-to-end processes, such as know-your-client, anti-money laundering and sanction lists in account opening.

For them, search technology is a way to integrate legacy systems — not an application that I thought of when I considered search — although it requires a deep knowledge of the business domain and the nature of the underlying data. Once the core integration structure has been created, additional data sets and applications can be added quickly and at a fairly low cost. The focus on event-driven business applications based on search results is where search really contributes value to their applications.

FASTforward: BI goes mainstream

Philip Bierhoff, Systems Manager at Proctor & Gamble, spoke about strategies to increase user adoption as business intelligence goes mainstream.

P&G’s Symphony project creates “decision cockpits”: dashboards based on specific roles and corporate divisions, and including information ranging from traditional BI reports to documents to news.

The underlying data landscape has moved from their first iteration of a common data warehouse in the mid-90s with regional servers plus ETL, storage and aggregation, where BI was driven by stored aggregations; to the current atomic data warehouse with a central server plus ETL and storage, where BI is driven by query rewrite — effectively, aggregation on the fly. They also have SAP generating data into SAP/BW; altogether, they have about 65TB in the data warehouse and 50TB in SAP/BW.

Originally, they used a traditional monolithic BI approach using a BI tool that acted as a portal as well as providing dashboards, graphs and reports as well as its own security layer. They had some challenges with scalability, and the boundary behavior when they pushed the size envelope. They felt restricted by the functionality of the tool, and by the lack of flexibility. Based on these restrictions, they developed a new vision of BI: one based on service-oriented architecture and industry standards, working directly from the corporate data warehouse instead of reporting data marts, and with a thin layer of BI as one of multiple layers of technology that could be changed out as required. The resulting solution uses an SOA-based (I assume JSR168-compliant) portal with various portlets, fed by FAST Radar for BI repots and graphs, and SharePoint for document storage. Under all of this is a service bus connecting to a data services platform, which in turn connects directly to the data warehouses and legacy applications. A WS-Security stack sits alongside to provide standard security and authentication.

They selected FAST Radar because it pairs simplicity with power for fast response times, fits into a portal architecture, downloads to common formats such as Excel and PDF, requires no data model maintenance and can use web services as a data source in additional to traditional database access. They’ve found that it needs very little training for creating and using reports, although they’re still using it only for predefined reports with drill-downs rather than allowing ad hoc report creation. The time to create complex reports has dropped from 30-60 seconds with the old BI tool to 5-10 seconds using FAST Radar.

The resulting architecture looks like a textbook example of how to do SOA correctly. They’re getting a good level of reuse from their services, especially the data access services, and they’re positioned for future extensions without completely overhauling the architecture. They’re able to provide customized dashboards for their 40,000-person user base in a reasonable time frame, in a large part because they’ve designed the underlying services and components for reuse.