Tag: fastforward

My regular live-blogging posts from the FASTforward conference are cross-posted here as well as the FASTforward blog, but check out the FASTforward blog for a ton of other content: other bloggers at the show, and some great video interviews by Jerry Michalski with the conference speakers and some of us bloggers. It’s the next best thing to being here!

Last breakout session of the day, and it was a tough decision: there was one on business intelligence built on search, but I opted for the enterprise mashups session with Arnt Schoning and Espen Sommerfelt of FAST, plus Stefan Sveen from Comperio, one of their partners that builds search solutions on top of the FAST platform.

They started with some background on service-oriented architecture (SOA) and mashups: SOA as a way to expose business functionality from siloed applications for reusability, and mashups as the Web 2.0 equivalent of SOA, using lighter-weight integration protocols such as REST and JSON instead of WS-*. At last year’s FASTforward, they created a business process in BPEL that consumed a couple of FAST services; this year, they wanted to show a more comprehensive mashup using their technology. There are about 20 services available out of the box with FAST, accessible using Java or SOAP/WSDL.

This turned into a fairly deep dive into the mechanics of how to implement mashups with their services, complete with Java snippets; a higher-level view of the importance of this would have been helpful as at least part of the presentation.

Interestingly, they put what they did last year in the context of purely automated integration-centric processes (hence their mischaracterization of their process as SOA rather than integration-centric BPM), whereas I can imagine that there are also uses of their services as called from processes that include human interaction.

They continued on to talk about mashups: visual versus non-visual, client-side versus server-side, assembly versus coding styles of implementation, and consumer versus enterprise usage. Mashups are ad hoc and situational, hence are primarily user-driven and used to empower users and knowledge workers on the long tail of application development: giving them the ability to create applications that IT would never get around to implementing because the audience is too small or because there’s only a temporary need.

They also talked about the difference between SOA (composite applications) and mashups; they distinguish mashups as being user-created and SOA as being IT-created, but I think that they’re off the mark here: SOA is really just the services layer, which can be consumed by a user-created application just as easily as an IT-created application. The true comparison would be between a composite application development environment and a mashup creation environment, and I think that the line between those is becoming increasingly indistinct.

They showed a demo of a mashup created using their enterprise search services, in which the user can enter any text, and any location information in the free form text is detected and mapped on a Google map. Since they used the location extraction service in their engine to parse the text, words similar to locations are not detected as locations as long as there is sufficient context to make that distinction (e.g., the phrase “Denzel Washington is an actor” did not result in Washington being mapped). Once the points are mapped, other information can be gathered related to those locations, such as weather data or Flickr photos at those locations; this is not really specific to a mashup using FAST services.

A second demo showed the creation of a simple blog with automatic tagging: their automatic tagging service generated the most appropriate tags based on the text of a blog entry. They also showed applying the automatic tagging using a large chunk of text from a real FASTforward blog post; for the second time in a year, I see a recent blog entry of mine being used in a demo during a conference presentation — a slightly odd experience, although they likely had no idea that I was in the audience.

Sveen from Comperio then discussed their best practices in developing search applications: they’ve created a lightweight framework, Comperio Front, that repackages FAST search functionality for easier reuse. The framework is service oriented, and models the search logic workflow and rules as well as encapsulating a number of reusable search tasks and simplifying some of the FAST APIs. It appears to be well-layered, such that (for example) additional adapters could be easily added to the core processing engine to augment the existing web services, HTTP and XML interfaces.

He showed a demo of how they could easily improve the user interface on a standard search portal by combining multiple navigation widgets, and a further demo that shows how search data can be dynamically filtered and displayed. Since I don’t know what the same functionality would look like directly in FAST, I can’t really judge how much easier it is using Comperio Front, although they have the definite benefit of exposing additional search functionality as services for consumption by composite applications.

I was delayed getting to the next session on using search as the next generation of enterprise portal by Kara Hansen of Disney, so missed a bit at the beginning. She described how they use FAST to crawl content across multiple enterprise sites to create a virtual view of their content, both enterprise wide and for specific subsites representing divisions within the enterprise. In total, they serve 55,000 employees, plus customers via an externally-facing portal.

One of their most significant challenges was security. First of all, the search crawler needs to have credentials for all of the content locations that it’s going to crawl so as to expose only information that should be exposed on the portal; although these are internal portals, there’s still security issues around content viewing. There’s also an externally-facing portal based on the same technology platform, limited by the specific collections and search profiles to generate a restricted view of the content. When any user executes a search, their security level is combined into the search behind the scenes to deliver only the information to which they should have access. This, in turn, results in different results depending on whether a user is logged in or not, which can frustrate users who don’t understand the implications that their user authentication has on their search breadth.

They also have a challenge with users finding applications that are typically exposed as portlets, but aren’t searchable so don’t end up in the search-based portal. In other words, the user is trying to reach a specific destination on the intranet, not information. To compensate for this, they created a simple XML document that links to the applications and allowed that document to be indexed, although this seems like a bit of a brittle approach since it presumably requires manual updating of that document to add, remove or modify links to applications. However, sometimes a manual workaround can bridge a major gap in functionality; queries for applications now forms a major portion of their executed searches.

They’ve seen a shift in communication culture within the organization: users proactively search for information that they want rather than wait for it to be fed to them. Search also acts as way to introduce user-driven applications, such as wikis and user-created profile pages, a change that Hansen refers to as “profoundly disruptive”, in a good way.

They did all of this with a team of five people — three Disney and two from FAST — although that didn’t include the content providers who manage the content that populates each divisional portal.

She ended her presentation by giving away some nice Disney swag: proving that Mickey is for adults too with some MM-themed bar accessories.

The breakout sessions have started, and I’m sitting in on the financial services track for a couple of sessions, since my primary customer base is in financial services. Adam Sussman at TABB Group is discussing how search technology will change the relationship dynamics of the institutional brokerage industry, where there is a much different relationship between brokers and clients because of the size of the institutional investment portfolios.

The current problem is that even large customers are devaluing the service provided by high-touch (full-service) traders, choosing to make their own trades directly and purchasing research on an ad hoc basis. However, the type of research provided by high-touch firms is really driven by search, providing information about movement in companies that are held within a portfolio, as well as other market conditions that might influence future trades.

More importantly, however, and the focus of Sussman’s talk, is that by implementing sophisticated search technology, the sell side will be able to recognize opportunities to revive high-touch revenues. In other words, use search technology to identify which customers are most likely to be profitable, and focus efforts on those customers.

You can read a bit more about Sussman’s research in this area on their site.

It’s the first session after lunch, and the first time that we’ve heard an actual FAST presentation, by Bjørn Olstad, their CTO, discussing next-generation innovation in search technology.

They have segmented their audience into two main groups:

• Monetization, which is driving revenue by creating unique user experiences that match customers to relevant assets
• Enterprise, which is driving productivity by creating unique user experiences unlocking actionable information across data silos

Unlike traditional content management systems, search focuses on the intentions of the consumers of the content, not the creators of the content. The search experience should constantly be evolving based on what the consumers are searching for and what they’re doing with the results, plus the constantly changing data underlying the search.

He looked at search as a research portal within an enterprise — which is also covered in a breakout session later today — and other uses that don’t fall into the “search box” view of search. Search is becoming the portal for user interaction, mapping the user intent onto the available content.

We then saw a demo of the new Content Integration Studio, which allows for an easy mapping of reader (data sources) to writer (consumption mechanism): a bit like using Yahoo Pipes to create a graphical mapping from inputs to outputs with various filters and transformations along the way, but using corporate databases and a wide variety of internal and external data sources. You can debug the flow on the fly, setting breakpoints then examining and/or changing data at the breakpoints to see how information will flow through.

They also have some new indexing and contextual matching algorithms in a new search core that allow for near-real-time availability of new content, which is essential for their media/news clients. They’re also providing tools for creating search-based portals to allow search to be the fundamental driver of the user interaction, and provide very customized experiences so that individual users will be presented with different content in their view of the portal.

The morning finished with Jared Spataro, a Microsoft product manager for SharePoint, responsible for enterprise search: namely, the group in which FAST will belong after the acquisition completes. He talked about why Microsoft is making this acquisition, how FAST will benefit, and what this means for customers.

Microsoft started to look at enterprise search about 18 months ago when they realized that their customers no longer saw search as just a feature, but that it was actually shaping the user experience. They found a huge gap in the market between high-end and low-end enterprise search segments, and created a mid-range search capability combined within the SharePoint platform. They also attacked the low end of the market by introducing Search Server Express, a free downloadable product. But that left the high end of the market, which Microsoft is approaching through the acquisition of FAST. So the answer to the question “what’s in it for Microsoft?”: total search world domination.

So what’s in it for FAST, besides $1.2B? It gives them scalability both in their ability to innovate and in the massive sales and marketing channel that will expand their product reach. Microsoft has recently bought in to the idea of distributed development, both to find and retain talent and because of non-US acquisitions, meaning that FAST’s development team can stay in Oslo, far from Redmond and hence a bit autonomous.

At the high end, of course, FAST runs on non-Microsoft platforms, and Microsoft is committed to continuing support and innovation across all platforms, a stance that should somewhat calm the fears of existing FAST customers. Spataro also talked about how it’s an advantage to customers that they offer complementary capabilities in enterprise search across the low, medium and high tiers; personally, I’ve never seen a big advantage to having multiple products in the same product class but based on different technologies available from the same vendor, since there will be confusion at the boundaries and some uncomfortable migrations.

He summed up with the overall SharePoint strategy, based on the pillars of business intelligence, collaboration, portal, search, content management and business processes. It’s not their intention to bury FAST inside SharePoint, although it will likely end up as part of the platform at least for branding purposes.

Tom Davenport presented on competing on analytics — he published an article with this title in 2006 in HBR. He believes that the planets are aligned for analytics:

• Powerful IT, and a new model for IT use
• Data critical mass: ERP, POS and web data
• Skills sufficiency, at least for the individual pieces
• Business need: a differentiated, personalized, well-informed easy use experience

He makes a distinction between reporting and analytics, and sees analytics as a way for companies to find the best customers and charging them the right price, or analyzing search logs to design a better website.

He sees the following attributes of a user-centric analytical strategy:

• Analytics are used to design and modify the customer experience
• Analytics are used to measure and maximize user engagement
• Analytics are used to determine what the user wants, and personalize to the user
• Analytics are employed across all customer channels
• Analytics are made available to the user

This includes companies such as Amazon and Netflix through their targeted recommendations, Google with customized ads and do-it-yourself analytics, eBay with behavioral targeting, Harrah’s and Marriott with loyalty programs, and Royal Bank of Canada and Capital One with targeted cross-selling across channels. Analytics can also be used for public service: New York City reduced crime through predictive analytics of where crimes were most likely to occur.

He stepped through what’s required for competition grade analytics: data (which he envisions in a data warehouse, counter to Weinberger’s arguments about the more serendipitous discovery of data), enterprise (widespread access to a centralized data store), leadership (executive commitment), targets (focusing analytic activity), and analysts (professional, semi-pro and amateurs, using all levels of analytical tools).

He finished up with the next steps for analytics, from the pursuit of new data types to search/BI combinations to better model management.

David Weinberger — author of Everything is Miscellaneous — spoke about the power of digital disorder, and how we need to unlearn what we think that we know about the best ways to organize information. He feels that we’re approaching the end of the age of information — by which he means a focus on rigidly structured information — and a move away from being “informationalized”, where we consider everything to be information even if they’re just symbolic representations of reality.

He looked at how many projects, typically physical projects, require a much greater degree of control as they increase in size, but contrasts that with the web, which has growth only because of the lack of control. Control doesn’t scale; we just thought that it did, and managed to scale with control by eliminating information.

There are two orders for how things are traditionally organized: in the case of physical objects, there’s the first order, namely, the physical order in which they are placed; and the second order, which is a catalog of the physical objects, like a card catalog in a library, which uses a limited and pre-determined taxonomy for categorizing the objects.

In the new world of organization, we allow for user-generated folksonomies to generate metadata that we couldn’t have possibly thought of for the objects. These multiple layers of metadata online allow for a physical object to virtually be in more than one place at a time; when the concepts are applied to online data rather than physical objects, the division between data and metadata starts to blur. All contents are also connections: everything leads to everything else, creating a wonderfully messy mass of interconnected data. The web, of course, excels at creating connections because of the basic premise of linking: we create hypertext links on pages to make connections that are important to us. The user revolution, therefore, is not just about us creating our own content; we also control the links, hence control the connections between content and the organization of that content. Digg, Twitter, your RSS feeds and other socially-created sites create our new “front page”, replacing the newspaper of old: why would you read someone else’s idea of what’s important, rather than self-select what you’re interested in reading?

A key thing is not to set arbitrary limits; he cites the example of the Library of Congress recently posting a huge set of historical photos on Flickr to allow open tagging by everyone, which has since run up against Flickr’s arbitrary limit of 75 tags per photo. Tagging like this helps to destroy stereotypes: the more data/metadata that you have about someone or something, the less likely you are to make a shallow characterization.

Weinberger’s a great speaker, and uses some really funny and compelling images in his slides.

Clare Hart, EVP of Dow Jones’ Enterprise Media Group spoke about rising expectations and new patterns for interacting with information. She started with the four macro trends that are impacting business and consumers today:

• Globalization
• Distributed workforce
• Follow the sun business cycle
• Information overload

Information is being exchanged around the world instantaneously, and this constantly flowing river of information is changing how companies react to events and fundamentally how they do business. Information overload — actually relevancy overload — makes it difficult to find the right information unless there are tools such as sophisticated taxonomies and tagging applied to properly filter the information. In my view, lack of relevancy is the most common reason that people ignore information that is available to them: some people don’t Google for information because they have a difficult time separating the wheat from the chaff.

A couple of years ago, Dow Jones acquired Factiva, which provides search and filtering of external news information to provide customized feeds to their customers; she stepped us through how their Factiva service presents this information in order to provide the most relevant information from traditional news sources as well as blogs and other non-traditional news sources.

She had an interesting cause-and-effect slide on how an event occurs, that triggers news, commentary and analysis that feeds into a discussion on that event and its analysis. For example, an investment banker interested in a specific industry would be presented with a screen of information in the event of a significant loss of a company in that industry: the original news story, internal and external research, market reactions and other related information that makes up the bigger picture driving potential investment in that industry. From this, he can drill through to more detailed contextual information on the specific company and its competitors, as well as general market information. For a different role, such as a wealth manager managing a client portfolio, the same information about a company’s losses has different meaning, hence is presented in a different way: exposure to that company within a client’s portfolio as well as specific client information and even news articles related to the client’s investment strategy and personal interests.

As she points out, this is search without the search box: the search is executed behind the scenes and the results pushed through to the recipient based on his known role and interests. The investment banker is looking to improve his investment-related reactions to the event; the wealth manager is looking to improve the relationship with his client.

She finished up with the benefits of search without the search box:

• Anticipatory discovery
• Contextual linking
• 360 degree view
• understand business from multiple angles
• Drive confidence
• Ability to act

The wifi continues to be flaky — good connection strength but no actual connectivity — but there’s power at the tables which almost makes up for it. Almost.

We started the day with John Markus Lervik (plus a light show, an enormous inflated ball, a small bit of theater and a live band) discussing the theme of the conference: the user revolution. There’s a strong relationship emerging between business intelligence and search, and the judicious use of enterprise search has moved beyond a commodity to become a competitive differentiator as well as playing a critical role in many government programs (the ones that keep tabs on all of us). Google made us all see that we can use search for finding information and relationships that were never available to us before; enterprise search such as FAST gives that same ability to employees inside an organization, allowing them to search the corporate information assets in order to do their jobs better. No longer are users stuck with a static view of information created by a report designer, but can create their own views for their own purposes.

John Hagel of the Deloitte & Touche Silicon Valley Research Lab spoke to us next about the impact of the user revolution on your organization: how it’s putting pressure on traditional organizations because of reduced product life cycles and other factors, contributing to life spans of companies.

He spoke about the forces that are driving the user revolution within business:

• The move from the scarce resource of limited shelf space for products and services to unlimited shelf space: what drives the long tail. The scarce resource is now our attention. There’s a much lower cost of interaction now, including finding information about vendors and switching to a new vendor. Customers are gaining more power.
• There’s a shift to intangible assets — mostly human talent — as key competitive differentiators rather than the old model of competition based on physical assets such as plants. There’s greater competition for this talent, since it’s much easier for people to move to another organization or even go out on their own; the key is to become a magnet for attracting talent by creating the right sort of environment, particularly an environment that builds talent. Talent is gaining more power, too.

He went on to discuss the change from push to pull business interaction models, and how there’s a move to some new performance metrics: return on attention (e.g., measuring profitability by customer rather than by product), return on information (e.g., what can we find out about participants/customers by their interactions with us, and how can we provide them with better information) and return on skills.

Hagel made a big deal at the beginning of his presentation about how he wasn’t using a PowerPoint deck, but then proceeded to read his presentation from notes in a mostly uninflected voice, making it difficult to stay focused on his talk — I’m sure that I missed a lot here. I never thought that I’d say this, but his presentation might have actually been improved by the use of PowerPoint.