FASTForward to Serendip (FF pt. 4)

"We have all become revolutionaries," said FAST ceo John Markus Lervik at the start of FASTForward08. This note was echoed by several speakers at the conference:

• Enterprise 2.0 will challenge monolithic corporate structures, said "Wikinomics" author Don Tapscott.
• "Corporations are going out of business at an accelerating rate," noted John Hagel, adding that average life of S&P 500 corporations has declined by 80% over the past 70 years and today averages 15 years.
• For JP Rangaswami of BT, power placed in consumers' hands is a major driver roiling the formerly more placid outlook of large stratified corporations.

The elephant-in-the-room is not exempt:

After starting the year near the $700 mark, Google shares have shed about one-third of their value. The stock slipped more than 8% in the past two days following the release of data from comScore that showed a decline in the growth rate of so-called "paid clicks," which takes place when Web users click on an ad-supported link. Marketplace

Only 1/3 of user queries scurrying across the web bother to go through web search engines, said IDC's Sue Feldman in her keynote. Once you've established a relationship with a gateway, hub or node, says Feldman, there's no need for the sort of search Google offers.

Which is where FAST and other companies come in. Beyond the $3 billion market Google has largely cornered via quick search + advertising, says FAST's Zia Zaman, there's another two to four billion market that will grow through advanced search and the monetization thereof.

So we'll take a peak here into advanced search and in a further post offer a similar glimpse of monetization.

Search is not changing something: it’s changing everything - JM Lervik

The claim for advanced search as the next big thing -- as far as I could make out from listening to the speakers at the conference, and talking with some of the analysts and vendors -- is in its ability to get to know the user, anticipate his/her wants and needs, and intervene with relevant information sensitive to the intent of the user and to the context in which the search is afoot.

Michael Cleary of Reuters spoke to Jerry Michalski about fusing user intent and advanced search content. Somewhat more elaborately, Zaman spoke of FAST's developing art of "interaction management," leading to a flexible, personalized mode of search built around the uniqueness of the individual user.

Rapidly ticking off four attributes of the new technology, Zaman told analysts and bloggers that search

• connects my world
• translates to my terms
• understands what I need
• finds what I want

It's a bold claim, made bolder when he notes that via a mode of "conversation" with the seeker, search will not only find what the seeker was looking for, but via the power of serendipity, change your decision by discovering "something you didn't know you needed."


There's a rich, suggestive tonality to these descriptions, reminiscent of the language of quest romance and psychoanalysis. More than an algorithm, what's being described is a deus ex machina probing the subterranean desires of the seeker even as he explores the web (rather than surfing it) to discover that which he doesn't yet know he wants.

It's as if search has undergone a profound augmentation. No longer is it simply and matter of factly, "seek and ye shall find," a la Google. Now, and moreso in the coming time, the "user" is on a quest, but in the course of the quest, he or she discovers what there is to seek. Search 2.0 = search squared, the turning of the explorative intelligence of IT upon the aura, the trace, the smudge of the seeker in the interactive slipstream.

If that's what's on offer, how does it work? Because let's face it, this isn't easy to do, as Zaman himself made clear when he told us, "It is terribly difficult to get at the true intent of the user."

For a hint, let's return to our friends at Lexalytics who shared some of their techniques with us the other day.

Unstructured search analyzes all sorts of information, however messy. It can locate and purports to assess the sentiment and tone of massive amounts of information.

According to Brian Pinette and Tim Mohler at Lexalytics, the art of sensing the tone of articles relies at least in part on algorithms that suss out emotive or value-laden words and relate them to nearby entities, persons or companies or products, with methods that distinguish which entities are being described or judged.

I was curious how the analysis could be certain that the words it was sensing belonged to the author of the document. Say, for example, there's a movie review in which the reviewer quotes another writer who loved the same film he's writing about. That encapsulated review presumably would glow with positive terms; but say it is within the reviewer's own strongly negative review -- same movie, two patches of words sharply differing in sentiment. What then?

That falls under the problem of anaphora, Pinette explained. Not the rhetorical kind, but the linguistic variety, defined as: coreference of one expression with its antecedent. The antecedent provides the information necessary for the expression’s interpretation. (link)

E.g., Mr. Defamer said he hated the film would be anaphoric in that it is relatively simple for us to carry the identify "Mr. Defamer" forward to be picked up by the pronoun, "he." Of course taken in isolation it's impossible to be sure that Mr. Defamer is not speaking about his friend Mr. Carp's view of the film, or for that matter, his dentist Dr. Scheissfinger's opinion of something found on his, Mr. Defamer's, front teeth. Then there's problematic anaphora.

Pinette said anaphora is a challenge but appeared confident it will be mastered -- along with cataphora, endophora, and the like. The rules of grammar and logic, while complex, seem after all to obey the rule of rules. But what, I wondered, of metaphor-a?

I was intrigued partly because the more complex texts, essays or literary forays that include not simply reported speech but implied speakers and the full panoply of rhetorical devices -- the stuff of rhetorical analysis which as a comparatist I used to do -- would appear to offer yet greater challenges. I suggested to Pinette that there might be in the forest of rhetorical terms some useful means of describing various linguistic structures. Whether they would help sort out the elusive, polysemous turns of sense flowing through sentences, chapters, or books remains to be seen.

Beyond the question of computing the reading of content --of sentiment, tone, and figurative speech -- is the question of sounding the intent of the seeker. How do you give him the thing he didn't know he was looking for? The kettles of fish this borders on -- questions of intentionality, meaning, the nature of mind and intelligence -- are likely to require more than the isle of Serendip.


More later