Where Next For Search?

Steve Arnold moderated his traditional closing panel discussing the future of searching.  The format was the same as in the past:  A series of questions from Steve with answers from the panelists who were:

  • Gregory Grefenstette, Chief Science Officer, Exalead
  • David Milward, Chief Technology Officer, Linguamatics
  • Dave Patterson, CEO, Sophia

(L-R) Gregory Grefenstette, Steve Arnold (Moderator), David Milward, Dave Patterson

Steve introduced the panel by noting that the participants all represent successful companies gaining in sales. 2012 will be a very challenging year for all companies; the changes in technology that are now underway are as dramatic as we have ever seen. Social change in findability is uncovering a growing interest in using people to find answers.

Below is an edited transcript of the conversation.

1. What is the major trend in enterprise search and content processing for 2012?

Greg: Enterprise search is different from web search because lots of information is in unstructured sources. The current trend is to include both internal and external information in searches, so there is a growing body of linked data available. People want to know everything about what is going on both inside and outside the enterprise.

David: Content is getting more diverse. The number of people available to sift through the data is getting smaller, so text mining and other technologies are necessary to analyze the data, and they are being used in a hidden way behind the scenes. With more automation, keeping things up to date is important. Text mining provides interactive information and can be used like a search tool, so we can combine it with search.

Dave: Our focus is on content, understanding its meaning, and letting organizations leverage more value from their content. Discovery is very important. People want to find out what they do not already know about and link it with information they have. Cloud-based computing will be prominent for the next few years.

Steve: The focus is dramatically different from basic retrieval we had in the last few years.  We have heard that Microsoft has made its search system available without charge as part of a bundle to large corporations, so in effect, search is free. Lucid Imagination is creating a free search system, and pricing pressure is increasing to where basic search has become a commodity.

2. In an environment of low cost systems, what is the value of a commercial system that costs much more?

David: We do not compete directly against search tools. Text mining finds indirect relationships, which you cannot do with a search tool. Information professionals want to use text mining to give added value over what end users would get. Terminologies are difficult and costly to create. Text mining tools are used with the technology.

Dave: Many free search tools are limited in functionality. Just finding some hits is no longer sufficient. If all you want is a flat list of documents, then the free tools are fine, but most enterprises want search tools that understand the content and leverage its value. Tools at the low end of the market will not return that value. Many organizations do not appreciate the time it takes to build something in-house using open source tools.

Greg: It is nice to have free search, and the tools are good at asking a question and getting an answer. We build applications where search is in the background and can connect to many resources. Another added value is the semantic processing we provide which is not available in the free tools. Modifications and fixes are either not available for the free tools or cannot be done quickly.

3. What is the impact of apps on search and findability and content processing?

Dave: We need to be more innovative in the way we present information to users. We cannot present results lists on the small screens of mobile devices, which puts more pressure on the intelligence of the search tool.

David: As you get to smaller devices, you need more understanding of the text and whether you can repurpose it differently. Faceted search and text mining allow us to structure the information and allow people to navigate through it. We may see more emphasis on push services that give people information wherever they are.

Greg: Users’ expectations are being raised; they want instantaneous response time, ease of use without training, and 24 hour availability. This is a challenge and Exalead has a solution. Mashup systems allow you to present information to different devices.

4. How is cloud reliability going to be addressed?

Greg: Search engine technology allows you to have constant availability.

David: We initially thought a cloud service would be interesting to small companies, but we found after launch that big companies were also interested. Peiople want to concentrate on core competencies. They do not want to worry about keeping external services up to date and want someone else to do that for them.

Dave: There are issues of security and reliability. Are there any statistics that in-house servers are more reliable than cloud ones? Are we panicking as a result of the massive failure of BlackBerries on October 10-12? (Google “Blackberry outage” for more information.) Security is also an issue. Companies protect their data as much as possible and are reluctant to put their data on the cloud. Employee behavior is a bigger risk to companies; laptops are frequently lost.

5. In the search and business intelligence space, former search companies are are repositioning themselves and are now saying they offering predictive analytics  or customer service. To an old person, this is search. Are these new terms helping or hurting?

Greg: Search-based applications can be very varied. There are many semantic technologies in search engines that allow these different applications.

David: It is always good to describe what you are doing to solve a problem instead of calling it just search. There is lots of subtility in language.

Dave: We are firmly focused on content and understand its meaning as well as new and better ways of helping people deliver it. Semantics is an example of true business intelligence. We need natural language processing to understand what we are doing. There are many ways of extracting meaning from unstructured business information.

This was an excellent concluding note for the conference on.  The old standby, search, continues to be change as technology advances.

Don Hawkins
Columnist, Information Today and Conference Circuit Blog Editor



  1. Search Engine Update - December 6, 2011

    […] TCC « Where Next For Search? Big Changes Coming Next Year […]