Real Time Search in the Enterprise

Real time search is a big fuzz in the global network called Internet. Major search engines like Google and Bing are now providing users with real time search results from Facebook, Twitter, Blogs and other social media sites. Real time search means that as soon as content are created or updated, it is immediately searchable. This might be obvious and seems like a basic requirement, but working with search you know that this is not the case most of the time. Looking inside the firewall, in the enterprise, I dare to say that real time search is far from common. Sometimes content is not changed very frequently so it is not necessary to make it instantly searchable. Though, in many cases it’s the technical architecture that limits a real time search implementation.

The most common way of indexing content is by using a web crawler or a connector. Either way, you schedule them to go out and fetch new/updated/deleted content at specific interval during the day. This is the basic architecture for search platforms these days. The advantage of this approach is that the content systems does not need to adapt to the search platform, they just deliver content through their ordinary API:s during indexing. The drawback is that new or updated content is not available until next scheduled indexing. Depending on the system this might take several hours. Due to several reasons, mostly performance, you do not want to schedule connectors or web crawlers to fetch content too often. Instead, to provide real time search you have to do the other way around; let the content system push content to the search platform.

Most systems have some sort of event system that triggers an event when content is created/updated/deleted. Listening for these events, the system can send the content to the search platform at the same time it’s stored in the content system. The search platform can immediately index the pushed content and make it searchable. This requires adaptation of the content system towards the search platform. In this case though, I think the advantages outweighs the disadvantages. Modern content systems of today are (or should be) providing a plug-in architecture so you should fairly easy be able to plug in this kind of code. These plug-ins could also be provided by the search platform vendors just as ordinary connectors are provided today.

Do you agree, or have I been living in a cave for the past years? I’d love to hear you comments on this subject!

The Business Case for Enterprise Search

1. Achieve higher employee efficiency levels by providing company-wide, swift access to relevant information

Every business day, employees need to access information stored in various enterprise applications and databases. Enterprise Search addresses this need by providing your co-workers with swift access to relevant information and by consolidating, ranking and presenting it properly. The value proposition of enterprise search is thus to promote core business by enabling co-workers to work more efficiently, to avoid redoing work done elsewhere and to produce better quality as the information they need can be found through one single search solution.

2. Make more money by providing revenue-driving business processes with tailored means to access and act on information

The larger the corporation, the more different information access needs. Besides providing large user groups with general access to corporate information, an Enterprise Search solution can be tailored to meet the specific needs of revenue-driving business processes such as solution sales, business intelligence, patent management and mergers and acquisitions. There might not be that many people working in these areas, but the outcome of their work can have a tremendous impact on the bottom line of your company.

3. Leverage the hidden value of existing IT investments

The return on investment of Enterprise Search is not only a matter of getting your money’s worth for the license and deployment costs of the Enterprise Search solution. As the solution makes all the information hidden in document repositories findable through one search solution, the Enterprise Search solution will in fact help you get a return on investment on content management investments already made.

4. Lower your IT costs by centralizing access to information

Reduce your license, maintenance and support costs by providing one centralized Enterprise Search platform to handle all information access requests. Most companies store information in various information systems such as intranets and web sites, collaboration portals, document management systems, CRM and ERP systems and many other enterprise applications and databases. A typical set-up is to have separate search tools for each of these systems. By using your Enterprise Search platform as a service, you can replace these siloed search functions with one centrally monitored platform that provides search to each of these applications. In this way, you can reduce the annual costs on licenses, maintenance and support for separate search applications.

The ROI of Enterprise Search—Where’s the Beef?

When faced with the large up-front investment of an Enterprise Search installation, executives are asking for proof that the investment will pay up. Whereas it is easy to quantify the value of search on an e-commerce site or as part of the company helpdesk—increased sales, shorter response times—how do you go about verifying that your million-dollar Enterprise Search application has the desired effects on your revenue stream?

Search engines on the Web have changed the landscape of information access. Today, employees are asking for similar search capabilities within the firewall as they are used to having on the Web. Search has become the preferred way of finding information quickly and accurately.

Top executives at large corporations have heard the plea and nowadays see the benefits of efficient Findability. However, it costs to turn the company information overload from a storage problem of the IT department to a valuable asset and business enabler for everybody. So how do you prove the investment worthwhile?

The Effects of Enterprise Search

Before you can prove anything, you need to establish the effects you would like your Enterprise Search solution to have on your organization. Normally, you would want an Enterprise Search solution to:

  •  Enable people to work faster
  •  Enable people to produce better quality
  •  Provide the means for information reuse
  •  Inspire your employees to innovate and invent

These are all effects that a well-designed and maintained Enterprise Search application will help you address. However, the challenge when calculating the return on investment is that you are attempting to have an effect on workflows that are not clearly visible on your revenue stream. There is no easy way to interlink saved or earned dollars to employees being more innovative.

So how do you prove that you are not wasting money?

There are two straightforward ways to address the problem: Studying how users really interact with the Enterprise Search application and asking them how they value it.

User Behavior through Search Logs

By extracting statistics from the logs of your Enterprise Search application, you can monitor how users interact with the tool. There are several statistic measures that can be interesting to look at in order to establish a positive influence on one or more of the targeted effects.

A key performance indicator for calculating if the Enterprise Search application enables people to work faster is to monitor the average ranking of a clicked hit in the result list. If people tend to scroll down the result set before clicking a hit and opening up a document, this implies the application does not provide proper ranking of the results. In other words, users are forced to review the result set, which obviously slows them down.

By monitoring the amount of users that are using the system, by following the number of different documents they open up through search and by observing the complexity of the queries they perform, you can estimate the level of information your users are expecting to find through searching.

If the application is trusted to render relevant, up-to-date results, more users will use it, they will carry out more complex queries and they will open up a wider range of different documents. If your users do not trust the system, however, they will not use it or they will only search for a limited set of simple things such as “news”, “today’s menu” or “accounting office”. If this is the case, you can hardly say your Enterprise Search application has met the requirements posed on it.

Conversely, if the users access a wide set of documents through search and you have a large number of unique users and queries, then this implies your Enterprise Search application is a valued information access tool that promotes information reuse and innovation based on existing corporate knowledge.

User Expectations through Surveys

Another way to collect information for assessing the return of investment of your Enterprise Search initiative is to ask the users what they think. If you ask a representative subset of your intended users how well the Enterprise Search application fits their specific purposes, you will have an estimate of the quality of the application.

There are a lot of other questions you can ask: Does the application help the user to find relevant corporate information? Are the results ranked properly? Does the application help the user to get an overall picture of a topic? Does it enable the user to get new ideas or find new opportunities? Does it help him avoid duplicating work already done elsewhere within the organization?

A Combination of Increased Usage and Perceived Value

As we have seen, the return on investment of an Enterprise Search initiative is often hard to quantify, but the impact such an application has on a set of targeted effects can be measured using search logs and user surveys. The data collected this way provides an estimate of the value of Findability within the firewall of an organization.

Nowadays, hardly anybody questions the marketing value of a good corporate web site or the impact email has on the way we communicate. Such channels and services are self-evident business enablers today. In this respect, the benefits of precise and quick information access within the corporation should be self-evident. The trick is to get the tool just right.