Tag Archives: Searching

The ROI of Enterprise Search—Where’s the Beef?

Posted on by
Reply

When faced with the large up-front investment of an Enterprise Search installation, executives are asking for proof that the investment will pay up. Whereas it is easy to quantify the value of search on an e-commerce site or as part of the company helpdesk—increased sales, shorter response times—how do you go about verifying that your million-dollar Enterprise Search application has the desired effects on your revenue stream?

Search engines on the Web have changed the landscape of information access. Today, employees are asking for similar search capabilities within the firewall as they are used to having on the Web. Search has become the preferred way of finding information quickly and accurately.

Top executives at large corporations have heard the plea and nowadays see the benefits of efficient Findability. However, it costs to turn the company information overload from a storage problem of the IT department to a valuable asset and business enabler for everybody. So how do you prove the investment worthwhile?

The Effects of Enterprise Search

Before you can prove anything, you need to establish the effects you would like your Enterprise Search solution to have on your organization. Normally, you would want an Enterprise Search solution to:

  •  Enable people to work faster
  •  Enable people to produce better quality
  •  Provide the means for information reuse
  •  Inspire your employees to innovate and invent

These are all effects that a well-designed and maintained Enterprise Search application will help you address. However, the challenge when calculating the return on investment is that you are attempting to have an effect on workflows that are not clearly visible on your revenue stream. There is no easy way to interlink saved or earned dollars to employees being more innovative.

So how do you prove that you are not wasting money?

There are two straightforward ways to address the problem: Studying how users really interact with the Enterprise Search application and asking them how they value it.

User Behavior through Search Logs

By extracting statistics from the logs of your Enterprise Search application, you can monitor how users interact with the tool. There are several statistic measures that can be interesting to look at in order to establish a positive influence on one or more of the targeted effects.

A key performance indicator for calculating if the Enterprise Search application enables people to work faster is to monitor the average ranking of a clicked hit in the result list. If people tend to scroll down the result set before clicking a hit and opening up a document, this implies the application does not provide proper ranking of the results. In other words, users are forced to review the result set, which obviously slows them down.

By monitoring the amount of users that are using the system, by following the number of different documents they open up through search and by observing the complexity of the queries they perform, you can estimate the level of information your users are expecting to find through searching.

If the application is trusted to render relevant, up-to-date results, more users will use it, they will carry out more complex queries and they will open up a wider range of different documents. If your users do not trust the system, however, they will not use it or they will only search for a limited set of simple things such as “news”, “today’s menu” or “accounting office”. If this is the case, you can hardly say your Enterprise Search application has met the requirements posed on it.

Conversely, if the users access a wide set of documents through search and you have a large number of unique users and queries, then this implies your Enterprise Search application is a valued information access tool that promotes information reuse and innovation based on existing corporate knowledge.

User Expectations through Surveys

Another way to collect information for assessing the return of investment of your Enterprise Search initiative is to ask the users what they think. If you ask a representative subset of your intended users how well the Enterprise Search application fits their specific purposes, you will have an estimate of the quality of the application.

There are a lot of other questions you can ask: Does the application help the user to find relevant corporate information? Are the results ranked properly? Does the application help the user to get an overall picture of a topic? Does it enable the user to get new ideas or find new opportunities? Does it help him avoid duplicating work already done elsewhere within the organization?

A Combination of Increased Usage and Perceived Value

As we have seen, the return on investment of an Enterprise Search initiative is often hard to quantify, but the impact such an application has on a set of targeted effects can be measured using search logs and user surveys. The data collected this way provides an estimate of the value of Findability within the firewall of an organization.

Nowadays, hardly anybody questions the marketing value of a good corporate web site or the impact email has on the way we communicate. Such channels and services are self-evident business enablers today. In this respect, the benefits of precise and quick information access within the corporation should be self-evident. The trick is to get the tool just right.

Designing for Search at Business to Buttons

Posted on by
Reply

I read an interesting interview where Peter Morville interviewed Luke Wroblewski about Search Patterns. They mentioned some search patterns.

Query Refinement

Most users won’t refine pre-query. They don’t know the size and range of the index. But they will refine post-query, and it provides tremendous value.

Search Assistance

Studies show a correlation between the number of words in the query and satisfaction with results. Search Assist improves query richness. Tips (e.g., did you mean?) improve confidence and suggestions (e.g., related concepts) can shift users into exploratory mode.

Layout

For query refinement tools, we found the right rail is the least discoverable. The top is the most obvious, but suggestions can get in the way of results. That’s why search assist is on top but hidden by default. Other examples include Local and Shopping.

Vertical Search

In verticals such as News and Games, we’ve found that people are fine with two search boxes, one for the vertical and one for web search. In each vertical, we’re selectively exposing structured metadata (e.g., pricing in Autos) to support the decision making process.

It’s always nice when other companies corroborate our findings and experiences.

If you want to know more about these topics and about designing good search experiences come and listen to us talk at Business to Buttons in Malmö. There will be something for the expert as well as those new to search technology. You can also come and talk to us in our monter during the breaks. Hope to see you there!

What Differentiates a Good Search Engine from a Bad One?

Posted on by
Reply

That was one of the questions the UIE research group asked themselves when conducting a study of on-site search. One of the things they discovered was that the choice of search engine was not as important as the implementation. Most of the big search vendors were found in both the top sites and the bottom sites.

So even though the choice of vendor influences what functionality you can achieve and the control you have over your content there are other things that matter, maybe even more. Because the best search engine in the world will not work for you unless you configure it properly.

According to Jared Spool there are four kinds of search results:

  • ‘Match relevant results’ – returns the exact thing you were looking for.
  • ‘Zero results’ – no relevant results found.
  • ‘Related results’ – i.e. search for a sweater and also get results for a cardigan. (If you know that a cardigan is a type of sweater you are satisfied. Otherwise you just get frustrated and wonder why you got a result for a cardigan when you searched for a sweater).
  • ‘Wacko results – the results seem to have nothing in common with your query.

So what did the best sites do according to Jared Spool and his colleagues?
They returned match relevant results, and they did not return 0 results for searches.

So how do you achieve that then? We have previously written about the importance of content refinement and information quality. But what do you do when trying to achieve good search results with your search engine? And what if you do not have the time or knowledge to do a proper content tuning process?

Well, the search logs are a good way to start. Start looking at them to identify the 100 most common searches and the results they return. Are they match relevant results? It is also a good idea to look at the searches that return zero results and see if there is anything that can be done to improve those searches as well.

Jared Spool and his colleagues at UIE mostly talk about site search for e-commerce sites. For e-commerce sites bad search results mean loss of revenue while good search results hopefully give an increase in revenue (if other things such as check out do not fail). Working with intranet search the implications are a bit different.

With intranet search solutions the searches can be more complex when information not items, is what users are searching for. It might not be as easy to just add synonyms or group similar items to achieve better search results. I believe that in such a complex information universe, proper content tuning is the key to success. But looking at the search logs is a good way for you to start. And me and my colleagues here at Findwise can always help you how to get the most out of your search solution.

Evolution of Enterprise Search

Posted on by
Reply

Yesterday I have had an exciting and challenging day. Caroline and I visited a new customer, a large global Swedish industrial enterprise, for whom Findwise is doing a prestudy of the customer’s search requirements. The guy, whom we met was very alert and challenged us in the evolution of enterprise search and if the technology is mature enough for “safe” project deployments. I have managed search projects throughout Europe since 2002 and today it became clear to me how the projects and hence the enterprise search technology has changed.

Back in the early days, connecting to different datasources and indexing a vast amount of different sources were the biggest problem. Most projects were managed with a focus on this and the users should be happy with what we delivered. Today connectors to most data sources as well as managing different formats are commodity. The projects have finally and gladly become user driven so my conclusion is, yes, the technique is definitely mature.

Thank you for an important insight and good challenging thoughts!

Search Driven Process to Increase Content Quality

Posted on by
Reply

Experience from recent and ongoing search and retrieval projects have shown that enterprises have got a better and deeper insight in their content when deploying a new search platform and search driven solutions. Not only in unstructured content repositories, but also in structured sources. As information is indexed and is visualized in a more user friendly way it doesn’t take much time before the people responsible find content issues that are brought out in the light. Content that e.g. is misplaced, tagged wrongly, documents with poorly defined security information etc. Issues that earlier were hidden due to lack of a holistic view of content. This leads to setting up search driven processes to increase the content quality.

It has been said that before enterprises should think of deploying an enterprise search solution one is recommended to get a completely clear picture of all it’s content; but maybe one should reformulate this and also think of an enterprise search solution as a supporting tool in the process when improving the content as well.

Taking it a step further would be to allow write-backs from the search engine to content sources to enrich and improve quality and completeness of stored information. Tune search quality and content quality at the same time! Make the content quality assurance process search driven!

The Evolution of Search in Video Media

Posted on by
Reply

Search is becoming more and more an infrastructure necessity and in some areas, and for some users, considered a commodity. However, the evolution of new areas for use of search is growing rapidly both on the web and within the enterprises. Google’s recent acquisition of YouTube is giving us one example of new areas. To search in video material is not simple and I believe we have just seen the very early stage of this new technique.

I am participating in an EU funded project – RUSHES. The project is within the 6th framework program. The aim of the project is among other things to develop techniques for automatic content cataloguing and semantic based indexing. So what impact will this have for the end users and search in video ?

Well, they won’t have to go to a category and search under for example “News and politics”, instead the users will be able to use keywords such as “president” and “scandals” to get clips about Nixon and the Watergate saga. The content provider, on the other hand, won’t have to see the video clip in order to annotate and meta tag it, they will just run the video through a “RUSHES” module and the program will handle the rest. These new scenarios in combination with the semantic web (Web 2.0), will enable new possibilities and business opportunities which we have not even dreamt of before! Like search in video!