Better Search Engines and Information Practices in Digital Workplaces

During this year I have worked on a research project that aims to facilitate the development and implementation of an enterprise search engine. By understanding the use and value of information at the digital workplaces, we hope to create even better preconditions for optimizing a search engine to the requirements of a specific organization.

We use a work-task based research approach where we study information practices – that is, the normalized ways we use to recognize information needs, look for information, and how it is valued and used. By studying such practices in real-life work tasks, we can outline the role that a search engine plays in relation to other work tasks as well as to other ways of finding information. In short, being engaged in a creativity-oriented work task initiates different types of information practices compared to the practices we use in everyday, routine-based work tasks …

The creativity-oriented work tasks involve a dimension of innovation, and concepts such as learning and development are often used to describe these activities. Uncertainty is something that is associated with curiosity and may be seen as a driving force behind information seeking. Information that is rich in nuances and that offers different, even contradictory explanations or descriptions is usually appreciated, and the task outcome is only vaguely discerned at first. Routine-oriented tasks, on the other hand, are focused on increasing effectiveness and reducing uncertainty as quickly as possible in the task outcome, which itself may be sketched out relatively clearly from the beginning. Information seeking is often directed to readily available facts. All this means that a search engine must support a variety of information practices at any given workplace!

The “we” in this project is myself together with a Findwise colleague Henrik Strindberg. The project is financially supported by the Swedish Foundation for Strategic Research, and while I am not working with the present project I am employed by the University of Borås.

Just now I am finalizing a presentation of the project for the ICKM conference in Pittsburgh, PA, USA, next week. The presentation is entitled “Interrelated use and value of information sources”, and will be available through the conference proceedings in due time.

Very exciting … and while there I will also attend the board meetings of the ASIS&T’s Board of Directors as a newly appointed Director-at-Large. Very exciting, too!

The 73rd Annual Meeting of ASIS&T focuses on “Navigation Streams in an Information Ecosystem”.

Why is Search Easy and Hard?

Last year my colleague Lina and I went to the Workshop on Human Computer Interaction and Information Retrieval (HCIR) in Washington DC. This year we did not have the possibility to attend but since all the material is available online I took part remotely any way. I wanted to share with you what I found most interesting this year. (Daniel Tunkelang who was one of the organizers also posted a good overview of the event on his blog.)

This years keynote speaker was Dan Russell, a researcher from Google. He talked about Search Quality and user happiness; Why search is easy and hard. The point I found most interesting in his presentation was how improvement is not only needed when it comes to tools and data but also improving the users’ search skills. My own experience from various search projects is similar; users are not good at searching. Even though they are looking for a specific version of a technical documentation for a specific product they might just enter the name of the product, or even the product family. (It’s a bit like searching for ‘camera’ when you expect to find support documentation on your Dioptric lens for you Canon EOS 60D.) So I agree that users need better search skills. In his presentation Russell also presented some ideas on how a search application can help users improve their search skills.

Search is both easy and hard. Perhaps this is one of the reasons for the introduction of the HCIR Challenge as a new part of the workshop . From the HCIR website:

The aims of the challenge are to encourage researchers and practitioners to build and demonstrate information access systems satisfying at least one of the following:

  • Not only deliver relevant documents, but provide facilities for making meaning with those documents.
  • Increase user responsibility as well as control; that is, the systems require and reward human effort.
  • Offer the flexibility to adapt to user knowledge / sophistication / information need.
  • Are engaging and fun to use.

The winner of the challenge was a team of researchers from Yahoo Labs who presented Searching Through Time in the New York Times. The Time Explorer features a results page with an interactive time line that illustrates how the volume of articles (results) have changed over time. I recommend that you read the article in tech review to learn more about the project, or try out the Time explorer demo yourself. You can also learn more about the challenge in this blog post by Gene Golovchinsky.

All the papers and posters from the workshop can be found on the new website.

Combining Search and Browse – Integrated Faceted Breadcrumbs

Finding information can be tricky and as I have written about in one of my previous posts improving findability is not about providing a single entrypoint to information. Users have different ways of finding information (browsing, searching and asking). They often combine these techniques with each other (berrypicking) and so they all need to be supported. Peter Morville states that.

“Browse and Search work best in tandem… the best finding interfaces achieve a balance, letting users move fluidly between browsing and searching.”

A lot of sites are improving their search experience through the implementation of faceted search. However, very few successfully integrate faceted search and browsing on their site. Searching and browsing are treated as two separate flows of interaction instead of trying to combine them which would provide the users with a much better experience.

That is why I was glad to learn about an idea from Greg Nudelman which he presented in his session at the IASummit which I attended last week. In his session Greg introduced his idea about Integrated Faceted Breadcrumb. According to him breadcrumbs are intuitive, flexible and resourceful and they are design elements that don’t cause problems but simply work. To test his idea he conducted usability tests on a prototype using the Integrated Faceted Breadcrumb. According to his evaluation the integrated faceted breadcrumb has a lot of advantages over other faceted solutions:

  1. Combine hierarchical Location & Attribute breadcrumbs
  2. Use Change instead of Set-Remove-Set
  3. Automatically retain relevant query information
  4. Label breadcrumb aspects
  5. Make it clear how to start a new search
  6. Allow direct keyword manipulation.

I find this idea interesting and I am currently thinking about whether it could be applied into one of my own projects. (According to Greg it has not been implemented anywhere yet even though the findings from the usability testing were positive.) However I wonder if this is a concept that works well only for sites with relatively homogeneous content or if it would also work on larger collections of sites such as intranets? Can it be used in an intuitive way with a large number of facets and can it cope with the use of more complex filtering functionalities? For some sites it might not be the best idea to keep the search settings when the user changes search terms. These are some things I would like to find out. What do you think about this? Could you apply it to your site(s)? I recommend that you have a look at Greg Nudelman’s presentation on slideshare and find out for yourself. You can also find an article about the Integrated Faceted Breadcrumb on Boxes and Arrows. I look forward to a discussion about whether this is any good so write me a comment here at the findability blog or find me on twitter.

Query Suggestions Help Users Get Unstuck

Several papers at the HCIR09 workshop touched on the topic of query suggestions. Chirag Shah and Gary Marchionini presented a poster about query reuse in exploratory search tasks and Diane Kelly presented results from two different studies that examined people’s use of query suggestions and how usage varied depending on topic difficulty. (Their papers are available for download as part of the proceedings from the workshop.)

According to Shah and Marchionini users often search for the same things. They reuse their previous queries e.g. search for the same things multiple times. Users use their previous searches to refind information and also to expand or further filter their previous searches by adding one or more keywords. There is also a significant overlap between what different users search for suggesting that users have a tendency to express their information needs in similar ways. These results support the idea that query suggestions can be used to help users formulate their query.  Yahoo and YouTube  are two of the systems that uses this technique, where users get suggestions of queries and how they can add more words to their query based on what other users have searched for.

Diane Kelly concludes that users use query suggestion both by typing in the same thing as shown in the suggestion and by clicking on it. Users also tend to use more query suggestions when searching for difficult topics. Query suggestions help users get “unstuck” when they are searching for information.  It is however hard to know whether query suggestions actually return better results. The users expectation and preferences do have an effect on user satisfaction as well. User generated query suggestions are also found to be better than query suggestions generated by the search system. So the mere expectation that the query suggestions will help a user could have an positive effect on his or hers experience…

Query suggestions are meant to help the users formulate a good query that will provide them with relevant results. Query suggestions can also work as with yahoo search where query suggestions both suggest more words to add to the query but also provides the users with suggestions for other related concepts to search for. So searching for Britney Spears will for example suggest the related search for Kevin Federline (even though they are now divorced) and searching for enterprise search will suggest concepts such as relevance, information management and off course the names of the different search vendors.

If you apply this to the enterprise search setting the query suggestion could provide the user with several different kinds of help. Combining the user’s previous searches with things other users searched for but also providing suggestions for recommended queries or concepts. The concepts will be high quality information and suggestions controlled by the team managing the search application. It is a way of combining quick links or best bets with query suggestions and a way to hopefully improve the experienced value of the query suggestions. The next step then is to work with these common queries that users search for and make sure that they return relevant results, but that is an entirely different topic…

Findwise in Cooperation with Borås University College Receives Research Grant

In recent decades Swedish and Western industry have had to adapt to the new paradigm. Moving from classical production industry organizations towards knowledge companies in which sales of services and knowledge are often bundled with a product – resulting in a complete solution. This change is vital for the survival of the Western world’s economy which previously has been built upon organizations of heavy industrial giants optimizing production processes and factory outputs by reducing overheads and increasing quality.

The threat to the industry from low cost countries, which no longer only compete just on low cost, but also with high quality and competence, forces Western organizations to develop new strategies to sustain their growth and competitive advantage. Cutting margins in order to compete with low cost countries is a downward spiral. Instead changing the model, to be able to provide knowledge and holistic understanding of customers needs and the ability to rapidly deliver a complete solution is now becoming the key competitive advantage. This however requires investment in IT and knowledge exchange tools. By moving away from selling physical products and components to solutions higher margins are possible because more business value is exchanged in the transactions.

The organizations adapting to this change, are identifiable by the fact that they consider knowledge and information as corporate assets – treated and cared for as any other asset. One example is the Swedish company SKF Group whose new vision is the “Knowledge Engineering Company” where the company going through a change from component supplier to a holistic supplier of both products and services.

A key success factor in this transformation from products to solutions is that the supply of knowledge and information to the employees is effective, easy to use and complete. The organization succeeds in providing that extra value, thereby allowing higher margins. Historical key performing indicators (KPIs) such as factory output, reduction of defects and increasing of quality, are dealing with physical production efficiency to ensure as little cost per unit manufactured and as high quality as possible. Individuals are used to measuring these KPIs and provide a way to manage the operational production processes. The turnover and efficiency of information and knowledge exchange lacks these models and measurement tools, thereby not allowing them to be managed. What you can’t measure, you can’t manage, or improve.

One technical solution which has the capabilities to enable complete, rapid and reduced turnover time for knowledge and information exchange, is Enterprise Search. This has been recognized by the Swedish Foundation for Strategic Research, which granted Findwise AB research funds to tackle this problem in cooperation with the College University of Borås in the Strategic Mobility program. The funded project will study the usefulness and value of a well functioning search engine for work-related information use. It will also identify performing indicators for information and knowledge exchange through search and to achieve results that systematically will illustrate the quantitative direct effects together with soft indirect effects.

The project will start early 2010 and run through the entire year. As part of the project, Dr. Katriina Byström will join Findwise and work together with Findwise employees in this joint research project. Findwise customers are invited to participate in the project and will have the availability to influence its direction. For more information on research at Findwise contact Henrik Strindberg.

About Dr. Katriina Byström
Dr. Katriina Byström is an associate professor in the Swedish School of Library and Information Science at the University College of Borås & Goteborg University, Sweden. She is one of initiators and director of the IA bachelor’s programme at Swedish School of Library and Information Science, and she is a chair for the programme with teaching involvement broadly across the curricula. Furthermore , Katriina is associate editor and co-founder at the Journal of Information Architecture. Katriina’s degree is in information studies, and her research focus on task-based information seeking, information retrieval and information architecture.

The Future of Information Discovery

I recently attended the third annual workshop on Human Computer Interaction and Information retrieval ( HCIR 2009) in Washington DC together with my colleague Lina. This is the first in a series of blog posts about what happened at the workshop. First up is the keynote about the Future of Information Discovery, by Ben Shneiderman.

Ben Shneiderman, professor at the University of Maryland and founding director of the Human Computer Interaction Laboratory held the workshop keynote. He started off by talking about what he called the elephant in the room, Google. Because whenever you talk about search these days you have to talk about Google. Google has become the baseline for search and the system that users relate other search experiences to. Almost all of our customers’ users has in one way or another asked “why can’t our intranet be more like Google?” (Read more about expectations to Googlify the company in a previous blog post by Mickel. You can also download the slides to Ben Shneidermans keynote presentation.)

As Ben Shneiderman said, Google does actually do the job, finding facts work. However searching for information can be dangerous. Google does well on handling simple fact-finding tasks but we need better tools to handle other types of searches such as:

  • Extended fact finding tasks where the queries are often vague
  • Tasks involving exploration of availability where the requested results can be vague
  • Open ended browsing and problem analysis where there can be hidden assumptions
  • Mismatch between the users information needs and the available metadata which will require exhaustive searching.

One of the points that I appreciated the most in this keynote was that systems that support searching for information not only need to support simple known-item searches, which Google does well. They also need to support other things:

  • Helping users enrich query formulation
  • Expanding result management
  • Enable long-term effort
  • Enhance collaboration

I am especially pleased by this statement since these are some of the important issues that we are working with in our customer projects. You will also learn more about query formulation in one of our upcoming blog posts from HCIR.

Supporting these cases are important for supporting users in their information seeking tasks and, according to Shneiderman, this should also be done while enabling users to deal with specific cases of search, concerning:

  • Completeness – Do I have all the information on a specific topic? This is especially important in for example legal or medical cases.
  • Absence of information – proving non-existence of information is very difficult but needed when applying for a patent or registering a trademark.
  • Outliers – making unexpected connections between information and finding and learning new  things that you would not have expected to find.
  • Bridging – Connecting different disciplines with each other.

This is very important because when users search the goal is not the information itself. No users go to a search interface just for the fun of searching for information. They need the information for a purpose. Search therefore needs to support things such as decision-making, collaboration, innovation and societal improvement. Search will only be of true value to users when it not only searches the simple fact-finding tasks but when it helps users solve the real problems in the real world. And good tools can force people to reframe their thinking and see things in a different light. That is the kind of tools that we should be designing.

Findwise is attending and Publishing a Paper at HCIR 2009

I’m glad to announce that Findwise is attending HCIR 2009, Human Computer Interaction with Information Retrieval, in Washington DC on October 23. Our paper about designing for Enterprise Search has been accepted to the conference so we (Maria Johansson and Lina Westerling) are going to Washington to attend the workshop and discuss HCIR with the researchers and practitioners most prominent in this area.

HCIR is a field bridging Human Computer Interaction with Information Retrieval. The design of usable search interfaces is off course a focus area in this field.

HCIR 2009 Article

The proceedings from the workshop has already been published on the HCIR 2009 conference website. You can also download our article “Designing for Enterprise Search in a Global Organization” from the Findwise website. We hope you enjoy it.

If you have any questions or topics you would like to know more about, send us an email with a question before October 21 and we’ll take it with us to the workshop. Stay tuned for more about what happened at HCIR 2009.

High Expectations to Googlify the Company = Findability Problem?

It is not a coincidence that the verb “to google” has been added to several renowned dictionaries, such as those from Oxford and Merriam-Webster. Search has been the de facto gateway to the Web for some years now. But when employees turn to Google on the Web to find information about the company they work for, your alarm bells should be ringing. Do you have a Findability problem within the firewall?

The Google Effect on User Expectations

“Give us something like Google or better.”

 

“Compared to Google, our Intranet search is almost unusable.”

 

“Most of the time it is easier to find enterprise information by using Google.”

The citations above come from a study Findwise conducted during 2008-2009 for a customer, who was on the verge of taking the first steps towards a real Enterprise Search application. The old Intranet search tool had become obsolete, providing access to a limited set of information sources only and ranking outdated information over the relevant documents that were in fact available. To put it short, search was causing frustration and lots of it.

However, the executives at this company were wise enough to act on the problem. The goal was set pretty high: Everybody should be able to find the corporate information they need faster and more accurately than before. To accomplish this, an extensive Enterprise Search project was launched.

This is where the contradiction comes into play. Today users are so accustomed to using search as the main gateway to the Web, that the look and feel of Google is often seen as equal to the type of information access solution you need behind the firewall as well. The reasons are obvious; on the Web, Google is fast and it is relevant. But can you—and more importantly should you—without question adopt a solution from the Web within the firewall as well?

Enterprise Search and Web Search are different

  1. Within the firewall, information is stored in various proprietary information systems, databases and applications, on various file shares, in a myriad of formats and with sophisticated security and version control issues to take into account. On the Web, what your web crawler can find is what it indexes.
  2. Within the firewall, you know every single logged in user, the main information access needs she has, the people she knows, the projects she is taking part in and the documents she has written. On the Web, you have less precise knowledge about the context the user is in.
  3. Within the firewall, you have less links and other clear inter-document dependencies that you can use for ranking search results. On the Web, everything is linked together providing an excellent starting point for algorithms such as Google’s PageRank.

Clearly, the settings differ as do user needs. Therefore, the internal search application will be different from a search service on the web; at least if you want it to really work as intended.

Start by Setting up a Findability Strategy

When you know where you are and where you want to be in terms of Findability—i.e. when you have a Findability strategy—you can design and implement your search solution using the search platform that best fits the needs of your company. It might well be Google’s Search Appliance. Just do not forget, the GSA is a totally different beast compared to the Google your users are accustomed to on the Web!

References

http://en.wikipedia.org/wiki/Googling

Six Simple Steps to Superior Search

Do you have your search application up and running but it still doesn’t quite seem to do the trick? Here are six simple steps to boost the search experience.

Avoid the Garbage in-Garbage out Syndrome

Fact 1: A search application is only as good as the content it makes findable

If you have a news search service that only provides yesterday’s news, the search bit does not add any value to your offering.

If your Intranet search service provides access to a catalog of employee competencies, but this catalog does not cover all co-workers or contain updated contact details, then search is not the means it should be to help users get in touch with the right people.

If your search service gives access to a lot of different versions of the same document and there is no metadata available as to single out which copy is the official one, then users might end up spending unnecessary time reviewing irrelevant search results. And still you cannot rule out the risk that they end up using old or even flawed versions of documents.

The key learning here is that there is no plug and play when it comes to accurate and well thought out information access. Sure, you can make everything findable by default. But you will annoy your users while doing so unless you take a moment and review your data.

Focus on Frequent Queries

Fact 2: Users tend to search for the same things over and over again.

It is not unusual that 20 % of the full query volume is made up of less than 1 % of all query strings. In other words, people tend to use search for a rather fixed set of simple information access tasks over and over again. Typical tasks include finding the front page of a site or application on the Intranet, finding the lunch menu at the company canteen or finding the telephone number to the company helpdesk.

In other words, you will be much advised to make sure your search application works for these highly frequent (often naïve) information access tasks. An efficient way of doing so is to keep an analytic eye on the log file of your search application and take appropriate action on frequent queries that do not return any results whatsoever or return weird or unexpected results.

The key learning here is that you should focus on providing relevant results for frequent queries. This is the least expensive way to get boosted benefit from your search application.

Make the Information People Often Need Searchable

Fact 3: Users do not know what information is available through search.

Users often believe that a search application gives them access to information that really isn’t available through search. Say your users are frequently searching for ”lunch menu”, ”canteen” and ”today’s lunch”, what do you do if you do not have the menu available at all on your Intranet or Web site?

In the best of worlds, you will make frequently requested information available through search. In other words, you would add the lunch menu to your site and make it searchable. If that is not an option, you might consider informing your users that the lunch menu—or some other popular information people tend to request—is not available in the search application and provide them with a hard-coded link to the canteen contractor or some other related service as a so called “best bet” (or sponsored link as in Google web search).

The key learning here is to monitor what users frequently search for and make sure the search application can tackle user expectations properly.

Adapt to the User’s Language

Fact 4: Users do not know your company jargon.

People describe things using different words. Users are regularly searching for terms which are synonymous to—but not the same as—the terms used in the content being searched. Say your users are frequently looking for a ”travel expense form” on your Intranet search service, but the term used in your official company jargon  is ”travel expenses template”. In cases like this you can build a glossary of synonyms mapping those common language terms people tend to search for frequently to official company terms in order to satisfy your users’ frequent information needs better without having to deviate from company terminology. Another way of handling the problem is to provide hand-crafted best bets (or sponsored links as in Google web search) that are triggered by certain common search terms.

Furthermore, research suggests that Intranet searches often contain company-specific abbreviations. A study of the query log of a search installation at one of Findwise’s customers showed that abbreviations—query strings consisting of two, three or four letters—stood for as much as 18 % of all queries. In other words, it might be worthwhile for the search application to add the spelled-out form to a query for a frequently used abbreviation. Users searching for “cp” on the Intranet would for example in effect see the results of the query “cp OR collaboration portal”

The lesson to learn here is that you should use your query log to learn the terminology the users are using and adapt the search application accordingly, not the other way around!

Help Users With Spelling

Fact 5: Users do not know how to spell.

Users make spelling mistakes—lots of them. Research suggests that 10—25 % of all queries sent to a search engine contain spelling mistakes. So turn on spellchecking in your search platform if you haven’t already! And while you are at it, make sure your search platform can handle queries containing inflected forms (e.g. “menu”, “menus”, “menu’s”, “menus’”). There’s your quick wins to boost the search experience.

Keep Your Search Solution Up-To-Date

Fact 6: Your search application requires maintenance.

Information sources change, so should your search application. There is a fairly widespread misconception that a search application will maintain itself once you’ve got it up and running. The truth is you need to monitor and maintain your search solution as any other business-critical IT application.

A real-life example is a fairly large enterprise that decided to perform a total makeover of its internal communication process, shifting focus from the old Intranet, which was built on a web content management system, in favor of a more “Enterprise 2.0 approach” using a collaboration platform for active projects and daily communication and a document management system for closed projects and archived information.

The shift had many advantages, but it was a disaster for the Enterprise Search application that was only monitoring the old Intranet being phased out. Employees looking for information using the search tool would in other words only find outdated information.

The lesson to learn here is that the fairly large investment in efficient Findability requires maintenance in order for the search application to meet the requirements posed on it now and in the future.

References

100 Most Often Mispelled Misspelled Words in English – http://www.yourdictionary.com/library/misspelled.html

Definition of “sponsored link” – http://encyclopedia2.thefreedictionary.com/Sponsored+link

Customer Service Powered by Search Technology

I was on the train, on my way to Copenhagen and UX intensive a four day seminar hosted by Adaptive Path. Looking forward to this week I was also contemplating the past year and the projects we’ve been working on. I recently finished a project at a customer service organization at a large company. The objective was to see if the agents (employees) helping customers could benefit from having a search platform. Would the search engine help the users in finding the right content to help their customers?

Our point was off course that it would, but it was up to us to prove it. And we did. The usages tests showed results better than I would have dared to hope for.

  • All users found it to be easier to search for information than browsing for it.
  • Searching helped the users not only in finding information faster, but finding information they didn’t know where to find or didn’t even know existed.
  • All users preferred using the search functionality instead of navigation for information.
  • The search functionality helped new employees learn the information they needed to know in order to help the customers, hence they were productive faster.Less time was spent asking for help from colleagues and support since users found the information they needed by searching for it.

These results are all very positive, but the most overwhelming thing for me in this project was the level of engagement from the users. They really enjoyed being a part of the evaluations, bringing feedback to the project team. They felt that they were a part of the process and this made them very positive to the change this project meant.

Change is often a hard thing in development projects. Even if the change is better for the end users of the system, the change in itself can still be problematic making people hostile to the idea, even though it is improving their situation. Involving users not only helps in creating a good product, but also in creating a good spirit around the project. I have experienced this in other projects as well. By setting up reference groups for the development process we have not only managed to get good feedback to the project but have also created a buzz about what’s happening. People are volunteering for being participants in our reference groups. This buzz spreads and creates a positive feeling about the change the project is bringing. Instead of dreading the users are welcoming the change. It’s user research at its best.

So the next time you are asking yourself why you should involve users in your project and not only business stakeholders – think about how not only the end product, but the project and the process as a whole, could benefit from this.