Gamification in Information Retrieval

My last article was mainly about Collaborative Information Seeking – one of the trends in enterprise search. Another interesting topic is the use of games’ mechanics in CIS systems. I met up with this idea during previously mentioned ESE 2014 conference, but interest is so high, that this year in Amsterdam a GamifIR (workshops on Gamification for Information Retrieval) took place. IR community have debated about what kind of benefits can IR tasks bring from games’ techniques. Workshops cover gamified task in context of searching, natural language processing, analyzing user behavior or collaborating. The last one was discussed in article titled “Enhancing Collaborative Search Systems Engagement Through Gamification” and has been mentioned by Martin White in his great presentation about search trends on last ESE summit.

Gamification is a concept which provides and uses game elements in non-game environment. Its goal is to improve customers or employees motivation for using some services. In the case of Information Retrieval it is e.g. encouraging people to find information in more efficient way. It is quite instinctive because competition is  an inherent part of human nature. Long time ago, business sectors have noticed that higher engagement, activating new users and establishing interaction between them, rewarding the effort of doing something lead to measurable results. Even if quality of data given by users could be higher. Among those elements can be included: leaderboards, levels, badges, achievements, time or resources limitation, challenges and many others. There are even described design patterns and models connected with gameplay, components, game practices and processes. Such rules are essential because virtual badge has no value until being assigned by user.

Collaborative Information Seeking is an idea suited for people cooperating on complex task which leads to find specific information. Systems like this support team work, coordinate actions and improve communication in many different ways and with usage of various mechanisms. At first glance it seems that gamification is perfect adopted to CIS projects. Seekers become more social, feeling of competence foster actions which in turn are rewarded.

The most important thing is to know why do we need gamified system and what kind of benefits we will get. Next step is to understand fundamental elements of a game and find out how adopt them to IR case. In their article “Enhancing Collaborative Search Systems Engagement Through Gamification”, researchers of Granada and Holguin universities have listed propositions how to gamify CIS system.  Based on their suggestions I think essential points are to prepare highly sociable environment for seekers. Every player (seeker) needs to have own personal profile which stores previous achievements and can be customized. Constant feedback on progress, list of successful members, time limitations, keeping the spirit of competition by all kinds of widgets are important for motivating and building a loyalty. Worth to remember that points collected after achieving goals need to be converted into virtual values which can distinguish the most active players. Crucial thing is to construct clear and fair principles, because often information seeking with such elements is a fun and it can’t be ruined.

Researchers from Finnish universities, who published article “Does Gamification Work?”, have broken down a problem of gamifying into components and have thoroughly studied them. Their conclusion was that concept of gamification can work, but there are some weaknesses – context which is going to be gamified and the quality of the users. Probably, the main problem is lack of knowledge which elements really provide benefits.

Gamification can be treated as a new way to deal with complex data structures. Limitations of data analyzing can be replaced by mechanism which increase activity of users in Information Retrieval process. Even more – such concept may leads to more higher quality data, because of increased people motivation. I believe, Collaborative Information Seeking, Gamification and similar ideas are one of the solutions how to improve search experience by helping people to become better searchers than not by just tuning up algorithms.

Why search and Findability is critical for the customer experience and NPS on websites

To achieve a high NPS, Net Promoter Score, the customer experience (cx) is crucial and a critical factor behind a positive customer experience is the ease of doing business. For companies who interact with their customers through the web (which ought to be almost every company these days) this of course implies a need to have good Findability and search on the website in order for visitors to be able to find what they are looking for without effort.

The concept of NPS was created by Fred Reichheld and his colleagues of Bain and Co who had an increasing recognition that measuring customer satisfaction on its own wasn’t enough to make conclusions of customer loyalty. After some research together with Satmetrix they came up with a single question that they deemed to be the only relevant one for predicting business success “How likely are you to recommend company X to a friend or colleague.” Depending upon the answer to that single question, using a scale of 0 to 10, the respondent would be considered one of the following:

net-promoter

The Net Promoter Score model

The idea is that Promoters—the loyal, enthusiastic customers who love doing business with you—are worth far more to your company than passive customers or detractors. To obtain the actual NPS score the percentage of Detractors is deducted from the percentage of Promoters.

How the customer experience drives NPS

Several studies indicate four main drivers behind NPS:

  • Brand relationship
  • Experience of / satisfaction with product offerings (features; relevance; pricing)
  • Ease of doing business (simplicity; efficiency; reliability)
  • Touch point experience (the degree of warmth and understanding conveyed by front-line employees)

According to ‘voice of the customer’ research conducted by British customer experience consultancy Cape Consulting the ease of doing business and the touch point experience accounts for 60 % of the Net Promoter Score, with some variations between different industry sectors. Both factors are directly correlated to how easy it is for customers to find what they are looking for on the web and how easily front-line employees can find the right information to help and guide the customer.

Successful companies devote much attention to user experience on their website but when trying to figure out how most visitors will behave website owners tend to overlook the search function. Hence visitors who are unfamiliar with the design struggle to find the product or information they are looking for causing unnecessary frustration and quite possibly the customer/potential customer runs out of patience with the company.

Ideally, Findability on a company website or ecommerce site is a state where desired content is displayed immediately without any effort at all. Product recommendations based on the behavior of previous visitors is an example but it has limitations and requires a large set of data to be accurate. When a visitor has a very specific query, a long tail search, the accuracy becomes even more important because there will be no such thing as a close enough answer. Imagine a visitor to a logistics company website looking for information about delivery times from one city to another, an ecommerce site where the visitor has found the right product but wants to know the company’s return policy before making a purchase or a visitor to a hospital’s website looking for contact details to a specific department. Examples like these are situations where there is only one correct answer and failure to deliver that answer in a simple and reliable manner will negatively impact the customer experience and probably create a frustrated visitor who might leave the site and look at the competition instead.

Investing in search have positive impacts on NPS and the bottom line 

Google has taught people how to search and what to expect from a search function. Step one is to create a user friendly search function on your website but then you must actively maintain the master data, business rules, relevance models and the zero-results hits to make sure the customer experience is aligned. Also, take a look at the keywords and phrases your visitors use when searching. This is useful business intelligence about your customers and it can also indicate what type of information you should highlight on your website. Achieving good Findability on your website requires more than just the right technology and modern website design. It is an ongoing process that successfully managed can have a huge impact on the customer experience and your NPS which means your investment in search will generate positive results on your bottom line.

More posts on this topic will follow.

/Olof Belfrage

Search Driven Navigation and Content

In the beginning of October I attended Microsoft SharePoint Conference 2011 in Anaheim, USA. There were a lot of interesting and useful topics that were discussed. One really interesting session was Content Targeting with the FAST Search Web Part by Martin Harwar.

Martin Harwar talked about how search can be used to show content on a web page. The most common search-driven content is of course the traditional search. But there are a lot more content that can be retrieved by search. One of them is to have search-driven navigation and content. The search-driven navigation means that instead of having static links on a page we can render them depending on the query the user typed in. If a user is for example on a health care site and had recently done a search on “ear infection” the page can show links to ear specialist departments. When the user will do another search and returns to the same page the links will be different.

In the same way we can render content on the page. Imagine a webpage of a tools business that on its start page has two lists of products, most popular and newest tools. To make these lists more adapted for a user we only want show products that are of interest for the user. Instead of only showing the most popular and newest tools the lists can also be filtered on the last query a user has typed. Assume a user searches on “saw” and then returns to the page with the product lists. The lists will now show the most popular saws and the newest saws. This can also be used when a user finds the companies webpage by searching for “saw” on for instance Google.

This shows that search can be used in many ways to personalize a webpage and thereby increase Findability.

Design Principles for Enterprise Search – The Philosophy of UX

In May I attended An Event Apart in Boston (AEA). AEA is a 2-day (design) conference for people who working with websites and was created by the father of web design Jeffrey Zeldman and the CSS guru Eric Meyer. The conference has a broad perspective, dealing with everything from how to write CSS3 and HTML5 to content strategy and graphic design. This post is about an AEA topic brought up by Whitney Hess: Create design principles and use them to establish a philosophy for the user experience.

Hess wants to create universal principals for user experience to communicate a shared understanding amongst team members and customers and to create a basis for an objective evaluation. The principles suggested by Hess are listed below along with examples of how these can relate to search and search user interfaces.

Stay out of people’s way

When you do know what people want stay out of their way

Google knows what to do when people visit their search at Google.com. They get out of the way and make it easy to get things done. The point is not to disturb users with information they do not need, including everything from modal popup windows or to many settings.

Create a hierarchy that matches people’s needs

Give crucial elements the greatest prominence

This means that the most used information should be easy to find and use. A classic example is that on most university webpages – it is almost impossible to find contact details to faculty members or campus address but very easy to find a statement of the school philosophy. But the former is probably what users mostly will try to find.

university website -  xkcd.com/773/

Limit distractions

This principle means that you should design for consecutive tasks and limit related information to the information you know would help the user with her current task. Don’t include related information in a search user interface just because you can if the information does not add value.

Provide strong information scent

There should be enough information in search results for users to decide if results are relevant. In an e-commerce site this would be the difference between selling and not selling. A search result will not be perceived as cluttered if the correct data is shown.

Provide signposts and cues

Always make it clear how to start a new search, how to apply filters and what kind of actions can be applied to specific search results.

Provide context

Let the user know that there are different kinds of search result. Display thumbnails for pictures and videos or show msn availability in people search.

Use constraints appropriately

Prevent errors before they happen. Query suggestion is a good way as it helps users correct spelling error before they happen. This saves time and frustration for the user.

Make actions reversible

Make it obvious how to removes filters or reset other settings.

Provide feedback

Interaction is a conversation so let the user know when something happens or when the search interface fetches new search results. Never let the user guess what happens.

Make a good first impression

You only have one time to make a first impression. It is therefore important to spend time designing the first impression of any interface. Always aim to make the experience for new users better. This could mean voluntary tutorials or fun and good-looking welcome messages.

So now what?

Are universal principles enough? Probably not. Every project and company is different and need their own principles to identify with. Hess ended her presentation with tips on how to create company principles to complement the universal principles. Maybe there will be future blog posts about creating your own design principles.

So what are your company’s principles?

Why is Search Easy and Hard?

Last year my colleague Lina and I went to the Workshop on Human Computer Interaction and Information Retrieval (HCIR) in Washington DC. This year we did not have the possibility to attend but since all the material is available online I took part remotely any way. I wanted to share with you what I found most interesting this year. (Daniel Tunkelang who was one of the organizers also posted a good overview of the event on his blog.)

This years keynote speaker was Dan Russell, a researcher from Google. He talked about Search Quality and user happiness; Why search is easy and hard. The point I found most interesting in his presentation was how improvement is not only needed when it comes to tools and data but also improving the users’ search skills. My own experience from various search projects is similar; users are not good at searching. Even though they are looking for a specific version of a technical documentation for a specific product they might just enter the name of the product, or even the product family. (It’s a bit like searching for ‘camera’ when you expect to find support documentation on your Dioptric lens for you Canon EOS 60D.) So I agree that users need better search skills. In his presentation Russell also presented some ideas on how a search application can help users improve their search skills.

Search is both easy and hard. Perhaps this is one of the reasons for the introduction of the HCIR Challenge as a new part of the workshop . From the HCIR website:

The aims of the challenge are to encourage researchers and practitioners to build and demonstrate information access systems satisfying at least one of the following:

  • Not only deliver relevant documents, but provide facilities for making meaning with those documents.
  • Increase user responsibility as well as control; that is, the systems require and reward human effort.
  • Offer the flexibility to adapt to user knowledge / sophistication / information need.
  • Are engaging and fun to use.

The winner of the challenge was a team of researchers from Yahoo Labs who presented Searching Through Time in the New York Times. The Time Explorer features a results page with an interactive time line that illustrates how the volume of articles (results) have changed over time. I recommend that you read the article in tech review to learn more about the project, or try out the Time explorer demo yourself. You can also learn more about the challenge in this blog post by Gene Golovchinsky.

All the papers and posters from the workshop can be found on the new website.

Evaluate Your Search Application

Search is the worst usability problem on the web according to Peter Morville (in his book Search Patterns). With that in mind it is good to know that there are best practices and search patterns that one can follow to ensure that your search will work. Yet, just applying best practices and patterns will not always do the trick for you. Patterns are examples of good things that often work but they do not come with a guarantee that your users will understand and use search simply because you used best practice solutions.

There is no real substitute for testing your designs, whether it’s on websites intranets or any other type of application. Evaluating your design you will learn what works and does not work with your users. Search is a bit tricky when it comes to testing since there is not one single way or flow for the users to take to their goal. You need to account for multiple courses of actions. But that is also the beauty of it, you learn how very different paths users take when searching for the same information. And it does not have to be expensive to do the testing even if it is a bit tricky. There are several ways you can test your designs:

  • Test your ideas using pen and paper
  • Let a small group of users into your development or test environment to evaluate ideas under development
  • Create a computer prototype that is limited to the functionality you are evaluating
  • You can also evaluate the existing site before starting new development to identify what things need improvement
  • Your search logs are another valuable source of information regarding your users behaviors. Have a look at them as a complement.

And the best part of testing your ideas with users is, as a bonus you will learn even more stuff about your users that will be valuable to you in the future. Even if you are evaluating the smallest part of your website you will learn things that affects the experience of the overall site. So what are you waiting for? Start testing your site as well. I promise you will learn a lot from it. If you have any questions about how to best evaluate the search functionality on your site or intranet, write a comment here or drop me an email. In the meanwhile we will soon go on summer holiday. But we’ll be back again in August. Have a nice summer everyone!

Combining Search and Browse – Integrated Faceted Breadcrumbs

Finding information can be tricky and as I have written about in one of my previous posts improving findability is not about providing a single entrypoint to information. Users have different ways of finding information (browsing, searching and asking). They often combine these techniques with each other (berrypicking) and so they all need to be supported. Peter Morville states that.

“Browse and Search work best in tandem… the best finding interfaces achieve a balance, letting users move fluidly between browsing and searching.”

A lot of sites are improving their search experience through the implementation of faceted search. However, very few successfully integrate faceted search and browsing on their site. Searching and browsing are treated as two separate flows of interaction instead of trying to combine them which would provide the users with a much better experience.

That is why I was glad to learn about an idea from Greg Nudelman which he presented in his session at the IASummit which I attended last week. In his session Greg introduced his idea about Integrated Faceted Breadcrumb. According to him breadcrumbs are intuitive, flexible and resourceful and they are design elements that don’t cause problems but simply work. To test his idea he conducted usability tests on a prototype using the Integrated Faceted Breadcrumb. According to his evaluation the integrated faceted breadcrumb has a lot of advantages over other faceted solutions:

  1. Combine hierarchical Location & Attribute breadcrumbs
  2. Use Change instead of Set-Remove-Set
  3. Automatically retain relevant query information
  4. Label breadcrumb aspects
  5. Make it clear how to start a new search
  6. Allow direct keyword manipulation.

I find this idea interesting and I am currently thinking about whether it could be applied into one of my own projects. (According to Greg it has not been implemented anywhere yet even though the findings from the usability testing were positive.) However I wonder if this is a concept that works well only for sites with relatively homogeneous content or if it would also work on larger collections of sites such as intranets? Can it be used in an intuitive way with a large number of facets and can it cope with the use of more complex filtering functionalities? For some sites it might not be the best idea to keep the search settings when the user changes search terms. These are some things I would like to find out. What do you think about this? Could you apply it to your site(s)? I recommend that you have a look at Greg Nudelman’s presentation on slideshare and find out for yourself. You can also find an article about the Integrated Faceted Breadcrumb on Boxes and Arrows. I look forward to a discussion about whether this is any good so write me a comment here at the findability blog or find me on twitter.

IASummit – Information Architecture and Search

This upcoming week my colleague Lina and I will participate in the IASummit in Phoenix Arizona. Search, information architecture and user experience and the relationships between them is the focus for us this upcoming week. We look forward to hearing a lot of great talks, meeting interesting people and enjoying the sunny weather in Arizona.

We will be blogging from the conference but if you don’t want to wait for that you can follow me, Maria on twitter or follow the hashtag for the IASummit #ias10 so see what everyone is tweeting about.

Faceted Search by LinkedIn

My RSS feeds have been buzzing about the LinkedIn faceted search since it was first released from beta in December. So why is the new search at LinkedIn so interesting that people are almost constantly discussing it? I think it’s partly because LinkedIn is a site that is used by most professionals and searching for people is core functionality on LinkedIn. But the search interface on LinkedIn is also a very good example of faceted search.

I decided to have a closer look into their search. The first thing I realized was just how many different kinds of searches there are on LinkedIn. Not only the obvious people search but also, job, news, forum, group, company, address book, answers and reference search. LinkedIn has managed to integrate search so that it’s the natural way of finding information on the site. People search is the most prominent search functionality but not the only one.

I’ve seen several different people search implementations and they often have a tendency to work more or less like phone books. If you know the name you type it and get the number. And if you’re lucky you can also get the name if you only have the number. There is seldom anyway to search for people with a certain competence or from a geographic area. LinkedIn sets a good example of how searching for people could and should work.

LinkedIn has taken careful consideration of their users; What information they are looking for, how they want it presented and how they need to filter searches in order to find the right people. The details that I personally like are the possibility to search within filters for matching options (I worked on a similar solution last year) and how different filters are displayed (or at least in different order) depending on what query the user types. If you want to know more about how the faceted search at LinkedIn was designed, check out the blog post by Sara Alpern.

But LinkedIn is not only interesting because of the good search experience. It’s also interesting from a technical perspective. The LinkedIn search is built on open source so they have developed everything themselves. For those of you interested in the technology behind the new LinkedIn search I recommend “LinkedIn search a look beneath the hood”, by Daniel Tunkelang where he links to a presentation by John Wang search architect at LinkedIn.