Tag Archives: Office 365

Customizing search in SharePoint Online

Posted on by
2

Search in SharePoint 2013 – Part 3: Customizing search in SharePoint Online

This post is the third in a series of four articles providing several best practices on how to implement and customise search in SharePoint. In the first post, we provided a brief overview of the differences in terms of search between the on-premise and cloud versions, and in the second blog post we discussed several things you should consider when migrating to the new SharePoint. In this post, we will mention several search features that can be configured in SharePoint Online, and we will be specifically referring to those available in the Enterprise Plan. If you need more information than is provided in this blogpost, feel free to visit our website or contact us!

Here is a summary of what customisations for search in SharePoint Online will be discussed:

  • Defining your own custom result sources, and hiding any that you are not using
  • Setting up hybrid search if you chose a hybrid solution
  • Defining which refiners to show and how to display them
  • Adding query suggestions that are related to your organisation
  • Adding query spelling corrections
  • Changing how the search results are displayed to show previews and additional metadata

Get ready to search ‘everything’

This is the uncustomized search box that you will see on your search center page. Please note that in some SharePoint Online plans the ‘Videos’ vertical is not available.

This is the uncustomized search box that you will see on your search center page.
Please note that in some SharePoint Online plans the ‘Videos’ vertical is not available.

Everything is the default scope when performing a search in the SharePoint search center and is returning every type of result from all of your site collections. There are a few other scopes (search verticals, or so-called Result Sources) that are included by default, People, Conversations, and Videos, and these are preconfigured to search on what you would expect.

  • You can add new result sources, say for example Reports, that shows only search results that are tagged with the keyword ‘Final Report’. You define yourself what the criteria for a result source should be.
  • If there is a result source that you are not using, say for example if you have no video content and don’t plan to have in the near future, it’s less confusing for the users if you simply not show it for now. It’s easy to add it back if you will need it in the not so foreseeable future.

If you choose a hybrid solution, your content is split between the online SharePoint and the on-premise SharePoint Server.

  • It’s possible to have one search that displays results from both locations. For example, to show results from the on-premise installation in SharePoint Online, you have to define a new result source that is able to retrieve the results from the on-premise. Then you can configure the search results page to show results from both result sources (everything from SharePoint Online plus everything from SharePoint on-premise that matches the search query).

Screenshot from the post Hybrid search by the Microsoft SharePoint Team Blog showing how results from the cloud are integrated in the search results page when the user searches from an on-premises SharePoint 2013 site. Notice also the new visual refiner for date interval in the refinement panel on the left.

Screenshot from the post Hybrid Search by the Microsoft SharePoint Team Blog showing how results from the cloud are integrated in the search results page when the user searches from an on-premises SharePoint 2013 site.
Notice also the new visual refiner for date interval in the refinement panel on the left.

Drill down into the search results

The search Refiners allow the users to drill down into the search results. There is a new type of refiner in SharePoint 2013, a visual refiner, by default used for the ‘Modified Date’.

  • The way in which the visualisation of the refiners is made has drastically changed, and you can define your own visualisation of the data if you want to. For example, what about a map as a refiner, instead of a list of city names?

By default, the refiners you will see would be the Result type (example values: Excel, Web page), Author (example values: John Doe, Jane Doe), and Modified Date (shown as a distribution of values).

  • If you edit the web part responsible for the refiners, you will be able to add other refiners as well. For example, company names are automatically extracted from your content, so it is easy to simply add that to your refiners.
  • Also, another useful refiner to show to your users is the Content Type, offering one level of detail more from the Result Type refiner.

Search guidance

Query suggestions are displayed as the user types.

Query suggestions are displayed as the user types.

As the user types a query in the search box, SharePoint is able to show Query Suggestions that help complete the query. SharePoint automatically creates a list of suggestions based on previous searches. When at least 6 search results are clicked for a specific query, that query will be added to the list of suggestions.

  • Besides the list that SharePoint creates automatically, you are able to add your own list of suggestions. This is especially useful when starting fresh with your installation, since a fresh installation will come with no query suggestions. You could help the users by adding your company name, product names or similar to the initial list of suggestions. You will also find manual adding of suggestions useful when reviewing the search logs, since these can give you a new perspective on what the users are looking for, and based on that input help guide your user to the relevant results using query suggestions.
  • You are also able to import a list of suggestions that are not intended to be shown in suggestions. Say for example that your testing team uses a specific keyword for testing content. In this case, it is very probable that the test keyword will soon appear as a suggestion for all users. To avoid this, simply add the keyword to the query suggestion exclusion list.

Similar to the query suggestions, another functionality whose purpose is to help the user in formulating the query is the Query Spelling Correction. An inclusion and exclusion list is used in this case as well, the only difference is that these are managed in the Term Store, while managing query suggestions is made by importing a plain text file.

  • You can add your own terms in the query spelling correction inclusion and exclusion lists. Probably one of the most often misspelled words is the word ‘business’. Or was it ‘bussiness’? After adding this term to the list of words to be included in the spelling suggestions, the correct form of the word would be shown under the ‘Did you mean’ functionality if the user misspells it.

Change how the search results are displayed

Screenshot from an Office Blogs post showing the hover panel for a PowerPoint document.

Screenshot from an Office Blogs post showing the hover panel for a PowerPoint document.

A final item on our list of proposed customisations for your search results is to change how the search results are displayed. In SharePoint 2013, it is the Display Templates that define how each element in the search results page is displayed. For example, there is a template for the refiner, another one for the hover panel of a PDF item, another one for the hover panel of a Word item, and so on.

  • A simple fix would be make sure that you have previews for PDF files in the hover panel. It is the Office Web Apps that power the previews for Office documents (such as Word, PowerPoint, Excel), but the preview for PDF files might not be visible for you. If so, what you can do is change the display template that is associated to the PDF result type.
  • You can also define what metadata to show for each result type. For example, for a Word document you would by default be able to see the Title, a text snippet and a URL, and in the hover panel the document preview, Last Modified date and author, as well as probably a list of the main headings from the document. However, if you have added additional metadata to your document, such as Location or Keywords, you can display these in the search results as well by modifying the right display template.

You can find more information about how to administer many of these search functionalities from this Microsoft Office page and from our search experts. Let us know how far you are in implementing SharePoint online for your organisation – we sure have a few more tips to how to configure and customize the search in SharePoint!

Cloud vs. on-premise SharePoint 2013 search

Posted on by
1

Search in SharePoint 2013 – Part 1: The difference between search within on-premise SharePoint 2013 and SharePoint Online

Cloud or on-premise? Findwise offers implementation and consulting services for both scenarios. This post is the first in a series of four articles providing several best practices on how to implement and customise search in SharePoint. The focus of this first post is introducing the difference between the cloud and on-premise SharePoint 2013 in terms of search features. If you need more information than you find in this blogpost, just stop by our website or contact us. 

“The cloud is on fire”

That is a quote from the Microsoft Office General Manager Jared Spataro during his keynote at the SharePoint conference in Las Vegas last month. At this conference, Microsoft revealed that 60% of the Fortune top 500 adopted Office 365 in the previous 12 months. While new versions of on-premise SharePoint and Exchange Server are promised to still come next year, Microsoft is adding more and more capabilities to the cloud version.

SPC14 Keynote summary

Fun random facts about SharePoint Online presented during the keynote at the SharePoint conference in Las Vegas this year (March 3rd 2014)

In addition to the numbers above, a market analysis report done by The Radicati Group on the adoption of Microsoft SharePoint reveals that almost a quarter of the worldwide users accessing deployments of SharePoint made during the year 2013 are using the cloud based SharePoint.

When deciding whether to go for the on-premise or cloud solution, a go-to resource for your IT team is the TechNet article describing the availability of features across the solutions. That article not only divides the features between on-premise and cloud, but also between the different Office 365 and SharePoint Online plans. What is the difference? SharePoint Online is the cloud version of the SharePoint Server, but it can be deployed as a standalone service or as part of the Office 365 suite, so different plans are usually listed for these different scenarios. There are also the Office 365 Dedicated plans, but these are out of the scope for this article. The Microsoft Office site has a more business oriented comparison of the different plans, including pricing. If not decided for one or the other, there is also the possibility of a hybrid solution!

 Availability Search feature Office 365 Small BusinessOffice 365 Small Business Premium Office 365 Midsize BusinessOffice 365 Enterprise E1 or K1Office 365 Education A2Office 365 Government G1 or K1 Office 365 Enterprise E3 or E4Office 365 Education A3 or A4Office 365 Government G3 or G4 SharePoint Online Plan 1 SharePoint Online Plan 2 SharePoint Foundation 2013 SharePoint Server 2013 Standard CAL SharePoint Server 2013 Enterprise CAL
Available within all plans
 Phonetic name matching Yes Yes Yes Yes Yes Yes Yes Yes
Expertise Search Yes Yes Yes Yes Yes Yes Yes Yes
Quick preview Yes Yes Yes Yes Yes Yes Yes Yes
RESTful Query API/Query OM Yes Yes Yes Yes Yes Yes Yes Yes
Result sources Yes Yes Yes Yes Yes Yes Yes Yes
Search results sorting Yes Yes Yes Yes Yes Yes Yes Yes
Ranking models Yes Yes Yes Yes Yes Yes Yes Yes
Query spelling correction Yes Yes Yes Yes Yes Yes Yes Yes
Refiners Yes Yes Yes Yes Yes Yes Yes Yes
Manage search schema Yes Yes Yes Yes Yes Yes Yes Yes
Available in all Office365 and SharePoint Online plans
 Deep links Yes Yes Yes Yes Yes No Yes Yes
Event-based relevancy Yes Yes Yes Yes Yes No Yes Yes
Graphical refiners Yes Yes Yes Yes Yes No Yes Yes
Recommendations Yes Yes Yes Yes Yes No Yes Yes
Search vertical: “Conversations” Yes Yes Yes Yes Yes No Yes Yes
Search vertical: “People” Yes Yes Yes Yes Yes No Yes Yes
Query suggestions Yes Yes Yes Yes Yes No Yes Yes
Query throttling Yes Yes Yes Yes Yes No Yes Yes
“This List” searches Yes Yes Yes Yes Yes No Yes Yes
Query rules—Add promoted results Yes Yes Yes Yes Yes No Yes Yes
Avail. in Office365 Advanced Content Processing Yes Yes Yes No No Yes Yes Yes
Hybrid search No Yes Yes Yes Yes Yes Yes Yes
Query rules—advanced actions No No Yes No No No No Yes
Search vertical: “Video” No No Yes No Yes No No Yes
Not available in any of the Office 365, SharePoint Online plans
 Search connector framework No No No No No No Yes Yes
Custom entity extraction No No No No No No No Yes
Extensible content processing No No No No No No No Yes

— Simplified view of the TechNet article, focusing on the search features availability across SharePoint solutions

Limitations in Office 365 and SharePoint Online plans

Is the cloud version good enough for your organisation when it comes to search features? The table above illustrates some of the things that you might be missing in terms of search, and in what follows we will discuss those whose availability varies amongst the Office 365 or SharePoint Online plans.

Query rules – advanced actions

In order to adapt the relevance of the search results to the user intent, SharePoint 2013 adds a new feature called query rules. A query rule is defined by a condition and a corresponding action to be taken when the condition is met. Within some SharePoint Online licenses, this functionality is limited to the possibility of adding promoted results, while more advanced actions are left out. The promoted results are similar to what was in previous SharePoint versions known as search keywords, or best bets, letting you promote specific results on top of the ranked search results. The more advanced actions could consist of for example changing the query or changing the ranking of the search results by promoting a certain group of results. You can read more about various usages of query rules in one of our previous blog post.

Search Connector Framework and Hybrid Search

Administrators of SharePoint Online will miss the feature of managing the different search connectors to content sources, since the search connector framework is not available. Only SharePoint content that is stored online is going to be indexed. Search results can only be retrieved from that content, or can be set up to retrieve from an Exchange Server, from a remote SharePoint, or from a search engine that uses the OpenSearch protocol. As an alternative approach to making content from other sources searchable, you can set up hybrid search. This feature is available in almost all Office 365 and SharePoint Online scenarios. It allows users to show search results from content available in the cloud and on-premise. So if you would like to index a content source that is not supported in SharePoint Online, you should be able to index it on the on-premise.

Custom Entity Extraction

The TechNet article describing features across solutions actually shows that this feature is only available with the enterprise licensing of SharePoint Server. This feature allows the extraction of custom-defined terms from your content and making them usable as search refiners. Say for example that you would like to extract all of your current product names from the content of your documents and then be able to refine your search results on the product name.

Content Processing Extensibility

The other search feature that is only available with the enterprise licensing of SharePoint Server is the content processing extensibility. In practice, this means there is an API that can be used to transform the data before it is stored in the index. For example, more advanced entity extraction can be made at this step. While the custom entity extraction discussed previously is able to identify names in the content based on a pre-defined list of names, through this API you can use a trained model to do entity extraction for example. Additional use cases could be cleaning or normalising the data according to predefined rules (for example, lowercasing all values in a property), or automatically tagging items based on the content.

It should be noted that the TechNet article is not a comprehensive list, and rather gives an overview of the major differences between solutions. Here is for example one more feature whose availability is limited.

Synonyms

One of the missing features in SharePoint Online that is available in the on-premise solution is the possibility of defining synonyms. Since it’s too easy to communicate the same thing with different words, defining synonyms or abbreviations for search phrases can help aggregate the results for the multiple ways of expressing the same information need. We hope that Microsoft will integrate this feature in the future versions of SharePoint Online as well.

Find the right documentation

When searching for which functionality is available across solutions on the Microsoft Office.com website or TechNet, make sure to check that the discussed functionality applies to your version of SharePoint. Articles usually indicate for which versions the functionality applies to.

Feature availability in MS articles

Articles on Office.com (left) and TechNet (right) indicate for which version
of SharePoint the discussed topic applies to.

Please note that things might change, new updates in SharePoint online can add functionality that was missing before.

To stay up-to-date, check the TechNet page once in a while, visit our website or contact us to help you map your requirements to the available search features across solutions.

Continuous crawl in SharePoint 2013

Posted on by
Reply

Continuous crawl is one of the new features that comes with SharePoint 2013. As an alternative to incremental crawl, it promises to improve the freshness of the search results. That is, the time between when an item is updated in SharePoint by a user and when it becomes available in search.

Understanding how this new functionality works is especially important for SharePoint implementations where content changes often and/or where it’s a requirement that the content should instantly be searchable. Nonetheless, since many of the new SharePoint 2013 functionalities depend on search (see the social features, the popular items, or the content by search web parts), understanding continuous crawl and planning accordingly can help level the user expectation with the technical capabilities of the search engine.

Both the incremental crawl and the continuous crawl look for items that were added, changed or deleted since the last successful crawl, and update the index accordingly. However, the continuous crawl overcomes the limitation of the incremental crawl, since multiple continuous crawls can run at the same time. Previously, an incremental crawl would start only after the previous incremental crawl had finished.

Limitation to content sources

Content not stored in SharePoint will not benefit from this new feature. Continuous crawls apply only to SharePoint sites, which means that if you are planning to index other content sources (such as File Shares or Exchange folders) your options are restricted to incremental and full crawl only.

Example scenario

The image below shows two situations. In the image on the left (Scenario 1), we are showing a scenario where incremental crawls are scheduled to start at each 15 minutes. In the image on the right (Scenario 2), we are showing a similar scenario where continuous crawls are scheduled at each 15 minutes. After around 7 minutes from starting the crawl, a user is updating a document. Let’s also assume that in this case passing through all the items to check for updates would take 44 minutes.

Continuous crawl SharePoint 2013

Incremental vs continuous crawl in SharePoint 2013

In Scenario 1, although incremental crawls are scheduled at each 15 minutes, a new incremental crawl cannot be started while there is a running incremental crawl. The next incremental crawl will only start after the current one is finished. This means 44 minutes for the first incremental crawl to finish in this scenario, after which the next incremental crawl kicks in and finds the updated document and send it to the search index. This scenario shows that it could take around 45 minutes from the time the document was updated until it is available in search.

In Scenario 2, a new continuous crawl will start at each 15 minutes, as multiple continuous crawls can run in parallel. The second continuous crawl will see the updated document and send it to the search index. By using the continuous crawl in this case, we have reduced the time it takes for a document to be available in search from around 45 minutes to 15 minutes.

Not enabled by default

Continuous crawls are not enabled by default and enabling them is done from the same place as for the incremental crawl, from the Central Administration, from Search Service Application, per content source. The interval in minutes at which a continuous crawl will start is set to a default of 15 minutes, but it can be changed through PowerShell to a minimum of 1 minute if required. Lowering the interval will however increase the load on the server. Another number to take into consideration is the maximum number of simultaneous requests, and this is a configuration that is done again from the Central Administration.

Continuous crawl in Office 365

Unlike in SharePoint 2013 Server, continuous crawls are enabled in SharePoint Online by default but are managed by Microsoft. For those used to the Central Administration from the on-premise SharePoint server, it might sound surprising that this is not available in SharePoint Online. Instead, there is a limited set of administrative features. Most of the search features can be managed from this administrative interface, though the ability to manage the crawling on content sources is missing.

The continuous crawl for Office 365 is limited in the lack of control and configuration. The crawl frequency cannot be modified, but Microsoft targets between 15 minutes and one hour between a change and its availability in the search results, though in some cases it can take hours.

Closer to real-time indexing

The continuous crawl in SharePoint 2013 overcomes previous limitations of the incremental crawl by closing the gap between the time when a document is updated and when this is visible in the search index.

A different concept in this area is the event driven indexing, which we will explain in our next blog post. Stay tuned!