Housekeeping rules within the Habitat

Posted on November 5, 2014 by Fredric Landqvist

This is the third post in a series (1, 2, 4, 5, 6, 7) on the challenges organisations face as they move from having online content and tools hosted firmly on their estate to renting space in the cloud. We will help you to consider the options and guide on the steps you need to take.

In the first post we set out the most common challenges you are likely to face and how you may overcome these. In the second post we focused on how Office 365 and SharePoint can play a part in moving to the cloud. Here we cover how they can help join up your organisation online using their collaboration tools and features.

When arranging the habitat, it is key to address the theme of collaboration. Since each of these themes, derives different feature settings of artifacts and services. In many cases, teamwork is situated in the context of a project. Other themes for collaboration are the line of business unit teamwork, or the more learning networks a.k.a communities of practice. I will leave these later themes for now.

Most enterprises have some project management process (i.e. PMP) that all projects do have to adhere to, with added complementary documentation, and reporting mechanisms. This is so the leadership within the organisation will be able to align resources, govern the change portfolio across different business units. Given this structure, it is very easy to depict measurable outcomes, as project documents have to be produced, regardless of what the project is supposed to contribute towards.

The construction of a habitat, or design of a joint workplace, all boils down to pragmatic steps that are aligned with the overarching project framework at hand. Answering a few simple Questions (Inverted Pyramid):

Who? will be participating, who will own (organisation) the outcome from the joint effort pulling together a project (dc.contributor ; dc.creator ; dc.provenance ) and reach ( dc.coverage ; dc.audience )
What? is the project all about, topic and theme (dc.subject ; dc.title ; dc.description, dc.type )
When? will this project be running, and timeline for ending the project. All temporal themes around the life of a project. (dc.date)
Where? will participants contribute. What goes where and why? (dc.source ; dc.format ; dc.identifier )
Why? usually defined in project description, setting common ground for the goals and expected outcome. ( dc.description )
How? defines used processes, practices and tools to create the expected outcome for the project, with links to common resources as the PMP framework, but also links to other key data-sets. Like ERP record keeping and masterdata, for project number and other measures not stored in the habitat, but still pillars to align to the overarching model. (dc.relation)

When these questions have been answered, the resource description for the habitat is set. In Sharepoint the properties bag (code) feature. During the lifespan of the on-going project, all contribution, conversations and creation of things can inherit rule-based metadata for the artifacts from the collections resource description. This reduces the burden weighing on the actors building the content, by enabling automagic metadata completion where applicable. And from the wayfinding, and findability within and between habitats, these resource descriptions will be the building blocks for a sustainable information architecture.

In our next post we will cover how to encourage employee engagement with your content.

Please join our Live Stream on YouTube the 20th November 8.30AM – 10AM Central European Time
Fredric Landqvist research blog
Mark Morell intranet-pioneer

Pace-Layering the Building Blocks in the Cloud

Posted on November 3, 2014 by Fredric Landqvist

This is the second post in a series (1, 3, 4, 5, 6, 7) on the challenges organisations face as they move from having online content and tools hosted firmly on their estate to renting space in the cloud. We will help you to consider the options and guide on the steps you need to take.

In the first post we set out the most common challenges you are likely to face and how you may overcome these. In this post we focus on how Office 365 and SharePoint online can play a part in moving to the cloud.

Let us be pragmatic and down-to-earth! It is time to roll up our sleeves and consider using Office 365, as one example of how organisations can make this transition from their estate to the cloud. Given that this is the collaborative space many organisations consider using, Office 365 is compelling as a one-size-fits-all, instant build and just roll-out enterprise-wide approach to take sometimes without an Information Architecture plan whatsoever!

In the Office 365 environment, one has to map the terrain, so that there are distinct districts to where things relate – the same goes for the structure of neighborhoods of clustered habitats. But where it gets tough is to have an agile and resilient city plan for the real-world experience. This is actually the pillar construction in a digital domain, aiming for resilience and emerging uses over the time… but with a simple and agreed upon game plan.

Pace-layering the information architecture

Most organisations have an ontology of entities, things, that are generic, as stated in the W3C Organisation schema. And these perspectives, domain models, vocabularies and ontologies, add up to become districts, and neighborhoods in the Information Architecture map, with a few angles:

Organisation Units (Business Unit, Division, Function, Group)
1. Clients/Customers/Partners/Competitors
2. Professional Networks
3. Governing agencies, or regulatory entities, intermediaries
Locations (Sites, Geographical places as /world/continent/country/region/city/address …)
Business Processes (Process & Activities)
Professions, and Disciplines ( Roles), Practices
Topics (derived from line of Business, and controlled vocabularies)
1. Products
2. Services
3. Tasks
Temporal (Events)

Regardless of line of Business for an organisation, these pan out as pretty good structural elements on which to build upon. Since an enterprise is a social construct with agreed borders, it is populated with people who act and interplay in various different ways, and have a multitude facets with regards to everyday work. Some entities change more frequently, generally in the organisational units further down in the leaves, and less so in the top main branches. The vocabularies within an organisation needs to be the center pillar, to reduce linguistic insecurities.

From an Information Architecture perspective, in using Office 365 or Sharepoint it is wise to use pace-layering to the building blocks, on to which navigational constructs are built upon. This means, using the highest level of the organisational unit tree branch, a pretty stable foundation for the site-structure can be built. This is where content and teamsites live. More fluid navigational themes (temporal, or topic entities) can then be added.

This goes for activities undertaken within daily practices, where a set of professions and disciplines interact. All of these activities lay out a tapestry of overarching business processes. The outcome or result, might be a thing that is detailed as topic taxonomies. For example, a product structure for a specific manufacturing industry. Since all organisations have actor networks in their ecology, it is preferable to add these entities into the structure, as clients, partners, competitor, regulatory agencies, social networks, communities of practice and so forth.

All of these set of terms, have to be maintained in a Managed Metadata Service, a.k.a TermStore. In most organisations there are other sources of their controlled vocabularies, hence mapping is key, to have aligned master term sets. Either through subscription models (batch) or enterprise-linked-data sets. All these actions, defines the terrain, so we map the ecosystem as taxonomic chartographers.

The Building Blocks: Artifacts and Collections

Office 365 comes with a pretty organised set of tools, themes and things to build upon. For more website related things, one could either use published web sites / portals, or enterprise wikis. The other main services are digital habitats, or collaborative spaces, team-sites. And lasty there are ESN (Enterprise Social Networks) like Yammer, and instant messaging tools like Lync, and Exchange Services like mail and calendar. Sharepoint Online and Office 365 is a Swiss Army Knife.

Within a collection, there are some key artifacts and services, content types:

documents
pages, web or wiki
calendars
to-do lists
blogs
forums
list object
and social objects to name a few.

If all of these included content types adhere to Dublin Core with a bare minimum of properties, the rest will follow just great.

We will cover more about how Office 365 and SharePoint collaboration tools can help join up your organisation online in our next post.

Please join our Live Stream on YouTube the 20th November 8.30AM – 10AM Central European Time
Fredric Landqvist research blog
Mark Morell intranet-pioneer

The search experience in SharePoint 2013: customised or targeted?

Posted on May 30, 2014 by

This post is the fourth in a series of four articles providing several best practices on how to implement and customise the search experience in SharePoint 2013. The previous posts listed the differences between the cloud and on-premise SharePoint, provided considerations when upgrading to SharePoint 2013, and dealt with the practicalities of configuring search in SharePoint Online. This fourth post handles the more advanced topic of ranking results and the future of search in SharePoint.

Managing ranking

We’ve previously mentioned the query rules as a way to change the ranking of the search results based on your requirements. These allow the promotion of certain search results or search result blocks on top of the ranked searched results, and more advanced query rules allow even changing the ranking of the search results based on what the query terms are.

By using query rules, customising the search results web part, and a few content by search web parts, you can change the behaviour of the search depending on what user is accessing it. That is, you would also need good metadata to make this work, but having a complete user profile (including the job title, department, and interests) is a good start. Based on such user information, you can define how the search experience for that user will be.

Changing ranking using query rules, however, requires a query rule condition, which describes the prerequisites that the query must fulfil in order for the query rule to fire. For changing the results for all queries, you can use the next approach.

If the default ranking does not satisfy your search requirements and you want to change the order of the ranked search results, SharePoint provides the possibility of changing the ranking models. It is a feature available in SharePoint Online as well, as described in the TechNet documentation: “SharePoint Online customers need to download and install the free Rank Model Tuning App in order to create and customize ranking models.”

A ranking model contains the features and corresponding weights that are used in calculating a score for each search result. Changing the ranking models might require a deeper and theoretical knowledge of how search works, and those that take the challenge of changing the ranking model are often dedicated search administrators or external specialised consultants.

The Ranking Model Tuning app is free on the App Store

The Rank Model Tuning App provides a user interface for creating custom ranking models, and can be used for both SharePoint Online and SharePoint Server, though in SharePoint 2013 Server there is also the possibility to use PowerShell to customise ranking models. New models are based on existing ranking models for which you can add or remove new rank features and tune the weight of a rank feature. It also allows for evaluating the new ranking model using a test set of queries. The set of test queries can be constructed from real queries made by users that can be gathered from previous search logs, for example. How to use the tuning app is explained step-by-step in the documentation on the Office site.

Changing the weight of certain file types (say for example for PowerPoint documents compared to Excel documents) might be enough for many search implementations, but depending on the content, the features that influence the ranking of the search results can become more elaborate. For example, a property defining whether documents are either official or work-in-progress might become an important factor in determining the ranking of search results. SharePoint provides the liberty to create new properties, so it makes sense that these can be used in search to improve the relevance.

It should be pointed out, however, that changing the ranking model influences all searches that are run using that ranking model. Though the main idea of changing the ranking model is to improve the ranking, it can become much too easy to make changes that can have an undesirable effect on the ranking. This is why a proper evaluation of ranking changes needs to be part of your plan for improving search relevance.

The office graph and the future of social

The social features introduced in SharePoint 2013 provide a rich social experience, which is interconnected with the search experience. Many social features are driven by search (such as the recommendations for which people or documents to follow), and social factors also affect the search (such as finding the right expertise from conversations in your network).

In the month of June 2012 Microsoft acquired the social enterprise platform Yammer. The SharePoint Server 2013 Preview has been made available for download since July 2012, and it reached Release to Manufacturing (RTM) in October the same year. The new SharePoint 2013 implements new social features (see for example the newsfeed, the new mysites and the tagging system), many of which are overlapping with those available in Yammer! This brings us to the question on everyone’s mind since the acquisition of Yammer: what is the future of social in SharePoint? Should you use SharePoint’s social features or use Yammer?

In March 2014, Microsoft announced that they will not include new features in the SharePoint Social but rather invest in the integration between Yammer and Office 365. The guidance is thus to go for Yammer.

“Go Yammer! While we’re committed to another on-premises release of SharePoint Server—and we’ll maintain its social capabilities—we don’t plan on adding new social features. Our investments in social will be focused on Yammer and Office 365” – Jared Spataro, Microsoft Office blog

Also at the SharePoint conference this March 2014, Microsoft introduced the Office Graph, and with it Oslo as the first app demo using it. During the keynote, Microsoft mentions that the Office Graph is “perhaps the biggest idea we’ve had since the beginning of SharePoint”. The office graph maps relationships between people, the documents they authored, the likes and posts they made, and the emails they received; it’s actually an extension of Yammer’s enterprise graph. The Oslo application is leveraging the graph, in a way that looks familiar from Facebook’s graph search.

The Office Graph, connecting people and information – Microsoft Office Blog

The new Office Graph provides exciting opportunities, and has consequences for how the search will be used. Findwise started exploring the area of enterprise graph search before Microsoft announced the Office Graph – see our post about the Enterprise Graph Search from January 2013.

Reluctant to go for the cloud?

Microsoft has hinted during the SharePoint conference keynote in March that they will be adding new functionalities to the cloud version first. Although they are still committed to another version of SharePoint server, new updates might come at a slower pace for the on-premise version. However, Microsoft also announced that with the SharePoint SP1 there is a new functionality in the administrative interface: a hybrid setting which allows you to specify whether you want the social component in the cloud/Yammer, or your documents on OneDrive, so that you don’t need to move everything to the cloud overnight.

Let us know how far you’ve come with your SharePoint implementation! Contact us if you need help in deciding which version of SharePoint to choose, need help with tuning search relevance, have questions about improving search, or would like to work with us to reach the next level of findability.

Cloud vs. on-premise SharePoint 2013 search

Posted on April 23, 2014 by

Search in SharePoint 2013 – Part 1: The difference between search within on-premise SharePoint 2013 and SharePoint Online

Cloud or on-premise? Findwise offers implementation and consulting services for both scenarios. This post is the first in a series of four articles providing several best practices on how to implement and customise search in SharePoint. The focus of this first post is introducing the difference between the cloud and on-premise SharePoint 2013 in terms of search features. If you need more information than you find in this blogpost, just stop by our website or contact us.

“The cloud is on fire”

That is a quote from the Microsoft Office General Manager Jared Spataro during his keynote at the SharePoint conference in Las Vegas last month. At this conference, Microsoft revealed that 60% of the Fortune top 500 adopted Office 365 in the previous 12 months. While new versions of on-premise SharePoint and Exchange Server are promised to still come next year, Microsoft is adding more and more capabilities to the cloud version.

Fun random facts about SharePoint Online presented during the keynote at the SharePoint conference in Las Vegas this year (March 3rd 2014)

In addition to the numbers above, a market analysis report done by The Radicati Group on the adoption of Microsoft SharePoint reveals that almost a quarter of the worldwide users accessing deployments of SharePoint made during the year 2013 are using the cloud based SharePoint.

When deciding whether to go for the on-premise or cloud solution, a go-to resource for your IT team is the TechNet article describing the availability of features across the solutions. That article not only divides the features between on-premise and cloud, but also between the different Office 365 and SharePoint Online plans. What is the difference? SharePoint Online is the cloud version of the SharePoint Server, but it can be deployed as a standalone service or as part of the Office 365 suite, so different plans are usually listed for these different scenarios. There are also the Office 365 Dedicated plans, but these are out of the scope for this article. The Microsoft Office site has a more business oriented comparison of the different plans, including pricing. If not decided for one or the other, there is also the possibility of a hybrid solution!

Availability	Search feature	Office 365 Small BusinessOffice 365 Small Business Premium	Office 365 Midsize BusinessOffice 365 Enterprise E1 or K1Office 365 Education A2Office 365 Government G1 or K1	Office 365 Enterprise E3 or E4Office 365 Education A3 or A4Office 365 Government G3 or G4	SharePoint Online Plan 1	SharePoint Online Plan 2	SharePoint Foundation 2013	SharePoint Server 2013 Standard CAL	SharePoint Server 2013 Enterprise CAL
Available within all plans	Phonetic name matching	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
	Expertise Search	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
	Quick preview	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
	RESTful Query API/Query OM	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
	Result sources	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
	Search results sorting	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
	Ranking models	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
	Query spelling correction	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
	Refiners	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
	Manage search schema	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Available in all Office365 and SharePoint Online plans	Deep links	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
	Event-based relevancy	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
	Graphical refiners	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
	Recommendations	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
	Search vertical: “Conversations”	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
	Search vertical: “People”	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
	Query suggestions	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
	Query throttling	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
	“This List” searches	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
	Query rules—Add promoted results	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
Avail. in Office365	Advanced Content Processing	Yes	Yes	Yes	No	No	Yes	Yes	Yes
	Hybrid search	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes
	Query rules—advanced actions	No	No	Yes	No	No	No	No	Yes
	Search vertical: “Video”	No	No	Yes	No	Yes	No	No	Yes
Not available in any of the Office 365, SharePoint Online plans	Search connector framework	No	No	No	No	No	No	Yes	Yes
	Custom entity extraction	No	No	No	No	No	No	No	Yes
	Extensible content processing	No	No	No	No	No	No	No	Yes

— Simplified view of the TechNet article, focusing on the search features availability across SharePoint solutions

Limitations in Office 365 and SharePoint Online plans

Is the cloud version good enough for your organisation when it comes to search features? The table above illustrates some of the things that you might be missing in terms of search, and in what follows we will discuss those whose availability varies amongst the Office 365 or SharePoint Online plans.

Query rules – advanced actions

In order to adapt the relevance of the search results to the user intent, SharePoint 2013 adds a new feature called query rules. A query rule is defined by a condition and a corresponding action to be taken when the condition is met. Within some SharePoint Online licenses, this functionality is limited to the possibility of adding promoted results, while more advanced actions are left out. The promoted results are similar to what was in previous SharePoint versions known as search keywords, or best bets, letting you promote specific results on top of the ranked search results. The more advanced actions could consist of for example changing the query or changing the ranking of the search results by promoting a certain group of results. You can read more about various usages of query rules in one of our previous blog post.

Search Connector Framework and Hybrid Search

Administrators of SharePoint Online will miss the feature of managing the different search connectors to content sources, since the search connector framework is not available. Only SharePoint content that is stored online is going to be indexed. Search results can only be retrieved from that content, or can be set up to retrieve from an Exchange Server, from a remote SharePoint, or from a search engine that uses the OpenSearch protocol. As an alternative approach to making content from other sources searchable, you can set up hybrid search. This feature is available in almost all Office 365 and SharePoint Online scenarios. It allows users to show search results from content available in the cloud and on-premise. So if you would like to index a content source that is not supported in SharePoint Online, you should be able to index it on the on-premise.

Custom Entity Extraction

The TechNet article describing features across solutions actually shows that this feature is only available with the enterprise licensing of SharePoint Server. This feature allows the extraction of custom-defined terms from your content and making them usable as search refiners. Say for example that you would like to extract all of your current product names from the content of your documents and then be able to refine your search results on the product name.

Content Processing Extensibility

The other search feature that is only available with the enterprise licensing of SharePoint Server is the content processing extensibility. In practice, this means there is an API that can be used to transform the data before it is stored in the index. For example, more advanced entity extraction can be made at this step. While the custom entity extraction discussed previously is able to identify names in the content based on a pre-defined list of names, through this API you can use a trained model to do entity extraction for example. Additional use cases could be cleaning or normalising the data according to predefined rules (for example, lowercasing all values in a property), or automatically tagging items based on the content.

It should be noted that the TechNet article is not a comprehensive list, and rather gives an overview of the major differences between solutions. Here is for example one more feature whose availability is limited.

Synonyms

One of the missing features in SharePoint Online that is available in the on-premise solution is the possibility of defining synonyms. Since it’s too easy to communicate the same thing with different words, defining synonyms or abbreviations for search phrases can help aggregate the results for the multiple ways of expressing the same information need. We hope that Microsoft will integrate this feature in the future versions of SharePoint Online as well.

Find the right documentation

When searching for which functionality is available across solutions on the Microsoft Office.com website or TechNet, make sure to check that the discussed functionality applies to your version of SharePoint. Articles usually indicate for which versions the functionality applies to.

Articles on Office.com (left) and TechNet (right) indicate for which version
of SharePoint the discussed topic applies to.

Please note that things might change, new updates in SharePoint online can add functionality that was missing before.

To stay up-to-date, check the TechNet page once in a while, visit our website or contact us to help you map your requirements to the available search features across solutions.

Continuous crawl in SharePoint 2013

Posted on April 15, 2014 by

Continuous crawl is one of the new features that comes with SharePoint 2013. As an alternative to incremental crawl, it promises to improve the freshness of the search results. That is, the time between when an item is updated in SharePoint by a user and when it becomes available in search.

Understanding how this new functionality works is especially important for SharePoint implementations where content changes often and/or where it’s a requirement that the content should instantly be searchable. Nonetheless, since many of the new SharePoint 2013 functionalities depend on search (see the social features, the popular items, or the content by search web parts), understanding continuous crawl and planning accordingly can help level the user expectation with the technical capabilities of the search engine.

Both the incremental crawl and the continuous crawl look for items that were added, changed or deleted since the last successful crawl, and update the index accordingly. However, the continuous crawl overcomes the limitation of the incremental crawl, since multiple continuous crawls can run at the same time. Previously, an incremental crawl would start only after the previous incremental crawl had finished.

Limitation to content sources

Content not stored in SharePoint will not benefit from this new feature. Continuous crawls apply only to SharePoint sites, which means that if you are planning to index other content sources (such as File Shares or Exchange folders) your options are restricted to incremental and full crawl only.

Example scenario

The image below shows two situations. In the image on the left (Scenario 1), we are showing a scenario where incremental crawls are scheduled to start at each 15 minutes. In the image on the right (Scenario 2), we are showing a similar scenario where continuous crawls are scheduled at each 15 minutes. After around 7 minutes from starting the crawl, a user is updating a document. Let’s also assume that in this case passing through all the items to check for updates would take 44 minutes.

Incremental vs continuous crawl in SharePoint 2013

In Scenario 1, although incremental crawls are scheduled at each 15 minutes, a new incremental crawl cannot be started while there is a running incremental crawl. The next incremental crawl will only start after the current one is finished. This means 44 minutes for the first incremental crawl to finish in this scenario, after which the next incremental crawl kicks in and finds the updated document and send it to the search index. This scenario shows that it could take around 45 minutes from the time the document was updated until it is available in search.

In Scenario 2, a new continuous crawl will start at each 15 minutes, as multiple continuous crawls can run in parallel. The second continuous crawl will see the updated document and send it to the search index. By using the continuous crawl in this case, we have reduced the time it takes for a document to be available in search from around 45 minutes to 15 minutes.

Not enabled by default

Continuous crawls are not enabled by default and enabling them is done from the same place as for the incremental crawl, from the Central Administration, from Search Service Application, per content source. The interval in minutes at which a continuous crawl will start is set to a default of 15 minutes, but it can be changed through PowerShell to a minimum of 1 minute if required. Lowering the interval will however increase the load on the server. Another number to take into consideration is the maximum number of simultaneous requests, and this is a configuration that is done again from the Central Administration.

Continuous crawl in Office 365

Unlike in SharePoint 2013 Server, continuous crawls are enabled in SharePoint Online by default but are managed by Microsoft. For those used to the Central Administration from the on-premise SharePoint server, it might sound surprising that this is not available in SharePoint Online. Instead, there is a limited set of administrative features. Most of the search features can be managed from this administrative interface, though the ability to manage the crawling on content sources is missing.

The continuous crawl for Office 365 is limited in the lack of control and configuration. The crawl frequency cannot be modified, but Microsoft targets between 15 minutes and one hour between a change and its availability in the search results, though in some cases it can take hours.

Closer to real-time indexing

The continuous crawl in SharePoint 2013 overcomes previous limitations of the incremental crawl by closing the gap between the time when a document is updated and when this is visible in the search index.

A different concept in this area is the event driven indexing, which we will explain in our next blog post. Stay tuned!

The Findability blog

the enterprise search and findability blog by Tietoevry Findwise

Tag Archives: Microsoft Office 365