Making your data F.A.I.R and smart

Posted on November 20, 2019 by Fredric Landqvist

This is the second post in a new series by Fredric Landqvist & Peter Voisey, explaining how your organisation could best shape its data landscape for the future.

How to create a smart data framework for your organisation

In our last post for you, we presented the benefits of F.A.I.R data, how to make data smarter for search engines and the potentials of an Information Commons. In this post, we’re giving you the pragmatic steps to make your data FAIR by creating and applying your own smart data framework. Your data-sharing dream, internally and externally, is possible.

A smart data framework, using FAIR data principles, encompasses the tooling, models and standards that govern datasets and the different context-specific information systems (registers, catalogues). The data is then ingested and processed (enriched/refined) into smart data, datasets and data catalogues. It can then be used and reused by different applications and e-services via open APIs. In this ecosystem, all actors and information behaviours (personas) interplay: provision agents, owners, builders, enrichers, end-user searchers and referrers.

A smart data & metadata catalogue

A smart data & metadata catalogue (illustrated below), provides an organisational capability that aligns data management with the FAIR data principles. View it not so much as one system to rule them all, but rather an ecosystem that is smart and sustainable. In order to simplify your complex and heterogeneous information environment, this set-up can be instantiated, as one overarching mechanism. Although we are describing a data and metadata catalogue here, the exact same framework and set up would of course apply also to your organisation’s content, making it smarter and more findable (i.e. it gets the sustainable stamp).

Smart Data Catalogue — The necessary services and component of a smart data catalogue

The above picture illustrates the services and components that, together, build smart data and metadata catalogue capabilities. We now describe each one of them for you:

Processing (Ingestion & Enrichment) for great Findability & Interoperability

(A) Ingest, harvest and operate. Here you connect the heterogeneous data sources for ingestion.

The configured input mechanisms describe each of the data sources, with their data, datasets and metadata ready for your catalogue search. Hopefully, at the dataset upload stage, you have provided a good system/form that now provides your search engine with great metadata (i.e. we recommend you use the open data catalogue standard DCAT-AP). The concept upload is interchangeable with either machine-to-machine harvester mechanisms, as with open-data, traditional data integration, or manual provision by human upload effort. (D) Enterprise Metadata Repository: here is the persistent storage of data in both data catalogue, index and graph. All things get a persistent ID (how to design persistent URI) and rich metadata.

(B) Enrich, refine analyze, and curate. This is the AI part (NLP, Semantics, ML) that enriches the data and datasets, making them smarter.

Concepts (read also entities, terms, phrases, synonyms, acronyms etc.) from the data sources are found using named entity extraction (NER). By referring to a Knowledge Graph in the Enricher, the appropriate resources are annotated (“tagged”) with the said concept. It does not end here, however. The concept also takes with it from the Knowledge Graph all of the known relationships it has with other concepts.

Essentially a Knowledge Graph is your encoded domain knowledge in a connected graph format. It is by reading these encoded relationships that the machine “understands” the meaning or aboutness of data.

This opens up a very nice Pandora’s box for your search (understanding query intent) and for your Graphical User Interface (GUI) as your data becomes smarter now through your ability to exploit the relationships and connections (semantics and context) between concepts.

You and AI can have a symbiotic relationship in the development of your Knowledge Graph. AI can suggest new concepts and relationships as new data is added. It is, however, you and your colleagues who determine the of concepts/relationships in the Knowledge Graph – concepts/relationships that are important to your department or business. Remember you can utilise more than one knowledge graph, or part of one, for a particular business need(s) or data source(s). The Knowledge Graph is a flexible expression of your business/information models that give structure to all your data and its access.

Extra optional step: If you can manage not only to index the dataset metadata but the datasets themselves, you can make your Pandora’s box even nicer. Those cryptic/nonsensical field names that your traditional database experts love to create can also be incorporated and mapped (one time only!) into your Knowledge Graph, thus increasing the machine “understanding” of the data. Thus, there is a better chance of the data asset being used more widely.

The configuration of processing with your Knowledge Graph can take care of dataset versioning, lineage and add further specific classifications e.g. data sensitivity, user access and personal information.

Lastly on Processing, your cultural and system interoperability is immensely improved. We’re not talking everyone speaking the same language here, rather everyone talking their language (/culture) and still being able to find the same thing. In this open and FAIR vocabularies further, enrich the meaning to data and your metadata is linked. System interoperability is partially achieved by exploiting the graph of connections that now “sit over” your various data sources.

Controlled Access (Accessible and Reusable)

(C) Access, search and visualize APIs. These tools control and influence the delivery, representation, exploration and consumption/use of datasets and data catalogues via a smarter search (made so by smarter data) and a more intuitive Graphical User interface (GUI).

This means your search can now “understand” user intent from just one or two keyword queries (through known relationship connections in the Knowledge Graph).

Your search now also caters for your searchers who are searching in an unfamiliar subject area or are just having a query off day. Besides offering the standard results page, the GUI can also present related information (again due to the Knowledge Graph), past related user queries, information and question-answer (Q&A) type material. So: search, discovery, learning, serendipity.

Your GUI can also now become more intuitive, changing its information presentation and facets/filters automatically, depending on the query itself (more sustainable front-end coding).

An alternative to complex scenario coding also includes the possibility for you to create rules (set in your Knowledge Graph) that can control what data users can access (when, how and where) based on their profile, their role, their location, the time and on the device they are using. This same Knowledge Graph can help push and recommend data for certain users proactively. Accessibility will be possible by using standard communication protocols, open access (when possible), authentication where necessary, and always with metadata at hand.

Reusable: your new smart data framework can help increase the time your Data Managers (/Scientists, Analysts) spend using data (and not trying to find it, the 80/20 data science dilemma). It can also help reduce the risk to your AI projects (50% failure rate) by helping searchers find the right data, with its meaning and context, more easily. Reuse will also be possible with the design that metadata multiple attributes, use licence and provenance in line with community standards

Users and information behaviour (personas)

Users and personas — User groups and services

From experience we have defined the following broad conceptual user-groups:

Data Managers, a.k.a. Data Op’s or Data Scientists
Data Managers are i.e. knowledge engineers, taxonomists and analysts.
Data Stewards
Data Stewards are responsible for Data Governance, such as data lineage.
Business Professionals/Business end-users
Business Users may have a diverse background. Hence Business end-users.
Actor System are different information systems and applications and services that integrate information via the rich open APIs from the Smart Data Catalogue

The outlined collaborative actors (E-H user groups) and their interplay as information behaviour (personas) with the data (repository) and services (components), together, build the foundation for a more FAIR data management within your organisation, providing for you at the same time, the option to contribute to an even broader shared open FAIR information commons.

(E) Data Op’s workplace and dashboard is a combination of tools supporting Data Op’s data management processes in the information behaviours: data provision agents, enrichers and developers.
(F) Data Governance workplace is the tools to support Data Stewards collaborative data governance work with Data Managers in the information behaviours: data owner.
(G) Access, search, visualize APIs, is the user experience to explore, find and interact with the catalogue and data in the information behaviours: searcher and referrer.
(H) API, is the set of open APIs to support access to catalogue data for consuming information systems in the information behaviours: referrer (a.k.a. data exchange).

Potential tooling for this smart data framework:

Search, integration and analytics: Findwise i3, Elastic, Fusion
Semantic tools: PoolParty, Synaptica, Smartlogic, TopQuadrant
Catalogue and integration: Entryscape, Informatica, Stibo, Talend, Marklogic

We hope you enjoyed this post and understand the potential benefits such a smart data framework incorporating FAIR data principles can have on your data catalogue, or for that matter, your organisational content or even your data swamps.

In the next post, Toward data-centric solutions with Knowledge Graphs, we talk about Knowledge Graphs (KG) and its non-proprietary RDF semantic web tech, how you can create your KG(s) and the benefits they can bring to your future data landscape.

Fredric Landqvist research blog

Peter Voisey

Data that really saves lives (and possibly your organisation)

Posted on November 12, 2019 by Fredric Landqvist

This is the first post in a new series by Fredric Landqvist and Peter Voisey, explaining how your organisation could best shape its data landscape for the future.

A Quest for a FAIR Information Commons

You might have heard recently of the phrase, “data that saves lives”. It certainly can, but just as you need to be in shape to do your work, so does data, to work its magic. Data too needs to be shaped by governing principles that we can apply along their life journey, in order that we can reap the consequential rewards and benefits that are there to be had. Data in shape, saves lives.

We all need to fix problems, usually quickly hence the presence of the closed model, data silos and data interoperability. It has had to happen this way, will continue to do so and there’s no shame in that. But if we can be part of a reliable data sharing community, whose data can help us to collaborate and solve better, well, we’d be foolish to turn it down.

So imagine a type of information commons. This isn’t so far-fetched, we just need to widen our horizons and collaborative ecosystem for it to happen, and perhaps take the same model advice internally for our own organisations.

The challenge of really saving lives with data requires new collaborators. As collaborators we require trust. In essence then to be part of this challenge we need to be willing to share data (we use the term data, content, information interchangeably here). Proof of that trust, is to sign an agreement to be part of an information commons, where data has certain principles (a.k.a. terms & conditions, T&Cs). In essence rules of engagement!

Declare interest
Sign a future rules of engagement to share and access data
Get ready to adhere to them

The T&Cs largely apply to the condition of the data being shared and the information about them. They match precisely how you would hope to find data in this new treasure trove. They may also be known as F.A.I.R. – data that is Findable, Accessible, Interoperable and Reusable. FAIR obviously alludes also to the fairness in collaboration and the F.A.I.R data principles originate from a good sharing place.
Here’s a great summary in image form, from Australian National Data Service [ANDS]:

How FAIR is your data today? Simply answer by following a brief checklist or later go for a more comprehensive description at Go FAIR.

Still here? Great! Let’s get started then with Findable!

Findable

We can only truly make data findable when we really think about the range of people who might want to find it and how they might want to use or reuse it (their need determines how they will ask for it). The reality of different data sources, formats, protocols and their possible attributes or descriptors, makes describing data for others problematic, plus, do you really have the time for tagging? Regardless of time, we’re not very good at putting ourselves in somebody else’s shoes (unless of course we’re selling something) and certainly not able to cover the variation in how people (with differing perspectives) search for data.

The best answer we have at the moment is to describe data or datasets by using agreed standards, perhaps that may vary a bit from domain to domain. Sharing or uploading data to the “ether” gives a different feeling to uploading data that matters to a known shared source and accessed by users who understand its value. In doing so it may inspire us to describe data according to a collective standard, with that feeling of having done something good for a bigger cause.

But hang on. Why is the onus on the end-user? We have the tech here now to automate much of this process. We just need a good sharing and upload design that can recognise the (hopefully changeable) standards of description (metadata). By processing data on upload, we can get a better understanding of data with reference to our standards and rules. Thus, according to what the machine recognises (pattern matching) or “understands” (by way of concept relationships in a knowledge graph) it can annotate the data, ready to serve the requests of data searchers and data applications, or at least be able to offer a related alternative.

Such processing is done using AI (NLP, ML etc.), but it’s not magic. We still have to teach our machines the agreed standards and rules in the first place. While that may sound cumbersome to some, it’s not like you keep having to teach them repeatedly. Conversely the student (AI) can also suggest new rules and annotations, keeping them current according to the data being processed. The beauty for most, is though, that we can employ more than one descriptive rule set for different data or datasets. Depending on data source, format and context, the machine can activate different metadata rule sets. The smart part for the uploader is the presentation of a semi-automated metadata form for their data, leaving them to confirm or alter it before hitting send. The “uploader” in this context, is a broad concept to address any agent that contributes with data to the shared information space, be they programmatic or human.

Let’s not forget we’re at the stage where we can use “search” not only for indexing and this automatic annotation, but for calculations on parsing to potentially annotate with even higher understanding. Such a solution fits well with the increasing demand for real-time data too.

So Findable, is really both about making data smarter, and findable.

Accessible

There’s nothing worse than finding something you want, only to be told you can’t use it.

While the premise of an Information Commons is sharing, it doesn’t necessarily have to mean that everything is accessible by everyone – the reason that why some readers left this page at the third paragraph.

Let’s be clever about this. There are lots of ways to automatically control accessibility and automatically police it. This could be technical, IP address, sign on, authorisation (classification of user) etc. But it could also be done by processing data on upload in determining the sensitivity level of data and/or the indicators of GDPR data.

Back to the end-user: they don’t want to see stuff they can’t use, but they also want to see from the go, if they need to get any new software to be able to access data that they are interested in.

Interoperable

Now for the hard part. The reality is that variety in data sources, protocols and formats ain’t going to go away any time soon. We have to accept that. We’ve just mentioned about the technical interoperability in Accessible. There’s also language interoperability (cultural and language) that can again be solved by using a knowledge graph with search (tinkering with knowledge graphs, just like Google does).

Lastly there’s data interoperability. Barriers preventing data and system interoperability are slowly being brought down through collaboration. In the meantime, it is possible for us to convert key data into the same data format, so AI and inferencing can be used on different (previously incompatible) datasets. The kind of thing that can lead to computation-derived insights that a human on their own couldn’t make. Converting data to RDF could be such a point in case, a real lingua franca of data, also connected to the Web.

Reusable

The “F.A.I.” part of FAIR, really already covers Reusable. We want to be able to find data that we can reuse. To do this, we need to be able to see related information on finding content, information, datasets and data catalogues as to the how, what, when, who, why, where of its potential usage. More: working on the shoulders of giants, less: reinventing the wheel. The information (rich metadata) associated with Reusable, also refers to its usefulness: value, age and provenance.

Healthcare Data Commons

There is an emerging FAIR Data paradigm shift within the health informatics and research professional communities, that has been sparked by those within the bio- and life science domains.

There are obvious regulatory constraints when speaking about patient data, or health data, that any data commons arena will have to nail, upfront.

Health Data: quality register data, EHRs data and the patient’s self-created data, together, would be a real gold mine in the pursuit of personalised medicine and health care. Patient-centric data and FAIR data governance will be key.

The outlined scenario for a FAIR Data Commons

The illustration above shows a FAIR data commons. It will be the foundation framework for all information systems (register) in the data ecology. These information systems need to harmonise and align to become FAIR. There is a set of generic agent information behaviour patterns (user personas):

Data provision agent, is an information behaviour with either a human actor that upload (provision data) or machine to machine data integration contributing to the datasets in the register.
Data owner, is an information behaviour relating to governance and ownership, stewardship to the datasets in the register.
Application builder, is an information behaviour relating to building capabilities with the use and reused datasets in the register.
Data enricher, is an information behaviour relating to expanding the models, and enriching the datasets. With i.e. use of linked-data, semantics and more to create richer metadata.
Searcher, is an information behaviour relating to finding and acting upon data.
Referrer, is an information behaviour relating to using data in information flows and data exchange to support different kinds of processes, activities and actions with other actors in the ecology.

The business value realised (effect) using the FAIR Data Commons will be via different means to e-services, used in the scenarios for searcher and referrer, but also in improved efficiency and improved data quality in the other information behaviours.

Next post in the series: Making Your data smart and F.A.I.R. Further reading to help inspire you:

The FAIRsharing.org provides very useful resources as building blocks in the creation of any context-specific data commons.
The National Institute for Health (NIH) in the USA have a Data Commons programme, with on-going pilots.
The Nordic NordForsk recent report on Nordic Commons with HealthData
Similar in the Nordics there are initiatives (Finnish catalog, Swedish register [RUT], HelseData in Norway and Danish Healthdata) coordinated via EU funded research programmes.
The Life Science industry together with Healthcare have some impressive initiatives e.g. Electronic Health Records 4 Clinical Research [EHR4CR], with its information platform, InSite (by TriNetX), in line with FAIR data.

Thoughts collected from LDSV 2019, Semantics 2019 Karlsruhe, HealthData Copenhagen, “Dagar om Lagar” [SFMI] and more…

Fredric Landqvist research blog

Peter Voisey

Plan for General Data Protection Regulation (GDPR)

Posted on August 25, 2016 by

Another new regulation from EU? Will this affect us? It seems so complex. Can’t we sit back and wait for the first fine to come and then act if necessary?

We have to care and act – start planning now!

I think we have to care and act now. Start planning now so you get it right. The GDPR is a good thing. This is not another EU thing about the right size of a strawberry or how bendy a banana could be. This is about the fact that all individuals should feel safe giving their personal information to business. Cyber security is a good thing, not protecting our data and our customers’ data is a bad thing for us. Credit card numbers and personal data leaks out from companies worldwide with large business risks, companies don’t just face fines or reputational damage, they can have their permission to issue credit cards and other financial services products withdrawn by the regulator and responsible employees faces imprisonment. We can only guess whether a company needs to be GDPR compliant or not to be allowed to compete in a bidding process?

What is the General Data Protection Regulation (GDPR)?

The General Data Protection Regulation (GDPR) is a new legal framework approved by the European Union (EU) to strengthen and unify data protection of personal information. GDPR will replace the current data protection directive (in Sweden Personuppgiftslagen, PUL) and applies from 25 May 2018.

Who is affected?

GDPR has global reach and applies to all companies worldwide that process personal data of European Union citizens.

Identify personal data and protect it

GDPR widely defines what constitutes personal data. Organisations needs to fully understand what information they have, where it is located and how it was collected. Discover, classify and manage all information, both structured and unstructured data and secure it.

Data breach notifications

GDPR requires organisations to notify the local data protection authority of a data breach within 72 hours after discovery.

Do you have the right to store this information? Explicit consent

Personal data should be gathered under strict conditions. Organisations need to ask for consent to collect personal data and they need to be clear about how they will use the information.

The right of access

Individuals will have the right to obtain access to their personal data and other supplementary information in a portable format. You must provide a copy of the information free of charge. GDPR also give individuals the right to have personal data corrected if it is inaccurate or incomplete.

The right to be forgotten

GDPR also introduces the right to be forgotten, or erased. Data are not to be hold for any longer than absolutely necessary, and data should not be used in any other way than it was originally collected for.

Penalties and fines

Companies that fails to protect customer data adequately will face significant fines up to €20m, or up to 4% of global turnover. This should be a serious incentive for companies to start preparing now.

First steps to GDPR compliance

Create awareness and allocate resources
First step is to make sure that your organisation is aware of the new EU legislation and what it means for you. How will your business be affected by the new regulation? You need to allocate enough resources, make sure you involve decision-makers and stakeholders in your organisation. Last, but not least, start today!
Content Inventory
The second step is to discover and classify all your information to identify exactly what types of personal identifiable data you have, where you have it and how it is collected.

Findwise can assist you in this process, please contact Maria Sunnefors and visit our website for more information.

Want to read more?

Read more about the GDPR at Datainspektionen (in Swedish) or at iCO.

Stay Cleaning and moving boxes for cloud

Posted on November 17, 2014 by Fredric Landqvist

This is the seventh post in a series (1, 2, 3, 4, 5, 6) on the challenges organisations face as they move from having online content and tools hosted firmly on their estate to renting space in the cloud. We will help you to consider the options and guide on the steps you need to take.

Do not even think about moving into the cloud apartment without a proper cleaning of the content buckets. Moving from an architected household to a rented place, taxes a structured audit. Clean out all redundant, outdated and trivial matter (ROT). The very same habit you have cleaning up the attic when moving out from your old house.

It is also a good idea to decorate and add any features to your new cloud apartment before the content furniture is there. It means the content will fit with any new design and adapt to any extra functionality with new features like windows and doors. This can be done by reviewing and updating your publishing templates at the same time. This will save time in the future.

Leaning upon the information governance standards, it should be easy to address the cleaning before moving, for all content owners who have been appointed to a set of collections or habitats. Most organisations could use a content vacuum cleaner, or rather use the search facilities and metric means to deliver up to date reports on:

Active / in-Active habitats
No clear ownership or the owner has left the building
Metadata and link quality to content and collections to be moved across to the cloud apartments.
Review publishing templates and update features or design to be used in the Cloud

When all active habitats and qualified content buckets have been revisited by their set of curators and information owners. The preparation and use of moving boxes, should be applied.

All moving boxes do need proper tagging, so that any moving company will be able to sort out where about the stuff should be placed in the new house, or building. For collections, and habitats, this means using the very same set of questions stated for adding a new habitat or collection to the cloud apartment house. Who, why, where and so forth, through the use of a structured workflow and form. When this first cleaning steps have been addressed, there should be automatic metadata enhancement, aligned with the information management processes to be used in the new cloud.

With decent resource descriptions and cleaned up content through the audit (ROT), this last step will auto-tag content based upon the business rules applied for the collection or habitat. Then been loaded into the content moving truck, or loading dock. Ready to added to the cloud.

All content that neither have proper assigned information ownership, or are in such a shape that migration can’t be done should persist on the estate or be archived or purged. This means that all metadata and links to either content bucket or habitat that won’t be moved in the first instances, should at least have correct and unique uri:s, address, to this content. And in the case a bucket or habitat have been run down by a demolition firm, purged. All inter-linkage to that piece of content or collection have to be changed.

This is typically a perfect quality report, to the information owners and content editors, that they need to work through prior to actually loading the content on the content dock.

Finally when all rotten data, deserted habitats and unmanageable buckets have been weeded out. It is time to prepare the moving truck, sending the content into its new destination.

Our final thread will cover how will the organisation and it habitants will be able to find content in this mix of clouds, and things left behind on the old estate? Cloud Search and Enterprise Search, seamless or a nightmare?

Please join our Live Stream on YouTube the 20th November 8.30AM – 10AM Central European Time
Fredric Landqvist research blog
Mark Morell intranet-pioneer

Placemaking, wayfinding and game rules in the Clouds

Posted on November 12, 2014 by Fredric Landqvist

This is the sixth post in a series (1, 2, 3, 4, 5, 7) on the challenges organisations face as they move from having online content and tools hosted firmly on their estate to renting space in the cloud. We will help you to consider the options and guide on the steps you need to take.

Starting from our first post we have covered different aspects you need to consider as you take each step including information structure and how it is managed using Office 365 and SharePoint as a technology example. We will cover more about SharePoint in this post, and placemaking in the cloud.

In SharePoint there are a set of logic chunks. One could decompose the digital workplace into intranet sites, as departmental and organisational buckets; team sites where groups collaborate, and lastly your personal domain being the my site collection. Navigating between these, is a mix of traditional information architecture and search driven content. When being within a such a habitat as a teamsite, it is not always obvious how to cross-link or navigate to other domains within the digital workplace hosted in Sharepoint.

One way to overcome this, is to render different forms of portals, based upon dynamic navigation. These intersections and aggregates help users to move around the maze of buckets and collections of the content. Sharepoint have very good features, and options to create search-based content delivery mechanisms.

A metadata and search-based content model, gives us cues for the future design of the digital workplace, with connected habitats and sustainable information architecture. Where people don’t get lost, and have wayfinding means to survive everyday work practices.

This is where how you manage the content in SharePoint and Office 365 is critical. As we said in our first post it is important you have a good information architecture combined with a good governance framework that helps you to transform your buckets of content from the estate into the cloud. We have covered information architecture so we now move more towards how governance completes the picture for you.

There are three approaches to the governance your organisation needs to have with SharePoint and Office 365. You don’t have to use just one. You can combine some of each to find the right blend for your organisation. What works best for you will depend on a number of different factors. Among them:

Restricting use – stopping some features from being used e.g. SharePoint Designer
Encouraging best practice – guidance and training available
Preventing problems – checking content before it is published

Each of these approaches can support your governance strategy. The key is to understand what you need to use.

Restricting use

You need to be clear why your organisation is using SharePoint and Office 365 and the benefits expected. This will shape how tight or loose your governance needs to be.

Once you are clear on this, you then need to consider the strategic benefits and drawbacks such as SharePoint Designer and site collection administration rights.

Benefits

You control what is being used.
You decide who uses a feature e.g. SharePoint Designer.
You manage the level of autonomy each site owner has.
You find out why someone needs to use a feature.
You monitor costs for licences, users, servers, etc.
You measure who is using what and why for reporting.

Drawbacks

You stifle innovation by not allowing people to test out ideas.
You stop legitimate use by asking for permission to use features.
You prevent people being able to share knowledge how they wish to.
You may be unable to realise the maximum potential of SharePoint.
You create unnecessary administration.
You risk adding costs without any value to offset them with.

You need to get the balance right with governance that gives you maximum value for the effort needed managing SharePoint and Office 365.

Encourage best practice

The goal from implementing SharePoint and Office 365 is to have an environment that enables employees to publish, share, find and use information easily to help with their work. They are confident the information is reliable and appropriate, whatever their need for it is. People also feel comfortable using these tools rather than alternative methods like calling helpdesks or emailing other employees for help.

Encouraging best practice by giving them the opportunity to test to meet their needs is one approach to achieving this. There are factors you need to consider that can help or hinder the success of using this approach.

Benefits

You inform employees of all the benefits to be gained.
You train people to use the right tools.
You design a registration process to direct people to the right tools.
You point employees to guidance on how to follow best practice.
You encourage innovation by giving everyone freedom of use.

Drawbacks

You can’t prevent people using different tools to those you recommend.
You risk confusing employees using content unsure of its integrity.
You can’t prevent everyone ignoring best practice when publishing.
You may make it difficult for people to share knowledge effectively.
Your governance model may be ineffective and need improving.

Getting the balance right between encouraging best practice and the level of governance to deter behaviour which can destroy the value from using SharePoint and Office 365 is critical.

Preventing problems

As well as encouraging best practice, preventing problems helps to reduce time and costs wasted on sorting out unnecessary issues. While that is the aim of most organisations the practical realities as it is rolled out can divert plans from achieving this.

You need to get the right level of governance in place to prevent problems. Is it encouraging innovation and keeping governance light touch? Is it a heavier touch to prevent the ‘wrong’ behaviour and minimise risk of your brand and reputation being damaged? How much do you want to spend preventing problems? What does your cost/benefit analysis show?

Benefits

People using SharePoint and Office 365 have a great experience (especially the first time they use it).
Everyone is confident they can use it for what they need it for without experience problems.
Employees don’t waste time calling the helpdesk because many problems have been prevented.
Effective governance encourages early adoption and increased knowledge sharing.
Costs spent preventing problems are justified by increased productivity and reduced risk of errors.

Drawbacks

People find registering difficult and lengthy because of extra steps taken to prevent problems and don’t bother.
People find it too restrictive for their needs and it stifles innovation.
People turn to other tools (maybe not approved) to meet their needs and ask other people for help to use them.
Too restrictive governance prevents most beneficial use by raising the barrier too high for people to use.
Costs of preventing problems are higher than benefits to be gained and not justified.

You need to consider the potential benefits and drawbacks before deciding on the level of governance that is right for your organisation.

Remember, it is possible and probably desirable to have different levels of governance for each feature. It may be lighter for personal views and opinions expressed in MyProfile and MySite but tighter for policies and formal news items in TeamSites.

That is the challenge! You have so much flexibility to configure the tools to meet your organisation’s needs. Don’t be afraid to test out on part of your intranet to see what effect it has and involve employees to feed back on their experience before launching it.

The way forward is to create a sustainable information architecture, that supports an information environment that is available on any platform, everywhere, anytime and on any device. A governance framework can show roles and responsibilities, how they fit with a strategy and plan with publishing standards as the foundation to a consistently good user experience.

Combining a governance framework and information architecture with the same scope avoids any gaps in your buckets of content being managed or not being found. It helps you transform from your estate to the cloud successfully.

In our last concluding posts we will dive into more design oriented topics with a helping hand from findability experts and developers. Adding migration thoughts in next post. But first navigating the social graph being people centric, leaving some outstanding questions. How will the graph interoperate if your business runs several clouds, and still have buckets of content elsewhere?

Please join our Live Stream on YouTube the 20th November 8.30AM – 10AM Central European Time
Fredric Landqvist research blog
Mark Morell intranet-pioneer

Content Governance – life cycle and reach

Posted on November 10, 2014 by Fredric Landqvist

This is the fifth post in a series (1, 2, 3, 4, 6, 7 ) on the challenges organisations face as they move from having online content and tools hosted firmly on their estate to renting space in the cloud. We will help you to consider the options and guide on the steps you need to take.

Content created within a context, as either a departmental site, or team habitat has usually only reach and bearing for the local context of fellow members of staff within this unit. Other pieces of content have a coverage that stretches all parts of the business. One simple example, is the bucket of content that makes up the management system, with governing principles, strategies, policies and guidelines that describes the core processes, activities, roles and so forth within an organisation.

Yet other content, as the outcome from a project, will build a bucket of content that either lives in a new context, improves a bucket of content or feeds into yet another following project.

From an information management perspective, it is vital that you have organising principles to all your content, where all these layers have been covered. Both reach, and the life cycle to the set of content.

You need a governance framework that reaches out to every bucket of content. This covers what is still on your estate as well as the growing amount in the cloud. All content needs to be managed to remove risks of leakage of sensitive information and prevent people having an inconsistent user experience as they move from one bucket of content in the cloud to another content bucket still on the estate.

You need to make sure people do not see the difference between buckets of content on the estate from content buckets in the cloud. People using your content to help with their work don’t need to know where the content is kept. They need to find it as easily as before, preferably even easier! Content in the cloud should feel the same and be a natural extension to the digital environment people are already used to. Manage it with a governance framework that covers every bucket of content and make it more easy to adopt quicker and use more often without caution or delay.

Part of your governance needs to cover publishing standards based on business needs so it is easy to access from any device e.g laptops, tablets and smartphones, and to view without unnecessary authentication levels. This helps to create that consistent good user experience that encourages people to use your content whether the bucket is in the cloud or not.

A professional team from group HR, might work in their local teamsite, with on-going conversations, work-in-progress documents and so forth. Pieces of their content production leads to governing policies that have a global reach within the organisation, and needs to be linked from the corporate intranet spaces. with versioning and good quality to resource descriptions (meta data). This practice and professional network of HR people, do also share content on a departmental site. With links and resources, that have direct impact on their internal processes. The group of people, have outreaching triggers, and in-bound conversations. And have to balance these two states.

When it comes to temporal content buckets, like a project team site. There are several considerations one have to capture. First where will the outcome and result be stored, when the project is finished. In which context will these content pieces contribute. Second, what should be captured from all on-going conversations (social elements) and work-in-progress and drafts developed during the projects lifecycle? Should a project habitat, be searchable after closing down? Or do the habitat change status, hence all documentation stay within the collection, but the overarching state to the habitat changes? Within Sharepoint these temporal states, versions, workflow and properties. All sum up the organising principles.

If these principles haven’t been ironed out, and been described and decided. Inevitable there will be emerging ghost towns, of dead habitats and lost collections of content. With no governance or ownership whatsoever. All this will become a digital landfill.

We will cover more about SharePoint in our next post in this series. Please visit Michael Sampson‘s recent slides where he takes you through strategy, planning, governance and user adoption for collaboration!
Please join our Live Stream on YouTube the 20th November 8.30AM – 10AM Central European Time
Fredric Landqvist research blog
Mark Morell intranet-pioneer

The Curator – how to cultivate the habitat

Posted on November 7, 2014 by Fredric Landqvist

This is the fourth post in a series (1, 2, 3, 5, 6, 7) on the challenges organisations face as they move from having online content and tools hosted firmly on their estate to renting space in the cloud. We will help you to consider the options and guide on the steps you need to take.

In the first post we set out the most common challenges you are likely to face and how you may overcome these. In the second post we focused on how Office 365 and SharePoint can play a part in moving to the cloud. In the third post we covered how they can help join up your organisation online using their collaboration tools and features.

In this post we will cover engagement and how sorting and categorisation of artifacts, according to a simple-to-understand and easy-to-use standard, will form the bits and parts of the curation and cultivation process.

All document libraries should have one standard listing of all items – with two very distinct audiences: being either actors within the habitat or the people contributing, acting and joining the daily conversation; and secondly, those visitors who pass-by the habitat to collect, link and act upon the content presented within the habitats realm.

This makes it very easy for visitors to find their way around a habitat, if the visitors’ area (business lounge) is pretty much aligned to the overarching theme of the site… and all artifacts that the project team like to share wider, have been listed in a virtual bookshelf, with major versions only. The visitors’ area, has all the relevant data, presented upfront. Basically the answers to the questions set when starting the project. The visitors’ area shouldn’t be a backdrop, but rather a storefront. The content has to be of good quality. Then there should be options to engage with the inner-living-room of the habitat, and enter the messy on-going conversations, depending on access-rights. But the default setting, should always be open for unexpected “internal” (within the realm of the organisation) visitors. If the visitors’ area is compiled in a nice and easy to use manner, most visitors are just happy to pick the best-read from the bookshelf, or at least raise a questions for the team! The social construct for this is “welcoming a stranger”, since that visitor might link to your team’s content, cross-linking into his social-spaces.

The habitat’s livingroom and social conversations, will address new context-specific organising principles. A team might want to add new list-items, sort categories or introduce very local what-goes-where themes. This may be especially so when the team consists of actors who have different roles and responsibilities with regard to the overall outcome. And because of this, there may be a certain mix of tools or services in this one habitat of many, where they hang-out for project tasks.

The contextual adjustment is where the curator has to work on a cultivation process that glues the team together. The shared terminology within a group conversation, is what match their practices together. At inception, the curator picks a bouquet of on-topic terms from the controlled vocabularies. Mixing this with everyday use, and contributions from all members, this can be the fruitful and semantically-enhanced conversations with end-user generated tags or “folksonomies”. The same goes for interior design of links, tools, chosen content types and other forms of artifacts that the team will be needing to fulfill their goals and outcome.

The governance of the habitat, leans very much on the shared experiences in the group, and assigned responsibilities for stewardship and curation – where publishing standards, guidelines and training should be part of the mix.

We will cover more on governance and how content should be managed in the cloud in our next post.
Please join our Live Stream on YouTube the 20th November 8.30AM – 10AM Central European Time
Fredric Landqvist research blog
Mark Morell intranet-pioneer

Housekeeping rules within the Habitat

Posted on November 5, 2014 by Fredric Landqvist

This is the third post in a series (1, 2, 4, 5, 6, 7) on the challenges organisations face as they move from having online content and tools hosted firmly on their estate to renting space in the cloud. We will help you to consider the options and guide on the steps you need to take.

In the first post we set out the most common challenges you are likely to face and how you may overcome these. In the second post we focused on how Office 365 and SharePoint can play a part in moving to the cloud. Here we cover how they can help join up your organisation online using their collaboration tools and features.

When arranging the habitat, it is key to address the theme of collaboration. Since each of these themes, derives different feature settings of artifacts and services. In many cases, teamwork is situated in the context of a project. Other themes for collaboration are the line of business unit teamwork, or the more learning networks a.k.a communities of practice. I will leave these later themes for now.

Most enterprises have some project management process (i.e. PMP) that all projects do have to adhere to, with added complementary documentation, and reporting mechanisms. This is so the leadership within the organisation will be able to align resources, govern the change portfolio across different business units. Given this structure, it is very easy to depict measurable outcomes, as project documents have to be produced, regardless of what the project is supposed to contribute towards.

The construction of a habitat, or design of a joint workplace, all boils down to pragmatic steps that are aligned with the overarching project framework at hand. Answering a few simple Questions (Inverted Pyramid):

Who? will be participating, who will own (organisation) the outcome from the joint effort pulling together a project (dc.contributor ; dc.creator ; dc.provenance ) and reach ( dc.coverage ; dc.audience )
What? is the project all about, topic and theme (dc.subject ; dc.title ; dc.description, dc.type )
When? will this project be running, and timeline for ending the project. All temporal themes around the life of a project. (dc.date)
Where? will participants contribute. What goes where and why? (dc.source ; dc.format ; dc.identifier )
Why? usually defined in project description, setting common ground for the goals and expected outcome. ( dc.description )
How? defines used processes, practices and tools to create the expected outcome for the project, with links to common resources as the PMP framework, but also links to other key data-sets. Like ERP record keeping and masterdata, for project number and other measures not stored in the habitat, but still pillars to align to the overarching model. (dc.relation)

When these questions have been answered, the resource description for the habitat is set. In Sharepoint the properties bag (code) feature. During the lifespan of the on-going project, all contribution, conversations and creation of things can inherit rule-based metadata for the artifacts from the collections resource description. This reduces the burden weighing on the actors building the content, by enabling automagic metadata completion where applicable. And from the wayfinding, and findability within and between habitats, these resource descriptions will be the building blocks for a sustainable information architecture.

In our next post we will cover how to encourage employee engagement with your content.

Please join our Live Stream on YouTube the 20th November 8.30AM – 10AM Central European Time
Fredric Landqvist research blog
Mark Morell intranet-pioneer

Pace-Layering the Building Blocks in the Cloud

Posted on November 3, 2014 by Fredric Landqvist

This is the second post in a series (1, 3, 4, 5, 6, 7) on the challenges organisations face as they move from having online content and tools hosted firmly on their estate to renting space in the cloud. We will help you to consider the options and guide on the steps you need to take.

In the first post we set out the most common challenges you are likely to face and how you may overcome these. In this post we focus on how Office 365 and SharePoint online can play a part in moving to the cloud.

Let us be pragmatic and down-to-earth! It is time to roll up our sleeves and consider using Office 365, as one example of how organisations can make this transition from their estate to the cloud. Given that this is the collaborative space many organisations consider using, Office 365 is compelling as a one-size-fits-all, instant build and just roll-out enterprise-wide approach to take sometimes without an Information Architecture plan whatsoever!

In the Office 365 environment, one has to map the terrain, so that there are distinct districts to where things relate – the same goes for the structure of neighborhoods of clustered habitats. But where it gets tough is to have an agile and resilient city plan for the real-world experience. This is actually the pillar construction in a digital domain, aiming for resilience and emerging uses over the time… but with a simple and agreed upon game plan.

Pace-layering the information architecture

Most organisations have an ontology of entities, things, that are generic, as stated in the W3C Organisation schema. And these perspectives, domain models, vocabularies and ontologies, add up to become districts, and neighborhoods in the Information Architecture map, with a few angles:

Organisation Units (Business Unit, Division, Function, Group)
1. Clients/Customers/Partners/Competitors
2. Professional Networks
3. Governing agencies, or regulatory entities, intermediaries
Locations (Sites, Geographical places as /world/continent/country/region/city/address …)
Business Processes (Process & Activities)
Professions, and Disciplines ( Roles), Practices
Topics (derived from line of Business, and controlled vocabularies)
1. Products
2. Services
3. Tasks
Temporal (Events)

Regardless of line of Business for an organisation, these pan out as pretty good structural elements on which to build upon. Since an enterprise is a social construct with agreed borders, it is populated with people who act and interplay in various different ways, and have a multitude facets with regards to everyday work. Some entities change more frequently, generally in the organisational units further down in the leaves, and less so in the top main branches. The vocabularies within an organisation needs to be the center pillar, to reduce linguistic insecurities.

From an Information Architecture perspective, in using Office 365 or Sharepoint it is wise to use pace-layering to the building blocks, on to which navigational constructs are built upon. This means, using the highest level of the organisational unit tree branch, a pretty stable foundation for the site-structure can be built. This is where content and teamsites live. More fluid navigational themes (temporal, or topic entities) can then be added.

This goes for activities undertaken within daily practices, where a set of professions and disciplines interact. All of these activities lay out a tapestry of overarching business processes. The outcome or result, might be a thing that is detailed as topic taxonomies. For example, a product structure for a specific manufacturing industry. Since all organisations have actor networks in their ecology, it is preferable to add these entities into the structure, as clients, partners, competitor, regulatory agencies, social networks, communities of practice and so forth.

All of these set of terms, have to be maintained in a Managed Metadata Service, a.k.a TermStore. In most organisations there are other sources of their controlled vocabularies, hence mapping is key, to have aligned master term sets. Either through subscription models (batch) or enterprise-linked-data sets. All these actions, defines the terrain, so we map the ecosystem as taxonomic chartographers.

The Building Blocks: Artifacts and Collections

Office 365 comes with a pretty organised set of tools, themes and things to build upon. For more website related things, one could either use published web sites / portals, or enterprise wikis. The other main services are digital habitats, or collaborative spaces, team-sites. And lasty there are ESN (Enterprise Social Networks) like Yammer, and instant messaging tools like Lync, and Exchange Services like mail and calendar. Sharepoint Online and Office 365 is a Swiss Army Knife.

Within a collection, there are some key artifacts and services, content types:

documents
pages, web or wiki
calendars
to-do lists
blogs
forums
list object
and social objects to name a few.

If all of these included content types adhere to Dublin Core with a bare minimum of properties, the rest will follow just great.

We will cover more about how Office 365 and SharePoint collaboration tools can help join up your organisation online in our next post.

Please join our Live Stream on YouTube the 20th November 8.30AM – 10AM Central European Time
Fredric Landqvist research blog
Mark Morell intranet-pioneer

Wagon Trains to the Cloud

Posted on October 30, 2014 by Fredric Landqvist

This is the first post in a series(2, 3, 4, 5, 6, 7) on the challenges organisations face when they move from having online content and tools hosted firmly on their estate to renting space in the cloud. We will help you to consider the options and guide you on the steps you need to take.

In this first post we show you the most common challenges that you are likely to face and how you may overcome these.

A fast migration path, to become tenants in a cloud apartment housing unfolds a set of business critical issues that have to be mitigated:

Wayfinding in a maze of content buckets and social habitats.
Emerging digital Ghost Towns due to lack of information governance.
Digital Landfills without organising principles for information and data.
Digital Litter with little or no governance or principles for ownership, with redundant, outdated and trivial (ROT) content.
With no strategy or plan, erodes any possibility to positive business outcome from moving to the clouds.

“WagonTrn” by Tillman at en.wikipedia – Transferred from en.wikipedia by SreeBot. Licensed under Public domain via Wikimedia Commons.

The way forward is to settle a sustainable information architecture, that supports an information environment in constant flux. With information and data interoperable on any platform, everywhere, anytime and on any device.

You need to show how everything is managed and everyone fits together. A governance framework can help do this. It can show who is responsible for the intranet, what their responsibilities are and fit with the strategy and plan. Making it available to everyone on the intranet helps their understanding of how it is managed and supports the business.

The main point is to have a governance framework and information architecture with the same scope to avoid gaps in content being managed or not being found.

Both need to be in harmony and included in any digital strategy. This avoids competing information architectures and governance frameworks being created by different people that causes people to have inconsistent experiences not finding that they need and using alternative, less efficient, ways in future to find what they need to help with their work.

Background

Building huts, houses and villages is an emerging social construction. As humans we coordinate our common resources, tools and practices. A habitat populated by people needs housekeeping rules with available resources for cooking, cleaning, social life and so on. Routines that defines who does what task and by when in order to keep everything ok.

A framework with governing principles that set out roles and responsibilities along with standards that set out the expected level of quality and quantity of each task that everyone is engaged and complies with, is similar to how the best intranets and digital workplaces are managed.

In the early stages with a small number of habitats the rules for coordination are pretty simple, both for shared resources between the groups and pathways to connect them. The bigger a village gets, it taxes the new structures to keep things smooth. When we move ahead into mega cities with 20+ million people living close, it boils down to a general overarching plan and common infrastructures, but you also need local networked communities, in order to find feasible solutions for living together.

Like villages and mega cities there is a need for consistency that helps everyone to work and live together. Whenever you go out you know that there are pavements to walk on, roads for driving, traffic lights that we stop at when they turn red and signs to help us show the easiest way to get to our destination.

Sustainable architecture and governance creates a consistent user experience. A well structured information architecture that is aligned with a clear governance framework sets out roles and responsibilities. Publishing standards based on business needs that supports the publishers follow them. This means wherever content is published, whether it is accredited or collaborative, it will appear to be consistent to people and located where they expect it to be. This encourages a normal way to move through a digital environment with recognizable headings and consistently placed search and other features.

This allegori, fits like a glove when moving into large enterprise-wide shared spaces for collaboration. Whether it is cloud based, on-premises or a mix thereof. The social constructions and constraints still remain the same. As an IT-services on tap, cloud, has certainly constraints for a flexible and adjustable habitual construction to be able to host as many similar habitats as possible. But offers a key solutions to instantly move into! Tenants share the same apartment building (Sharepoint online).

When the set of habitats grow, navigation in this maze becomes a hazard for most of us. Wayfinding in a digital mega city, is extremely difficult. To a large extent, enterprises moving into collaboration suites suffer from the same stigma. Regardless if it is SharePoint, IBM Connections, Google Apps for Work, or a similar setting. It is not a discussion of which type of house to choose, but rather which architecture and plan that work in the emerging environment.

Information Architecture for Digital Habitats

If one leans upon linked-data, linked-open-data, and emerging semantic web and web of data standards, there are a set of very simple guidelines that one should adhere to when building a Digital Village or Mega City. The 5 stars, our beacon of light!

All collections and shared spaces, should have persistent URI:s, which is the fourth star in the ladder. When it comes to the third star of non-proprietary formats it obviously becomes a bit tricky, since i.e. MS Sharepoint and MS Office like to encourage their own format to things. But if one add resource descriptions to collections and artifacts using Dublin Core elements, it will be possible to connect different types of matter. With feasible and standardised resource descriptions it will be possible to add schemas and structures, that can tell us a little bit more about the artifacts or collection thereof. Hence the option to adhere to the second star. The first star, will inside the corporate setting become key to connect different business units, areas with open licenses and with restrictions to internal use only and in some cases open for other external parties.

Linking data-sets, that is collections or habitats, with different artifacts is the fifth star. This is where it all starts to make sense, enabling a connected digital workplace. Building a city plan, with pathways, traffic signals and rules, highways, roads, neighborhoods and infrastructural services and more. In other words, placemaking!

Placemaking is a multi-faceted approach to the planning, design and management of public spaces. Placemaking capitalizes on a local community’s assets, inspiration, and potential, with the intention of creating public spaces that promote people’s health, happiness, and well being.

We will cover more about how this applies to Office 365 and SharePoint in our next post.

Please join our Live Stream on YouTube the 20th November 8.30AM – 10AM Central European Time
Fredric Landqvist research blog
Mark Morell intranet-pioneer