Project Cortex Overview ClearPeople

Project Cortex is the next generation technology for document management (DM) and knowledge management (KM) in Office 365. It enables the mining and extraction of knowledge from data, and organises the information into a Knowledge Network of shared topics (such as customers, projects, matters, cases or other).

Introduction

Forgive me for acting like a child in a sweet shop – but that is exactly how it feels.

For years we have been looking through the window of the sweet shop, marveling at what could be, and it feels like finally the time has come. We can walk through the door and have a taste or two very, very soon….

Undoubtedly, for those that work in the field of Knowledge Management (KM) or are Subject Matter Experts (SMEs), it really must (should) feel like “our time has come”. I dedicate this blog (and any other blogs on Project Cortex) to you and patience for waiting for all these years.

(If you are interested in the topic of knowledge management then you might want to check out the webinar here on collective knowledge)

Most of the information in this blog is a summary, extraction and interpretation of content presented and released during Microsoft Ignite 2019 (03 Nov - 08 Nov). In particular I found the sessions delivered by CJ Tan (BRK3004), Chris McNulty and Naomi Moneypenny (CLB50), and Sean Squires (BRK3005) really useful to gather an overview. The images have mainly been lifted from CJ Tan's deck (thanks for sharing CJ).

For those working with data, document management and knowledge management in SharePoint since the early days, I think it is fair to say that we all must have had more optimism in our bones than not. Each time a new big release of SharePoint was announced we were filled with hope that “this is it”. Finally we will be able to deliver exactly what our customers are asking for, and build the solutions that the various industries are crying out for. And each time, realism hit soon after: “Hey, it’s a lot better but not quite there yet.”

The high level discipline and process for knowledge management is generally agreed to be something along the lines of the visual below:

Personally, I remember coming back after the SharePoint 2010 Conference where a lot of new shiny SharePoint features were released to public preview and we could start demonstrating it. MMS (Managed Metadata Services) and the Term Store was announced and it was cool man! For me, it was the best thing to have hit SharePoint,…. EVER!

These was exciting times and MMS/Term Store became a key feature of nearly every single solution ClearPeople has developed or delivered since. In the meantime, Microsoft appeared to have forgotten about MMS and the Term Store. We waited patiently – but no further updates came along. It just kept ticking along with each new SharePoint update. And as we all started swinging into Office 365 and SharePoint Online, the trusty old Managed Meta Service followed along year after year – looking older and more tired with each new Office 365 feature release.

Until now and Project Cortex.

Why now?

For many it has been natural to assume that with Office 365 it would become easier to work and search more efficiently across everything within the Microsoft tool set. The reality has been different and even more silos have been created within Office 365.

In the background however, Microsoft have continued to work on the long-term ambition to deliver a more coherent data strategy, consolidated data services and a more universal approach to search. Many of those efforts now appear to be coming to fruition, in parallel with AI technologies maturing, such as machine learning and Natural Language Processing with LUIS (Language Understanding Information Services). Importantly, Microsoft are introducing the concept of “Machine Teaching” (which is not Machine Learning) into the available AI stack. This in particular is relevant to knowledge management as it does not require massive data sets or data scientists, but rather – smaller sets of sample data, using the knowledge manager or subject matter expert as the teacher.

And finally, the Microsoft Graph should not be underestimated in terms of its importance to make a lot of new developments and work come together. While still a young data service technology, it is maturing fast and it is the glue behind the scenes for organising the representation of data.

In my view we are seeing this big leap forward now with Project Cortex, as a result of a whole series of technologies that have been innovated, evolved, matured and are now coming together.

What is Project Cortex?

Project Cortex is primarily focused on converting data into valuable knowledge.

Project Cortex is the next generation technology for document management (DM) and knowledge management (KM) in Office 365 and enables the mining and extraction of knowledge from data. It organises information into a Knowledge Network of shared topics. Such as customers, projects, matters, cases or organisational information buckets.

Have a look at Microsoft's own intro video to Project Cortex:

It is clear that Microsoft is pushing this as an enabler for changing the way people work. It is about enabling people to use the knowledge available within an organisation, allowing “individuals to learn and work faster together”.

According to the released information and the sessions I have gleaned, Microsoft appears very clear in terms of the business challenges and benefits that are being addressed with Project Cortex. In particular, Project Cortex seeks to address some of the deep seeded knowledge management challenges those in the field experience every day:

Information is everywhere and is often unstructured
Information is stuck in silos
Information is hard to find, and the findability will depend on whether you know anything about it or whether it is knowledge you don’t even know how to ask about.
Knowledge needs capturing to be useful and findable - which often does not happen.
Not collecting knowledge is costly, especially when tacit and when someone leaves with uncaptured knowledge.

The benefits that Project Cortex delivers include:

Empower people with knowledge and expertise across the apps we use everyday
Automatically connect and organize knowledge across teams and systems
Manage and protect content with AI and built-in security & workflow

The message from Microsoft is also clear when to comes to the expectations of AI. The combination of AI and human curation by the crowd and/or by experts is still very much the best approach for the (long-term) foreseeable future.

The tooling allows organisations to engage the technology for the right reasons and to get better outcomes, whether the approach is manual, automated or assisted with AI.

What is the Knowledge Network?

I plan to dedicate a specific blog on this particular topic. At this point the most important aspect to understand is that the Knowledge Network is a standardised and structured way of storing, representing and relating information that can be accessed via a common interface. It allows for a more natural and logical approach to interact with and delve into data. In order words, when data is "converted" to knowledge - it is structured and persisted within the Knowledge Network for a range of applications (including Search) and user interfaces to take advantage of. Ultimately leading to that information being presented and relevant to the end user.

The Knowledge Network builds on the Knowledge Graph which in turn extends the Microsoft Graph, and therefore is in effect the "gateway" to data and intelligence in Microsoft 365. It provides a unified programming model that developers can use to access the huge amount of data in Office 365 (and even within Windows 10 and the security products).

Project Cortex roadmap (Nov 2019)

While some related features are planned for release before end of 2019, the first early "release" pf Project Cortex is expected to be ready for General Availability during first half of 2020. My colleagues or I will naturally be following this closely.

Please refer to this blog for the latest updates (and my comments) to the roadmap: Project Cortex Roadmap

I look forward to exploring this topic more!