A Repository Agnostic Approach to ECM

Today’s Business Information Environment: Complicated, Chaotic and Exploding

Managing business information is more complicated today than ever. The amount of data is exploding—according to The Foundation for Scientific and Industrial Research, the largest independent research organisation in Scandinavia, over 90 % of the world’s data has been generated in the last two years—and it’s not slowing down. As a result, individual businesses are finding it harder and harder to effectively manage the information they need to get the job done and stay competitive.

And if dealing with such large and growing amounts of information was difficult enough, it’s often scattered across a variety of different systems and repositories, including shared network drives, email, traditional document management and enterprise content management (ECM) systems, such as SharePoint, OpenText and Documentum, as well as emerging file sync and share services, such Box, Dropbox and OneDrive.

Some of these older legacy systems, are basically on life support and being phased out, while others are new services with frequent updates. Confusing matters further, multiple different interfaces slow down user adoption that decrease efficiency and productivity. 

In short, today’s business information environment is chaotic, complicated and expensive, both in terms of the costs of the actual systems and the necessary IT resources to maintain them. What is clear is that simply adding another traditional repository or system is not the answer.

The “Old” Approach: Data Migration, Information Silos and Vendor Lock-in

The “old,” or traditional, approach to managing information (how it is classified, organised, processed and secured) has been primarily based on where content or information is stored. In fact, for essentially all information management systems (e.g., enterprise content management, document management, records management, etc.), in order to get value from the system, the content must reside in that system. As such, one of the first hurdles to getting an ECM system into production is data migration, and this isn’t limited to just older systems, even new services like Box and Dropbox are predicated on the requirement that information must first be migrated into these systems.

While it takes significant time and money just to move the information, deploying a new system also requires substantial effort to manage change, as often many users and managers don’t want to change, not because they’re particularly happy with the old system, but because they’d rather live the devil they know than the one they don’t.

The aversion to change and the problems surrounding migration often leads to vendor “lock-in,” which shackles businesses to the fate of the vendors they’ve chosen. Furthermore, as new needs arise, rather than dealing with the difficulties of change management and migration, a new system is purchased to solve that requirement, leading to a proliferation of systems and “information silos” with different user interfaces and limited interoperability that inhibit productivity and collaboration. Unfortunately, the bigger the company, the more silos there typically are.

Almost all ECM systems today use a location-centric, folder-based paradigm. Essentially, how information is managed is almost entirely based on where the information is stored (e.g., in which folder, site, library, etc). This is a methodology that is over 30 years old, a lifetime in technology. Even relatively new file sync and share services, such as Box and Dropbox, are basically just moving a traditional folder system into the cloud.

The folder paradigm has two fundamental flaws.

The first problem is information must be in more than one place. The location varies based on the role of the person needing the information and where the information is in its lifecycle. For instance, a contract might be meaningful to a sales person based on the customer or client it’s related to, but for someone in the legal department, its relevance might be based on the type of contract it is, if it’s coming up for renewal, or possibly part of a pending legal action. Now multiply this variation across the whole range of content and stakeholders in a business. It’s essentially impossible to create a single folder structure that can effectively address this situation. The result is that the resulting system works in a less than optimal way for everyone.

The second problem is that folder structures are entirely subjective. People approach problems differently, and generally organise things in ways that make sense to them individually. Hence, folder structures created by different people are always different. Consider how that might vary across departments in the same company, or across different companies in the same industry, not to mention across different companies in different industries. This cannot be reconciled without forcing some static, less than ideal structure in a top down fashion. This is the old way.

The New Approach to Information Management: Context is King

The new approach is all about context, what the information is and whether it is relevant to the individual’s or business’s current need. That need can be, and generally is, different for others. In the modern, “what it is vs. where it’s stored” approach, any information can “show up” in multiple “places” based on the context without duplication of content. These places aren’t fixed, they’re dynamic. A single, unique contract can show up in a search with other customer-related information for sales also shows up with pending legal contracts, independent of customer, for the legal department.

Further, the new approach is completely objective. A contract is a contract whether it is in sales or the legal department. Moreover, this fact is constant not only across departments in the same company, but across different companies in the same industry, and even in different companies across different industries. From this objective foundation comes precision. And what’s more, it’s conceptually intuitive. Everyone knows what they are working on. However, guessing where something is or should be stored is wholly dependent on who set up the folder structure. Inefficiencies result when staff is forced to spend time looking for information or trying to ascertain where to put information. 

This is the new approach, and it’s highly personalised, and it’s about a whole new dynamic way to organise, process, secure, retain and dispose of information.

Metadata – the Foundation of the New Approach

So, what drives this dynamic, personalised, context-aware future? Metadata. Metadata is the driver of the new modern information management architecture and it enables defining what something is and what it’s related to in an objectively precise and intuitive way.

For instance, rather than storing a proposal related to a certain account or project in the account or project folder (recall the dilemma of information needing to be in more than one place), one simply tags it with the account and the project, as well as other important information, such as date, or any other important attribute to help organise, secure and process it. 

Once information is decoupled from location, the focus shifts to what it is, and hence an entirely new world of information management opens. Information can be in the cloud or on-premises. Or it could be in any repository or system, such as Office365 or SharePoint online, a network file share, Box or Dropbox, or Documentum, or OpenText. It can also reside as structured data, in other core business systems, such as Salesforce, Microsoft Dynamics, NetSuite and SAP. 

Bridging the Gap between Core Business Systems and Content 

ECM systems have traditionally been focused on managing documents such as contracts, proposals, invoices and so on—often referred to as “unstructured content.” But other core business systems (CRM, ERP, HR, etc.) also contain information vital to managing documents and other content and establish context and relevance. 

For instance, a proposal or contract might be relevant based on the account or customer to which it is related, and that account or customer is typically managed in the company’s CRM. Likewise, an invoice might be considered important because it is related to a work order for a vendor working on a key project, all of which is managed in the company’s ERP. The new approach to ECM embraces these core systems to create a unified, 360° view of information, so no matter what system you’re using, you always find what you need based on the context. 

A Repository Agnostic Approach to ECM 

The new approach to information management can be thought of as a repository agnostic “intelligent metadata layer” that unifies information across the enterprise based on context, not on the system or folder in which it is stored. 

This new methodology allows content and data to remain in place, undisturbed, so that users of existing system can continue to work uninterrupted, while also allowing information to be enriched for evolving needs and new use cases. 

This revolutionary new approach combines the following three core characteristics; namely it is: 

  • Metadata-driven 
  • Repository agnostic 
  • Intelligent 

The Intelligent Metadata Layer

The middle layer, also referred to as the “Intelligent Metadata Layer” (IML), is the central component of the architecture, where the metadata-driven and intelligence components reside, along with multi-repository search.

All the typical capabilities of an ECM system are supported here, features such as search, version management, workflow, security and collaboration and check-in/check-out. In addition, search is federated from the typical ability to search one repository to encompass the notion of “enterprise search” or the ability to crawl and index content and data in other repositories for quick search and retrieval.

Also supported are metadata-driven ECM capabilities such as “Dynamic Views,” that are dynamically generated “virtual folders” based on metadata. This is the realisation of the notion described earlier wherein content can “show up” in multiple “places” without duplication. Think of it like how the iPhone handles music, for instance, a single unique song can show up by artist, album, genre or date, although with IML the variations are essentially unlimited.

In addition, this layer supports analytics to provide automatic classification and metadata definition. This aspect of the architecture is also open, such that third-party “metadata providers” can be plugged into the solution to address the needs of specific industries, use cases or geographic regions and languages. This layer is also designed to support not only text analytics, but technologies such as machine learning to improve performance over time based on user behaviour, wherein content and information can be “recommended” to users. This could be envisioned as something akin to “Netflix for the enterprise,” wherein content like that one has retrieved before, or content that others in similar roles have frequently accessed, is also suggested.

Multi-Repository Backend

The lower layer of the architecture is known as the multi-repository backend. This layer supports the ability to connect with other repositories and systems via “connectors.” While a core set of connectors will be provided “out of the box,” such as for network files shares, Office 365, SharePoint Online, Box and Salesforce, it also allows third parties to develop connectors for a variety of other repositories and systems. These connectors are designed to be lightweight, and easy to develop and maintain, rather than heavy, complex interfaces that attempt to drive all the capabilities of other repositories.

Unified User Experience Layer

Traditional ECM is typically complex and difficult to learn and use, and with each system there’s yet another interface to be learned. The top layer of the architecture is intended to address ease-of-use by providing a unified user experience, delivering a consistent and familiar interface to information regardless of the original repository or system in which it is stored and managed. This provides simple, intuitive access from any device, including native mobile apps for smartphones and tablets, in addition to traditional PCs. Additionally, offline access is also supported when an internet connection maybe unavailable.

Value-based Information Management

One of the key concepts behind this new approach is referred to as “value-based information management,” which means that information is managed differently based on its value to the business. This follows the premise that managing all content in the same manner is doomed to fail. This premise is based on two ideas. The first being that the volume of organisational information is so large and growing that it will eventually overload any system. The second is that some information is more business critical than the rest. For instance, a picture or image for a marketing brochure is less important, than compliance, legal or financial information.

To address this issue, IML defines two broad categories of content: “unmanaged” and “managed.” Unmanaged content can also be thought of as casual content and is primarily discovered through full-text search. Managed content is business critical and is highly valuable information that is precisely classified with metadata.

Promoting Content from Unmanaged to Managed

As alluded to above, unmanaged content is indexed and can be quickly found with the multi-repository search capabilities supported by IML. More valuable information, for whatever purpose, can be easily and quickly “promoted” to the managed state, by simply tagging it with metadata. This then creates a metadata record in the IML database, but it does not move the content from its original repository, nor does it disturb or hinder its use by others in the original system or related processes.

This allows the information in other systems to be enriched and re-purposed in new ways for changing needs and use cases without incurring the traditional penalties that such repurposing entails—such as data migration and change management. Once promoted to the managed state, the rich power of the system’s metadata-driven architecture becomes available to the content, regardless of the sophistication of the original repository. For instance, capabilities such as version management, workflow, advanced access permissions, mobile or offline access now become available for content even in a repository with no information management or metadata capabilities, such as a network file share.

Moreover, consider the difficulties an advanced repository such as Documentum when it serves as a validated system in a highly regulated industry. It might be very problematic to add a few metadata properties to information without “breaking” the validated system. However with IML, those properties can easily and simply be applied to the content with no impact on the validated solution.

Personalisation: Start Small and Expand…or Stay Small 

Choosing an ECM system is usually a top down decision that, requires organisational change, and managing can be daunting. Unfortunately, resistance to change can often start in smaller pockets within the business. With IML, it is possible to enable small departments and workgroups to enrich and repurpose information to support new initiatives for specific business needs, such as in legal, accounting and human resources, without disrupting existing systems and processes. 

Consider a project team working on a project with a specific account or customer that has important project-related data located across various systems and repositories. Maybe some of the data is in network shared drives, for instance drawings from engineering. Other information, such as contracts or agreements, could be in Office 365 as required by the legal department. Lastly, other documents such as status reports from external contractors is stored in Box. By simply tagging all of these documents with the customer and project, which could be managed in Salesforce and SAP, respectively, a single project-focused, unified view of this information is created. It’s based on context and what is relevant to the needs of the project team. 

IML does this by leaving the content in place, without impacting anyone else in the organisation, or disturbing any use or processes dependent on the original systems. Moreover, this can be done by different groups and departments to create their own independent “views” into the information they need. 

Data Migration Becomes a Relic of the Past 

And what becomes of the big hurdles to ECM systems discussed earlier: data migration and change management? 

IML enables innovation in those aspects of the business by sidestepping some of the biggest hurdles to ECM in the past. Existing business processes and systems can continue uninterrupted, until and if, the organisation chooses to move forward with change on a larger scale. 

The Future of Information Management is Bright 

The Intelligent Metadata Layer dramatically changes the business information environment from one that is complex, chaotic and static, to one that is intuitive, organised and dynamic. The possibilities are truly exciting, offering organisations the opportunity to unlock the value of existing information and maximise investments in existing systems to drive innovation improved business performance. 

M-Files is leading the way to this exciting future and we are ready to explore the possibilities for your business today. 

For more information on how we can help your organisation please email: peter@documentmanagementsoftware.com.au or visit www.documentmanagementsoftware.com.au


Peter Ellyard

Having spent over 20 years immersed in the document management software industry I have found that by offering a simple to use, highly effective electronic document management solution (knowledge management software) we increase productivity dramatically. Typically by an hour per person, per day! This is not rocket science, just a simple way to streamline your day to day information needs.