Introduction: The Challenge Of IT System Changes
A typical business IT system will be used daily for several years. During this time, there will be inevitable changes, including software version updates, end users switching devices, mitigating new security threats, and modifications to the existing system use to meet new business requirements.
However, changing anything about a key IT system naturally introduces a risk. Does everything still function after a software update? Will the integrations with other systems continue to function? How will our users cope with the new user interface? Can we decommission our aging servers and move the entire system to Cloud? If— despite all preparations— the update does introduce an unforeseen disaster, can we roll back to pre-update state? If a completely new business need arises, can we somehow extend the use of our current system, or should we introduce a completely new one?
This concern is even greater in regulated, compliance-driven environments. Regulatory guidelines mandate that key IT systems are first validated and then placed under change control. This means the user organization must plan, test and approve changes in advance to avoid unintended consequences. Everything must be thoroughly documented, turning a software update into a resource-hungry project of its own. This significantly raises the threshold for changing or updating an IT system, to the point where an outdated IT system or a web browser version with known security holes remains in use to avoid the burden of a post-validation change.
Purpose Of This Paper
This paper provides the non-technical reader with a broad overview of how changes and updates can be applied to an M-Files system with minimal risk. This includes, but is not limited to, validated M-Files systems in regulated environments. Example screenshots to M-Files administrator tools are being shown to give the reader an idea of how exactly a certain task is executed. However, hands-on use of such administrator tool is not covered in this paper.
This paper is organized around a single example in which the user organization adds a new process to their existing M-Files system. The example is deliberately a simple one, but the overall process would be similar for more demanding, higher-risk changes and upgrades.
How Controlled Is Controlled Enough?
M-Files implementations vary significantly from one user organization to the next.
Companies that use M-Files vary significantly in size, from small start-ups to global enterprises. Some companies have in-house IT, while others outsource their IT as a service.
There are both highly regulated companies who must comply with external regulation and are subject to regulatory inspections, and less regulated businesses with more freedom in choosing their own best practices.
Within regulated environments there is quality-critical IT system use, and low-risk, generic business use.
All these criteria effect how changes to an M-Files system should be conducted, and what level of risk-analysis, testing and documentation, if any, is produced for each change or update. The example case in this paper most likely looks overly controlled to some readers, and not controlled enough to others. The reader is asked to tailor the best practices presented in this paper to match their own environment, internal process controls and applicable regulation.
Change-Control Friendly M-Files Features
IT systems vary significantly in how change-tolerant they are. M-Files is considered to be a change-friendly software system due in part to these features:
The entire M-Files Vault structure, document and data classification, workflows and business functionality is essentially one big configuration. This configuration can be exported and stored elsewhere as a single file, and later used to compare a vault before and after a change.
It is possible to make exact live copies of an M-Files Vault for development, testing and training use via vault copy or backup-relocate-restore routine. Developers can change a Vault and then immediately test the changes live.
Selected content or structure that contain the desired changes can be exported from the development environment into a Change Package. This Change Package can then be applied to other environments. This allows for moving new or updated functionality from the development environment to other environments in a controlled manner.
There are tools that automatically document the vault structure in a human-readable MS Word format.
Change-Control Friendly Hosting Setup
A good IT practice is to have a separate test sandbox where the effects of the planned changes can be safely evaluated. A three-vault approach is proposed in this paper.
DEV is where new M-Files structure is technically developed and where initial live testing is done.
TEST is where formal testing is conducted against the user organization’s requirements and regulatory requirements. TEST may be also used for User Acceptance Testing (UAT) where a regular end user, armed with standard system training or user guide only, attempts to conduct expected tasks.
PROD is the actual production Vault and the only one visible to all users.
DEV, TEST, and PROD vaults are created during the initial deployment, and due to previous activities are structurally close to being identical. Once the change is complete the three vaults should be identical once again, as explained later in this paper.
The changed project will also result in documentation (both written documents and data files) which should not be stored to the system itself that is being changed. Thus there needs to be a means to store and manage all the change documentation.
Step-by-Step Execution of an M-Files System Change
Our example story: The Need For Change Emerges.
“Here at Incredi-o-Product Inc. we use M-Files in several areas. Our M-Files solution consists of four vaults, two of them subject to validation: a QMS Vault for all quality work, and an Operations Vault with all the R&D and manufacturing documentation. Due to their validated status we claim that these vaults contain electronically signed original records. We routinely send these records out to customers and present during audits directly from the vault.
We are adopting a new practice in which a sample of each production batch is sent to an external service provider for quality control. The service provider will send us the test results back as a PDF certificate document. Based on the testing outcome our product manager either releases the production batch or puts the batch on hold.
We want to change the structure of the Operations Vault to include a dedicated document class for the test certificates, and a forced workflow that automatically assigns each new certificate for the designated product manager. We want the system to help ensuring that a product batch cannot be released without a test certificate and product manager’s approval signature.”
Hands-on Change Execution Begins
The expected change to the Operations Vault is discussed in a managers’ weekly meeting, and the decision documented to the meeting minutes. Next, change planning takes place and resulting documents are stored in the change documentation. Eventually all planning and preparation is done and hands-on execution may start.
Record The Pre-change System
Before any change it is crucial to document the current status of the system. Using M-Files Admin tool, the IT administrator or M-Files consultant manually exports the full structure of the PROD vault
The resulting full vault configuration is turned into a single .zip file and stored into the change documentation as a pre-change document. As a rule of thumb, any relevant working document, data file, signed record or important email created or received during the change project should be stored immediately into the change documentation.
Update The Structure In The DEV Vault
The actual desired change is developed in the DEV vault. A trained M-Files specialist adds the new External Test Certificate document class with suitable metadata structure. Many existing elements, such as Product, Supplier and Manager, are used as metadata for the new document class.
Next, a new External Certificate Processing workflow is designed using M-Files workflow designer. In this example screenshot an electronic signature is put in place for the state transition when the product manager, after inspecting the test certificate, approves the test results.
The change may also require an update to the M-Files Compliance Kit configuration. For example, a new error-preventing data validation rule may be put in place via the Unique Object Enforcement module to check automatically that each production batch does not have more than one test certificate.
During development the current M-Files Views are also updated so that the new certificate documents appear in current, familiar places. This is actually the only change to the existing configuration – all the rest is about adding something new.
As development proceeds it’s good to test the functionality right away. With M-Files there is no need to compile a new software build or commit the changes, but developers can simply switch over to the M-Files end-user interface and try out the new features immediately. After a few iterative develop-test cycles a new updated prototype structure is in place in the DEV vault that meets the business need.
Assess Risks & Update Tests
Next it should be assessed if this change introduces any risks. This can be done in a number of ways, including documenting the risk scenarios in a review meeting and storing the result to the change documentation. The risk analysis phase typically includes a decision on what level of testing, re-validation, or additional training is needed. Updated vault functionality may also call for some updates on the user guide, training slides or a written procedure. In this case it is decided that only the added features need to be tested and that an existing Product Release SOP (Standard Operating Procedure) needs to be updated to cover the new process.
Next, key personnel will look at the current validation test material from the initial validation effort and revise them for the to-be-updated M-Files system. A few new requirements are added to cover the new use case. If satisfied via testing, they will give enough evidence that the new External Test Certificate process works exactly as intended: certificates can’t be falsified, releasing a product batch indeed requires the product manager’s approval, and so on. A new test script is authored for the new process, with references to the requirements.
Export Changes From The DEV Vault & Import To The TEST Vault
Next, the new structure and functionality (and only the new or changed structure) is carefully exported out of the DEV vault
The resulting package is again stored as a new change document, as well as imported to the TEST vault using M-Files Structure Import feature. If DEV and TEST are on different M-Files servers, then the package must first be copied over to the correct one.
When a structure import is initiated in the M-Files Admin Import Tool, the tool generates a report about what will be changed in the target vault. Conveniently, this is a “what if” report: at this point nothing is yet changed in the target vault. The report shows that for example a new External Certificate Processing workflow would be added (only a small part of the full report shown)
Administrator will now carefully inspect this summary report line by line (especially looking for warnings, errors and conflicts), and if satisfactory, save the report as a PDF document into the change documentation and proceed with the change.
Since we also changed the Compliance Kit configuration to add a new validation rule for test certificates, we must also move the updated Unique Object Enforcement module configuration from DEV vault to TEST. This can be done by using the Module Configuration Import/Export tool
This Compliance Kit configuration export should be stored with the standard export package created earlier in this step. After the import of the Compliance Kit configuration files, there is now a live TEST vault with the new Test Certificate document class, connected with its matching new External Certificate Processing workflow, ready for testing.
Conduct The Tests
It’s time to test the new updated structure. How formally the test results are recorded depends on company preferences and applicable regulations. Some companies prefer to print the pre-approved test script on paper and record each passed test with testers’ handwritten initials. Others simply record test outcome electronically into a test script document.
Before testing starts, the TEST vault’s current full audit trail / event log should be exported and stored as new change document. This records the pre-testing state of the TEST vault and provides a clean event log for testers. The idea is that after test completion the TEST vault will contain a full audit trail of all testing activities.
If there are errors during testing, the user organization decides whether the failed test results are acceptable. Typically, they are not, in which case we go back to DEV vault to fix the problem and repeat the previous steps.
Once testing is completed, the event log is again exported from the TEST vault. This, and the all-test documentation produced by testers, is stored into the change documentation.
Update Documentation & Provide Training
As agreed earlier, the new External Test Certificate process is described in a revised version of a Product Release SOP. New document learning requirements may be generated so that users read the updated SOP, or a classroom training may be organized to train the new process. M-Files QMS vault may be used for all three activities: updating the SOP, generating document learning assignments to staff, and providing the training.
Eventually all necessary documents are revised, there is evidence that users know the new process, and we’re finally ready to execute the change in the production environment.
Import Changes To The PROD Vault & Go Live
The final step is to repeat the import process that was used for the TEST vault with the PROD vault. The same change package that was used for updating the TEST vault is used to update the PROD vault to ensure the exact same set of changes just verified in TEST now occur in PROD. Despite all efforts, there is always a risk that something can go wrong in this phase, especially if the PROD vault is integrated to other IT systems or is hosted differently than the test server. Thus, there should be extra measures, such as.
- All users are notified well in advance
- Key IT staff and the M-Files account managers familiar with the environment are available as backup, just in case
- PROD vault is closed from use and all users are disconnected – this is often done over the weekend or late at night
- An additional, full backup is taken of the M-Files production system
- Change is implemented, verified and documented just as the DEV -> TEST replication explained earlier in this paper
- The administrator opens the Event Log of the PROD vault, checks the latest events and verifies all new, correct elements show in the system audit trail
- PROD vault goes back online
- Key users log on to the PROD vault and inspect that it is technically intact, previous content is in place, the familiar Views work as expected, search results look accurate and so on
- The updated PROD vault is now back in use
Document The Post-change Status
Before closing the change, the post-change status is recorded. This includes comparing once more the pre- and post-change structure of the PROD vault. The full structure is exported into an XML file once again and stored into the change documentation.
Any suitable file comparison tool may be now used to verify the exact change done to the PROD vault by comparing the pre- and post-change structure files. In this example the freely available WinMerge (http://winmerge.org/) is used to highlight changes in workflow permission settings. Eventually the case can be closed. The Operations Vault is back in production use with the new Test Certificate process. The three instances of the vault (DEV, TEST and PROD) are again structurally nearly identical, given that the change is conducted correctly and carefully. DEV and TEST vaults are taken offline.
Final documentation is approved according to organization’s own practices. The final validation report is perhaps printed on paper for managers’ handwritten signatures. Finally, all the evidence of a well-executed change is stored and available for management reviews or audits.