Considering a commercial off the shelf (OCTS) CRM solution for your organisation? Stephen Bounds warns that without systems support, complex and ad hoc workflows will fall outside of the CRM into an unmanaged space ...

The edict came down from the government agency’s IT Strategic Committee after a long, fiery afternoon session. “No more custom applications. We’re going COTS!”

“COTS?” asked Jonathon. “Commercial off the shelf software,” sighed Sarah. “It means that the CIO is sick of paying our developers to fix their buggy code for our custom uFix case management system. Some sales person must have got into his ear and promised that using COTS would mean he can ‘do everything through configuration, not code’.”

“So we’re buying someone else’s case management product?”

“Perhaps. But I expect the CIO either wants to leverage our existing CRM (customer relationship management) and EDRMS (electronic document and records management system), or to replace everything with a giant ERP (enterprise resource planning) platform,” said Sally.

“Aren’t ERP platforms very expensive and complex to implement?”

“Yes, they’re normally overkill for a mid-sized agency like ours. We wouldn’t get the ROI to make it worthwhile. So my money is on integrating our CRM and EDRMS systems. The idea is sound in theory, since the idea is to leverage the strengths of both tools, but …” Sally shrugged.

“It’s harder to do well than it looks, right?”

“It always is. The big problem is that vendors generally over-promise on the integration they can deliver. I’ve seen performance issues, user adoption issues, data integrity issues, two-way synchronisation issues … there are lots of traps. Unfortunately you generally don’t find them until you’ve actually implemented the system and start testing with real data at large volumes.”

“Hmm. What a pain in the …” 

“Yes.”

Case management is a common core business process for government agencies. While volumes and complexity can very immensely, most agencies have a responsibility to provide a variety of services to industry, businesses, or the public. Having an effective and efficient means for delivering those services is where case management comes in. While case management systems can be standalone or part of a (CRM) platform, the basic issues remain similar.  The term CRM should be read as covering both types of system for the remainder of this article.

Let’s take a typical case management process handled by our fictional agency:

1. A paper form is received by the agency requesting a service

2. New case created and fields filled in by agency staff

3. Standard response to client created from template and sent

4. Management of case through a variety of stages to conclusion

5. Outcome records and customer case closed

Most CRM systems struggle in the fourth step, which typically involves ad hoc contact via mail, telephone, and email. These may be internal collaboration between agency staff, supervisors and/or senior executives or external contacts with the client or other organisations. Most correspondence sent outside the agency will be based on templates, but even these are generally customised to some extent.

Creating a document from a template, and even allowing editing of the generated document after creation are standard features of CRM systems.  However, unusual, important or sensitive responses correspondence typically require multiple cycles of authoring and review, and possibly multiple approval stages, before being sent. This is where the basic document management capabilities of a CRM becomes most apparent.

Without systems support, complex and ad hoc workflows will fall outside of the CRM into an unmanaged space, with staff reverting to email and/or shared drives to handle these situations.  The lack of accountability and record-keeping compliance in these ad hoc processes leaves agencies wide open to administrative or legal challenges if a case process goes awry.

The need for a CRM and an EDRMS now becomes obvious. Having a system that excels at flexibly managing structured data and another one that specialises in unstructured content management would seem to yield the best of both worlds.

Often these systems become de facto integrated through a simple reference, such as storing a corresponding EDRMS file number against a case. This is functional but has a number of drawbacks, including that:

• users have to learn multiple interfaces

• coordination of workflows across both systems is nearly impossible

• the risk of storing documents into the wrong record-keeping file is far greater

• new EDRMS files need to be manually created and stored separately from the CRM

• documents generated by CRM need to be manually saved and cross-referenced in the EDRMS

CRM + EDRMS integration approaches must address these issues in a simple and robust manner in order to be viable. The four major challenges typically faced in integrating CRM + EDRMS systems boil down to user experience, platform limitations, metadata management and process automation.

A number of vendors now claim an integrated approach.  For example, Microsoft has improved the integration of its Dynamics CRM and SharePoint platforms.  Other vendors have modular systems that can be configured to support both requirements, such as EMC’s Documentum.  

However, you should evaluate these claims with a skeptical eye.  As a rule of thumb, the more generic the platform is, the more “configuration” it will require, and the higher the likelihood of custom code to deliver critical business requirements.

User experience

The user experience when switching between the CRM and EDRMS systems is crucial. The most common approach to integration uses CRM as the starting point for accessing data, so we will follow this assumption. 

There are four basic approaches — popup windows, IFRAME embedding, vendor-built interfaces, and custom-built interfaces. The first two use the actual interface of the EDRMS software and so can appear ‘out of place’, while the last two provide an interface that appears more ‘native’ to the CRM system.

The risk of re-using the EDRMS interface is that users can generally navigate outside of the file area reserved for use with that particular case. This requires additional user training. On the other hand, users can easily leverage the full functionality of the system within the interface.

Conversely, the drawback of the native interfaces (whether vendor-provided or custom) is that they tend to only provide the most essential functionality required to store and retrieve documents. However, the experience can be tightly controlled and requires less understanding of the fact that two systems are involved.

Platform limitations

One of the characteristics of case management is a high volume, flat structure.  In other words, having thousands or millions of case records which are relatively independent of each other.  However, EDRMS systems work best in hierarchical arrangements, with no more than several hundred folders or documents at any particular level of the hierarchy.  A naive mapping of cases to a flat structure in the EDRMS is likely to cause severe performance problems.

Often this requires choices in filing arrangements to be made to avoiding overburdening the EDRMS database design.  For example, cases might be classified by year, by surname initial, or by case type.  This design may not be supported out of the box by the vendor systems and needs to be approached with care.

Metadata management

Enterprise search and document discovery is a high profile issue at the moment. The inquiry into the Fitzgerald files is just the latest example of where difficulties in locating documents, understanding their provenance, and controlling their access has had serious consequences. 

To maximise discoverability and appropriate handling, relevant metadata for the case needs to be stored against the case documents in the EDRMS. However, this introduces difficulties in synchronisation and governance and may require custom code to be written. A particularly sticky issue is permissions. Often vendors don’t apply the permissions on viewing a case to the documents stored in the case file. The governance approach to keeping these permissions in sync needs to be carefully considered.

Process automation

One of the most common uses of CRM systems is to standardise on a common process approach to case management. However, it is difficult to target objects outside the CRM using out of the box facilities. For example, if a draft document is authored and saved to the EDRMS, it would be ideal to track the workflow tasks relating to review and approval of that document within the CRM.

However, since the CRM and EDRMS have little infrastructure in common, it is difficult to ensure both systems remain fully aware of the status of this document.  As a simple example, if a CRM assigns a task to a user to approve a document, that approval in the EDRMS won’t automatically close the CRM task since the CRM doesn’t “know” about the approval in the EDRMS. 

To avoid this cross-system state management problem, often a third-party workflow tool based on Windows Workflow Foundation (WWF) or Business Process Execution Language (BPEL) is deployed. However, this introduces a whole extra layer in administrative and licensing costs which should not be underestimated.

By no means should these issues discourage your organisation from pursuing the goal of integrated CRM and EDRMS systems. The productivity and governance benefits can be substantial.  However, it is a process that requires careful thought and testing at volume to ensure that the integration will scale to the production requirements of your organisation.  You would be wise to ensure that the cost-benefit payoff of your customisations can be justified.

Stephen Bounds is an Information and Knowledge Management Specialist with a wide gamut of experience across the government and private sectors.  As founding director of knowquestion Pty Ltd, Stephen provides strategic thought leadership in the development and implementation of modern information systems. Contact him at sb@knowquestion.com.au