We Create A Lot of Data, But We Don’t Put It All Together

In the past decade, the quantity of data created and transmitted has grown at an exponential rate. The consulting firm PwC estimates the volume of data grew from 4.4 trillion terabytes of data in 2013 to 44 trillion terabytes of data in 2020. The number of data source systems, types of delivery systems, and the amount and type of payload content sent across these sources continues to swell. Organizations create and exchange documents and data on new and more powerful line of business systems, ubiquitous desktop software, web applications, mobile devices, and increasingly digitized fax and scan solutions.

As a knowledge economy, the strategies and business practices of our most advanced organizations rely on this ever-growing mountain of data. However, regardless of their sophistication, all enterprises continue to transact data in three basic formats: electronic copy, soft copy, and hard copy. Several steps in a business transaction often include a mix of documents and data in more than one medium, distributed into and out of systems multiple times along the way. For data driven business operations to remain efficient, we need reliable methods to process this valuable currency regardless of the initial or eventual data structure.

We’re Missing A Lot of Data That Goes to Users at Their Desk

One of the most significant vulnerabilities in our ability to manage data today stems from the fact that large volumes of data and documents are delivered directly to the knowledge worker at their desktop. Workers exchange “soft” copy documents through email, electronic content management folders, and other document exchange systems. Are either of these scenarios familiar?

Scenario A: All day the staff receives documents by email or opens them from a line of business system. To move the transaction along, users open the document and review the content. They may need to edit the document, add data, or capture some information. Users print, scan, or save their work documents, possibly entering some of the data to another system, and then save it to a shared folder. They or someone else will then need to search for and retrieve the document later.

Scenario B: An electronic document management system automatically identifies and processes documents, but not everything makes it through on the first pass. Exceptions go into an “unprocessed” folder or are misrouted to an incorrect folder. Additional documents stack up in the folder until someone is assigned to review, sort, and fix the data or documents in the folder. From time-to-time, we find a common problem and make an update to reduce known exceptions.

Both scenarios are typical today and both create unnecessary costs, reduce quality, and present strategic risks to the organization. In Scenario A, the business shifts the function of the knowledge worker away from value-added tasks and towards document and data processing. In Scenario B, automation created efficiency on the front end at the expense of quality and workload on the back end. One element these situations have in common is the need to combine electronic, soft, and hard copy data and documents.

Combine and Automate

Most companies have an excellent opportunity to integrate soft copy documents with the data managed in hard copy and electronic copy formats. The first step is not news to anyone: improve data and document capture. Hard copy documents captured through office and mobile devices do not have to remain static. Advanced scan and e-fax solutions should support document recognition and capture of critical discrete reference data. Also, do not underestimate the value of “freeing” the electronic data to feed less automated sources. Expect to use data from hard copy documents and electronic systems to trigger or supplement the soft copy document workflow. In addition to forms, use software with machine learning to recognize and capture recurrent user-level documents with a focus on source type and format, reference data, and data triggers.

The second step is to expand expectations about what it means to “process” the data and documents. The primary automated trigger for soft copy document identification and workflow is a watch folder service to initiate and route work. After receipt of a structured or unstructured document, operational transactions frequently involve three additional activities. First, users and systems evaluate the documents. This often involves some type of judgement or decision about the content. Based on their review, staff may need to manipulate the documents by adding or deleting data, combining documents, or rearranging them into a new package. Finally, processes involve distribution to one or more destination: tasks along the workflow, transactional systems, analytics and reporting systems, document folders, and data storage. Routing rules distribute the data in accordance with business requirements and store documents to the appropriate system or folder.

Step Three: Plan for and Automate the Exceptions

Finally, it is imperative to manage the exceptions process. During standard operations, the system will not always handle all documents. Poor document quality, an unfamiliar document or data element, or an unspecified destination workflow can cause the automated model to reject the activity. Use software to flag these to an exception handling queue. Exception handling addresses two needs: to assure the operational completion of flagged documents; and, to prioritize documents and workflows to update the data, document, or workflow library. Here are three workflow options for document exception management to consider.

The most automated approach is to establish a “learning” service with manual or automated “ticketing” and prioritization of documents and workflows flagged as exceptions. The learning service updates the document management library and routes unresolved issues. At the other end of the spectrum, the organization may elect to have end-users address the operational documents through a prompt and respond model. This approach alerts users to exceptions on the spot or requires the business to assign exception handling specialists who access folders to perform resolution activities. The middle ground is to assign an internal administrator to review the exceptions queue. The internal administrator selects the appropriate resources to address the operational requirements and can either “train” the document workflow, assign the work to an internal team, or assign exceptions to the workflow service through a ticketing system.

Automating the integration of soft copy documents management with electronic and hard copy data and workflow yields a high return on investment. Better data capture and distribution from the desktop means better data retrieval and usage. Leveraging existing technology investments and operational activity cuts redundancy and errors. The result is more effective use of staff time, at a lower cost. This frees up knowledge workers to focus on decisions and to manage transactions that allow businesses to thrive.