Automating Capture to Transform Unstructured Data Processes

Registration is free. Login or register to view/download this content.


Director / owner, Lithe IT Ltd
Anthony is a BCS Chartered IT professional, with over 15 years IT experience, working mostly in Financial Services and the UK Public sector. Having worked as an IT Director, he is passionate about helping businesses successfully use technology to increase agility. He formed Lithe IT to help further this aim.

We live in a fast paced world where consumers demand high quality and rapid services. When we buy an item on-line we expect a confirmation email in seconds or we start to be concerned that something has gone wrong.

The desire for rapid and meaningful response is also growing in more complex transactions such as a mortgage or insurance application, even though these often require documents to be supplied that contain the data needed to decide on the claim or application. Consumers can now take and send pictures on mobile devices or scan documents from home, and are attracted by the simplicity and availability of 24/7 e-business.

When transforming these types of business process a capture strategy is needed to enable documents to be delivered and processed in a fast and low cost means, with a consistent high level of quality.

Within this article I contrast processes that are initiated with structured and unstructured data, review the challenges and costs associated with unstructured data, and suggest how automating the extraction of unstructured data can deliver competitive advantage.

What is Data Capture and Why is it Important?

The execution of each distinct electronic business process is entirely dependent on its specific data values. Capture is the method of gathering the input data that each process instance uses. There are three common approaches to capturing seed data for processes. The data is either:

  • Imported from a line of business system either directly or via an automated transfer mechanism,
  • Manually entered by a customer via an application such as a website or an app,
  • Extracted from semi-structured documents such as invoices or completely unstructured content such as emails, letters or unknown forms.

The first two are examples of highly automated processes launched by structured data at the point the data is delivered; where manual intervention is only required if the process requires it for fulfilment. Consider the example of buying a dress online. The shopper selects the specific dress they want and provide additional information such as colour and size. Data capture is complete the moment the customer elects to buy the item, with address and delivery details subsequently supplied. Manual effort is only needed to box and post the correct dress.

These processes result in a consistent and rapid execution which provides a predictable and repeatable customer experience. The process typically has a low cost to execute due to minimal need for checking or manual hand-offs.

The third is an example of processes using unstructured data in the form of documents. These include traditional paper based applications and letters, but are increasingly dealing with documents  being sent electronically as pictures from mobile phones, or attachments within emails. Modern technology allows content to arrive from customers almost immediately, but that only increases the expectation from the customer for a speedy response.

Unstructured data is not in an accessible format that can be immediately used to launch and add to automated business processes. A strategy is needed to extract the data and transform it into a useful format.

Approaches to Extracting Unstructured Data

Achieving customer satisfaction is increasingly important and difficult to achieve. Satisfied customers are 50% more likely to listen to a sales offer, while dissatisfied customers — if they even choose to listen — are twice as likely as satisfied customers to decline an offer after listening to it.[i] Customers do not expect poorer or slower services just because documents are provided as process inputs.

There are a number of ways to transform the data. A technology-free approach is possible where printed documents are given to workers to read, extract and re-key data or launch actions based on the content. This traditional approach requires knowledgeable workers who are trusted to take the correct action. There is a need for quality assurance, measurement and training to execute these steps in a consistent way. This dependence on manual resource makes these processes slow to complete and typically are less reliable than processes seeded with structured data. It is difficult to see how this approach can deliver a competitive value. The alternative is to invest in technology.

Technology for Extracting Unstructured Data

Extraction technology is now a remarkably mature and reliable choice that is relatively inexpensive to implement and benefit from. These technologies use:

  • ICR (Intelligent Character Recognition) to translate handwritten words,
  • OCR (Optical Character Recognition) to extract machine characters such as those in this article and,
  • IMR (Intelligent Mark Recognition) to be able to determine if a check box has been selected on a form.

Capture technology is most successful and easiest to implement when data is extracted from structured forms where a template is known. But it can also be highly effective when dealing with semi or unstructured documents too, but this does take more effort.

With an automated capture process, it is possible to limit human touches to only verify and validate that the data automatically extracted from the documents is accurate. These tasks tend to be completed on an exception basis only and require minimal effort from an end user. In many instances it is also possible for someone to complete the task that has no expertise in the subject matter as they are only confirming characters have been extracted correctly.

Leading capture technology solutions also allow for techniques such as double blind verification, where two people confirm the same data, separate from each other, to maximise the likelihood that extracted seed data is accurate.  Although this adds extra time and cost, it is typically much faster than the traditional method.

This technology typically interfaces well with leading BPMS, allowing an end to end capability for automating business processes.


When an effective capture strategy is implemented that minimises the time and cost of transforming unstructured into structured data there are many benefits to gain. These include:

  • Significantly reducing the time taken to respond to customers who are required to provide documents as process inputs. Use cases include insurance claims and mortgage requests, where drop off rates between completing the structured data application form and then submitting paper documents remains high using the traditional model of “snail mail”.
  • Removing the need for documents to be sent via the post, by uploading via an app or website instead reduces storage costs as there is no need to manage physical content.
  • Shrinking the cost of processing as an effective capture strategy will reduce the human interactions required to extract and validate the data. Reduced manual involvement also reduces the likelihood of error which will please the customer.
  • The maturity of the technology allows for fast implementation, allowing for rapid ROI.

The net effect is a faster, reliable and consistent process that can transform the interaction with customers and deliver a tangible competitive advantage.


Within the document I have highlighted problems that unstructured data creates when attempting to improve processes. A data capture strategy is required that quickly and reliably transforms this data so that it can be processes in a structured and repeatable way. A technology based capture solution will provide the greatest benefit and should therefore be given due consideration when improving these types of process.


[i] Financial Services Customer Experience Survey, Maritz, 2008. (quoted in “Insurance Providers: Improving Customer Retention through the Contact Center” by Jacada, 2008).


Similar Resources

Featured Certificate: BPM Specialist

Everyone starts here.

You're looking for a way to improve your process improvement skills, but you're not sure where to start.

Earning your Business Process Management Specialist (BPMS) Certificate will give you the competitive advantage you need in today's world. Our courses help you deliver faster and makes projects easier.

Your skills will include building hierarchical process models, using tools to analyze and assess process performance, defining critical process metrics, using best practice principles to redesign processes, developing process improvement project plans, building a center of excellence, and establishing process governance.

The BPMS Certificate is the perfect way to show employers that you are serious about business process management. With in-depth knowledge of process improvement and management, you'll be able to take your business career to the next level.

Learn more about the BPM Specialist Certificate





  • Business Process Management Specialist
  • Earning your Business Process Management Specialist (BPMS) Certificate will provide you with a distinct competitive advantage in today’s rapidly evolving business landscape. With in-depth knowledge of process improvement and management, you’ll be able to take your business career to the next level.
  • BPM Professional Certificate
    Business Process Management Professional
  • Earning your Business Process Management Professional (BPMP) Certificate will elevate your expertise and professional standing in the field of business process management. Our BPMP Certificate is a tangible symbol of your achievement, demonstrating your in-depth knowledge of process improvement and management.


BPM Certification

  • Make the most of your hard-earned skills. Earn the respect of your peers and superiors with Business Process Management Certification from the industry's top BPM educational organization.




  • Operational Excellence Specialist
  • Earning your Operational Excellence Specialist Certificate will provide you with a distinct advantage in driving organizational excellence and achieving sustainable improvements in performance.


OpEx Professional Certificate

  • Operational Excellence Professional
  • Earn your Operational Excellence Professional Certificate and gain a competitive edge in driving organizational excellence and achieving sustainable improvements in performance.



  • Agile BPM Specialist
  • Earn your Agile BPM Specialist Certificate and gain a competitive edge in driving business process management (BPM) with agile methodologies. You’ll gain a strong understanding of how to apply agile principles and concepts to business process management initiatives.  

Business Architecture



  • Business Architecture Specialist
  • The Business Architecture Specialist (BAIS) Certificate is proof that you’ve begun your business architecture journey by committing to the industry’s most meaningful and credible business architecture training program.

  • Business Architecture Professional
  • When you earn your Business Architecture Professional (BAIP) Certificate, you will be able to design and implement a governance structure for your organization, develop and optimize business processes, and manage business information effectively.

BA CertificationCertification

  • Make the most of your hard-earned skills. Earn the respect of your peers and superiors with Business Architecture Certification from the industry's top BPM educational organization.




  • Digital Transformation Specialist
  • Earning your Digital Transformation Specialist Certificate will provide you with a distinct advantage in today’s rapidly evolving business landscape. 


  • Digital Transformation Professional
  • The Digital Transformation Professional Certificate is the first program in the industry to cover all the key pillars of Digital Transformation holistically with practical recommendations and exercises.



  • Agile Business Analysis Specialist
  • Earning your Agile Business Analysis Specialist Certificate will provide you with a distinct advantage in the world of agile software development.


  • DAS Certificate
  • Decision Automation Specialist
  • Earning your Decision Automation Certificate will empower you to excel in the dynamic field of automated decision-making, where data-driven insights are pivotal to driving business innovation and efficiency.