Handwritten Form Interpretation System
Machine Learning for Digital Inspections
A Global Power Research Institute
In Partnership with Black Ice LLC
The Problem
Regular inspection of key components in nuclear power plants has been a critical part of their operation since at least the 1960s. Before digital media, inspection reports were handwritten on standard forms. Attempting to analyze trends that started in this pre-digital era was near impossible.

The Solution
Leveraging multiple machine-learning techniques, we developed an application that automatically identifies the correct handwritten forms, isolates the key fields of interest, and interprets the handwriting in those fields, ultimately storing the data in a database consistent with later digital inspections.
Research and Analysis
This project initially required breaking down the problem into components and researching the latest tools and techniques available in the rapidly evolving field of machine learning to accomplish each component. For example, first, the system had to identify particular forms within many page PDFs. Next, it had to straighten out the form image in order to normalize all field positions. After that, it had to identify field positions of interest, excise images from those locations, and interpret the handwritten text in them.

As a research project, all work and deliverables were subject to review and presentation to our client's members for future funding. 
Key Takeaways 
  • Early decomposition of complex problems reveals areas for focused research
  • The science of interpreting handwritten text is rapidly evolving beyond the character-based approach that was state-of-the-art only a few years ago
Get in touch
Have ideas or questions? Let’s talk. We’d love to help.
Contact YoJonesy