Add ability detect checkboxes or radio buttons to OCR and Handwritten text
I've successfully been able to use vision to extract handwritten text out of fixed forms by knowing the coordinates of each form field. However many of my forms have checkboxes and/or radio buttons that the users will be filling in with pen. It doesn't seem that vision has a way to detect this type of content.
Tables and checkboxes and radio buttongs are available in Form Recognizer product from the OCR team – learn more at https://azure.microsoft.com/en-us/services/cognitive-services/form-recognizer/
More specifically, check out the Layout API quickstart at https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/quickstarts/python-layout?tabs=v2-0
Steven Williams commented
I've previously mapped the coordinates of each of my form fields.
So each form field I already know the Xposition(topLeft), Yposition(topLeft), Width, & Height properties.
I then use the following linq query against the HandwritingRecognitionOperationResult object to find all the words within that rectangle.
string FoundWords = hwResults.RecognitionResult.Lines.Where(L => L.BoundingBox >= (MinX)
&& L.BoundingBox <= (MaxX)
&& L.BoundingBox >= (MinY)
&& L.BoundingBox <= (MaxY)
).SelectMany(w => w.Words).OrderBy(o => o.BoundingBox).Select(t => t.Text).ToArray();
How are you able to specify a field to the vision API? I'm assuming the form is an image and the coordinates are pixels coordinates?
Thanks in advance!