Remote Inspection and Grading Pilot Project

Background

The ongoing global shortage of veterinary personal and other specialists required for food inspection tasks in meat plants has increased pressure to use existing staff more efficiently. Leveraging computer vision and imaging technologies could enable some staff to work remotely. This would benefit the industry at large, as well as reduce inspection costs and travel times for smaller and more remote facilities who face particularly acute workforce shortages. It may also be possible for a single remote specialist to serve multiple facilities.

This research, conducted with the assistance of the Canadian Food Inspection Agency (CFIA) and industry partners, was designed to provide an initial perspective on the feasibility of remote postmortem inspection of for organs on the red offal line (head, heart, lungs, kidneys, and liver) and beef carcasses on the held rail.

Objectives

Phase I: Procure and instal cameras on the red offal line and held rail in a beef processing facility and initial evaluation of image quality and capture parameters.

Phase II: Explore the use of images to remotely detect defects in the liver, lungs, kidney, heart and head. Develop software to store and view images and document defects, which could be used to identify whether individual organs are suitable for human consumption or rendering, and track these numbers for individual producer shipments and the total slaughter period (shift, day, week, etc.).

Phase III: Explore the use of images to remotely detect carcass defects on the held rail. Develop software to store and view images and document defects.

A key aspect of phase III involved combining the images of the whole carcass and specific areas of interest alongside images of it’s organs. The combined image set would then be remotely viewed by a veterinarian to determine if a decision could be made regarding its disposition (condemnation status). The findings of the remote personnel could then be compared to those of onsite CFIA personnel who would be considered the gold standard.

What They Did

Two imaging stations were set up to capture front and back views of the offal, including LED lighting units, RGB cameras with enclosures and sensors for tracking offal locations, and green backdrops. Computer vision algorithms were developed to identify organ types from images obtained from both cameras and to determine if the organ was marked as condemned. The ability to detect two types of ink (red and blue) as well as the presence or absence of a paper tag with an “x” were evaluated. A report to share the percentage of organs condemned from each producer was designed and implemented for the participating beef processing facility.

Similar equipment was used for the held carcass rail station, including a touchscreen PC and enclosure, an enclosed HD RGB stationary camera and enclosed handheld camera, LED lighting units, and a Samsung Galaxy smartphone. The performance of the enclosed handheld camera and smartphone cameras were compared. The smartphone’s superior ability to adapt to dynamic lighting conditions in the plant and variable focal ranges for both wide and detailed photography led to its use for the main rounds of image collection. An associated mobile app was developed to organize the collected images into a standard format established with the CFIA. Images were collected through coordination between the onsite veterinarian and the smartphone equipped technician for each held carcass. The images were then supplied to three remote CFIA Operational Specialist Unit (OSU) veterinarians, and independent remote diagnoses and dispositions were conducted by each of these veterinarians. A consensus between the individual remote veterinarians, and between the remote veterinarians and the onsite veterinarian was used to evaluate the feasibility of remote defect detection in organs and carcasses.

What They Learned

Access to computer vision and imaging technicians from Germany were limited by COVID-19 border restrictions. Access to local CFIA personnel were limited by their reallocation to the Avian Influenza outbreak.

During the completion of the objectives outlined in phase I, the meat plant environment proved challenging for precision imaging. A concerted effort went into mitigating the issues of blood spatter, corrosive chemicals, steam, vibration, and limited space. Ultimately, all the cameras were made operational, integrated into the plant IT system and images from the same organ taken from the two offal stations were synchronized using specialized software and sensors.

In phase II, the computer vision algorithms were unable detect the red ink used by CFIA to indicate “condemned” status. Instead, algorithms were developed to detect an “x” inscribed on paper tags that other establishments use to indicate condemned status. These computer vision algorithms were successfully implemented. The detection of blue ink to illustrate OTM status was possible using computer vision algorithms, and image data was successfully used to generate reports of usable organ quantities by producer and by total slaughter to help inform plant procurement personnel.

In phase III, two or more of the three remote veterinarians agreed on the diagnosis for 91% of the held carcass image sets (out of image sets from 92 carcasses), and agreement between all three remote veterinarians on the held carcass disposition were obtained for 82% of the sets. Confidence in the disposition (diagnosis) based on the remote images was measured on a scale of 1 (least) to 5 (most confident), and a mean confidence of 2.53 ± 0.78.

Challenges identified during phase III included the need for a larger dataset, as well as better standardization in the completeness and presentation of the carcass images sets. While pathology was somewhat easy to identify, its extent (severity) was not as clear cut. Pathology may be more confidently assessed if regional lymph nodes were included in the assessment.

A pilot trial to evaluate the ability of a robot arm to aim a camera at a precise point of interest on the carcass image (using a mouse pointer) was also undertaken to explore the potential of providing image data to remote veterinarians without the need for an onsite operator.

What it means

The preliminary results obtained from this pilot exploration suggest that remote approaches to veterinary inspection have potential, with the 82% agreement obtained between OSU veterinarians and over 90% agreement between OSU and the on-site veterinarian being particularly encouraging. In addition to further development and validation of the technology, implementing it would require legislative change in both domestic and international markets. However, the increasing adoption of telemedicine and telepathology in human and veterinary medicine bodes well for these changes.

The application of imaging technology for remote inspection of offal also showed promise, with the detection of offal type and condemnation status through computer vision providing valuable insight to plant procurement and possibly allowing for remote review by feedlot veterinarians. With further research into carcass manipulation methods, the camera mounted robot arm could allow for a remote veterinarian to examine tissues and provide a potential mitigation to the issue of personnel shortages.