Surface inspection on a mobile arm

This is a simulation of a surface-inspection cell we are developing for a prospective customer: a camera head with its own raking lights, on a UR10e arm, carried round the part on an RB-Theron mobile base.

+3 mins reading, +7 images

The part here is a 2 × 1 m solid oak table, and it is only a stand-in. The customer's part is something else, and we are keeping it out of the pictures to protect their IP.

The stand-in: a 2 × 1 m solid oak table on X legs, the legs set in so the camera can reach under the edge.

The head over the top of the table: one of 319 exposures.

The job

What carries over from the customer's part is the job.

The entire top needs to be scanned, as well as the sides and a border around the bottom.

Another thing about the customer is that they are a large manufacturer with many other products. In this case a mobile robot was selected to be able to tour the factory and inspect various items in queue.

In this case, each move is planned with OMPL and checked for collisions against the table and the robot itself. The simulation environment is NVIDIA Isaac Sim, and the camera is modelled on a standard 12 MP machine-vision camera.

Aligning after each move, and reading the label

The cart only has to park roughly. Each time it stops, the head first looks down to calculate alignment.

The first thing the job reads is the serial-number label, a QR code under the table, so every image that follows is filed against that part.

The arrival shot: the table's edge and the codes on the mat, taken each time the cart stops.

Reporting a finding

We have planted six defects of three types: scratch, dent and discoloration. In this case we are cheating for the detection part as the detection models will likely be sourced from a 3rd party.

A frame from the inspection camera: a dent in the oak, circled where it was found.

The fold-out map: every surface unfolded, one tile per camera frame. Green: scanned and clear; red: a finding; yellow ring: the defect.

How the lights show a scratch

The head lights each tile five ways: straight on, and raking from each side at about 13°. A scratch catches the raking light differently from each side, while the wood's colour and grain stay the same, so comparing the frames pulls the shape out and leaves the grain behind.

The scratch under the five lights: straight on, then raking from each side. 80 × 80 mm.

Below, what that comparison brings out: the difference between opposing lights, how much each pixel changes as the light goes round, the slope and height recovered from all five frames (photometric stereo), the colour on its own, and the frame against a clean reference. A dent shows up the same way; a stain shows in none of the shape maps, only against the reference.

From the five frames: the difference between opposing lights (X, Y), relief, slope, height, colour only, and against a clean reference.

Dents work the same way and discoloration is currently based on an acceptable colour palette - which is a problem for natural woods. However, it's not a problem for the actual target customer - who is using a different material entirely.

Until next time.