FingerVision is an optical tactile sensor , which provides the RGB image of the object within the fingers in addition to the contact forces from the optical markers.
Since the sensor captures the image of the background also in addition to the object, we need to seperate out the object and the background.
to update later
A deep learning approach is used to segment the objects and the background. Training data is collected, manually labelled and a U-Net based network is trained. The training acheives good accuracy in a couple of tens of minutes. Transfer learning methods could be used to quickly adapt to new environments.
It works, as expected.
The left side shows the raw sensor image and the right one is the segmented image output.