|
Hundreds of materials, such as drugs, explosives, makeup, food additives, are in the form of powder. Recognizing such powders is important for security checks, criminal identification, drug control, and quality assessment. However, powder recognition has drawn little attention in the computer vision community. Powders are hard to distinguish: they are amorphous, appear matte, have little color or texture variation and blend with surfaces they are deposited on in complex ways. To address these challenges, we present the first comprehensive dataset and approach for powder recognition using multi-spectral imaging. By using Shortwave Infrared (SWIR) multi-spectral imaging together with visible light (RGB) and Near Infrared (NIR), powders can be discriminated with reasonable accuracy. We present a method to select discriminative spectral bands to significantly reduce acquisition time while improving recognition accuracy. We propose a blending model to synthesize images of powders of various thickness deposited on a wide range of surfaces. Incorporating band selection and image synthesis, we conduct fine-grained recognition of 100 powders on complex backgrounds, and achieve 60%~70% accuracy on recognition with known powder location, and over 40% mean IoU without known location. |
| Publications
"Multispectral Imaging for Fine-Grained Recognition of Powders on Complex Backgrounds"
Tiancheng Zhi, Bernardo R. Pires, Martial Hebert and Srinivasa G. Narasimhan
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019.
[PDF] [Supp] [Oral] [Dataset] [Poster] [Code]
|
Dataset
SWIRPowder: A 400-1700nm Multispectral Dataset with 100 Powders on Complex Backgrounds
[Real Data]:
Download: Patches and Scenes as described in the paper (background images without powder are also available)
[Synthetic Data]:
Download: Synthetic powder and background images
|
Illustrations
 |
Image Acquisition System. RGB, NIR, and SWIR cameras are co-located using beamsplitters. The target is imaged through a 45 mirror.
|
 |
Hundred powders. Thick RGB patches, NIR patches and normalized SWIR signatures are shown.
|
 |
Patches example. Thin powders are put on the same black background material. Patches are manually cropped for thick powders, thin powders, bare background, common materials, and white patch.
|
 |
Scenes example. The ground truth mask is obtained by background subtraction and manual annotation.
|
 |
Theoretical spectral transmittance of 4 selected bands (different colors). NNCV has a good band coverage.
|
 |
Examples of (a) thick powder RGB, (b) thin powder RGB, (c) SWIR signature, and (d) kappa signature. The two signatures of many powders are negatively correlated.
|
 |
Powder against background data synthesis. (a)(c) are RGB regions from NYUDv2 [34] and (b)(d) are their segmentation labels. We obtain the shading (e) via intrinsic image decomposition, and the background image (f) by filling segments in (b) with patches from Patch-common. We obtain the powder thickness map (g) via smooth shading estimation, and the thick powder image (h) by filling segments in (d) with patches from Patch-thick, only for pixels with positive thicknesses. The final image (i) is obtained by blending background (f) and thick powder (h) using Beer-Lambert Blending with thickness (g), and applying shading (e). The ground truth (j) is obtained by thresholding thickness (g).
|
|
Results
|
Acknowledgements
This work was funded in parts by an NSF grant CNS-1446601 and by ChemImage Corporation. We thank the Chemimage Corporation for the DPCF-SWIR camera used in this work. |
