Automatic Recognition of Food and Assessment of Diet
This project is to apply the multimedia technology to the study of obesity. We are developing technologies that capture and process
multimedia data that diet information. The ongoing research includes:
Multimedia Information Processing and Summarization
We are developing technologies to facilitate a human observer effectively analyzing these data. More specifically, we are developing
algorithms and software modules for: 1) automatic dietary activity recognition from the chronically recorded video data, 2) data management
and organization, and 3) user interfaces in the graphical, thumbnail, and storyboard forms for rapid data analysis in obesity research and clinical
study. For example, an American, on average, spends one and a half hours on eating and drinking activities. We would like to able to detect
these activities and summarize them into five minutes of the data. We will further provide analytic tools for the medical professionals for dietary
assessment.
Privacy Protection in Video
In this project, the video camera is configured to record the same scene as the wearerĄ¯s perspective, which would raise privacy concerns.
Although privacy protection in video is a poorly defined problem, we adopt the common measure in practice that protects the identity of people
and content of objects from being recognized during the video playback. We are developing techniques to address privacy protection problems
for both the subject who wears the device and other people. For example, the device may capture other people in the scene and patientĄ¯s own
computer screen when he/she is using a computer. We will mask all human faces and computer screens in the captured video using object
detection techniques.
Fast Food Database
Building a food database is a starting point for developing and testing food recognition programs for obesity study. Collaborating with Intel
Pittsburgh research lab, we have built a fast food dataset,
PFID (Pittsburgh Fast-food Image Dataset). The dataset contains a total of 4,545 still
images, 606 stereo pairs, 303 3600 videos for structure from motion, and 27 privacy-preserving videos of eating events of volunteers.
The dataset is public available at http://pfid.intel-research.net/
Related Publications:
-
M. Chen, K. Dhingra, W. Wu, L. Yang, R. Sukthankar, J. Yang, PFID:Pittsburgh Fast-food Image Dataset,
Proceedings of International Conference on Image Processing (ICIP 2009).
- W. Wu and J. Yang, Fast Food Recognition from Videos of Eating for Calorie Estimation, Proceedings
of IEEE International Conference on Multimedia & Expo (ICME), 2009.
- Q. Wang, J. Yang, Drinking Activity Analysis from Fast Food Eating Video Using Generative Models,
ACM Int. Conf. on Multimedia (SIGMM) - Workshop on Cooking and Eating Activity, 2009
- L. Yang, N. Zheng, H. Cheng, J. D. Fernstrom, M. Sun, J. Yang, Automatic Dietary Assessment from Fast
Food Categorization, Proceedings of the IEEE 34th Annual Northeast Bioengineering Conference, 2008.
- S. Greiner and J. Yang, Privacy Protection in an Electronic Chronicle System, Proceedings of the
IEEE 34th Annual Northeast Bioengineering Conference, 2008.
- L. Yang, J. Yang, N. Zheng, H. Cheng, Layered Object Categorization, Proceedings of 19th International
Conference on Pattern Recognition, 2008.
- D. Chen, Y. Chang, R. Yan, and J. Yang, Tools for Protecting the Privacy of Specific Individuals in Video,
EURASIP Journal on Advances in Signal Processing, Vol. 2007, doi:10.1155/2007/75427, 9 pages, 2007.
- N. Yao, R. J. Sclabassi, Q. Liu, J.Yang, J. D. Fernstrom, M. H. Fernstrom, and M. Sun, A Video Processing
Approach to the Study of Obesity, Proceedings of 2007 IEEE International Conference on Multimedia and
Expo (ICME 2007), pp.1019-1022, 2007.
- Y. Chang, R. Yan, D. Chen, J. Yang, People Identification with Limited Labels in Privacy-Protected Video,
Proceedings of IEEE International Conference on Multimedia & Expo 2006 (ICME 2006), pp. 1005-1008, 2006.
Back to the previous page
Last updated January, 2010