Personalized Dietary Self-Management using Mobile Vision-based Assistance

Daily appropriate decision making on nutrition requires application of knowledge where it matters, and being adjusted to the individual requirements. We present a highly personalized mobile application that assists the user in appropriate food choices during grocery shopping, while simultaneously incorporating a personalized dietary recommender system. The application can be used in video based augmented reality mode, where a computer vision algorithm recognizes presented food items and thus replaces tedious search within the food database. The recognition system employs a shallow Convolutional Neural Network (CNN) based classifier running at 10 fps. An innovative user study demonstrates the high usability and user experience of the application. The vision classifier is evaluated on a newly introduced reference image database containing 81 grocery foods (vegetables, fruits).