Food Image Recognition and Dietary Assessment

Food image recognition and dietary assessment combine advances in computer vision, machine learning and nutrition science to automate the identification of food items and the estimation of their nutritional content. By analysing photographs taken with smartphones or wearable cameras, these systems aim to overcome the limitations of self‐reported dietary records—such as underreporting and recall bias—by providing objective, real‐time measurements of food type, portion size and nutrient composition. Central techniques include convolutional neural networks for food classification, depth sensing and view synthesis for three‐dimensional reconstruction and traditional image processing for segmentation. These approaches have been applied across diverse settings—from smartphone apps that help individuals manage chronic conditions to population studies in low‐resource environments. The integration of large, well‐annotated food image databases and continuous online learning has accelerated model accuracy, while ongoing work addresses challenges in cultural food diversity, mixed‐dish recognition and explainable AI. The global significance of this field spans public health surveillance, personalised nutrition and the design of interventions to curb diet‐related diseases.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Food Image Recognition and Dietary Assessment publication trend

The graph below shows the total number of articles in food image recognition and dietary assessment across all publications each year (not limited to Nature Index journals).

Technical terms

Convolutional Neural Network (CNN): A class of deep learning models that applies convolutional layers to detect visual patterns and features in images, commonly used for food classification.

Portion Size Estimation: The process of quantifying the volume or weight of food items captured in images to estimate nutrient intake.

Egocentric Vision: A first‐person perspective imaging approach, typically using wearable cameras to capture daily food intake with minimal user intervention.

Depth Map: A representation of distance information for each pixel in an image, used to reconstruct three‐dimensional structure of food items.

View Synthesis: A technique that generates novel viewpoints of an object from a single image to aid in three‐dimensional reconstruction and volume estimation.

References

1. Deep learning in food category recognition. Information Fusion (2023).
2. A Review of Image-Based Food Recognition and Volume Estimation Artificial Intelligence Systems. IEEE Reviews in Biomedical Engineering (2024).
3. AI-enabled wearable cameras for assisting dietary assessment in African populations. npj Digital Medicine (2024).
4. “Snap-n-Eat”. Journal of Diabetes Science and Technology (2015).
5. Food Volume Estimation Based on Deep Learning View Synthesis from a Single Depth Map. Nutrients (2018).