- In an image annotation project conducted for a Swiss analytics company, the aim was to create labeled image datasets to train the client’s model in analyzing food waste.
- In addition to categorizing food waste, we enriched the images with information like day, time, and other metrics.
- Hitech’s multi-step image annotation process enhanced the accuracy of the client’s ML model in analyzing the food wastage patterns.
Table of Contents
Image annotation plays a crucial role in training and improving the accuracy and performance of AI and machine learning models. In this article, we will detail the steps of a real-life image annotation for food waste detection project we executed for a Swiss company specializing in waste assessment and food waste analysis.
We will cover each stage of preparing and labeling image datasets, from project assessment and tool selection to finalizing metrics, training annotators, implementation, quality control and continuous improvement.
Through this customer journey of image annotation for food waste detection project, we provide insights into the key elements that are foundational to each of our successful image annotation projects and help us deliver positive outcomes.
Navigating a customer’s journey through an image annotation project.
The client
Our client, a Swiss analytics company, focuses on reducing food waste through innovative solutions that enable restaurants and hotels to instantly analyze food waste. To achieve this, they developed powerful machine learning models tailored to the needs of the global hospitality industry.
The business need
Thousands of photos of food garbage, restaurant waste, kitchen waste, discarded, and leftover food, captured in different settings and containers needed to be labeled and readied as training data for food wastage analysis models.
Beyond accurate recognition, tagging, and labeling of food waste, the training data needed enrichment with additional information to help analyze and extract relevant insights.
The client is onboarded
Project assessment and documentation
We began by holding meetings and discussions to understand the client’s goals and decide how this project differed from other image annotation projects we had previously handled.
Based on these discussions, our image annotation specialists defined and documented the Scope of Work (SOW) which required us to:
- Map food product images against the client’s existing food item repository
- Tag and segment images based on defined standards, and
- Validate images with uncertain annotations
Defining benchmarks and metrics
We then worked with the client to define tasks, key project metrics, deliverables, reporting frequency, delivery schedules, communication and security protocols. A comprehensive project plan with detailed timelines, benchmarks, budgets, milestones, and individual roles and responsibilities was created.
We also worked out additional enrichment data that was needed, file formats, and storage. Responsibilities were assigned and points of contact (POCs) were designated to facilitate seamless communication between the client and the annotation team.
Based on initial photos of discarded food and relevant information shared by the client, our experts identified information gaps and potential challenges, prompting further collaboration with the client to enhance context and refine workflows.
To ensure consistency and quality across all annotation tasks, we developed and implemented Standard Operating Procedures (SOPs). These SOPs provided clear instructions on labeling and categorizing food waste images, defined quality benchmarks, and outlined error-handling procedures.
We also included detailed guidelines for tool usage and structured training modules to ensure proficiency and adherence to standards among annotators. Regular updates were made to the SOPs based on feedback and new insights, ensuring their continued relevance and effectiveness throughout the project lifecycle.
With these SOPs in place, we maintained high-quality standards and consistency, ensuring the project met all established metrics and delivered the desired outcomes for the client.
Assessing project challenges
Some project challenges assessed and documented by our team included:
- Images contained a mix of food items jumbled together in trays, and individual food items had to be recognized accurately.
- Discarded food had to be distinguished from non-food debris and then categorized.
- Annotators needed to recognize and categorize the food waste according to states of decay and contamination.
- Day, time, and other parameters had to be recorded with the images, for the ML models to learn how to properly analyze food wastage patterns.
- Annotators needed special training to help them recognize diverse cuisines, raw and decayed food elements, etc.
- Annotators needed knowledge of lesser-known European food, including regional varieties and food industry terminology.
Project implementation phase: Image annotation starts
We worked on the client’s platform and used tools suggested by the client for identification, categorization, and labeling of the food images.
Preparatory phase before commencement of the image annotation process
The images were annotated through a three-step process:
-
Labeling against existing repository:
Image annotation specialists labeled the food garbage images using an existing database of food item names presented in a drop-down menu within the client’s portal. This involved cross-referencing visual data of food being discarded and comparing it with the predefined list of food items.
The annotators reviewed the images from various angles to ensure the accurate identification of food objects, matching them with the available food item names.
-
Tagging and segmentation:
The annotation process included tagging and segmentation of images according to the client’s specified product list and the quality standards set by restaurants and hotels.
This step involved categorizing and outlining food items within the images, ensuring that they conformed to the defined standards. This included identifying and highlighting areas of interest within an image, applying the chosen annotation type (e.g., bounding boxes or polygons), and assigning appropriate labels or class names to those annotated objects or regions.
-
Handling low confidence images:
Images that exhibited uncertainty or “fuzziness” in terms of food object identification were segregated and handled separately.
These images, which raised doubts or had annotations with low confidence levels, were flagged for validation by the client. This step aimed to ensure the highest level of accuracy and reliability in the annotated data.
QC and validation
Our quality control processes were rigorously executed against pre-defined benchmarks and metrics, ensuring that all deliverables met the predefined quality standards.
We deployed advanced tools and methodologies and a multifaceted approach that enabled precise monitoring throughout the project lifecycle.
- First, a parallel labeling activity was conducted by the client’s in-house team to cross-verify annotations and ensure consistency.
- Any discrepancies or deviations in the labeled data were identified, serving as feedback for further QC, training, and refinement.
- Mislabeled images were identified and re-labeled following feedback.
- Additionally, our annotators adhered to rigorous standards and guidelines throughout the annotation process, ensuring that each food image was accurately labeled against the client’s master repository.
- Any doubts or uncertainties in image identification were proactively addressed through client validation before final labeling.
Client review, feedback, and monitoring
Customer feedback involved regular updates and clarifications. This ensured the project stayed on course, minimizing any information gaps.
Our image annotation team worked on client software and the client’s in-house team worked parallel with us for continual validation and course rectification which helped ongoing process improvement and training.
Client feedback received during our scheduled governance calls also ensured alignment with client expectations.
Project monitoring and coordination
At Hitech, our Project Monitoring strategy uses KPIs, metrics, and tracking systems, designed to offer a multidimensional view of a project’s health, progress, and performance. This approach allows us to anticipate challenges and manage risks effectively.
- Key Performance Indicators (KPIs) – We established KPIs that aligned with the specific goals and objectives of our clients. These KPIs provided quantifiable targets to ensuring the project team always focused efforts on client-defined success metrics. For example, KPIs included the accuracy rate of annotations, the volume of images processed per week, and adherence to delivery schedules.
- Metrics – We utilized a broad range of metrics to gain deeper insights into aspects such as team performance, resource utilization, and process efficiency. This granular view allowed us to identify areas where micro-adjustments enhanced overall project performance. Metrics such as time spent per annotation, error rates, and feedback turnaround times were critical in our evaluations.
- Tracking – Real-time tracking tools allowed us to monitor the progress of each task and phase of the project, enabling us to identify any deviations from the plan promptly. This helped us to take corrective actions and keep the project on track. Tracking tools provided visibility into task completion rates, upcoming deadlines, and resource allocation.
Through this integrated approach of regular meetings, rigorous KPI monitoring, detailed metrics analysis, and real-time tracking, we ensured that the project stayed aligned with its goals, timelines, and quality standards. Our ability to adapt and respond to the dynamic needs of the project was key to delivering exceptional results for our client.
Final deliverables
As final deliverables, we ensured:
- Labelled and categorized food images were automatically uploaded to the client’s portal for machine training.
- Reports and dashboards provided required insights into the quantity and categories of annotated images.
- Images were enriched with various metrics like the day and time of food wastage, food types wasted, and the ratio of waste to prepared food, that were needed for advanced analysis.
Conclusion
Our accurate and thorough annotation of food waste images helped the client’s ML models achieve their objectives.
Annotating thousands of complex images expedited training of the food waste analysis model, enabling quicker insights. With efficient workflows, and by diligently working together in this complex scenario, we could help our client realize the potential of visual data. Through this project, our team also successfully demonstrated its ability to learn and adapt.
Turn images into insights with high-quality annotation
From categorization to enrichment, we build powerful training datasets for AI.
