Annotating pre-recorded and live video streams provide accurate training data to power machine learning models for a California based data analytics company
Millions of video frames accurately labeled with objects of different categories with their movement trajectories.
Client successfully deployed a dashboard of directional traffic volumes that provided live data and alerts based on historical volumes for city traffic management.
Client was able to prove the capabilities of their video analytics solution for both live as well as historical videos.
Client Profile.
An integrated data analytics company from San José’s, CA providing solutions to government agencies in the energy, water and communications arena. To assist Department of Civil and Environmental Engineering, they were developing a machine learning solution to predict traffic congestions, prevent traffic collisions and improve road planning by better estimating transit demand.
Business Need.
Use the widely deployed traffic camera feeds, pre-recorded and live video streams, to produce directional counts of traffic users to understand study and assess traffic congestion and plan lane movement accordingly. The client was looking to identify, categorize and label thousands of vehicles based on turning movement, direction of approach and mode; as a preparatory step towards its video annotation activities.
They partnered with HitechDigital in the technology development venture by studying and labeling vehicle images in pre-defined criteria. The labeled images were further to be used to train client’s machine learning models and evaluate if video analytics can detect queues, track stationary vehicles, and tabulate vehicle counts from live video feeds.
Challenges.
Having data annotators well versed with standard automobile classification.
Deploying annotators across shifts, expert at capturing moving vehicles on the computer screen from live video streams.
Labeling vehicle images from traffic videos that become unsteady due to severe illuminations, orientation variations, complicated backgrounds and obstructions.
Counting pedestrians and bicycles from in-pavement loops not differentiated for directions of movement (going straight and turning right)
Hiring and training data specialists experienced at annotating images from videos captured in:
Day and night, different lighting conditions, and weather conditions.
Different traffic volumes; busy times as well as light traffic times.
Identifying a targeted vehicle in different cameras with disjoint views, aka vehicle re-identification.
Solution.
Identified, categorized and labelled hundreds of thousands of vehicle and pedestrian images, from both live as well as historical traffic video feeds, from across major cities in US and Canada. Human annotated images were further used as extensive training data to be fed into machine learning models.
Approach.
After initial assessment of vehicle and pedestrian images in pre-recorded traffic videos and URLs to live video streams, team of data annotators at HitechDigital documented a workflow to expedite image labeling. The images were annotated through a five-step process:
Data Sources /Data Inputs
Videos footages were received in two forms: pre-recorded videos as well as URLs to live video streams.
Pre-recorded videos were uploaded to OneDrive City wise.
To access live video streams, human annotators securely accessed the VPN using pre-provided credentials to log into the City’s traffic camera network.
Labeling and Segmentation
Label vehicles by its category, model name, color of the vehicle and direction of the vehicle.
Classify objects in 14 categories including:
Car, SUV, small truck, medium truck, large truck, pedestrian, bus, van, group of people, bicycle, motorcycle, traffic signal-green, traffic signal yellow, and traffic signal-red.
Classify, tag and segment vehicles:
By turning movement (through, left or right).
By direction of approach (northbound, southbound, etc.).
Vehicles that appeared in one-third of the distant scenes, with less visible characteristics, were defined as small targets and not labeled.
Vehicles with more than 1/2 of view obstructed were categorized as occluded and were not labeled.
Fuzzy or low confidence footage due to poor weather or lighting conditions etc., were routed to be re-validated by the client.
Annotation technique used
Line-based technique was used to uniquely count vehicles and other objects. The line-counter captured the “state transition” of the lines, i.e., the state of the line changes from unoccupied to occupied, and then back to unoccupied, before it increases the count for said line. It allowed us to obtain count of vehicles in individual lanes.
The red line in the figure represents the demarcation line of the labeled area, and the small vehicle located outside the red line is not labelled.
Audit and review
As part of a multi-layered quality check process, more than 10% images annotated were audited by a team of senior annotators.
Resultant data was used to validate anomalies in the labeled images submitted by operational level annotators in the process.
Our data annotators used the deviations for QC and training purposes.
Erroneously labeled or missed out images were re-labeled appropriately.
Deliverables
City wise upload of labeled images on OneDrive to serve as machine training data.
Granular reports on number and type of vehicles annotated.
Value added analytical insights by factoring in numerous metrics such as the day, time and type of vehicle, count of type of vehicles that passed against total number of vehicles that passed a particular lane etc.
Technology Used
Secure Login to City’s traffic camera network through VPN using pre-provided credentials
Ask the Experts.
Schedule a free 30 minute consultation with our experts. We’d love to talk to you!
