You’ve probably seen this scenario play out on a police procedural show on television: A crime has been committed and officers are tasked with looking through security footage to see if any of it was caught on camera. On TV, they can cut away to commercial and have the answer back as soon as they return.
In real life, however, analyzing huge quantities of video data is a task that’s rarely accomplished effectively by human operators. There’s just too much data to sift through, and the cost for the man hours required is too high.
But that problem is being overcome with machine learning and video analytics. Video analytics is the process of extracting information, meaning, and insights from video footage. And video analytics can do everything that image analytics can do, plus a bit more. Whereas image analytics looks at a still image – be it a photograph or a medical scan – and seeks to find patterns and anomalies, or identify faces in pictures, video analytics also can measure and track behavior.
Traditionally, video data was only really gathered on closed-circuit TV for security purposes to monitor retail or business premises for theft, malicious damage, or employee wrongdoing. The purpose of the video footage was to protect the business and provide evidence if something happened. If nothing happened, the recordings would be erased so the tape or digital hard drive could be re-used over and over again. All that data wasn’t saved because there was too much of it and there was no way to use it.
But all that has changed. Increases in storage capability and new analytics techniques mean that all that video footage is now very useful. And we have an awful lot of it. Nearly every person on the street has a cell phone equipped with a video camera. Three hundred hours of video are uploaded to YouTube every minute. And even inexpensive security cameras now come with facial-recognition software.
How Video Analytics is Being Used
Video analytics can be used for identification (face recognition), behavior analysis, and situational awareness. Businesses use video analytics if they want to know more about who is visiting their store or premises, and what those people are doing when they get there. Facial recognition can be used to help maintain security, but it also can be used to find out more about a business’ customers.
And because video data is dynamic, not static like image data, you can also use it to monitor your customer’s behavior and learn more about how they react to offers, etc. For example, you can collect data from different closed-circuit TV cameras in a retail environment and analyze the footage to see how your customers behave and how they move through the store. This data can help you see how many people stop at a particular product display or offer, how long they engage with it, and whether or not it is working and converting into sales.
Video analytics are also being used in other fields, including law enforcement, security, and even marketing. Body cams for police officers are a great example of this. The New York City police department alone generates more than a million hours of video footage per week; that’s a lot of data that could be useful to the department. But much of this data is never reviewed, even when it could provide investigative leads.
Proper video analytics could add rich tagging and indexing to the video to aid in future searches. It also can search video from certain time periods and for individuals with certain characteristics to develop leads; and even can be taught to recognize patterns and predict vulnerabilities.
Airports, stadiums, and other major hubs for transportation and gatherings are using video analytics to identify hotspots and help alleviate congestion and lines. Airports in Asia already are testing video analytics that allow them to monitor lines and congestion and deploy employees and messaging to redirect passengers to areas with shorter lines.
Video-based Predictive Analytics
Just recently, MIT released a report on a new algorithm they created in which the computer can predict human actions and interactions from what the people are doing in the seconds before. The current algorithm was shown 600 hours of YouTube videos and asked to predict whether the actors’ next action would be a handshake, a hug, a high-five or a kiss. But the potential for this type of technology is vast. Perhaps computers could eventually be taught to predict when someone in a crowd will be injured, or even when a crime is about to take place.
In addition, these kinds of video analytics will be necessary for robots that will interact with humans. We humans do a lot of predicting in our everyday lives, and robots will need to predict and react similarly if they want to interact with us seamlessly.
And for marketers, Veenome has a YouTube analytics tool that analyzes the content of videos to help advertisers choose which videos on which to place their advertisements. Brian Fitzgerald, president of Evolve, told Adweek, “It is critical that publishers get out of a pure impression-based economy and offer brands accountable media.”
A customer of mine is Prozone, a leader in sports analytics. The company collects data gathered from a number of cameras placed around a football or hockey pitch, for example, to track players. The system creates over 10 data points per second for each player on the field and allows coaches to analyze all activities, on and off the ball, to answer questions like miles covered by each player, successful and unsuccessful passes or tackles for each player, and even which players best attract opposition players away and thereby create new spaces and attacking opportunities.
Video analytics can also assist decision making in complex, highly fluid situations such as aviation, air traffic control, ship navigation, power plant operation, and emergency services. Using technology and video footage to alert personnel to changes or anomalies can help to save lives and prevent crime.
The Video Gray Area
This type of analytics, where the collection and review of the data can all happen without a person’s permission, is currently a gray area in current law. But there will come a time when it won’t be. Companies shouldn’t be afraid of diving into video analytics, but should do so with a mindset to deliver best practices, treat all data with respect and privacy, and ensure that if they are using customers’ video data, they are making sure the outcome is ethical and adds value to the customer, not just the business. The trick is to make extra sure you stay on the right side of the law and use the data wisely and ethically.