First, why we do what we do at Vubble
Too much content exists online. We are drowning in a sea of information, misinformation and disinformation, an indecipherable blend of fact and opinion. Too little of the information we access is reliable, well-sourced, and relevant to us. Vubble creates solutions for this new information economy. We boost and strengthen the capacity of organizations that create and distribute evidence-based quality information.
Our computer vision challenge
This problem of too much information online is especially true for video - the dominant medium for information consumption online. This creates a significant technical challenge - understanding what’s happening in a video.
Automatic video categorization is a hard and active challenge in the computer vision research community. Scenes from the same video category are recorded at different lighting conditions, in different environments, with various backgrounds, from different viewpoints, or at various temporal or spatial resolutions. On the other hand, sometimes video sequences from different categories may be very similar and hard to distinguish from each other. Each of these challenges raise the level of complexity faced by the research and development community in designing and building automated systems to enable video categorization.
Currently, companies across industry sectors use data farms, usually offshore centres, with minimally-paid people to tag and label videos, often with violent and offensive content. This current solution fails in correctly identifying ambiguous information. For example: is a particular video providing political information? Is it humorous? Is it propaganda? Is the information credible? Current solutions also fails in correctly identifying contextual information - i.e. is a video of a breastfeeding mother pornography or maternal health advice?
Through an ongoing multi-year collaboration with Seneca College, Vubble has developed and implemented a video categorizer system, which we call the Vubble Video Categorizer. Customized for video curators, the Categorizer automates the data labelling of ambiguous and contextual information based on both visual and audio cues. Vubble’s solution can categorize ambiguous and contextual information, and it does that at scale.
That’s where 5G comes in.
Vubble’s 5G experiments
5G networks allow high-quality broadcast and reliable transmission of massive amounts of data. This has the potential to improve the precision and speed of Vubble’s automatic categorization pipelines. Vubble's technical team has optimized the video Categorizer system for the 5G testbed in order to run a series of pre-recorded and live video categorization tests. These tests were funded in part by Ontario Centre of Innovation (OCI) and supported by the Communitech ENCQOR team. They include:
Test 1 - In this experiment, we installed the Vubble Categorizer system on a personal computer and connected the computer to the 5G network. Then we put the Categorizer to work, automating the data labelling of several hundred videos we downloaded from YouTube.
The objective of this test was to provide a baseline for future tests. The 5G network allowed the processing of multiple videos simultaneously for a period of two hours. Video categorization requests ran in intervals from two to 20 seconds. This performance speed is similar to that of physical infrastructure, like a cable connection. One limiting factor was the performance of the computer.
This test provided valuable performance data on the simultaneous download (parallelization) of videos through a 5G network. It also helped the Vubble team identify areas of the Categorizer system that needed improvement, specifically, parts of the code that needed to be rewritten to avoid a situation where the system ran out of RAM memory because of the increased download speed. Finally, the test helped to identify the best balance of the Categorizer system components (the optimum number of downloaders, audio categorizers and video categorizers).
Test 2 - In this experiment, a client accessed the Vubble Categorizer system in the cloud and requested remote categorizations of hundreds of videos.
The objective of this test was to gather comparison data for Test 1 and to identify the limit of parallel categorizations of the Categorizer system in the cloud infrastructure, when compared with a local deployment of the Categorizer system code on a laptop.
Again the 5G network allowed a high communication speed - sending video categorization requests and receiving the output quickly. With this test, the Vubble team identified the limit of the number of requests that could be managed by the Vubble Categorizer system. This limit was caused by the longer processing time of the component that produced video transcriptions. The team solved this problem by improving the number of replicas of the API and the video Categorizer pods to factors of 3 and 8 respectively. This reduced wait time, improving clients' experience of the system.
Test 3 - In this experiment, a client with video footage on a device pushed the video to the cloud-based Vubble storage and then requested categorization of video.
The objective of this test was to simulate a situation in which the client records videos locally, (e.g. video on surveillance cameras or motion-detection cameras), and then pushes the video to the cloud-based Vubble storage prior to categorization.
The 5G performance for pushing videos to Vubble storage was comparable to the speed of upload experiences using a cable network. This experiment revealed that it is possible to provide an online categorization service for clients connected to a 5G network. The bottleneck problem frequently encountered when uploading huge files in 3G and 4G networks disappeared on the 5G network.
Test 4 - In this experiment, a client on the 5G network streamed video from Vubble storage.
The objective of this test was to compare the performance of video streaming for videos stored in Vubble storage to other storage services such as YouTube’s. The test ran one-hour videos stored both on YouTube and on Vubble to evaluate the quality of transmission on a 5G network.
The test result proved that it is possible for Vubble to offer private video streaming to clients connected to 5G networks.
Test 5 - In this experiment, a client streamed a video to Vubble’s storage for live categorization.
In this final test, the Vubble team evaluated the categorization of live video. The idea was to evaluate the performance of streaming a local camera to the Vubble cloud infrastructure. In a second step, these streamed frames were processed and categorized.
The 5G network allowed the streaming of live video without any issues. However, the test showed that the Vubble categorization system did not have the capacity to receive a transmission of more than six frames per second. The speed of the 5G network was more than the Vubble system was capable of processing live.
The team concluded that the Vubble Video Categorizer needs improvements to be able to categorize live video on a 5G network.
- The Vubble Categorizer system can be deployed on a stand-alone device and connected to the 5G network.
- A client on the 5G network can request remote categorization of videos from the cloud-based Vubble categorization system without having to deploy the system on their device.
- Vubble can store clients’ videos and provide video streaming of those videos.
- Given the speed of the 5G network, a client can push videos to Vubble’s storage for categorization without a local deployment of the Vubble Categorizer on their device.
- A client can send live footage from remote cameras to the Vubble system for categorization in the cloud. However, the Vubble categorization system code must be modified to address a problem where the code kept reading and writing files, slowing performance.
What's next for the Vubble Categorizer?
This 5G test allowed the Vubble team to create two new solutions: The first was the creation of a stand-alone Categorizer system that can be deployed in the cloud and receive videos from cameras and then categorize those videos without a client installing the Vubble Categorizer system on his or her local device. The second was the creation of Vubble video storage so that Vubble can host and stream videos without the need for third-party services such as YouTube.
The performance on the 5G network demonstrated that it is possible to process more videos than previously expected. With this is mind, the Vubble research team will:
- Scale-up the current deployment of the Categorizer system with the addition of more replicas of the audio Categorizer and the addition of more nodes (servers) to the current cluster.
- Test the use of GPU-capable machines to improve the speed of video data labelling.
- Improve the code of the camera live-stream to increase the number of frames available for processing in the Vubble Categorizer.
Photo by Umberto on Unsplash