Intelligent Video Redaction is a Reality Today

Tom Scanland, SVP Sales & Marketing, redactX

Privacy protection has been a hot topic the last few years.  Laws and mandates have been put in placeto ensure that the privacy of individuals is protected in a number of circumstances.  For instance, in the healthcare industry it is required that patient information be protected under the Health Insurance Portability and Accountability Act (HIPAA).  This law is complex and applies to all forms of data (structured data, documents, videos, images, audio, etc.)  For instance, a video recording of a surgical operation may be created and shared for training purposes, or perhaps submitted for doctor certification purposes.  When that happens, the identity of the patient, doctors and nurses in the operating room should not be disclosed.  This means creating a copy of the original video where faces, text writings on whiteboards, name badges, etc. are blurred.  Anything that could be used to identify a person needs to be blurred out as per HIPAA.  In another example, if the doctor holds a telehealth session with a patient, and that session’s video is recorded and stored, then any subsequent disclosure should blur-out patient identifying information (their face, the faces of others in the background, spoken information such as the patient’s name, chart number, address, etc.).  As the healthcare industry embraces the use of video in its operations, the ability to comply and demonstrate compliance with HIPAA requirements have become a significant consideration in terms of cost and time.

Similar requirements apply to other industries.  For instance, in the case of law enforcement, body-worn cameras record officers’ work throughout the day.  Often these videos are used by both prosecutors and defense attorneys in court, and if so the privacy of bystanders, informers, officers, etc. needs to be protected.  Faces, license plates, spoken words, etc. that are not strictly relevant to the case need to be redacted out before the material can be shared with the courts, juries, and the public.  Typically, police are allowed only a handful of days to respond to requests for evidence, so quick and accurate de-identification is important.  Similarly, Government organizations are required to respond to a growing number of FOIA requests.  This entails a manual redaction process that takes time and particular care/approvals when sensitive information is involved.  The task only grows in complexity when media files are involved.

The need to protect privacy extends to other environments such as newscasts and the reality-TV industry, where faces of minors or bystanders cannot be shown without specific authorization obtained from the affected individual.

It’s clear that requirements about personal privacy protection are here to stay and will only grow going forward.  Meeting these requirements when dealing with images and videos is a complex and time-consuming matter.

For the most part, video editing software has been a manual process, drawing boxes around the things we want to obfuscate.  Unfortunately, there are many frames in a video, so lots of boxes must be drawn to cover the video sequence.  This is extremely time consuming and labor intensive.  To further complicate matters, there is a significant learning curve associated with operating most video editing software packages.  In other words, one needs specialists to do this work and not everyone can adjust to it easily or quickly.  Some level of automation is available in the more sophisticated video editing systems.  Some features try to automatically track a group of pixels from one frame to the next until it disappears or morphs into something visually different.  The basic idea is that the user marks the start and endpoint of a rectangle on the screen draws and the software tries to fill in the gap.  Unfortunately, video tracking is a complex technical subject, and it seldom works well, in particular when the sequence is longer than just a few frames, or when the characteristics of the video are not ideal (brightness, contrast, resolution, texture, etc.).  It is common for tracked rectangles to “drift” and blur something other than what the user intended.    

A cottage industry of vendors that perform video-redaction-as-a-service has sprung up over the years.  These suppliers use highly skilled staff and modern video editing software to redact files according to user specifications.  As one can imagine the customer pays dearly to have files processed quickly.  Charges reflect the amount of work that an editor has to perform to redact the file.  This means not only the duration of the video, but also the number of things that must be redacted within it.  (a video with many faces will cost more to process than a video with a single face).  Other considerations that come into play when deciding whether to outsource the include the issue of control, as it may not be possible to send the video out due to secrecy or confidentiality requirements.

Some dedicated software solutions have emerged but they have limited functionality and are primarily restricted to use cases that involve police body-worn cameras.  Camera vendors have bundled these software applications with their hardware offerings so the software can only process videos from their specific cameras.  One finds that the accuracy of these applications is quite limited, so the user still spends significant time doing manual edits.  The level of functionality is also quite limited.  For example, audio redaction is purely manual.  Text redaction is not available.  Objects redaction is limited to less than a handful of items (cars, license plates), and so on.

 At redactX, we have created the industry's most comprehensive and accurate video redaction software package 

At redactX we have created the most comprehensive and accurate video redaction software package in the industry.  The basic design premise of the software was that it should be enterprise class, accommodate many users, be agnostic about the source of the videos, accommodate many formats, and provide comprehensive redaction capabilities in each of four areas:  faces, text, objects, audio.  In addition, we wanted the software to provide true workflow automation and an unparalleled level of user control in so far as what gets automatically redacted and how the output should look.

In each redaction-capability area, redactX pushes the state of the art by a considerable margin:

● Faces – redactX can detect faces at any angle (frontal, side, back, top, upside down or at any rotated angle in the z-plane).  It detects partially occluded faces, as well as faces with Covid-19 style face masks.  It is robust to a range of lighting conditions and face sizes.

● Text strings – redactX can detect and recognize text in over 90 languages.  It is able to do partial text string matches, as the user defines.

● Objects – redactX incorporates a growing library of standard object classes that it can recognize and redact.  For non-deformable objects, redactX also lets users define and name their own specific objects.

● Audio – redactX lets users redact all audio or only specific segments of the audio stream.

Each of these functions has deep underpinnings in terms of the level of image analysis performed.  The combination and integration in a single package greatly simplifies what users can do with just a few clicks of the mouse.  For instance, users can specify to redact all or only specific faces, all or only specific text strings, all or only specific objects, all or only specific words in the audio.  The inverse of each can also be accomplished (ex:  blur all faces except for these two).

Users can define and manage re-usable redaction templates each defining a set of criteria that can be applied to multiple redaction jobs.  Users can create and manage recurring tasks (for example, monitor the contents of a directory and automatically process newly added files).  Users can choose to receive an e-mail notification of job completion, so they can do other work in the meantime.  Users can view in a side-by-side display both the original and the redacted versions of a video and if satisfied download the output directly from the browser UI.  They can also make changes and resubmit a job as they inspect the results. 

Most of the features in redactX arise from a desire of automating as much as possible and make it so that anyone, without special skills, can create their desired output with a much lower level of effort than alternatives (think from hours and days down to minutes).  Based on customer response, it appears that redactX has succeeded in this endeavor, with more improvements and advancements to come as the product evolves.