Amazon’s on-premises device for vision apps, AWS Panorama Appliance, launches publicly

Amazon today announced the general availability of the AWS (Amazon Web Services) Panorama Appliance, a device that allows customers to use existing on-premises cameras and analyze video feeds with AI. Ostensibly designed for use cases like quality checks and supply chain monitoring, Amazon says that the Panorama Appliance is already being used by companies including Accenture, Deloitte, and Sony.

“Customers in industrial, hospitality, logistics, retail, and other industries want to use computer vision to make decisions faster and optimize their operations. These organizations typically have cameras installed onsite to support their businesses, but they often resort to manual processes like watching video feeds in real time to extract value from their network of cameras, which is tedious, expensive, and difficult to scale,” Amazon wrote in a press release. “Most customers are stuck using slow, expensive, error-prone, or manual processes for visual monitoring and inspection tasks that do not scale and can lead to missed defects or operational inefficiencies.”

By contrast, the Panorama Appliance connects to a local network to perform computer vision processing at the edge, Amazon says. Integrated with Amazon SageMaker — Amazon’s service for building machine learning models — the Panorama Appliance can be updated and deployed with new computer vision models. Companies that opt not to create their own models can choose from solutions offered by Deloitte, TaskWatch, Vistry, Sony, Accenture, and other Amazon partners.

To date, customers have developed models running on the Panorama Appliance for manufacturing, construction, hospitality, and retail, Amazon says. Some are analyzing retail foot traffic to inform store layouts and displays, while others are identifying peak times in stores to pinpoint where staff might be needed.

The Cincinnati/Northern Kentucky International Airport in Hebron, Kentucky, is using the Panorama Appliance to monitor congestion across airport traffic lanes. With the help of Deloitte, The Vancouver Fraser Port Authority has applied the Panorama Appliance to track containers throughout its facilities. And Tyson has built models on the device to count packaged products on lines for quality assurance.

“Organizations across all industries like construction, hospitality, industrial, logistics, retail, transportation, and more are always keen to improve their operations and reduce costs. Computer vision offers a valuable opportunity to achieve these goals, but companies are often inhibited by a range of factors including the complexity of the technology, limited internet connectivity, latency, and inadequacy of existing hardware,” VP of Amazon machine learning at AWS Swami Sivasubramanian said in a statement. “We built the Panorama Appliance to help remove these barriers so our customers can take advantage of existing on-premises cameras and accelerate inspection tasks, reduce operational complexity, and improve consumer experiences through computer vision.”

Privacy implications

Since its unveiling at Amazon’s re:Invent 2020 conference in December, experts have raised concerns about how the Panorama Appliance could be misused. While the purported goal is “optimization,” the device could be coopted for other, less humanitarian intents, like allowing managers to chastise employees in the name of productivity.

In the promotional material for the Panorama Appliance, Fender says it uses the product to “track how long it takes for an associate to complete each task in the assembly of a guitar.” Each state has its own surveillance laws, but most give wide discretion to employers so long as any equipment they use to track employees is plainly visible. There’s no federal legislation that explicitly prohibits companies from monitoring staff during the workday.

Bias could also arise from the computer vision models deployed to the Panorama Appliance if the models aren’t trained on sufficiently diverse data. A study conducted by researchers at the University of Virginia found that two prominent research-image collections displayed gender bias in their depiction of sports and other activities, showing images of shopping linked to women while associating things like coaching with men. Even differences in the sun path between the northern and southern hemispheres and variations in background scenery can affect model accuracy, as can the varying specifications of camera models like resolution and aspect ratio.

Recent history is filled with examples of the consequences of training computer vision models on biased datasets, like virtual backgrounds and automatic photo-cropping tools that disfavor darker-skinned people. Back in 2015, a software engineer pointed out that the image recognition algorithms in Google Photos were labeling his Black friends as “gorillas.” And the nonprofit AlgorithmWatch has shown that Google’s Cloud Vision API at one time automatically labeled thermometers held by a Black person as “guns” while labeling thermometers held by a light-skinned person as “electronic devices.”

Amazon has pitched — and employed — surveillance technologies before. The company’s Rekognition software sparked protests and pushback, which led to a moratorium on the use of the technology. And Amazon’s notorious “Time Off Task” system dings warehouse employees for spending too much time away from the work they’re assigned to perform, like scanning barcodes or sorting products into bins.

An Amazon spokesperson recently told the BBC that the Panorama Appliance was “designed to improve industrial operations and workplace safety” and that how it is used is up to customers. “For example, AWS Panorama does not include any pre-packaged facial recognition capabilities,” the spokesperson said. All its machine learning functions can happen on the device, they added, “and [relevant data] never has to leave the customer’s facility.”

The Panorama Appliance is now available for sale through Amazon’s AWS Elemental service in the U.S., Canada, U.K., and E.U.