How Draganfly brought a ‘pandemic drone’ to the U.S.

Like the rest of the world, Canadian drone maker Draganfly has been anxiously watching the spread of the novel coronavirus. And when COVID-19 cases started springing up across Washington nursing homes in mid-February, the team began brainstorming. By March, Draganfly had licensed the machine vision and AI tech needed to offer social distancing and health monitoring services from the air. Demand to test the technology was “insatiable,” not just from government and law enforcement, but also from health care, airline, cruise, hospitality, theme park, and other commercial industries. By mid-April, the police department in Westport, Connecticut had a pilot underway, the first of its kind in the U.S. Moreover, Draganfly had three to seven more U.S. pilots planned. By April 23, the Westport pilot was dead.

But the story doesn’t end there. We spoke with Draganfly CEO Cameron Chell before and after the abrupt termination of the Westport pilot. The drone company has two more pilots scheduled to start in less than two weeks. Chell says Draganfly has been “inundated” with requests from other jurisdictions, while the numbers on the private side “are even more prolific.” Indeed, the next couple of U.S. pilots will be in the private sector. One is drone-based, and the other is facility-based. Additional U.S. public sector pilots will start “relatively soon.” As for Canada, Chell said “a couple of institutions” are also interested, particularly in the transportation industry.

As federal and local governments wrestle with the coronavirus pandemic — from tracking the spread of COVID-19 to gauging when to lift restrictions — everyone is taking a closer look at autonomous technologies like drones and robots. The public and private sectors are desperate for technology that can help limit human contact and provide early detection data on the implementation and effectiveness of measures like social distancing. Any business that relies on human interaction, whether with customers or between employees, will be hungry for data to understand health trends. And drones could play a critical role in detecting and tracking outbreaks, safeguarding both public health and business operations.

Deploying drones since ’98

Unlike most drone companies, Draganfly has decades of experience. It was founded in 1998, and Chell prides himself on leading “the oldest commercial drone manufacturer in the world.” The Canadian company has some 25 employees and is based in Saskatoon, Saskatchewan, with offices in Vancouver; Los Angeles; and Raleigh, North Carolina. Until this month, Draganfly was arguably best known for developing the first drone credited with saving a human life, in 2013.

Cheap consumer drones have become popular in recent years, thanks to market leaders DJI and Parrot. But none of Draganfly’s four revenue streams is consumer-related. The first is contract engineering (primarily for tier one U.S.-based military contractors). The second is original systems manufacturing, meaning building drones that are fixed-winged and can perform vertical takeoff and horizontal landing, along with ground robots and other specialized products. That business line encompasses software design and development for other companies’ drones and includes the health measurement system that has been all over the news. The third line of business is managed services, when Draganfly essentially becomes the data collection and drone services arm of a business. The fourth line, which is still emerging, covers data analytics and management.

While Draganfly does not build consumer drones itself, it has designed various payloads, gimbal attachments, and software integrations for its customers that do. The company prefers highly specialized work for the likes of U.S. marshals and border patrol. Forget cheap drones — think batteries that operate in colder weather, specialized sensors, and a North American supply chain. In the past five years or so, Draganfly’s work has started to skew toward the public safety arena.

Draganfly customers “generally have a higher performance requirement,” Chell told VentureBeat. “They have some specialized needs that our engineering is attuned to. Also, they tend to not necessarily buy as much foreign product. The buyers in that area are a bit more conscientious, certainly on a military level, of security concerns and potential foreign parts and things like that. But even as that trickles down into public safety and law enforcement, they tend to have a bit more of a skew toward a North American or a NATO-based solution. So we’ve naturally ended up migrating that way.”

Vital Intelligence Project

Some 12 weeks ago, when “things got scary in Washington with nursing homes,” the Draganfly team was trying to figure out how drones could help. But they wanted to do more than just use drones to yell at people from the skies.

“We were like, ‘Put a loudspeaker on a drone?’ Big deal. Really? That’s not innovative,” Chell declared. “We were thinking if this hits, we need to be able to provide more value than very typical use cases like that.”

The company realized it did not have the AI chops to pull off what it really wanted to do. So it started talking to its partners.

“We went looking for it,” Chell said. “We were thinking ‘Oh, thermal cameras!’ Probably every person in the world has thought about thermal cameras. And we debunked the use of those very quickly. Thermal doesn’t measure core temperature, which is what is required to understand if there’s a potential fever present. We were like, ‘We [have] got to find something much different.’ And lo and behold, and very fortunately [for] us, it was the University of Southern Australia (UniSA), which is more coincidental than anything. But given the fact that they bought their first drone from us in 1999 — the trust relationship [was there, and] we could move very quickly.”

The Canadian company researched, built use cases, ran tests, and in the space of a few weeks had signed a deal. Draganfly paid $1.5 million to license a health and respiratory monitoring platform, the Vital Intelligence Project, developed in a collaboration between UniSA and the Australian Department of Defence Science and Technology Group. Draganfly would commercialize and deploy the computer vision technology.

The Vital Intelligence Project can help estimate the distances between people, but it can also monitor temperatures, heart rates, and respiratory rates of individuals in crowds and workforces. Draganfly envisioned the tech being deployed by airlines and cruise ships; for potential at-risk groups, like seniors in care facilities; and in convention centers; at border crossings; and within critical infrastructure facilities.

“We licensed it for camera networks and for drones,” Chell explained. UniSA built the core technology — “the specific machine vision and AI in a non-productized form.” Draganfly simply happened to have the public research university in its Rolodex. The company then developed the productized form, including camera networks and drones.

“That includes everything from going out and doing the policy development work through to what the GUI needs to look like,” Chell said. “Both software and mechanical engineering to provide stabilization on the drones so that they’re optimized to collect this data. That piece of IP, go to market, and actual commercialization piece is all in-house [at] Draganfly. But the hardcore research and IP behind the machine vision and AI up until this point has been [at] the University of South Australia. A bunch of test data and learning that we’re bringing in — we’re codeveloping that portion of the IP now, with them. However, they’re the hardcore Ph.D.s that are doing the AI work.”

Still, some repurposing was required, as the Vital Intelligence Project was not exactly being used to monitor groups of people.

“They were using it so that they can fly helicopters over … disaster relief zones and pick up the vital signs of survivors on the ground,” Chell said. “They could determine what resources they needed to apply where or the severity of survivors’ current situation, and did they need to get them right at that moment. They also ended up using it to monitor wildlife. You have a migration happening and you might have fires or drought. Wildlife officials need to see, ‘What is the health of the herd, and do we need to take any action?’ They also used it for prenatal babies, where they didn’t want a lot of people coming in and out of the ward because of potential introductions of infections, and also in that situation where probes and monitors being taped to babies don’t typically stay on or are uncomfortable in some way. Those are the journaled, peer-reviewed use cases that are out there.”

Draganfly and the university took the technology and adapted it for social distancing and health monitoring. To be clear, the Vital Intelligence Project had never been strapped to a drone and pointed at a crowd before the Westport pilot.

“The previous use case that would be most similar to this one was designed to be used in disaster relief areas to get the vital signs of survivors on the ground,” Chell said. “Those happen to be the same set of vital signs that we can now pick up anonymized in a crowd to determine if there’s infectious or respiratory challenges.”

Westport: Flatten the Curve Pilot Program

Draganfly started test flights in Westport, Connecticut to identify social distancing and detect symptoms. The city is in Fairfield County, adjacent to New York City and considered the epicenter for the spread of the coronavirus in Connecticut.

The three-phase pilot was supposed to validate the technology’s use and have officials develop public safety policy around it. The next step would have been to test those policies. The whole process was supposed to take some 60 days, during which Draganfly hoped to initiate additional pilots.

Phase one: Social distancing

Phase one was to test whether the technology could be an effective resource multiplier, letting officials cover more ground to see if social distancing is being effectively adhered to, for example. Instead of sending a few cruisers out and having officers walk around, they could put one camera up in the sky and assess where to apply resources.

“Social distancing, which we’ve shown on the videos, that’s actual visual data,” Chell said. “It gives the operator of the camera, typically the officer, the real-time data. They should make operational decisions at that point if they need to separate a crowd. Or everything is fine [and] they don’t need to go in and waste their time there.”

Westport First Selectman Jim Marpe called it the “Flatten the Curve Pilot Program.” It was supposed to help the community “practice safe social distancing, while identifying possible coronavirus and other life-threatening symptoms.” Police Chief Foti Koskinas said at the time: “Using drones remains a go-to technology for reaching remote areas with little to no manpower required. Because of this technology, our officers will have the information and quality data they need to make the best decision in any given situation.” The hope was to deploy at town and state-owned beaches, train stations, parks and recreation areas, and shopping centers. “It will not be used in individual private yards, nor does it employ facial recognition technology,” the police department said.

There was no health monitoring in phase one. Before the Westport pilot ended, Chell was already calling phase one a “success,” so we asked what exactly that meant. “The technology worked in a real-world environment,” Chell said. “So that was successful. We were able to get very good operational social distancing data. And the working relationship between the public safety officials and us was also a success.”

Until it wasn’t.

In announcing the end of the pilot just a couple of days later, Marpe said, “in our good faith effort to get ahead of the virus and potential need to manage and safely monitor crowds and social distancing in this environment, our announcement was perhaps misinterpreted, not well-received, and posed many additional questions. We heard and respect your concerns, and are therefore stepping back and reconsidering the full impact of the technology and its use in law enforcement protocol.” Koskinas added: “We thank Draganfly for offering the pilot program to Westport and sincerely hope to be included in future innovations once we are convinced the program is appropriate for Westport.”

When we spoke to Chell a few days later, he seemed to understand why the pilot had to end.

“The official pushback was around health monitoring, and the misunderstanding around how it works, what it does, and what it’s for,” he said. “And so at this point, Westport just feels politically that they just don’t want to move forward with the project. They had been extremely helpful. They provided us great insight, great policy framework, and all the rest of it, but they’re not going to move to go forward with us, at least not at this time, which is totally fine. There’s lots of people for us to move forward with, and [the people in Westport have] been totally professional and great to work with.”

No part of the Vital Intelligence Project employs facial recognition technology, Draganfly has consistently said. Still, we wondered if the resulting output from phase one could be repurposed to do so. Could someone take the video feed and run a facial recognition algorithm on top of it?

“No more than you could do that on a security camera system today,” Chell explained. “The requirements to run social distancing, in terms of resolution and stabilization in the video platform are minute compared to what you have to have in order to run the health measurement platform. So while today we can take existing security networks and do social distancing, you couldn’t take that same video feed and do heart rate, respiratory rate, or even often facial recognition stuff, which we don’t use.”

Phase two: Anonymized health measurement

The pilot never got to phase two, which was the anonymized health measurement. The plan was to test crowds as sample sets — how many are coughing, sneezing, have a fever (considering heart rate, respiratory rate, high blood pressure, and biometric measurements based on skin tones). We asked if the health monitoring tech had been tested with a variety of skin tones, given AI’s issues in the past.

“It has. There’s some challenges with that at times,” Chell admitted. “So darker skin tones and different types of lights and the rest of it can create some problems. If you have somebody walking up to a kiosk and using this type of technology, it’s a different scenario because it’s a controlled environment, you control the lighting, and you can go from there. As opposed to, if you’re trying to do it across a large room that’s got 300 or 2,000 people in it, you’re not going to get every person, every time. But you are going to get a very meaningful population sample, especially as you do it more and more over time. You’re going to get 85% of the people — 15% of people, because of the lighting, because they [have] a hoodie on, or [a] certain type of skin tone, it’s just not going to catch. But again, it’s not meant to catch an individual.”

That’s not a problem, Chell insists, because the technology is not meant to identify people. The whole point is to measure the health of a population.

“When you combine things like fever, coughing, elevated heart rate, particular respiratory rates, then you’ve got a picture of health,” Chell said. “You’re not diagnosing if somebody has COVID-19 or not, but you are doing a health measurement and getting a pretty clear idea of the rate of infectious or respiratory diseases potentially in an area. If it’s under 0.02%, we’re in great shape. If it’s 0.02% yesterday, and then tomorrow in a similar-sized sample in the same area it’s at 1%, and the day after it’s a 3%, you’re on top of it. You’ve got some information now that can correlate with [the question of whether] social distancing [is] going to be required. Or you can certainly have policy developed. We’re not caught in a situation where we’ve got something being spread pandemically and we don’t even know it yet.”

One big difference between phase one and phase two that led to major confusion is that the latter does not output video. Draganfly published videos to show how the technology worked, but that was misleading. The health measurement system does not record the subjects at a location that the drone “saw.”

“It just comes back and says in this particular geographic location, where you did the health measurement data, there were 22 people in the field view. Here are the heart rates, here are the respiratory rates, here are the fevers. Here’s the likelihood, and percentage of, infections and/or respiratory disease.”

The system takes “a stable 15 seconds” to acquire the data.

“You need at least that much data time to understand respiratory rate,” Chell explained. “In that timeframe, you also have your heart rate beating so you’re able to collect that data concurrently. You can get biometric measurements of skin tone data quite quickly as well in that timeframe. Within reason, if you’ve got good skin tone exposure, you get core temperature, along with these other things for anybody that’s in the field of view. So if there’s 20 people in that field of view, and you’ve got a good angle on those 20 subjects, in that 15 seconds you can collect 20 sets of data.”

The drone sends the data it collects back to the cloud (Draganfly uses AWS and Fortinet) for processing. “All of that happens in the cloud through encrypted lines. So that is a secure cloud environment where all of that AI happens. If the drone goes down, there’s no SD card that you can pull out with a bunch of great data.”

Interestingly, Westport wasn’t using Draganfly drones — the plan was to deploy its Commander series in phase three. In the interest of time, the first two phases were to rely on third-party drones, which Westport already had. The police department launched its drone program in early 2016 to support its dive team operations when locating submerged objects or victims. It later expanded to accident investigation, documentation of scenes, search and rescue, public works projects, and pre-event planning.

Draganfly claims its technology works at up to 190 feet away from subjects. “At 190 feet, you’re talking about a $35,000 drone, just because of the cameras and the additional sensors and stabilization. On a $600 drone, that doesn’t have really optimized stabilization software and such with it, it’s more like 20 feet. Using a drone that has optimized stabilization and a zoom lens — any distance, in theory, is possible. We are currently working with different ranges, and 190 feet has worked well.”

But again, that was meant for phase three. The Draganfly system Westport piloted could have potentially worked from almost 10 times further away. For now, it doesn’t look like Westport will ever verify those claims. Other towns and companies, however, want to try.