How Amazon safety tested its new MK30 delivery drone

Last November, we took the latest big step in our Prime Air journey, launching our MK30 drone for commercial delivery service in the West Valley of Arizona’s Phoenix metro area and in College Station, Texas.

As we expected, delivering packages to customers across these different environments provided valuable operational insights. For example, we knew that we’d experience more dust in the air in Phoenix than we would in other places, and that proved to be true. While we’d already designed the drone to be safe in those environments, the data from these flights still gave us opportunities to learn and build even more safety measures into our drones. One example of that is in an altitude sensor that we use—we saw that environmental factors like dust can sometimes interfere with its readings.

While we never experienced an actual safety issue with it, we learned that—in extremely rare cases—this could have caused the drone to receive an inaccurate reading about its position in relation to the ground. Even though this is a highly unlikely edge-case scenario, we saw no reason to take risks. Safety is and always will be our top priority.

So, we took those learnings and decided to proactively enhance the fleet—even when we hadn’t encountered any actual safety issues in flights. And while we’re making those improvements, we’ve voluntarily paused operations across the fleet. This is a normal part of our rigorous internal safety and engineering processes, but some people misunderstood what we were doing, and inaccurately suggested that there was a safety issue.

We have complete confidence in the underlying safety of the drone, which is the result of the rigorous testing regime that the MK30 went through, and the certification from the Federal Aviation Administration (FAA) that ensures the MK30 can safely integrate into the airspace and safely conduct deliveries to our customers.

Experimental delivery drones lined up on tarmac

Push it to the limit

These real-world learnings are a helpful part of the process, but they only happen after extensive testing in labs and on closed courses. If we back up too much earlier in that process, how do we test a design like the MK30 and ensure that it’s safe? The simple answer is that we push it to and beyond its limits in safe environments. Over and over again. That allows us to find potential issues and fix them, and understand where the limits of the technology are—so we can be sure we won’t reach them in real-world scenarios.

Experimental drone with worker in safety vest

This might sound straightforward, but new innovations require new ways of proving they’re safe and effective. So, as we created the MK30, we also created a way of testing it. It’s a process that requires a blend of creativity and rigor.

“That is one of the most challenging, but also most rewarding, parts of our job,” says Phil Hornstein, who leads system safety for Prime Air, focusing on the drone’s design and engineering. “Our aim at Prime Air is to establish and meet a safety bar that is higher than what is required by regulators.”

A big part of the test process is running lots and lots of flights—5,166 to date, to be exact, totaling 1,300 hours in the air. These flights generate invaluable data and learnings that ensure not just that the drone can stay aloft (our longest single flight covered over 15 miles), but that it can stay aloft even when problems arise.

The challenging part is finding and eliminating the drone’s weaknesses, while also providing the evidence required to demonstrate that the drone meets the high safety bar. This involves simulating static and dynamic obstacles, including erecting a crane and flying a helicopter in the drone’s flight path.

Amazon Prime Air drone flying over rural landscape

Standards and challenges

Creating a new test regime for a new aircraft begins by setting the right benchmarks.

Because the drone industry doesn’t have decades of established safety protocols to build upon, the team borrowed heavily from traditional aviation standards and processes for commercial airplanes—the highest safety bar in the industry.

They also drew ideas from the automotive industry’s expertise in testing smaller vehicles and self-driving car developers’ approach to highly automated features. Following these protocols helped the team do things like identify and design out points of single failure, such as using the same power source to supply both a component and its backup.

With goals set, the next step was to think through every hurdle along the way. Those fall into three general categories. Functional hazards are things that can go wrong with the drone’s software or hardware. Human factors account for any mistakes an operator might make (a helpfully limited area in this highly automated system). And the environmental bucket catches everything the drone’s liable to encounter, from winds and hail to obstacles in a customer’s backyard.

Amazon Prime Air drone flying past large tree in countryside

Injected risk

For every given hazard, the Amazon team creates a test to re-create the problem and see whether their drone can handle it.

For instance, the “MEP out test” (that’s motor, electronic speed controller, and propeller) is meant to test the limits of the drone’s hardware by using software to stop one of the MK30’s motors, to verify the drone can stay in flight and safely return to the launch site. Switchover testing pushes the limits of the drone’s software system by “injecting” a failure that knocks out the drone’s primary flight computer, to verify it can switch to a backup that can take control and safely get the drone back to base. They run both these tests over and over, blitzing the drone at different points of a mission, in horizontal and vertical flight, right after launch and at the moment of delivery.

Professional deep in thought while working on laptop

Safely integrating our drone with other aircraft in the same airspace is one of the most important safety imperatives for the program, and requires skilled operators who understand the mechanics of drone flight. Our operators run through tests to ensure teams know and follow our standard operating procedures for pre- and post-flight checks.

Lastly, to address obstacle avoidance, the team created a test regime that pushes the drone to understand what’s happening below it, and know when it shouldn’t deliver a package (it delivers while hovering 12 feet up). Before a flight, Prime Air’s Airspace and Mission Orchestration System generates a high-fidelity model of the operating area. That model creates flight paths that avoid known obstacles like buildings, bridges, and power lines.

Drone over letter-covered mat surrounded by QR codes

As a rule, the operations team keeps the drones to specific airspace that should be clear (it flies under 400 feet), and monitors the signals many aircraft use to broadcast their position. The drone’s ability to spot and dodge an aircraft may be its last line of defense, but that makes it all the more important to test, and to do so in as realistic a way as possible.

“When we test the detect and avoid capability, we’re flying an airplane at the drone,” says Adam Martin, who runs Prime Air’s flight test and safety organizations. “We’ll fly a helicopter at it.” And they’ll do it again many times, with different routes, angles, heights, speeds, and scenarios. “We ran many, many tests of that type,” he says.

Looking ahead

While we’re confident in the testing we do before we get FAA certification and start making real-world flights, we don’t stop when we hit those milestones. Testing is a game of one-upmanship, where each evolution in the design or use of a product requires a similar evolution in how it’s proven safe, and where we’re always asking ourselves “what other situation could we possibly encounter, and how do we get ahead of it?”

We’re always working to learn more, study more, and get ready to ace the next test.