SpaceX points to leaky valve as culprit in Crew Dragon test accident

A prototype of SpaceX’s Crew Dragon capsule completed a pad abort test at Cape Canaveral in May 2015. Credit: SpaceX

Investigators believe a leak of propellant inside the Crew Dragon spacecraft’s propulsion system led to the capsule’s explosion April 20 during a ground test at Cape Canaveral, and a senior SpaceX official said Monday that delays are making it “increasingly difficult” to fly astronauts on the commercial spaceship before the end of the year.

Engineers are replacing valves inside the Crew Dragon’s launch abort propulsion system to prevent similar leaks from happening in the future, according to Hans Koenigsmann, SpaceX’s Vice President of build and flight reliability.

The explosion sent a cloud of toxic vapors into the sky over Cape Canaveral Air Force Station, but winds drove the material offshore, and no one was injured in the accident.

In a press briefing Monday, Koenigsmann described the preliminary findings of a joint SpaceX-NASA investigation board set up to determine the cause of the April 20 explosion, which occurred on a test stand at Cape Canaveral Air Force Station during a test of the Crew Dragon’s SuperDraco launch abort engines.

A leaky check valve inside the propulsion system allowed nitrogen tetroxide oxidizer, which the Crew Dragon’s eight SuperDraco rocket engines consume mixed with hydrazine fuel, to enter high-pressure helium tubes during ground processing, SpaceX said.

The helium system is used to quickly pressurize the propulsion system, allowing the SuperDraco thrusters to fire up during a launch emergency and push the Crew Dragon and its astronaut crew away from a failing rocket.

“If you have a propellant tank, and you fill that tank, and you do have a check valve, it’s conceivable that the check valve leaks backwards … and you push propellant into the pressurization system,” Koenigsmann said. “The amount might be a cup or something like that, or more than a cup, it depends on how the system is being built up. And then it’s there for a while after loading, and when you pressurize you basically open the valves really, really fast.”

The abort propulsion system needs to pressurize up to 2,400 pounds per square inch to force propellant into the SuperDraco thrusters, which SpaceX designed to nearly instantaneously ignite and power up to 16,000 pounds of thrust. The eight SuperDraco engines are clustered around the Crew Dragon capsule in four pods.

The Crew Dragon carries 16 smaller Draco thrusters for in-space maneuvers. They operate at lower pressure than the SuperDraco escape engines.

SpaceX accomplished a successful test of the lower-pressure Draco thrusters before pressing on to a SuperDraco hot fire test April 20. As the abort system pressurized, roughly 100 milliseconds before the SuperDraco engines were set to ignite, “we think that this slug (of nitrogen tetroxide) was driven back into the check valve,” Koenigsmann said Monday.

“Imagine a lot of pressure driving back a slug of liquid (that) has significant force, and that basically destroyed the check valve and caused an explosion,” Koenigsmann said.

The same Crew Dragon spacecraft, after its successful return to Earth, was mounted on a test stand at the Cape Canaveral Air Force Station for a test firing of its Super Draco abort engines. An instant before ignition on April 20, the vehicle was destroyed in a catastrophic explosion. Credit: NASA

The explosion destroyed the Crew Dragon spacecraft, the same vehicle that successfully launched to the International Space Station on an unpiloted test flight March 2. The capsule returned to Earth six days later and splashed down in the Atlantic Ocean.

The April 20 test-firing was key milestone as SpaceX teams readied the same spacecraft for an in-flight abort test, in which the SuperDraco engines will be ignited to push the capsule away from a Falcon 9 rocket about a minute after liftoff.

Koenigsmann said workers recovering debris from the spacecraft found burns within the faulty check valve, and the investigation’s preliminary findings line up with video and telemetry data. The SuperDraco thrusters were recovered intact after the accident, and SpaceX officials are confident the engines themselves were not at fault in the explosion.

In a written statement, SpaceX said the failure of the check valve — made of titanium — in a high-pressure NTO (nitrogen tetroxide) environment was “sufficient to cause ignition of the check valve and led to an explosion.”

SpaceX engineers tested the check valve failure hypothesis at the company’s test site in McGregor, Texas.

“We found out that … when the pressure is high, and you drive a slug with a lot of energy into a titanium component, that you can have this rather violent reaction,” Koenigsmann said.

He added that the result was surprising. Engineers did not expect titanium, a material commonly used for decades on space vehicles around the world, could react so explosively in such an environment.

“We still are not done with the testing,” Koenigsmann said. “We have preliminary results, basically, but we know enough … that you should actually make sure that no oxidizer can move over to the pressurant side, and then cause problems when you pressurize the system for flight.”

A diagram of the Crew Dragon spacecraft. Credit: SpaceX

SpaceX is replacing four check valves in the Crew Dragon’s abort system with burst disks, which seal off the flow path between the propellant tanks and the plumbing for the abort system’s gaseous pressurization system.

“A burst disk is basically a device that is completely sealing left from right, and only opens when you have pressure that exceeds its rating, and then it opens and works,” Koenigsmann said. “That is basically the functionality that we need for the escape system for it to work properly in the case of a vehicle abort.”

Koenigsmann said the hardware changes inside the propulsion system will be “relatively small.”

“The burst disk we have now is definitely the safer approach overall going forward,” Koenigsmann said. “We didn’t really expect this to be a problem prior to that (accident), but that’s what you learn when you test. You find out some components might be better off exchanged with other components.”

The check valves are designed with a spring to open and close as needed.

“The problem is that sometimes the spring is a little bit sticky,” Koenigsmann said. “The valve has moving parts, and so that’s why things sometimes, especially at low pressure, are not quite sealing as well as they’re supposed to in check valves.”

On rockets and spacecraft, burst disks are designed for a single use. The burst disks block the pathway between the propellant and pressurization systems until they rupture during the engine startup sequence.

SpaceX originally intended the Crew Dragon spacecraft to use its SuperDraco thrusters for propulsive helicopter-like landings on the ground, assuming they were not needed for a launch abort. The company nixed those plans entirely in 2017, electing to use parachutes for the capsule to splash down in the ocean.

Koenigsmann said the investigation into the accident is still ongoing. He estimated the SpaceX-led inquiry, which includes NASA participation, is about 80 percent complete.

SpaceX wants to make sure the failure mode in the nitrogen tetroxide pressurization system will not cross over to other parts of the Crew Dragon spacecraft, such as the fuel system that feeds hydrazine to the SuperDraco thrusters.

“Through this process, we will continue to learn things that will help us fly safer,” said Kathy Lueders, NASA’s commercial crew program manager.

“When you have something like a test anomaly like this one — as big as this one — then you look at other systems,” Koenigsmann said. “You make sure you don’t have vulnerabilities in … other systems, on the fuel side, and so on.

“We must characterize the basic physics of this how does this happen? How do NTO and titanium ignite, and what does it mean for flammability,” he said. “So, in general, we still have work ahead of us.”

Schedules for commercial crew test flights remain murky

In a conference call with reporters Monday, Lueders declined to offer a schedule for SpaceX’s next two Crew Dragon test flights.

After completing the unpiloted test flight to the space station in March, known as Demo-1, SpaceX was gearing up for the in-flight abort test when the capsule exploded at Cape Canaveral. SpaceX was building a second space-worthy Crew Dragon spacecraft for the first test flight with astronauts to the International Space Station, known as Demo-2.

At the time of the April 20 explosion, SpaceX and NASA aimed to conduct the in-flight abort test in July, followed by the Demo-2 mission with astronauts as soon as September.

Lueders was non-committal when asked about the chances of astronauts flying to space on a Crew Dragon this year.

“Obviously, there’s always a chance that we’re going to fly crew on a SpaceX vehicle this year,” she said. “But I think, right now, we’re playing very close attention to the work thats being done coming out of the anomaly investigation, along with … getting through parachute testing and other key tests, along with conducting our in-flight abort test. All those things need to occur before we’re going to be confident that these systems will safely fly our crews.”

SpaceX also suffered a parachute mishap during a drop test in April to demonstrate the Crew Dragon’s ability to safely land with three parachutes, in case one of the four main chutes failed to unfurl. The remaining parachutes did not fully open during the April drop test over Nevada, and a metal test sled built to simulate the weight of a Crew Dragon capsule impacted the ground.

SpaceX is expected to redo the abort system hotfire test on the next Crew Dragon spacecraft before proceeding to the in-flight escape test.

“We’re going to fly when we feel like the certification work’s been done to be able to safely fly our crews,” Lueders said. “I hope it’s this year, but we’re going to fly when it’s the right time, and when we know that we’ll be flying our crew safely.”

Koenigsmann acknowledged the delays are making it less likely SpaceX can launch astronauts for the first time before the end of the year.

“I want to say I’m pretty optimistic at this point in time because we have a good path forward, but like I said, we’re still not quite done. We could find other things, so there’s always the unknown between now and then.

“End of the year? I don’t think it’s impossible, but it’s getting increasingly difficult, too,” Koenigsmann said.

NASA astronauts Bob Behnken and Doug Hurley are assigned to the Demo-2 mission, in which they will blast off from pad 39A at the Kennedy Space Center in Florida on top of a Falcon 9 rocket and dock with the International Space Station for a mission planned to last one-to-two weeks.

They will return to Earth for a parachute-assisted splashdown in the Atlantic Ocean east of Florida.

The Demo-2 flight will pave the way for crew rotation missions to the station, each carrying four astronauts. The Crew Dragon is designed for stays of up to 210 days at the space station, providing a lifeboat for crews before their regularly-scheduled trips back to Earth.

SpaceX is shuffling the lineup of Crew Dragon capsules on the company’s assembly line in Hawthorne, California, to replace the spacecraft lost in the April 20 accident. The capsule previously assigned to the Demo-2 mission will launch on the in-flight abort test, and the vehicle intended for the first crew rotation flight to the station will now launch on Demo-2.

NASA has awarded more than $3.1 billion in funding to SpaceX to develop the Crew Dragon spacecraft since the commercial crew initiative began in 2010. In a similar arrangement, the space agency has signed a series of commercial crew agreements and contracts with Boeing worth more than $4.8 billion over the same time period.

Boeing has also run into trouble during ground testing of abort engines on its CST-100 Starliner crew capsule. The company resumed ground firings of the Starliner’s service module rocket engines in May after halting the test series following a propellant leak in June 2018.

The Starliner capsule will take off on United Launch Alliance Atlas 5 rockets, and return to landings in the Western United States, slowed by parachutes and cushioned by airbags.

NASA said last year it would provide updates on launch planning dates for the commercial crew program “approximately monthly.” But NASA released the last formal update to the Boeing’s Starliner target launch dates April 3, when the agency said the Starliner’s unpiloted demonstration flight to the space station was set for no sooner than August, followed by the Starliner’s first test flight with a crew in late 2019.

The August schedule for the Starliner’s unpiloted demonstration mission to the space station, named Orbital Flight Test-1, is no longer achievable, at least partly due to slips in the Atlas 5 launch in ULA’s manifest. An Atlas 5 rocket was scheduled to blast off from Cape Canaveral on June 27 with the U.S. Air Force’s AEHF 5 communications satellite, but technical concerns have delayed the launch until no earlier than Aug. 8.

After the AEHF 5 launch, ULA ground teams will safe the launch pad and roll the Atlas 5’s mobile launch platform back to the Vertical Integration Facility south of the pad. Once the mobile launch table is inside the VIF, ULA needs about six weeks to stack the two-stage Atlas 5 rocket, its twin solid rocket boosters, and hoist the Starliner capsule on top of the launch vehicle.

The six-week timeline also includes time for a wet dress rehearsal, in which ULA will transfer the Atlas 5 to Cape Canaveral’s Complex 41 launch pad for a countdown fueling test. ULA does not conduct such a fueling test on most launch campaigns.

The time required to configure the launch pad after the liftoff of the AEHF 5 mission, assemble the new Atlas 5 launcher, conduct the wet dress rehearsal, and accomplish other unique tasks on the first space-bound Starliner means the OFT-1 launch will likely not occur before late September, or more likely early October.

That schedule does not account for any processing or testing issues encountered by Boeing’s team working on the Starliner spacecraft.

“We’re looking at both providers’ schedules pretty closely now,” Lueders said. “We’re working through with Hans and the SpaceX folks on the activities that need to occur and the timing for those as we’re moving towards our Demo-2 flight.

“We’re also working with Boeing as they’re getting ready for the uncrewed mission. I don’t think today I’m going be talking about either one of those schedules just because we’re taking the time right now to make sure we’re understanding all the work, and making sure we’re bringing forward joint (planning) dates over this next period of time.”

Email the author.

Follow Stephen Clark on Twitter: @StephenClark1.