Spaceflight Now





Space station hit by multiple computer glitches
BY WILLIAM HARWOOD
STORY WRITTEN FOR CBS NEWS "SPACE PLACE" & USED WITH PERMISSION
Posted: February 21, 2010


Bookmark and Share

Multiple computer glitches aboard the International Space Station triggered intermittent communications blackouts early Sunday, officials said, and while the three command and control computers seem to be working, engineers do not yet understand what has caused the machines to repeatedly transition from one to the other.

The station's command and control software was updated before the shuttle Endeavour's flight to account for the new Tranquility module and it's possible the computer failures, or transitions, are software related. Issues with computers in Tranquility also have been noted, but it's not yet clear whether they are related to the command and control issues.

During normal operations, one of the three C&C computers, known as a multiplexer/demultiplexer, operates as the "prime" machine, allowing station astronauts and ground controllers to send commands to various systems and providing critical telemetry.

A second C&C computer operates in backup mode, ready to take over if the prime computer suffers a problem, and the third machine operates in standby in a domino-like software architecture. The computer system is critical to all aspects of space station operation.

Early Sunday, the prime C&C computer ran into a problem, forcing a transition to the backup. A NASA source familiar with the troubleshooting said the backup "came on and was prime for about a second and then it failed a diagnostic, the equivalent of the blue screen of death on a Windows machine. The third one, which was in standby, it came up and was prime for about an hour and just a little while ago it also transitioned and stopped work. The first one that went down, the original prime, it's back up right now."

"But now that this has cycled through all three machines it is very clear there is some generic, probably software related problem," he said.

NASA mission control commentator Pat Ryan outlined a slightly different sequence of events, saying when the prime computer malfunctioned "the backup computer moved assume its responsibilities and then the standby computer moved to backup. In the process, Houston lost S-band communications with the crew on board."

Commander Jeff Williams called down at 9:55 a.m. EST "and confirmed there had been a primary and backup failure of the C&Cs," Ryan said. "However, two of the three computers were healthy. Later this morning, just before (11 a.m.), there was data here in mission control that indicated there had been another C&C transition, this time still with just two computers healthy.

"But after a third transition this morning, all three computers were indicating they were healthy, although they were now operating in different slots among the primary, backup and standby computers. About 20 minutes ago (just after noon EST), space station commander Jeff Williams indicated there had been another transition, this one only impacting the backup and standby computers.

"But we are still in a situation currently where all three computers are healthy but the team here in mission control is still scratching its head and trying to determine what's been causing the repeated transitions," Ryan said. "There has been no impact to station life support systems while this was going on and the crew is in good shape."

As of 12:30 p.m., command and control computer No. 3 was prime, with C&C No. 2 in backup and No. 1 in standby.

"We're thinking we might need another day off," Williams joked with mission control as the troubleshooting continued.

"Copy and concur, Jeff," replied Stan Love in Houston. "We are talking in the room, we still do not know what has been causing these transitions. We are toying with the idea there might be something related to commanding. But we are not sure, it's just speculation at this point."

Love later said engineers had detected a fault in a computer system in the Tranquility module, but it was not immediately known if that was related to the C&C issue.

Spaceflight Now Plus
Additional coverage for subscribers:
VIDEO: PREVIEW OF FLIGHT DAY 13 ACTIVITIES PLAY

VIDEO: FLIGHT DAY 12 HIGHLIGHTS MOVIE PLAY
VIDEO: FRIDAY MORNING'S MISSION STATUS BRIEFING PLAY
VIDEO: SHUTTLE AND STATION CREWS BID THEIR FAREWELLS PLAY
VIDEO: RIBBON-CUTTING CEREMONY FOR STATION'S CUPOLA PLAY
VIDEO: IN-FLIGHT CREW NEWS CONFERENCE FOR U.S AND JAPAN PLAY

VIDEO: FLIGHT DAY 11 HIGHLIGHTS MOVIE PLAY
VIDEO: THURSDAY MORNING'S MISSION STATUS BRIEFING PLAY
VIDEO: OXYGEN GENERATOR RELOCATED TO TRANQUILITY PLAY
VIDEO: STATION'S TOILET INSTALLED IN THE NEW MODULE PLAY
VIDEO: WATER GENERATING EQUIPMENT MOVED INTO TRANQUILITY PLAY
VIDEO: PRESIDENT OBAMA CALLS SHUTTLE AND STATION CREWS PLAY

VIDEO: DAZZLING VIEWS FROM INSIDE CUPOLA PLAY
VIDEO: CUPOLA WINDOW SHUTTERS ARE TESTED PLAY
VIDEO: STATION ASTRONAUT'S CAMCORDER VIEWS OF CUPOLA PLAY
VIDEO: FLIGHT DAY 10 HIGHLIGHTS MOVIE PLAY
VIDEO: WEDNESDAY MORNING'S MISSION STATUS BRIEFING PLAY
VIDEO: CUPOLA OPEN FOR VIEWING PLAY
VIDEO: FREEING THE SHUTTERS PLAY
VIDEO: CUPOLA IS UNVEILED! PLAY
VIDEO: FIRST HALF OF CUPOLA UNCOVERED PLAY
VIDEO: WIRING UP DOCKING ADAPTER ON TRANQUILITY PLAY
VIDEO: ACTIVATING TRANQUILITY'S OTHER COOLING LOOP PLAY
VIDEO: SPACEWALKERS FLOAT OUT OF AIRLOCK PLAY
VIDEO: STEP-BY-STEP PREVIEW OF SPACEWALK NO. 3 PLAY
VIDEO: PREVIEW OF FLIGHT DAY 10 ACTIVITIES PLAY

VIDEO: FLY WITH THE ASTRONAUTS IN THE COCKPIT PLAY
VIDEO: LOOKING OUT THE PILOT'S FRONT WINDOW PLAY
VIDEO: LEFT-HAND BOOSTER CAMERA LOOKING UPWARD PLAY
VIDEO: LEFT-HAND BOOSTER CAMERA LOOKING INBOARD PLAY
VIDEO: LEFT-HAND BOOSTER CAMERA LOOKING DOWNWARD PLAY
VIDEO: RIGHT-HAND BOOSTER CAMERA LOOKING UPWARD PLAY
VIDEO: RIGHT-HAND BOOSTER CAMERA LOOKING INBOARD PLAY
VIDEO: RIGHT-HAND BOOSTER CAMERA LOOKING DOWNWARD PLAY
VIDEO: EXTERNAL TANK CAMERA FROM LIFTOFF TO SEPARATION PLAY

VIDEO: FLIGHT DAY 9 HIGHLIGHTS MOVIE PLAY
VIDEO: TUESDAY MORNING'S MISSION STATUS BRIEFING PLAY
VIDEO: DOCKING ADAPTER PLACED ONTO TRANQUILITY PLAY
VIDEO: PREVIEW OF FLIGHT DAY 9 ACTIVITIES PLAY

VIDEO: FLIGHT DAY 8 HIGHLIGHTS MOVIE PLAY
VIDEO: NEW YORK, ST. LOUIS AND MEMPHIS MEDIA INTERVIEWS PLAY
VIDEO: MONDAY MORNING'S MISSION STATUS BRIEFING PLAY
VIDEO: OBSERVATION PORTAL RELOCATED TO NEW HOME PLAY
VIDEO: CUPOLA PLUCKED FROM OUTBOARD PORT PLAY
VIDEO: ROBOTIC ARM GRAPPLES CUPOLA PLAY
VIDEO: CREW'S EDUCATIONAL EVENT PLAY
VIDEO: PREVIEW OF FLIGHT DAY 8 ACTIVITIES PLAY
VIDEO: SUNDAY AFTERNOON'S FLIGHT DIRECTOR INTERVIEW PLAY

VIDEO: FLIGHT DAY 7 HIGHLIGHTS MOVIE PLAY
VIDEO: SUNDAY MORNING'S MISSION STATUS BRIEFING PLAY
VIDEO: PREPPING CUPOLA'S NEW HOME PORT PLAY
VIDEO: SPACEWALKERS COMPLETE OTHER OUTFITTING WORK PLAY
VIDEO: AMMONIA BEGINS FLOWING TO COOL TRANQUILITY PLAY
VIDEO: PATRICK'S SPACESUIT CONTAMINATION CHECKS PLAY
VIDEO: SPACEWALKERS INSTALL SECOND COOLANT LOOP PLAY
VIDEO: FIRST SET OF AMMONIA LINES HOOKED UP PLAY
VIDEO: PREVIEW OF FLIGHT DAY 7 ACTIVITIES PLAY
VIDEO: STEP-BY-STEP PREVIEW OF SPACEWALK NO. 2 PLAY

VIDEO: FLIGHT DAY 6 HIGHLIGHTS MOVIE PLAY
VIDEO: OPENING CUPOLA'S HATCHWAY BRIEFLY PLAY
VIDEO: ENTERING TRANQUILITY FOR FIRST TIME PLAY
VIDEO: SPACEWALKERS ANSWER TWITTER QUESTIONS PLAY
VIDEO: ASSOCIATED PRESS, CBS AND REUTERS MEDIA INTERVIEWS PLAY
VIDEO: SATURDAY MORNING'S MISSION STATUS BRIEFING PLAY
VIDEO: PREVIEW OF FLIGHT DAY 6 ACTIVITIES PLAY

VIDEO: FRIDAY AFTERNOON'S MANAGEMENT TEAM UPDATE PLAY
VIDEO: FRIDAY MORNING'S MISSION STATUS BRIEFING PLAY
VIDEO: FLIGHT DAY 5 HIGHLIGHTS MOVIE PLAY
VIDEO: STEP-BY-STEP PREVIEW OF SPACEWALK NO. 1 PLAY
VIDEO: PREVIEW OF FLIGHT DAY 5 ACTIVITIES PLAY

VIDEO: FLIGHT DAY 4 HIGHLIGHTS MOVIE PLAY
VIDEO: WEDNESDAY NIGHT'S MISSION STATUS BRIEFING PLAY
VIDEO: SACRAMENTO, MOBILE AND ST. LOUIS MEDIA INTERVIEWS PLAY
VIDEO: WEDNESDAY AFTERNOON'S MANAGEMENT TEAM UPDATE PLAY
VIDEO: WEDNESDAY MORNING'S FLIGHT DIRECTOR INTERVIEW PLAY
VIDEO: PREVIEW OF FLIGHT DAY 4 ACTIVITIES PLAY

VIDEO: FLIGHT DAY 3 HIGHLIGHTS MOVIE PLAY
VIDEO: WEDNESDAY MORNING'S MISSION STATUS BRIEFING PLAY
VIDEO: SHUTTLE CREW WELCOMED ABOARD STATION PLAY
VIDEO: HATCHWAY OPENED BETWEEN TWO SPACECRAFT PLAY
VIDEO: ENDEAVOUR DOCKS TO THE SPACE STATION PLAY
VIDEO: SHUTTLE FLIES OUT IN FRONT OF STATION PLAY
VIDEO: ENDEAVOUR PERFORMS 360-DEGREE BACKFLIP PLAY
VIDEO: STUNNING SHOT OF SHUTTLE AGAINST HORIZON PLAY
VIDEO: STATION'S VIEW OF SHUTTLE ENGINE FIRING PLAY
VIDEO: PREVIEW OF FLIGHT DAY 3 ACTIVITIES PLAY
VIDEO: TUESDAY AFTERNOON'S MANAGEMENT TEAM UPDATE PLAY

VIDEO: AMAZING LAUNCH FOOTAGE FROM COCKPIT CAMERA PLAY

VIDEO: FLIGHT DAY 2 HIGHLIGHTS MOVIE PLAY
VIDEO: TUESDAY'S MISSION STATUS BRIEFING PLAY
VIDEO: TIME-LAPSE OF WING INSPECTIONS PLAY
VIDEO: INSPECTION BOOM READIED FOR USE PLAY
VIDEO: PREVIEW OF FLIGHT DAY 2 ACTIVITIES PLAY
VIDEO: NARRATED TOUR OF ENDEAVOUR'S PAYLOAD BAY PLAY

VIDEO: FLIGHT DAY 1 HIGHLIGHTS MOVIE PLAY
VIDEO: THE FULL STS-130 LAUNCH EXPERIENCE PLAY
VIDEO: SHUTTLE ENDEAVOUR BLASTS OFF! PLAY | HI-DEF
VIDEO: GO BEHIND THE SCENES IN MISSION CONTROL PLAY
VIDEO: JETTISONED EXTERNAL FUEL TANK TUMBLES AWAY PLAY
VIDEO: PAYLOAD BAY DOORS OPENED FOLLOWING LAUNCH PLAY

VIDEO: CREW FINISHES GETTING SUITED UP PLAY
VIDEO: ASTRONAUTS LEAVE CREW QUARTERS PLAY | HI-DEF
VIDEO: CREW ARRIVES AT LAUNCH PAD 39A PLAY
VIDEO: ASTRONAUTS BOARD THEIR SPACECRAFT PLAY

VIDEO: LAUNCH REPLAY: VAB ROOF PLAY | HI-DEF
VIDEO: LAUNCH REPLAY: PRESS SITE PLAY | HI-DEF
VIDEO: LAUNCH REPLAY: PAD PERIMETER PLAY | HI-DEF
VIDEO: LAUNCH REPLAY: BEACH TRACKER PLAY | HI-DEF
VIDEO: LAUNCH REPLAY: PAD CAMERA 070 PLAY | HI-DEF
VIDEO: LAUNCH REPLAY: PAD CAMERA 071 PLAY | HI-DEF
VIDEO: LAUNCH REPLAY: UCS-23 TRACKER PLAY | HI-DEF
VIDEO: LAUNCH REPLAY: PLAYALINDA BEACH PLAY | HI-DEF
VIDEO: LAUNCH REPLAY: PAD FRONT CAMERA PLAY | HI-DEF
VIDEO: LAUNCH REPLAY: BANANA CREEK SITE PLAY | HI-DEF

VIDEO: NARRATED REVIEW OF SHUTTLE'S PREPARATIONS PLAY | HI-DEF
VIDEO: NARRATED REVIEW OF PAYLOADS' PREPARATIONS PLAY | HI-DEF

VIDEO: EXPLANATION OF WEATHER PROBLEMS PLAY | HI-DEF
VIDEO: LOW CLOUDS SCRUB FIRST COUNTDOWN PLAY | HI-DEF
VIDEO: ASTRONAUTS DEPART QUARTERS FOR PAD 39A PLAY | HI-DEF
VIDEO: CREW GETS SUITED UP FOR LAUNCH ATTEMPT PLAY | HI-DEF
VIDEO: PAD SERVICE GANTRY RETRACTED PLAY | HI-DEF
VIDEO: TIME-LAPSE OF MOBILE TOWER ROLLBACK PLAY

VIDEO: INTERVIEW WITH COMMANDER GEORGE ZAMKA PLAY
VIDEO: INTERVIEW WITH PILOT TERRY VIRTS PLAY
VIDEO: INTERVIEW WITH MISSION SPECIALIST 1 KAY HIRE PLAY
VIDEO: INTERVIEW WITH MISSION SPECIALIST 2 STEVE ROBINSON PLAY
VIDEO: INTERVIEW WITH MISSION SPECIALIST 3 NICK PATRICK PLAY
VIDEO: INTERVIEW WITH MISSION SPECIALIST 4 BOB BEHNKEN PLAY

VIDEO: ENDEAVOUR'S PRE-LAUNCH NEWS CONFERENCE PLAY
VIDEO: THURSDAY STATUS AND WEATHER UPDATE PLAY
VIDEO: COUNTDOWN PREVIEW BRIEFING PLAY
VIDEO: ASTRONAUTS ARRIVE FOR LAUNCH PLAY | HI-DEF

VIDEO: LAUNCH DATE SET AT FLIGHT READINESS REVIEW PLAY
VIDEO: PAYLOAD BAY DOORS CLOSED FOR LAUNCH PLAY | HI-DEF
VIDEO: CREW SEES TRANQUILITY LOADED INTO SHUTTLE PLAY | HI-DEF

VIDEO: SHUTTLE EVACUATION PRACTICE PLAY | HI-DEF
VIDEO: ASTRONAUTS BOARD ENDEAVOUR PLAY | HI-DEF
VIDEO: THE LAUNCH DAY SIMULATION BEGINS PLAY | HI-DEF
VIDEO: PAD BUNKER TRAINING FOR THE CREW PLAY | HI-DEF
VIDEO: CREW BRIEFED ON EMERGENCY PROCEDURES PLAY | HI-DEF
VIDEO: TEST-DRIVING AN EMERGENCY ARMORED TANK PLAY | HI-DEF
VIDEO: NIGHTTIME APPROACHES IN TRAINING AIRCRAFT PLAY | HI-DEF
VIDEO: ASTRONAUTS CHAT WITH REPORTERS AT PAD 39A PLAY
VIDEO: SPACEWALKER UPDATES COOLING HOSE FIX PLAY | HI-DEF
VIDEO: ROBINSON'S THOUGHTS ON SHUTTLE RETIREMENT PLAY | HI-DEF
VIDEO: ASTRONAUTS ARRIVE FOR PRACTICE COUNTDOWN PLAY | HI-DEF

VIDEO: TRANQUILITY DELIVERED TO PAD 39A PLAY | HI-DEF
VIDEO: PAYLOAD TRANSPORTER GOES UPRIGHT PLAY | HI-DEF
VIDEO: PACKING UP PAYLOAD FOR LAUNCH PLAY | HI-DEF

VIDEO: SHUTTLE ENDEAVOUR'S FRIGID ROLLOUT TO PAD PLAY | HI-DEF
VIDEO: ENDEAVOUR HOISTED FOR ATTACHMENT TO TANK PLAY | HI-DEF
VIDEO: CRANE ROTATES THE ORBITER VERTICALLY PLAY | HI-DEF
VIDEO: ENDEAVOUR MOVES TO ASSEMBLY BUILDING PLAY | HI-DEF
VIDEO: TIME-LAPSE SHOWS ENDEAVOUR ASCENDING IN VAB PLAY
VIDEO: TIME-LAPSE SHOWS THE MOVE TO ASSEMBLY BUILDING PLAY

VIDEO: ORBITER READY TO LEAVE HANGAR PLAY | HI-DEF
VIDEO: EXTERNAL TANK ATTACHED TO BOOSTERS PLAY | HI-DEF
VIDEO: ENDEAVOUR'S MAIN ENGINE INSTALLATION PLAY | HI-DEF
VIDEO: ASTRONAUTS VISIT THEIR SPACECRAFT PLAY | HI-DEF
VIDEO: CREW INSPECTS MISSION PAYLOADS PLAY | HI-DEF
VIDEO: FUEL TANK UNLOADED FROM THE BARGE PLAY | HI-DEF
VIDEO: EXTERNAL TANK ARRIVES AT SPACEPORT PLAY | HI-DEF
VIDEO: FORWARD THRUSTER POD CHECKED OUT PLAY | HI-DEF
VIDEO: ENDEAVOUR TOWED OFF RUNWAY FROM STS-127 PLAY | HI-DEF

VIDEO: TRANQUILITY HATCH SEALED FOR LAUNCH PLAY | HI-DEF
VIDEO: CUPOLA ATTACHED TO TRANQUILITY PLAY | HI-DEF
VIDEO: THE SPACE STATION'S NEW CUPOLA PLAY | HI-DEF
VIDEO: TRANQUILITY UNPACKED IN FLORIDA PLAY | HI-DEF
VIDEO: NEW MODULE ARRIVES FROM EUROPE PLAY | HI-DEF
SUBSCRIBE NOW



MISSION INDEX