9-603-062
REV. OCTOBER 29, 2002

DOROTHY LEONARD
DAVID KIRON

Managing Knowledge and Learning at NASA and the Jet Propulsion Laboratory (JPL)
Downsizing at NASA over the last decade through attrition and buyouts has resulted in an imbalance in
NASA’s skill mix.1
— The President’s Management Agenda, Fiscal Year 2002
By the end of this decade, many of the most experienced scientists and engineers at NASA and JPL are going to retire. If we don’t have systems in place to retain more of what they know, our institution is going to suffer. — Jeanne Holm, Chief Knowledge Architect for NASA
In the spring of 2002, Jeanne Holm, Chief Knowledge Architect for the National Aeronautics and
Space Administration (NASA) and its Jet Propulsion Laboratory (JPL), was giving a tour of JPL.
Stopping at a viewing stage above JPL’s mission control center, Holm explained the growing need for knowledge management at NASA:
Almost 40% of JPL’s science and engineering workforce is currently eligible for retirement.
In just four years, half of NASA’s entire workforce will be eligible. Many of these people are the most experienced project managers—the people who worked on Apollo (the mission to the
Moon) and built the first space shuttle. Yet, we have few programs designed to bring their wisdom into our institutional memory.
In the past 10 years, the budgets on our missions have been radically reduced, missions have multiplied ten-fold, and our scientists and engineers have been pushed to the limits.
Three years ago, we endured the highly publicized failure of two missions to Mars. NASA as a whole, and JPL in particular, have really struggled to find the right balance between mission performance and cutting-edge space exploration. With some of our most experienced scientists and engineers poised to leave in the coming years, these issues have the potential to become even more severe.

1 President’s Management Agenda, Fiscal Year 2002, , p. 13,

accessed September 23, 2002.
________________________________________________________________________________________________________________
Professor Dorothy Leonard and Senior Researcher David Kiron of the Global Research Group prepared this case. HBS cases are developed solely as the basis for class discussion. Cases are not intended to serve as endorsements, sources of primary data, or illustrations of effective or ineffective management.
Copyright © 2002 President and Fellows of Harvard College. To order copies or request permission to reproduce materials, call 1-800-545-7685, write Harvard Business School Publishing, Boston, MA 02163, or go to http://www.hbsp.harvard.edu. No part of this publication may be reproduced, stored in a retrieval system, used in a spreadsheet, or transmitted in any form or by any means—electronic, mechanical, photocopying, recording, or otherwise—without the permission of Harvard Business School.

This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

In 2000, the U.S. Congress, which funds NASA, called on the agency to develop appropriate knowledge management (KM) tools to reduce the risk of future mission failures and to address impending retirements. Until 2002, most of NASA’s knowledge management tools were information technology systems. NASA’s most senior-level KM champion, Chief Information Officer Lee
Holcomb, had sponsored the development and implementation of these systems. But as Holm explained, IT systems alone could not fully address NASA’s mandate; to meet that challenge, the agency had to change its culture:
IT systems, such as Internet-based databases and portals that organize and make more accessible lessons learned have been relatively easy to pitch and to sell to people at the agency to fund. (See Exhibit 1 for annual KM budget.) The problem is, IT systems don’t address the critical need for the most experienced people to mentor and train others or to share tacit knowledge from one mission to another. Without this critical knowledge, missions will fail or be delayed or cancelled. For example, assessing risk in designing spacecraft components is one of the most difficult tasks; novices can learn this only from experienced engineers and scientists, or trial and error. What we need is a culture shift in the way experiential knowledge is cultivated and managed.
Our team is really faced with a dilemma. Do we continue on the IT track, successfully pitching IT systems that improve NASA’s ability to capture and distribute knowledge? Many senior managers are most comfortable supporting these initiatives. Or do we take a riskier tack? Do we go to the Chief Administrator of NASA, Sean O’Keefe, and say, “Look, you know us for IT, and that’s what we’ve been doing, but what we really need to do is change the knowledge-sharing culture at the agency.” This second option would require higher funding levels, not only for us in Knowledge Management, but also across the agency. (See Exhibit 2 financial information about each option.) It would also be a harder option to pitch, especially given that NASA is four billion dollars over-budget on the International Space Station.
I wouldn’t be able to show Mr. O’Keefe a specific return on investment. I can only promise him that if he makes the investment, we’ll be a better agency. The risk is, if the argument is unsuccessful, we lose credibility at a time when we need it the most.

NASA Background
NASA was conceived at a time of crisis for the United States. On October 4, 1957, the former
Soviet Union launched Sputnik, the first man-made satellite to orbit the Earth. Surprised by the technological feat and alarmed by the prospect of falling behind its Cold War adversary, U.S.
Congress established NASA on October 1, 1958, just days before the anniversary of Sputnik’s success.
NASA was assembled from several existing federal agencies and organizations. The largest of these, the National Advisory Committee for Aeronautics (NACA), supervised the nation’s rocket, jet, and space systems. Primarily a research organization, NACA contributed 8000 employees, an annual budget of $100 million, three research laboratories—Langley Aeronautical Laboratory, Ames
Aeronautical Laboratory, and Lewis Flight Propulsion Laboratory—and two smaller test facilities.
NASA also integrated several other groups, including the Jet Propulsion Laboratory which was managed by the California Institute of Technology for the Army, the Army Ballistic Missile Agency in
Huntsville, Alabama, and space scientists from the Naval Research Laboratory in Maryland.

2
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

603-062

The Apollo Era—Mission to the Moon
On May 25, 1961, three weeks after Russia’s Yuri Gagarin became the first man in space, then
President John F. Kennedy announced “I believe that this nation should commit itself to achieving the goal, before this decade is out, of landing a man on the Moon and returning him safely to Earth.”2
According to one political scientist, the “Moon Project was chosen to symbolize U.S. strength in the head-to-head competition with the Soviet Union.”3
Despite little public support,4 federal funding for the Moon Project quickly reached astronomical levels. The space agency’s budget grew from $500 million in 1960 to $5.2 billion in 1965, or 5.3% of the federal budget.5 The cost of a roundtrip mission to the Moon was estimated at $20 billion ($179 billion in 2001 dollars). Staffing for the Moon Project grew proportionally. In the mid-1960s, NASA staffing levels peaked: 36,000 civil servants in 1966 and 376,700 contractors in 1965.6 One in 50 Americans worked on some aspect of the Apollo program. For the United States, only the Manhattan Project (a military project) and the building of the Panama Canal (a civilian project) were comparable in terms of scale and cost.7
At the time, all NASA centers had experience managing small-scale research projects, but had little experience managing large-scale projects and operations. One former manager remarked,
“NASA had considerable technical depth, but almost no program management experience.”8 NASA established an Apollo program office at its D.C. headquarters and imported an Air Force management group that had supervised the Minutemen Intercontinental Ballistic Missile program.
By centralizing authority for design, engineering, procurement, testing, construction, manufacturing, spare parts, logistics, training, and operations at headquarters, the Apollo management team forced adjustments in the technical cultures that existed at NASA centers.
Skeptical of external input and distrustful of outsiders’ work, centers became more dependent on private industry and headquarters. Centers also prioritized performance, schedule, and cost. Joseph
Gavin, the engineer who led the lunar module project9 for Grumman, recalled:
It took us only a couple of months to learn that there really wasn’t any tradeoff. You absolutely had to give priority to performance. Then you did the best you could to meet the schedule. Costs came . . . third. That may sound irresponsible, but when you think about it, that’s the way things had to be for something like Apollo. If a major project is truly innovative, you cannot possibly know its exact cost and its exact schedule at the beginning. And if in fact

2 Stephen J. Garber and Roger D. Launius, “A Brief History of the National Aeronautics and Space Administration,” June 2001,

, accessed April 21, 2002.
3 Cited in Roger D. Launius, NASA: A History of the U.S. Civil Space Program (Melbourne, FL: Krieger Publishing Company,

1994), p. 61.
4 In May 1961, before Kennedy’s announcement, a Gallup public opinion poll found that 58% of the population was against

(vs. 38% for) the United States spending up to $40 billion to send a man to the Moon. Howard E. McCurdy, Inside NASA: High
Technology and Organizational Change in the U.S. Space Program (Baltimore: Johns Hopkins University Press, 1993), p. 102.
5 Launius, p. 70.
6 McCurdy, p. 101.
7 Launius, p. 55.
8 McCurdy, p. 92.
9 The lunar module was a vehicle that landed astronauts on the Moon and rendezvoused with the orbiting Apollo spacecraft.

3
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

you do know the exact cost and the exact schedule, chances are that the technology is obsolete.10 In July 1969, riding Apollo 11, astronauts Neil Armstrong and Buzz Aldrin became the first humans to land on the Moon and return safely home.11

Post-Apollo Era
In 1970, flush with Apollo’s success, NASA sent a budget proposal to then president Richard
Nixon that included funding for a manned Mars mission, a lunar space station, and a 50-person
Earth-orbiting station serviced by a Space Shuttle.12 In the midst of the Vietnam War, Nixon recognized a diminished public appetite for space exploration spending and limited NASA’s big budget items to the Shuttle program. Congress, similarly attuned to public sentiment, was skeptical of funding even the remaining shuttle component. Top NASA administrators had to sell the shuttle to Congress as a project that would, in large part, pay for itself. The Shuttle, they argued, would be like a bus, routinely ferrying scientists, military and scientific payloads, and commercial satellites back and forth in space. Congress funded the Shuttle, but cut NASA’s overall budget in half.
During the 1970s, NASA accomplished several projects, including Skylab, Apollo-Soyez Test
Project, Project Viking, and Project Voyager. Skylab was an experimental, three person, Earthorbiting space station that used a reconfigured stage of the Saturn V rocket stage as its basic component. The Apollo-Soyez Test Project tested the compatibility of U.S. and USSR docking systems and symbolized lessening tensions between the two cold war adversaries. Project Viking, a billion-dollar mission to Mars, landed two spacecraft on the red planet as well as an orbiting device that returned useful information about Mars’s atmosphere. Project Voyager sent two unmanned spacecraft past Jupiter, Saturn, and the outer planets. Voyager’s pictures of the solar system revolutionized astronomers’ understanding of the outer planets.
In 1977, NASA began building a fleet of four reusable space shuttles. For the Shuttle program,
NASA increased outsourcing to private industry. With management decentralized, some centers returned to their pre-Apollo technical culture. Project managers for certain elements of the Shuttle program felt more accountable to their center than to the Shuttle program.13 Vital program information frequently bypassed the Shuttle Program Manager, who lacked the authority to compel the information-sharing necessary for decision-making. Preoccupied with controlling costs on the
Shuttle, NASA resisted advancing technology, streamlined management, and adopted commercialstyle business practices. The most experienced Center engineers, those who had built the Apollo spacecraft, became involved more with integration and contractor oversight and less with design.
With government workers overseeing contractors’ every move, NASA developed an aversion to risk.14 10 MIT Alumni Association, “Fly me to the moon—Interview with Joseph G. Gavin, Jr., head of the lunar module project,”
Technology Review, July 1, 1994.
11 Just three years later, in 1972, Apollo 17 completed the program’s final mission to the Moon.
12 Rogers Commission Report on space shuttle disaster, , accessed May 21, 2002.
13 McCurdy.
14 Ibid.

4
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

603-062

On April 12, 1981 the first shuttle, Columbia, was launched, and hailed as a sign of continued
American superiority in space technology. After the fourth shuttle flight in 1982, senior NASA officials declared, under congressional pressure, that the Shuttle program was “operational”—i.e., ready to begin routine flights.15 This declaration riled many within the agency who believed that the
Shuttle, given its inherently risky technology systems, would never warrant that description. One
Shuttle program executive said, “to talk about spinning that off as an operational vehicle is kind of crazy to me.”16 In 1985, NASA planned to launch nine shuttle missions and projected an annual rate of 24 flights by 1990. The Shuttle program however, was unprepared for this launch schedule.
In January 1986, the Shuttle Challenger exploded 73 seconds after liftoff, killing all seven crew members. Investigations discovered that the cause was failure of the O-ring, a faulty joint in the
Shuttle’s right solid rocket motor, which had eroded over the nine missions. A flood of government reports, however, also revealed significant flaws in managerial decision-making leading up to
Challenger’s loss, including its risk-management procedures.17 The Shuttle program was grounded for more than 2.5 years as NASA reviewed the Shuttle’s management, design, and manufacture.
A year after Challenger’s loss, NASA was struggling to regain its confidence. Describing the
American space program at that time, the chairman of the National Academy of Sciences Committee on Planetary and Lunar exploration, Robert Pegin, said, “We’re really in a mess.”18 Former astronaut
Eugene Cernan said, “Low morale and frustration is the description of the U.S. space program.”19 By
October 1987, on the anniversary of Sputnik’s successful launch into space, the United States found itself back where it was in 1957—lagging behind the Soviet Union’s space program.20
Corrective measures were attempted. NASA centralized space shuttle authority in Washington, as it had done with the Apollo project. An executive Shuttle team board reviewed technical specifications, not just budgets as it had been doing. In November 1988, after several postponements, the Shuttle Endeavor launched and returned safely to Earth. For NASA, success was short-lived. In
1990, NASA sent the first of its four great space observatories into orbit, the $3 billion Hubble
Telescope, with a flawed mirror that could have been detected and fixed prior to launch. In 1992,
NASA was assigned a new administrator, Daniel Goldin, who described the agency he took over:
“We were reeling from massive failures. The Challenger had blown up. The Hubble was blind in orbit. The weather satellites weren’t working. The shuttle was plagued with mechanical problems. It

15 Safety regulators, however, advised that the shuttle was a developmental, not an operational, system.
16 McCurdy, p. 144.
17 Rogers Commission Report, Appendix F, , accessed July 10, 2002.

According to one of the commissioners on the report, the contractor engineers at Thiokol and NASA management came to accept erosion and blow-by as unavoidable and an acceptable flight risk. NASA and Thiokol accepted escalating risk apparently because they “got away with it last time.” As Commissioner Feynman observed, the decision-making was “a kind of Russian roulette. . . . (The Shuttle) flies (with O-ring erosion) and nothing happens. Then it is suggested, therefore, that the risk is no longer so high for the next flights. We can lower our standards a little bit because we got away with it last time. . . .
You got away with it, but it shouldn’t be done over and over again like that.” (Rogers Commission Report, Chapter 6,
, accessed July 10, 2002.)
18 Lowther, “Soviets racing to final frontier,” Toronto Star, October 4, 1987.
19 Ibid.
20 Ibid.

5
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

would take off on schedule only 23% of the time, and when it did, everyone would hold their breath.”21 The Goldin Era
Goldin’s mandate was to increase mission performance, cut costs, and reduce NASA’s size.
Between 1993 and 1999, Goldin reduced NASA’s workforce by 28% from 25,000 to 18,000 employees
(civil servants). With a virtual hiring freeze in effect, NASA relied increasingly on outside contractors.22 Goldin decentralized agency management and placed shuttle management in the hands of commercial enterprise, giving oversight to Johnson Space Center. For the International Space Station,
Goldin assigned management to Marshall Space Flight Center. Goldin’s most noteworthy management reform was to move NASA away from producing billion-dollar missions that required a decade to develop, build, and launch (e.g., the Hubble Telescope) and toward less expensive and more innovative projects that were produced on a faster timetable. Nicknamed “Faster, Better,
Cheaper” or “FBC,” the reforms were intended to unleash dormant creativity and reduce aversion to risk. According to Goldin:
A lot of our programs were so big and expensive that there was this incredible fear of failure. So I said. . . . we’ll make failure acceptable by breaking programs into smaller pieces and increasing the number and diversity of programs. That way, if there was a failure, we wouldn’t be losing a whole program. . . . You have to set up systems so that when failure occurs it doesn’t propagate across the whole organization or disable an entire single-mission spacecraft.23 Faster, Better, Cheaper
Goldin gave senior managers the freedom to implement the FBC reform as they saw fit: “I set the
‘faster, better, cheaper’ vision, but they came up with the implementation approach.”24
Unfortunately, program managers and project directors did not know how or whether it was possible to implement FBC at the program level. Donna Shirley, retired manager of the Mars exploration program at JPL from 1994 to 1998, recounted a widely held view: “We’ve never been able to define what “better” is in any meaningful way. What is better? More science with simultaneous observations? Incredible resolution with no coverage? You need both. As the joke goes, you can’t have faster, better, and cheaper. Pick two.”25
While the term “better” was open to interpretation, “cheaper” was clearly defined. For example, in 1992, the JPL-managed Mars program received a $260 million budget to research, design, and build the Mars Pathfinder. This budget was a small fraction of the billion dollars NASA had spent on the previous Mars mission, the failed Mars Observer, which disappeared in 1992 as it approached the red
21 “Leading Ferociously,” a conversation with Daniel Goldin, Harvard Business Review (May 2002): 22–25.
22 Mark Carreau and William E. Clayton Jr., “Significant job cuts for NASA/Employees offered early retirement or face termination, Houston Chronicle, March 1, 1995.
23 “Leading Ferociously,” Harvard Business Review, May 2002.
24 Ibid.
25Beth Dickey, “Midcourse Correction,” Government Executive, September 1, 2000, accessed from Factiva, .

6
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

603-062

planet. The Mars program spacecraft were to save on launch costs by using Delta launch rockets, which were smaller than the frequently used Titan rocket (the Mars Observer used a Titan launch vehicle). The smaller-sized Delta limited both the size and weight of subsequent Mars missions. As cost became a high priority, missions were subject to cancellation if budgets exceeded their targets by as little as 15%.
“Faster” meant a compressed development and launch schedule for all FBC projects. Missions would take a few years (at a maximum) to design, build, and launch rather than a decade or more, as with traditional NASA projects. The Mars program was an exquisite experimental setting for this approach to spacecraft development. Because Mars came closest to Earth every 26 months, this periodic alignment created regularly occurring optimal launch opportunities. In 1992, Mars program st officials envisioned trips to Mars every 26 months well into the 21 century.26 One JPL project manager with more than 20 years management experience noted that this development schedule influenced how quickly knowledge could be transferred from one mission to the next:
It takes about 26 months to build a payload or to build an instrument. It takes another year or two to plan, so you certainly can’t learn something in one encounter that will be of any benefit to you in the next. We can’t learn something in the 2001 encounter that will help in
2003. It may help us operate a 2005 mission, but it won’t help us even design a 2005 mission.
To have a design impact, we’ll need to go out yet another two years. So, we need to take advantage of every opportunity every 26 months, but if we make a mistake in one, we can’t usually correct it before the next one.

What Happened with FBC
Initial results of the FBC program were promising. One of the first FBC missions was a widely publicized success. The $260 million Mars Pathfinder spacecraft and its Sojourner rover successfully explored the surface of Mars during the summer of 1997, returning stunning photographs of the planet.27 One JPL senior manager with decades of project management experience recalled the reasons for Pathfinder’s success:
Mars Pathfinder was a great success for many reasons. Anthony Spear, project manager for the mission, was a very senior, experienced person, who had worked on at least a half a dozen projects during his career. He knew, like the back of his hand, successful management and engineering practices. He also knew that the Pathfinder job was very challenging—so difficult that if he got nothing but experienced people, they would tell him that it couldn’t be done. So he had to find enough bright, energetic people, without enough experience to know that what he was asking them to do couldn’t be done. He surrounded himself with an advisory group and a few—very few—people who knew what needed to be done. His team thought that they had thrown away the rulebook and that this was faster, better, cheaper. But Spear had his finger very closely on the whole process. That’s why Pathfinder worked.
Spear’s team never got much coaching. A lot of those people came away from that experience thinking that they knew how to operate successful projects. And so the blessing of
Mars Pathfinder was that we had a tremendously successful mission that answered the challenge of faster, better, cheaper. The curse of it was that we had a lot of people who thought
26 Spacecraft with different propulsion systems averaged about seven months travel time to Mars, when Mars and Earth came closest to one another.
27 The following year, the low-cost Lunar Prospector detected evidence of water ice on the Moon. Howard E. McCurdy, Faster,
Better, Cheaper (Baltimore: Johns Hopkins University Press, 2001), p. 5.

7
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

that they understood exactly how to manage projects when they didn’t. It took us a while, I think, to recover from that.
As project leaders from Pathfinder made their way into management positions on other FBC projects, later FBC missions were unable to repeat consistently Pathfinder’s success. Between 1992 and January 1, 2000, NASA attempted 16 FBC missions. Of these, six failed. (See Exhibit 3.) This success rate was “significantly below spacecraft norms and well below the expectations for reliability that [FBC] advocates placed on the initiatives.”28 In 1999, four FBC projects failed. In March 1999,
NASA lost the Wide-Field Infrared Explorer (WIRE), a cryogenically cooled telescope, when the spacecraft’s protective cover prematurely separated and drifted away. In September, the Mars
Climate Orbiter was lost as it approached Mars. In December, Mars Polar Lander and the twin Deep
Space 2 microprobes that rode with it disappeared without a trace.29 The Mars mission failures cast a pall over the agency, especially at JPL where the spacecraft were developed.

Jet Propulsion Lab
With a 2001 budget of $1.3 billion and a staff of 5,500, JPL was a widely recognized world-leading aerospace and robotics research institution based in southern California, 12 miles northeast of Los
Angeles. JPL began earning its high reputation with its work for the Department of Defense during
World War II. JPL’s name derived from its research and design of the first rocket-assisted take-offs for military aircraft carriers. During the 1970s, 1980s, and 1990s, its successful unmanned missions to planets in the solar system, including Viking, Voyager, Pioneer, Mars Pathfinder, and Cassini missions, revolutionized interplanetary astronomy. After these acclaimed successes, the Mars failures were both surprising and disappointing.
JPL operated much like a university with independent departments, competition for talent among project teams, and highly competent, achievement-oriented personnel. In this environment, two cultural factors stood out. First, a “not-invented-here” attitude, a legacy from the pre-Apollo days, was pervasive; project components developed outside of JPL were viewed with more suspicion than those created at JPL. Second, projects competed for talent. Engineers and scientists were informally ranked internally as Team A, B, or C players. “A-team” staff had the highest talent levels.

Stretched Thin
With the advent of FBC, JPL projects jumped from four to 40 within the space of five years. Holm described the impact of the increase on JPL staffers.
We’ve had to take some really junior folks and put them in relatively senior positions. In these positions they have to learn their trade on the job, with little help from the most experienced staffers. We had 80- to 90-hour workweeks as the norm on some projects. So the idea of people additionally spending a little time mentoring somebody was ludicrous. At the same time, we could not conduct normal risk management, when experienced engineers perform reviews during the project lifecycle. Senior folks were stretched thin as the number of reviews escalated by a factor of ten. On top of that, in the late 1990s and into 2000, people were retiring and experienced personnel were leaving for Internet companies, a process we called the “Dot-gone Syndrome.”
28 McCurdy, Faster Better Cheaper, p. 5.
29 Ibid.

8
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

603-062

The most experienced scientists and engineers went to the largest projects, such as the billiondollar Cassini mission to Saturn, leaving fewer experienced scientists and engineers to head, staff, and review small-scale projects. Smaller projects were forced to compete for experienced staff for their project element teams and review boards, a pivotal and standard part of the design and testing phases. With the most experienced personnel unevenly distributed across projects, both operations and design suffered.
For instance, operation of the 1999 Mars Polar Lander suffered when an overburdened navigator failed to successfully guide the Lander to the surface. In 1999, this one scientist was simultaneously responsible for navigation, design, and operation of several Mars missions that were in different stages of development. He was the only person in charge of navigating the 1999 Mars Polar Lander as it approached Mars. At the same time, he was supervising navigation system development for the 2001
Mars Odyssey and designing the navigation system for the 2003 Mars Exploration Rover mission. He was also expected to participate in peer reviews, write papers, and attend conferences and meetings.
The lack of experienced personnel affected mission design. For instance, the lead investigator on the WIRE mishap review board, Matt Landano (Director of Flight Safety at JPL), pointed to a lack of oversight by experienced personnel over WIRE’s design:
My investigation found a very simple, straightforward fault. It was a fragile design, one that clearly violated an obvious [design] principle. If we had had some of our more experienced individuals on the job—individuals who had done this kind of work before—
I guarantee the flawed design would not have gotten through; they would not have approved it. But many of the old guys were working on Cassini, a big unmanned mission to Saturn, so we didn’t have enough of those guys to spread among these other projects. And the WIRE project had to get done in a very short time, so the amount of reviews and independent reviews one could do was shrunk by the pace and number of projects. The WIRE staff all wanted to do the right thing, they all worked hard. But they didn’t have the experience to know when they were doing something wrong. Sometimes you don’t know enough to even know that there is a problem.
Even when review boards were staffed with appropriate personnel, there was little guarantee that the experts could be effective. Though review boards were part of the standard course of mission development, experts sometimes had difficulty “getting up to speed” on the different projects, given their responsibilities for their own projects and the fast pace and inherent complexity of the projects under review. One senior manager who supervised several reviews overheard project managers complaining, “We’ve been over this ground before. They didn’t tell us anything we didn’t already know.” When experts were right on the issues, the project team sometimes resisted their feedback.
One unit director said, “That’s the kind of disconnect that can occur. The challenge for the project manager is to make sure that those messages [from the experts] are heard, that they get worked on right way. Those problems didn’t happen so much in the past because these experts would have been on the project; there wasn’t this separation of internal expertise and external expertise.”

Short-changing the Future
With demands to cut costs and maximize mission performance, decisions favored present over future missions. From a program perspective, this orientation was counterproductive. A senior JPL manager explained:
The successful 1997 Mars Pathfinder had a radio beacon on it—a sort of “I’m still OK” signal. As it went through the atmosphere, it sent out a ping, and you could follow it as it
9
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

traveled through the atmosphere. You couldn’t see it in real time, but after you did some processing you could see it bounce and roll as it hit the ground. The beacon was there primarily to let the team know that it made it to the surface. If something screwed up, at least they knew that they made it to the ground. The Mars Lander did not have a similar beacon and as it hit the atmosphere, we lost transmission, and we never heard from it again. Nobody knows what happened, or whether it got all the way to the surface. The same sort of beacon on the Mars Lander would have eliminated a whole set of possibilities. The Mars Lander managers considered a beacon, but their investment criteria were to maximize mission success, so they didn’t put it on because the beacon just reports facts; it wouldn’t help the mission be successful. The beacon can help only the next mission. So, because the beacon wasn’t on the
Mars Lander, we don’t know how to prevent something similar from happening again. The project managers were thinking about project success, not program success.
Unable to learn from the Mars Polar Lander, Mars program officials would not attempt a powered descent to the Martian surface for many years. The 2003 Mars mission would employ the successful
1997 Pathfinder descent method, which utilized parachutes and airbags.

Using (Ignoring) the Past
Program managers and project leaders differed over the significance of knowledge capture and its uses. One senior manager at JPL believed that, “advancing the creation of new knowledge is more important than capturing old knowledge.” This was especially true for software and IT development, said the manager. In practice, this belief sometimes led to inefficiencies. Another manager recalled that software engineers created new programming for successive Mars-related navigation systems because they “questioned whether they understood the programming that had been used before.”
The manager said that creating new programming was a “waste of resources.” He added, “Some of the push for getting others to rely on others’ work has to be top-down from upper management. But it’s culturally challenging and against the grain of how we’ve done past missions. It’s asking project managers to swallow a different kind of risk—to trust stuff that others have produced.”

Call for Better Knowledge Management (FBC Reviewed)
After the three Mars failures, federal and independent investigations reviewed FBC missions and the Mars program. As Exhibit 4 shows, they found several recurring problems with FBC and other projects conducted during the 1990s: Thomas Young, a retired Lockheed Martin executive who led a team that examined NASA’s Mars program, said that the failed 1999 Mars missions were at least 30% under-funded. Mission engineers were forced to work 60 hours a week or more. Many had little time to recheck their work in the painstaking way they would have preferred. “The mistakes that took place on Mars Climate Orbiter and Mars Polar Lander were mistakes in areas where we know how to do things correctly,” said Young. “Taking risks with things you know how to do are not the kind of risks to take.”30 Another report pointed to an environment where increasing time and budget pressures broke down lines of communication and prevented people from internalizing and applying previous lessons.31 In his March 1999 review of FBC missions, Anthony Spear, the retired project manager for the successful 1997 Mars Pathfinder mission, said, “We need to slow down some, not

30 Rex Graham, “Missing the Mark,” Astronomy, July 2000, accessed from Factiva, .
31 See the report at , accessed June 20, 2002.

10
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

603-062

rush too quickly into important programs and projects, plan and implement them more carefully, and move away from fixations on costs and near-term gain.”32 Holm added:
We’re in the middle of trying to figure out how to conduct innovative space exploration on constrained timetables, with limited scientific payloads. We’ve had calls to move more slowly, but JPL staff can be successful only if they apply for grants and bid for projects. Yet, if they win too many, they become overburdened. That’s what’s happened over the past few years. In some ways, we’ve been victims of our own success.
John Casani, a senior manager with 40 years of project management experience who was brought back from retirement in 2001 to prevent mishaps such as those on the 1999 Mars missions, added:
With more frequent missions, there is an issue of knowledge transfer from one mission to the next. One issue is, if you’ve got the same people doing the next mission, then the knowledge is embedded in those people, but that’s very rarely possible to do and rarely happens. So you rely on the institution to capture that knowledge, by whatever means, and then to pass it on effectively to the next group of people. Before FBC, we had time to do that.
We had written down the plans and the implementation approaches having to do with quality assurance, with testing, with requirements definition, with functional definitions, with the whole spectrum of what it takes to plan and execute a project. These documents probably occupied 10 or 15 inches on a shelf. Before we started the next mission, we would get the project team together and would literally redline all of those documents, updating them with lessons from the previous project. Now when you compress the cycle time, that’s not readily possible. A partial solution would be to capture more information on the fly and not wait to the end of the project to gather up the knowledge that was learned from that project and then try to apply it to the next. Some information is not available until you’re nearly through the mission.
So that can only be a partial answer. But even if we don’t have the whole answer, we cannot be tyrannized by the notion that the only way to conduct missions is to spread them out five or ten years.

Impact of Impending Retirements
As some of the most successful and experienced NASA employees neared retirement, some JPL and NASA staffers were concerned about what their loss would mean to the space agency. Landano,
Director of Flight Safety at JPL, was concerned about risk assessment:
Less experienced team leaders may take a lot longer to come to an answer to a problem, or to an issue. They’ll think about it, talk about it, and analyze it. These are the right things to do, but when you’re under a tight schedule, you need a solution that is probably 90% right in one day, as opposed to a solution that is 95% or 98% right in three weeks. And if those people are not experienced enough, they could end up solving the wrong problem from a riskmanagement point of view. Or if the risk is identified too late in the project, they may not be able to repair it or to mitigate it, and then we’re in a position of having to accept that risk
After you’ve worked through a project or two, you appreciate how all the components interact. When you change A, you have a sense of how it affects B, C, and D. But once you think you understand it, something tends to bite you—what I call the “unknown unknowns.”

32 Cited in Dickey, p. 2.

11
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

You need to have the technical margin to cover the unknown unknowns in your system designs. If you hold too much margin, you don’t accomplish what you ought to. But if you don’t hold enough margin, you could find yourself at risk. You have to balance risk management and cost. And that balance is very difficult to learn without first-hand experience on several projects.
John Casani was concerned that losing experienced personnel would lead to the loss of knowledge that would be not only difficult to capture, but useful for future missions.
The Mars Observer was lost just a few days before it was scheduled to go into orbit around
Mars. We spent a long time trying to understand what the cause of failure was. And finally we figured out that the tanks to the propulsion system had a regulator that failed. The regulator was supposed to pressurize the contents of the tanks, but, for reasons we couldn’t figure out, the regulator failed though we knew it had been exposed to nitrogen tectoxide. Years afterwards, sitting around the table with a couple of people discussing this problem, one retired fellow said, “Well, what was the regulator?” And he was told, and he said, “Well that regulator has an inlet filter, and has some flux that is residual from the manufacturing process that’s incompatible over the long haul with nitrogen tectoxide. The regulator would fail in a long-duration mission.” The same regulator in the same propulsion system had been flown at least six times, but in missions lasting weeks, not months. The chemical reaction that took place takes six or seven months to be a problem, and that’s exactly how long it took for the Mars
Observer to travel to Mars. Now, this was a piece of knowledge in that one man’s head, and he would be very unlikely to volunteer that piece of information, even if you asked him, “tell me everything you know.”

Knowledge Management at NASA
In January 2000, NASA launched its agency-wide knowledge management initiative. Lee
Holcomb sponsored the project and appointed Holm team leader, following her success with knowledge management programs at JPL. Holm’s 40-member team (comprising part-time participants from across the agency) was charged with developing a strategic plan and coordinating its implementation with NASA centers. (See Exhibit 5 for a description of NASA’s 2001 knowledge management goals and activities.) Each NASA center was responsible for developing its own knowledge management systems. By 2002, seven of ten NASA centers had a formal KM initiative in place. Holcomb explained that the agency-wide KM plan needed to cultivate a more open technical culture and garner political support:
Competition among centers for projects and funding can be an engine for creativity, but it also sustains a culture of privatizing knowledge. Scientists and engineers sometimes don’t include material in their reports that might compromise their competitive advantage. One human resources-led initiative is to recognize and reward knowledge sharing and mentoring.
But this effort has yet to gain widespread political support within the agency. No one is openly against it, but in some meetings I’ve attended, some are for it, and others are silent. To support this kind of initiative with various internal constituencies, I’ve had to become a chameleon with a cast-iron skin.
Another senior manager with more than 30 years at NASA identified succession planning as a place to target KM initiatives.

12
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

603-062

We have no formal process for transferring knowledge from people who are leaving highlevel management positions, such as Associate Administrators (AAs) who oversee NASA centers and program level engineers. (See Exhibit 6 for NASA organization chart.) There are lots of informal agreements between AAs and sometimes between Center directors. I know of several occasions where a deal between two AAs fell apart when one guy retired. Technical
AAs are being replaced with generalists, which may work from the standpoint of implementing policy, but is less effective from the standpoint of creating policy.
The same manager believed that about 75% of retirees remained available to the agency as consultants. Yet, as Holm explained, “that may be true at headquarters and at some centers, but it’s not true at every center and certainly won’t be true for the agency in a few years when these NASA retirees retire from the consulting life.” In the words of one senior manager at JPL, the loss of this collective wisdom made the agency “less wise.”
In her strategic plan for NASA’s knowledge management system, Holm summarized the challenges facing the space agency:
!

How do we successfully transfer information gleaned from one mission to other missions and across generations of people who come in and out of the agency? Some of our missions will take 50 years to complete. Long-duration projects tend to have older technologies that require extensive knowledge of their original construction. We really need to be able to capture, organize, and store knowledge so that it’s accessible to the right people at the right time.

!

How do we get hugely diverse, geographically distributed communities to work together on complex space missions? Technically that’s not too hard, but politically it’s tricky because all sorts of other government agencies like the Department of State come into play. So we have to understand how and when to share technical information with our international and industrial partners who often are also our competitors.

!

How do we manage the knowledge that we already have? NASA has four million public Web pages. We get two billion hits per month on our Web sites. We have a huge amount of information that the American public is interested in seeing. How do we get the information into the hands of people so that they can write school reports, do business with NASA, bid on new proposals and participate in our future space missions?

Responding to several constituencies, Holm developed a strategy based on a performanceoriented definition of knowledge management: “Knowledge management is getting the right information to the right people at the right time, and helping people create knowledge and share and act upon information in ways that will measurably improve the performance of NASA and its partners.”33 At the project level, this meant (a) providing an engineer with the history of design decisions on previous projects; (b) giving a project manager access to the best risk-management practices and tools when he or she needs them; and (c) providing the time for a senior scientist to mentor a promising young star. Holm said, “Some of this can be accomplished through clever information technology solutions and improved access to NASA’s already rich, explicit information.
The larger part of this relates to capturing the tacit knowledge of our workforce and effecting cultural changes that will encourage people to share what they know.” Toward these ends, Holm supported a three-pronged set of initiatives that included:

33 2001 NASA Knowledge Management Strategic Plan.

13
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Improving Documentation—The Lessons Learned Information System (LLIS)
NASA’s formal mechanism for sharing lessons learned across the agency was its Lessons Learned
Information System or LLIS, which contained lessons from the operation or design of particular missions and project elements. (See Exhibit 7 for a sample of a Lessons Learned entry in the JPL LLIS database.) Project managers were required to review LLIS on an ongoing basis. Several NASA centers, including JPL, maintained LLISs geared toward their own staff. Holm’s team focused on redesigning the LLIS to address a variety of concerns. In a 2002 report, the General Accounting Office recognized that managers were reluctant to share negative lessons for fear that they might not be viewed as good project managers, and that there was little time for lessons learning to take place. One manager stated, “Until we can adopt a culture that admits frankly to what really worked and didn’t work, I find many of these tools to be suspect.”34 The GAO report also revealed significant inadequacies in the effectiveness of LLIS:
A survey we conducted of all NASA program and project managers revealed fundamental weaknesses in the collection and sharing of lessons learned agency-wide. Although NASA’s processes and procedures require that program and project managers review and apply lessons learned throughout a program’s or project’s life cycle, our survey found that managers do not routinely identify, collect, or share lessons. Respondents indicated that LLIS . . . is not the primary source for lessons learning. Instead, managers identified program reviews and informal discussions with colleagues as their principal sources for lessons learned. One reason
LLIS is not widely used, according to one center official, is because its lessons cover so many topics that it is difficult to search for an applicable lesson. Another respondent indicated that it is difficult to weed through irrelevant lessons to get to the few jewels.35

Developing a Web-based Portal
NASA’s Web site, www.nasa.gov, encompassed more than four million Web pages, thousands of databases and electronic repositories, petabytes of mission and planetary data, and millions of online reports. In 2001, NASA Web sites counted close to two billion hits a month. It was up to the KM group to make this vast amount of information useful to internal and external agency constituents.
For internal use, Holm’s team developed Inside JPL and Inside NASA pages—customizable portals that staffers used to search efficiently for information they needed on an ongoing basis. (See Exhibit
A-1 for screen capture of a sample page.) These pages could be tailored to access project-related information from all over NASA, universities, and industry. In addition, several Web-based tools were being developed to facilitate collaboration among project team members who were spread across centers.

Academy of Program and Project Leadership (APPL)
Available to all NASA staff, APPL offered individual career development opportunities, online management tools, and performance support for project teams. Through its Knowledge Sharing
Initiative (KSI), APPL supported best-practice communities and a culture of sharing knowledge through grass-roots efforts across the agency. KSI worked with centers to identify best-of-best managers and bring them together to share project-related stories in a series of workshops. Once or

34 U.S. General Accounting Office, “NASA: Better Mechanisms Needed for Sharing Lessons Learned” (GAO-02-195), January
2002, p. 5.
35 Ibid.

14
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

603-062

twice a year, KSI brought a group of such managers from all over NASA to meet one another. The head of APPL, Edward Hoffman, contended that APPL was crucial to NASA’s efforts to capture and share expertise that was impossible to load onto databases:
Databases and IT can’t address mission complexity—there are just too many ways for things to go wrong. Through our career development programs, project managers are exposed to living models of excellent NASA managers. Some of these same experts are made available as consultants to ongoing projects. And with KSI, managers get to know each other in a way that promotes collaboration and networking.

Moving Forward
Jeanne Holm stepped away from the viewing platform in JPL’s mission control center and summarized some of the outstanding issues facing her KM team:
Some of my own team members do not seem to understand the significance of the knowledge loss issue. One team member interviewed several JPL project managers, none of whom thought that knowledge loss was a big issue. I said, “Are those the project managers working on three- to five-year missions?” Yes. Well, they don’t think it’s a big issue because they have a very short-term task ahead of them. It’s from an institutional perspective, an organizational memory perspective, that it’s a problem. “Can we replicate Apollo?” No. Well, is that a problem? Yes, actually it is. Not that we want to do Apollo over again, but we want to be able to reuse the knowledge from components of that mission, and we can’t because we never captured it. We don’t want that to happen with the shuttle and the space station, which are going to be around a long time and will depend on parts that have already been around for many years.
The fact that many of our most experienced personnel are about to leave the agency is recognized at the highest levels, but opinions differ as to how to address this fact. In large part, our IT initiatives are creating a virtual presence of these individuals—their management styles, decisions, and decision-making—on the Internet. Many senior managers are most comfortable supporting these initiatives, such as directories of experts and LLIS. But other initiatives need senior-level support, such as expanding our mentoring programs, creating oral histories, and creating the time and incentives for mentoring.
So, recently I have tried to expand our base of support within the agency for knowledge management initiatives that will revamp the culture of knowledge management. I recruited two other senior-level sponsors—the chief engineer to support cultural changes in the engineering processes, and the head of our HR and education department to support cultural change across the agency. The problem is, to begin the cross-agency cultural changes necessary to make this work, we’ll probably need a larger budget (see Exhibit 2). But this will be ineffective unless the agency, at many different levels, owns the problem. If we fail to make the argument for cultural change and a larger budget at the right time and in the right way, we’ll lose credibility and delay the needed change. And delays are what we cannot afford at this point. 15
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Exhibit 1

Annual NASA KM Budget (Selected Years: 2001–2003)

NASA
Fiscal Years
2001

2002

2003

Exhibit 2

Activities
Lessons learned information system redesign.
Develop an experts’ directory to locate people with specific expertise.
Develop an enterprise portal.
Expand portal prototype to an internal
Agency-wide pilot.
Deploy collaborative tools for teams.
Deploy NASA public portal and Web redesign.
Capture design knowledge and decisions.

Annual
Allocations

Annual
Operating Costs

$150,000

$100,000

$190,000
$175,000

$150,000
N/A

$500,000
$300,000

$275,000
$400,000

$3,000,000
$300,000

$800,000
Unknown

Budget Recommendation Options

OPTION 1
IT Budget
Proposal
2003–2005

Total KM
Allocation
Included additional IT projects

$2–$4 million

Integrating document-management solutions and processes for accessing and archiving project information.
Adoption and integration of standards for exchange of engineering data between NASA and its partners.
Integrating a variety of systems to provide better responses to searches (when you find a document, you can contact the author, related experts, and find related discussions or material). Creating decision support systems for flight project engineers.
Deploying state-of-the-art e-learning technologies.

OPTION 2
Culture Change
Budget Proposal
2003–2005

$5.85 million
Formal KM organization (such as a Chief Knowledge Officer or
KM Program) to ensure infusion of KM practices across the organization and embedding them in the processes.
Ongoing operational costs for pre-existing KM systems.
Changes in the incentives program to encourage knowledge sharing. Providing time for key employees to share and mentor.
Capturing key employee knowledge about inexplicit systems.

Source:

Internal NASA documents, Jeanne Holm.

16
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Exhibit 3

603-062

The First “Faster, Better, Cheaper” Projects and Outcomes

Project

Dates

Purpose

Outcome

NEAR (Near Earth
Asteroid Rendezvous)

Launched February 17,
1996.

Orbital insertion achieved in February 2000.

Mars Pathfinder

Launched December 4,
1996.

Lunar Prospector

Launched January 6,
1998.

Stardust

Launched February 7,
1999.

Completed flyby of asteroid Mathilde in
June 1997 preparatory to orbiting asteroid
Eros.
Landed on Mars; dispatched Sojourner rover to conduct studies of Ares Vallis floodplain.
Orbited the Moon; discovered evidence of water ice at the Moon’s north and south poles.
Scheduled to encounter comet Wild 2 in 2004 and return samples of comet material to Earth in 2006.

Flight-tested 12 new technologies, including an ion propulsion engine. Contained two microprobes designed to penetrate the Martian subsurface and search for evidence of water ice. Damaged during encounter with asteroid
Braille in July 1999.

Encountered difficulties with aerobraking maneuvers; delayed entry into final orbit.
Lost when it passed too close to Mars, a result of miscommunication about the units of measurement used to calculate position of the spacecraft.
Lost during the entrylanding phase.

DISCOVERY PROGRAM

Landed on Mars
July 4, 1997.

Mission completed
July 1999.

N/A

NEW MILLENNIUM
PROGRAM
Deep Space 1

Launched October 24,
1998; three months late. Deep Space 2

Launched with the Mars
Polar Lander on
January 3, 1999.

Both probes disappeared during the landing phase. MARS SURVEYOR
PROGRAM
Mars Global Surveyor

Launched November 7,
1996. Reached Mars
September 11, 1997.

Undertook extensive mapping activities of
Mars.

Mars Climate Orbiter

Launched December 11,
1998.

Scheduled to arrive
September 1999, and provide detailed information about the surface and climate of
Mars.

Mars Polar Lander

Launched January 3,
1999. Scheduled to land on December 3,
1999.

Investigate an area near the south pole of Mars.

17
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Project

Dates

Purpose

Outcome

Solar, Anomalous, and Magnetospheric
Particle Explorer
(SAMPLEX)

Launched into Earth orbit
July 3, 1992.

Conducted wide-ranging investigations of local interstellar matter and solar material.

Successful

Fast Auroral Snapshot
Explorer (FAST)

Launched into Earth orbit
August 21, 1996; two years late due to problem with the
Pegasus XL launch vehicle. Launched December 4,
1998; 3.5 years late due to problems with the Pegasus XL launch vehicle.
Launched April 2, 1998.

Conducted extensive investigations of physical processes that produce aurora.

Successful

Conducted telescope investigations of the composition of dense interstellar clouds where new stars form.
Employed a special telescope to take spectacular highresolution photographs and conduct investigations of the
Sun’s atmosphere.
Contained a cryogenically cooled telescope and arrays of infrared detectors to study the evolution of galaxies.

Successful

Contained advanced
Earth-sensing
instruments.

Spun out of control four days after launch; total loss. Designated to carry a variety of instruments to study the Earth and
Sun, including a veryhigh-resolution camera for taking stereo images of the Earth.

Project terminated
February 1998 due to cost overruns and schedule delays.

SMALL EXPLORER
PROGRAM (SMEX)

Submillimeter Wave
Astronomy Satellite
(SWAS)

Transition Region and
Coronal Explorer
(TRACE)

Wide-Field Infrared
Explorer (WIRE)

Launched March 4, 1999.

Successful

Declared a total loss after the telescope’s protective cover was prematurely ejected shortly after launch, causing the frozen hydrogen needed to cool the telescope to vent into space.

SMALL SATELLITE
TECHNOLOGY
INITIATIVE
Lewis

Launched into Earth orbit
August 22, 1997; one year late due to problems with the
Lockheed Martin
Athena launch vehicle. Clark Earth Observing
Satellite

Source:

Adapted from Howard E. McCurdy, Faster, Better, Cheaper (Baltimore: Johns Hopkins University Press, 2001).

18
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Exhibit 4

603-062

Reasons for Several NASA Mission Failures

MAJOR MISHAP REVIEWS

REASONS FOR FAILURE
Cost and Schedule
Constraints

WIRE

Mars
Climate
Orbiter

Mars
Polar
Lander

!

Insufficient Risk Assessment and Planning

Lewis

SOHOa

!

!

!

Insufficient Testing

!

!

Poor Team Communication

!

!

Inattention to Quality and Safety

DC-Xb

!

!

!

!

!

Underestimation of
Complexity and Technology
Maturity

!

!

!

!

!

!

!

!

!

!

!

!

Inadequate Review
Process

!

!

!

Design Errors

!

!

!

Inadequate System
Engineering

!

!

!

!

!

Inadequate or Under-trained
Staff

Source:

Mars
Observer

!

!

!

!

!

!

!

!

!

Adapted from U.S. General Accounting Office, “NASA: Better Mechanisms Needed for Sharing Lessons Learned”
(GAO-02-195), January 2002, p. 12.

aSOHO: Solar and Heliospheric Observatory. bDC-X: Delta Clipper–Experimental.

19
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Exhibit 5

NASA Knowledge Management (KM) Goals and Activities

NASA FIVE-YEAR KM STRATEGY
(2001–2006)
Purpose

Goals
!

Encourage storytelling to share lessons learned.

!

Enhance knowledge capture.

!

Activities

Exploit expert systems for better decision making.

!

APPL (Academy of program and project leadership) and its
Knowledge Sharing Initiative,
KSI), HR-led Recognition
Management Study to reward mentoring. !

Identify and capture the information that exists across the Agency.

Process-Based Mission
Assurance collects video nuggets and best practices for risk management, redesign of
Lessons Learned Information
System (LLIS).

!

LLIS restructuring, portal development, integrated directories, portal development,
InsideNASA customized Web pages. Source:

More efficiently and effectively manage current information.
Enhance system integration and data mining.

!

Develop techniques and tools to enable remote teams to collaborate. !
!

Help to efficiently manage the
Agency’s knowledge resources.

Utilize intelligent agents to deliver “just-in-time” information.

!

“Know-bots” to search Web for detailed information.

!

Enable remote collaboration through tools and team training.

!

!

Support communities of practice through electronic and traditional processes.

Developing Web-based collaborative environments, such as a document manager that has an action-item tracker, a calendar, a team directory, a threaded discussion tool, and an activity log.

Adapted from NASA Knowledge Management Strategic Plan, 2001.

20
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Exhibit 6

603-062

NASA Organization

By 2002, NASA was composed of nine field centers, five field facilities, and the Jet Propulsion
Laboratory. NASA’s missions operated under five separate Enterprise offices, which supervised activities and projects under their purview. The five Enterprise offices were as follows:
!

Aerospace Technology—Mission was to pioneer identification, development, verification, transfer, application, and commercialization of high-payoff aeronautics and space transportation technologies. !

Biological and Physical Research—Mission was to conduct basic and applied research to support human exploration of space and to take advantage of the space environment as a laboratory for scientific, technological, and commercial research.

!

Earth Science—Mission was to use outer space to provide unique information about Earth’s environment. With research and industry partners, this Enterprise supported complex environmental policy and future economic investment decisions.

!

Human Exploration and Development of Space—Mission was to explore, use, and enable space development and to expand human experience into the far reaches of space.

!

Space Science—Mission was to solve mysteries of the universe, explore the solar system, discover planets around other stars, search for life beyond Earth, chart evolution of the universe, and understand its galaxies, stars, planets, and life.

Source:

NASA Web site, , accessed July 22, 2002.

21
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Exhibit 7

Lessons Learned Information System (LLIS) Sample

Mars Climate Orbiter Mishap Investigation Board—Phase I Report

!
!
!
!

Lesson Number: 0641
Lesson Date: 01-dec-1999
Submitting Organization: HQ
Submitted by: Pete Rutledge

Description of Driving Event:
The Mars Climate Orbiter (MCO) Mission objective was to orbit Mars as the first interplanetary weather satellite and provide a communications relay for the Mars Polar Lander
(MPL) which [was] due to reach Mars in December 1999. The
MCO was launched on December 11, 1998, and was lost sometime following the spacecraft’s entry into Mars occultation during the Mars Orbit Insertion (MOI) maneuver.
The spacecraft’s carrier signal was last seen at approximately
09:04:52 UTC on Thursday, September 23, 1999.

Lesson(s) Learned:
The MCO Mishap Investigation board (MIB) has determined that the root cause for the loss of the MCO spacecraft was the failure to use metric units in the coding of a ground software file, “Small Forces,” used in trajectory models. Specifically, thruster performance data in English units instead of metric units was used in the software application code titled SM_FORCES (small forces). A file called Angular
Momentum Desaturation (AMD) contained the output data from the SM_FORCES software. The data in the AMD file was required to be in metric units per existing software interface documentation, and the trajectory modelers assumed the data was provided in metric units per the requirements.
During the nine-month journey from Earth to Mars, propulsion maneuvers were periodically performed to remove angular momentum buildup in the on-board reaction wheels (flywheels). These Angular
Momentum Desaturation (AMD) events occurred 10–14 times more often than was expected by the operations navigation team. This was because the MCO solar array was asymmetrical relative to the spacecraft body as compared to Mars Global Surveyor (MGS) which had symmetrical solar arrays. This asymmetric effect significantly increased the Sun-induced (solar pressure-induced) momentum buildup on the spacecraft. The increased AMD events coupled with the fact that the angular momentum (impulse) data was in English, rather than metric, units, resulted in small errors being introduced in the trajectory estimate over the course of the nine-month journey. At the time of Mars insertion, the spacecraft trajectory was approximately 170 kilometers lower than planned. As a result, MCO either was destroyed in the atmosphere or re-entered heliocentric space after leaving Mars’ atmosphere.
The Board recognizes that mistakes occur on spacecraft projects. However, sufficient processes are usually in place on projects to catch these mistakes before they become critical to mission success. Unfortunately for MCO, the root cause was not caught by the processes in place in the MCO project. A summary of the findings, contributing causes, and MPL recommendations are listed below. These are described in more detail in the body of this report along with the MCO and MPL observations and recommendations.

22
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

603-062

Root Cause: Failure to use metric units in the coding of a ground software file, “Small Forces,” used in trajectory models
Contributing Causes:
1.
2.
3.
4.
5.
6.
7.
8.

Undetected mis-modeling of spacecraft velocity changes
Navigation Team unfamiliar with spacecraft
Trajectory correction maneuver number 5 not performed
System engineering process did not adequately address transition from development to operations
Inadequate communications between project elements
Inadequate operations Navigation Team staffing
Inadequate training
Verification and validation process did not adequately address ground software

Recommendation(s):
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.

Verify the consistent use of units throughout the MPL spacecraft design and operations
Conduct software audit for specification compliance on all data transferred between JPL and
Lockheed Martin Astronautics
Verify Small Forces models used for MPL
Compare prime MPL navigation projections with projections by alternate navigation methods
Train Navigation Team in spacecraft design and operations
Prepare for possibility of executing trajectory correction maneuver number 5
Establish MPL systems organization to concentrate on trajectory correction maneuver number 5 and entry, descent, and landing operations
Take steps to improve communications
Augment Operations Team staff with experienced people to support entry, descent, and landing
Train entire MPL Team and encourage use of Incident, Surprise, Anomaly process
Develop and execute systems verification matrix for all requirements
Conduct independent reviews on all mission critical events
Construct a fault tree analysis for remainder of MPL mission
Assign overall Mission Manager
Perform thermal analysis of thrusters feedline heaters and consider use of pre-conditioning pulses
Reexamine propulsion subsystem operations during entry, descent, and landing

Evidence of Recurrence Control Effectiveness:
N/A
Applicable NASA Enterprise(s):
Space Science
Applicable Crosscutting Process(es):
Provide Aerospace Products & Capabilities: Implementation
Additional Key Phrase(s):
Configuration Management
Flight Operations
Flight Equipment
Software
Spacecraft
Test and Verification
Mishap Report Reference(s):
Mars Climate Orbiter Mishap Investigation Board Phase I Report
Approval Info:
Approval Date: 01-dec-1999
Approval Name: Eric Raynor
Source:

NASA Web site, , accessed July 12, 2002.

23
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Appendix
JPL Knowledge Management Initiatives
Knowledge management at JPL began two years before the Mars failures. Many of its KM solutions were a model for all NASA centers. Holm, whose office was at JPL, split her time between knowledge management projects for the entire agency and those for JPL. According to Holm, “While
I live at the Jet Propulsion Laboratory, I also lead the NASA efforts.” No one was devoted full-time for NASA knowledge management programs. JPL’s KM initiatives comprised an assortment of formal, pilot, and informal projects. These initiatives supported several KM goals (see Exhibit A-1).

Formal KM Projects
One of JPL’s first KM projects was Project Libraries (see Exhibit A-2), which effectively organized project-related documents. It also served as a place to maintain information for delayed projects.
Holm described some processes and advantages of the Libraries:
Project Libraries took a commercial off-the-shelf package (Xerox's DocuShare) and bundled
JPL-specific processes around it in a way that met key project needs. We worked with projects to select a "librarian" who was already on their staff and developed a community forum for training and help for these people. They became extended members of the KM team on each project. Projects often spent 6 to 10 months and $200K per year putting their documents in an online place where people could share them. Project Libraries, which are on a full-cost recovery and so charge for their services, cost projects on average $10K per year for a more capable product that meets regulatory compliance, and is up and running in five days. This laid the groundwork for our long-term electronic archive, which allows people to quickly find examples of good documents, decisions, or practices used on previous projects—information that was previously unavailable after a project was completed. In one example, a Mars vehicle, the Athena Rover, was under development when we had a set of back-to-back failures on our
Mars missions. The Athena Rover was cancelled. We worked with their Project Librarian to sort through the information in their Project Library and save appropriate information. We placed them in the electronic archive, which allowed other projects and our investigation teams full access to their documentation, but locked it down at the point of archive. Several months later, work began on the successor to Athena and their library was up and running in a couple of hours. Such a practice was unthinkable previously.
These documents were accessible through Project Libraries and Inside JPL (see Exhibit A-3 for a representative screen capture). Other formal initiatives included:
!

An Internet-based KnowWho Directory of 1200 key experts willing and able to answer questions from JPL staff (see Exhibit A-4 for a representative screen capture).

!

An Internet-based Technical Questions Database that attempted to create a virtual presence of
JPL’s best engineers and managers for project teams preparing for reviews. During project development and JPL’s formal review process, managers were required to use the Technical
Questions database, which was built upon questions solicited from JPL technical experts.
Holm was also experimenting with several pilot projects to capture an expert’s experience.
(See Exhibit A-5 for a representative screen capture.) Holm described some of the uses of this database: 24
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

603-062

These are the kind of systems people use at specific points in the project lifecycle to get a better understanding of what they need to do next. So we see cycles of use on these systems related to the reviews of key milestones on our projects. We originally thought that we’d get a lot of younger people looking at the systems, but what we found is a lot of mid-level folks, who have grown in the organization without the rich experience of deputy positions—people with gaps in their knowledge. They come to look at the system when they’re preparing to serve as a reviewer. So they come to refresh their knowledge on areas about which they don’t have as much experiential knowledge. So we see unexpected behaviors, but we know that all the users find the system beneficial.
!

A two-volume series on design principles and practices that apply to basic components of interplanetary missions, such as communication, navigation, and propulsion. According to
Matt Landano, author of the series:
There are specific things that you may learn on one mission that may not be applicable to another mission because each mission is unique in certain aspects.
But the fundamentals are basically the same: Here’s the mission. Here’s what we must accomplish. Make sure that if something goes wrong something else can replace its function—what we call ‘redundancy.’ Some project elements should be redundant but not others. These books help readers judge what to make redundant in different situations.

Pilot KM Initiatives
These initiatives were designed to capture specialized knowledge from experienced personnel and perspectives on project-based achievements from project teams.
!

Personal knowledge organizers gathered oral histories from experienced individuals about the impact of significant events on JPL culture and missions (such as the loss of the 1999 Mars missions), timing issues that were critical to their job success, and key artifacts and publications they produced or referenced frequently.

!

Legacy reviews, conducted at the end of each project, catalogued the physical or intellectual legacy of a project (such as a new lab, a new process, or key understandings of a planet). KM personnel attended design meetings and captured the “decision tree” of particular discussions
(e.g., what launch vehicle or what power source to use). As Holm explained: “The decision tree was useful to current and future projects. It documented for everyone why a decision was reached (no more wondering later in the design process whether another option had been investigated). For future projects, it offered a history of project decision-making.”

Informal KM Initiatives
JPL established seven Centers of Excellence to develop specialized knowledge, hardware, and software in disciplines that were key to enabling new classes of future NASA missions. (See
Exhibit A-6 for a description of each Center.)

25
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Exhibit A-1

JPL KM Initiatives

Category

Goal

Activities

Document and Data Management

Support the entire lifecycle of project information As of 2001, Project Libraries
(aka DocuShare)
Moving to an electronic archive and integrated authoring environment

Expert Connections

Help find people with the answers

Includes profiles of ~1200 technical experts Standards (http://step.jpl.nasa.go)

Adopt standards for core metadata, name spaces, and engineering models

Advocacy

Knowledge Capture

Investigate how teams create, capture, and share knowledge

Oral Histories and Legacy reviews

Knowledge Navigation

Use portals, taxonomies, and enhanced searching to help gather information for individuals and communities

Technical Questions Database
Inside JPL portal
Inside NASA pilot

Source:

Jeanne Holm, “Knowledge Sharing at NASA,” PowerPoint presentation, Competia Public Service Symposium, 2002.

26
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Exhibit A-2

Source:

603-062

Screen Capture of Inside JPL

JPL Web site, , accessed September 23, 2002.

27
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Exhibit A-3

Project Libraries

Project Libraries
The new project manager of Mars ‘05 was meeting with the Project Library customer representative. She had a stack of papers three feet high—it included NPG 7120, ISO 9001, NPG 2810 on
Information Technology Security, JPL’s Policy on Document and Data Control, NARA Guidelines, and many, many others.
She told him, “You and all your project members need to be experts on every requirement in this stack, and then find, buy, and operate a system that lets you securely share information with your international partners. Oh, and there are overlaps and redundancies in many of these requirements. On the other hand, you can pay a small monthly fee for a Project Library, which we will operate for you.
While it’s your responsibility to make sure these requirements are met, using a Project Library will streamline meeting those requirements.”
“Does it cover the archive requirements?” “Yes.”
“What about controlled records?” “It’s in there.”
After a handshake, a service agreement, and one week, the Mars ‘05 Project had an online, shared workspace, tailored for their specific partnerships and work breakdown structure. Built upon Xerox’s
DocuShare system, the structure supports ISO 9000 through automatic creation of a Master Controlled
Documents List. At the end of the mission, project documents will be moved to an online archive, incorporating requirements for records management, so that other projects will be able to learn from the
Mars ‘05 successes.
—Manson Yew, Jet Propulsion Laboratory
Source:

Adapted from , p. 17, accessed July 12, 2002.

Exhibit A-4

Source:

Screen Capture of KnowWho Directory Web Page

Jeanne Holm, “Knowledge Sharing at NASA,” PowerPoint presentation, Competia Public Service Symposium, 2002,
p. 17.

28
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Exhibit A-5

Source:

603-062

Screen Capture of Technical Questions Database

Jeanne Holm, “Knowledge Sharing at NASA” PowerPoint presentation, Competia Public Service Symposium, 2002,
p. 15.

29
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.

603-062

Managing Knowledge and Learning at NASA and Jet Propulsion Laboratory (JPL)

Exhibit A-6

JPL Centers of Excellence

Center for Space
Microelectronics Technology
(est. 1987)

This center conducted research and advanced development in microdevices, microsystems, and revolutionary computing. It focused on microtechnology that was unique to such space applications as sensors for those portions of the electromagnetic spectrum not accessible from Earth; micro-instruments and microelectronic systems for miniature spacecraft; and revolutionary computing, both in space and on the ground, for space system autonomy, mission data analysis and visualization.

Center for Space
Interferometry
(est. 1996)

This center developed and maintained world-class, leading-edge capability in optical interferometric imaging and astrometric technology. It nurtured world-class science experiments in extra-solar system exploration and astrophysics, and provided lightweight space telescopes, interferometers, and detectors for astrophysics missions.

Center for In Situ Exploration and Sample Return
(est. 1996)

Enhanced JPL’s scientific, technological, and system-development capabilities for in situ and sample return missions to solar system bodies.
Enabled JPL to carry out sample return missions to Mars and comet nuclei and in situ missions to Europa, Titan, Venus, and the outer planets.

Center for Integrated Space
Microsystems (CISM)
(est. 1998)

This center was the focal point for the system architecture, core technology development, and system-level integration of spacecraft subsystems into small-scale units for use in relatively small spacecraft.

Center for Space Mission
Architecture and Design
(est. 1997)

This center supported JPL’s ability to design and implement missions at the system and mission level. It ensured the value of a mission’s scientific and technological content, costs, and strategic conception. The center focused on aggressive development of processes, tools, and people.

Center for Deep Space
Communications and
Navigation Systems
(DESCANSO)
(est. 1997)

This center provided technical leadership for programs in the communications and navigation disciplines within JPL and coordinated with other NASA centers, universities, and industry to enable NASA to meet its goals in deep space exploration. DESCANSO focused on deep-space communications link technology, deep-space networking strategies, deepspace navigation and position location, distributed operations across the solar system, and coordinated use of autonomous systems in space.

Center for Space Mission
Information and Software
Systems (est. 1999)

The IT center of excellence was established to unify information technology efforts and strategic planning across the laboratory. The center’s focus was to create a mission software-development process and build a world-class information technology community at JPL.

Source:

Adapted from JPL 2000 Implementation Plan.

30
This document is authorized for use only in GMBA-EN-BL_Abr2015_A1 - Innovation and project management by Jose Luis Fernandez Borrero, IE Business School from February 2016 to
January 2017.