Training the National Security Space Workforce in Systems Architecting
Mark Maier
Aerospace has developed a curriculum that effectively teaches the mindset required for successful system architecting.
For the past eight years, Aerospace has been developing an educational program designed to help move highly skilled, technically oriented engineers into the more nebulous, ill-structured world of systems architecture. Although this program—the Aerospace Systems Architecting Program—originally concentrated on internal capabilities, it evolved to accommodate students from the larger national security space community. The program uses an intellectual core of material in ill-structured problem solving and conceptual design to address areas usually regarded as "soft," or more art than science. While this material is not traditionally presented in engineering programs, it is in fact possible to teach—and after years of refinement, Aerospace has developed an effective means of doing so.
Program History
The idea for a formal training program in systems architecture within Aerospace began in the mid-1990s. It was evident to a number of managers that the rapid changes brought about by the end of the Cold War were requiring a new set of skills in program conception. The stable background of objectives and requirements was falling away, and new programs had to deal with understanding root purpose, and how root purpose can be influenced by technology, in ways that had been uncommon for nearly 20 years. At about the same time, a former Aerospace CEO, Eberhardt Rechtin, founded an academic program in systems architecting at the University of Southern California. Several Aerospace employees had been exposed to the program during graduate school, and awareness of the approach was growing in the corporation. A decision was made to create an internal training program in systems architecting focused on the specific needs of national security space (see sidebar, What is Systems Architecting).
The first version of the Aerospace Systems Architecting Program was a 120-hour sequence of lectures and case exercises spread over several months in 1997 and 1998. Based on the pilot experience, this was rearranged into four one-week blocks to assist students with managing time away from their jobs. The single case exercise was expanded into two complete case exercises, each lasting approximately half the course. This "production course" was then offered annually through 2002 to audiences drawn from most company sites and divisions.
As the core audience completed the course, demand trailed off. With that in mind, the course was rearranged into a more modular format, one that could be completed over a longer period and that lent itself to greater customization for particular audiences. This was done by building a core sequence of three short courses, each capable of standing alone, followed by a set of elective courses tailored to particular project areas.
One factor that was evident from the beginning, but became more obvious over time, is that executive leadership plays a central role in the success or failure of an architecture project. While managers do not need the same set of skills as practitioners, they do need to understand what architecting is and how to organize and manage it. With that in mind, Aerospace began working in 2002 with the chief scientist of the Air Force, who was putting together a three-day course in architecture-based systems engineering for upper level Air Force personnel. This course provided a strong foundation for a version of the Aerospace course targeted at executives and managers.
In 2005, Aerospace was approached by another government agency that was developing an architecting program with very similar goals. Aerospace agreed to provide an adapted version of the modular program, and this version is now regarded as the baseline for effective architecting education. Its fundamental structure is a two-week course, with integrated case exercises and case studies, on the foundations of systems architecting, along with some coverage of special topics. This is followed by a second two-week block, again with integrated case exercise and case studies, on more complex situations specifically tailored to the project needs of the agency.
Program Structure
Today, the Aerospace Systems Architecting Program is one of three specializations within the broader Aerospace Systems Architecting and Engineering Certificate Program. Students can select this specialization after completing a set of core courses. They may then enroll in an on-the-job training program, which involves completing two projects overseen by a technical mentor (usually a previous course graduate or instructor). The two projects must be for different customers and involve an area of business different from the student's previous experience. At the end of the training period, students present a briefing on lessons learned, typically to a management audience and a forum of previous course graduates.
The goal of the program is not simply to convey knowledge, it is to develop active skills and even change student attitudes with respect to complex and ill-structured tasks. These skills can only be developed (and their acquisition evaluated) in realistic task settings. One has to let the student perform a relevant task, observe the result, critique it, and do it again. Because skills learned in the course apply to large-scale design problems, they can really only be developed and assessed through exercises of realistic scope. To make this possible in a time-limited course setting, specially selected case studies and case exercises are used.
Case studies are retrospective analyses of past projects. They provide students with insight into real decision-making processes and allow them to critically analyze the results. Each case study involves three-to-four hours of classroom work and includes both instructor presentation and group consideration of alternative approaches. Four major case studies (and several smaller ones) have been developed through a combination of archival research and interviews with primary sources. Two of the case studies are available for general use (one on the DARPA Grand Challenge and one on the Global Positioning System), while two others are restricted to particular audiences. Three new case studies are being planned for a future version of the course. In addition, a number of minor case studies (mostly on systems of historical interest) are used to illustrate particular points.
Case exercises are architecture design problems executed by the students, derived from real (or at least realistic) projects. Several standard exercises have been developed over the life of the course. The design of a case exercise has proven particularly challenging, and those in use have been extensively refined over time. The first was based on the Triana Earth-observing satellite program and was intended to end in a specific satellite system concept, even though the given scenario was politically complex and lacked any firm requirements. More recently, two new case exercises have been developed—one based on a hypothetical command and control system for coalition peacekeeping operations, and the other based on a project to integrate business-oriented networks with R&D networks. As an illustration of their realism, solution elements of the R&D network case exercise have been reused on several real projects conducted by students or instructors after the class.
The systems architecture program has two tracks, which differ slightly for Aerospace students and government students. Both begin with the same introductory classes, "The Art and Science of Systems Architecting" and "Foundations of Systems Architecting." The next phase focuses on core knowledge. For the Aerospace track, this phase includes the courses "Architecture Frameworks" and "Architecture Design and Development." For the government track, it includes the courses "Communicating Complex Architectures" and "Complexity, Uncertainty, and Decision Making in Architecting." The third phase, the capstone, entails one or more courses in which the methods are applied to specific problems that students are engaging. For Aerospace personnel, there are several elective versions available or planned. For government students, there is currently one version on "Architecting Technical Intelligence Systems."
The Aerospace Systems Architecture Program has two tracks, which differ slightly for Aerospace and government students. Both begin with the same introductory classes. The next phase focuses on core knowledge. The third phase is known as the capstone application. It includes various electives for Aerospace personnel. |
Course Descriptions
"The Art and Science of Systems Architecting" is a one-day general introduction to the subject. Its purpose is twofold: First, it provides an accessible introduction to the core ideas for wide audiences. Second, it helps students decide whether they should enroll in a longer course, which would represent large investments of their time and their organization's resources. Many strong engineers are uncomfortable in the environment of ill-structured problem solving and will not become much more comfortable with time—even with training. Helping those people identify themselves, and steering them to other advanced professional educational programs, is good for them and good for the architecting program.
"Foundations of Systems Architecting" is a two-week course that provides fundamental training in the methods of systems architecting and their application to realistic situations. The first week presents all the main technical elements of the program, organized as a single pass through a spiral process (see sidebar, The Aerospace Systems Architecting Method). All of the instructional elements are applied in parallel in a case exercise. In practice, it turns out that a rapid introduction of the material with immediate application is the best way to achieve the learning objectives. It does lead to a sense of discomfort and some frustration in many students, but most find working through that frustration useful (and an accurate reflection of the nature of these projects). During the second week, the students brief the instructors on their first pass through the case exercise; they receive extensive critique, and redo the process they have been taught, culminating in a final briefing on the last day of class. Instructors present major case studies along with the case exercises, inviting the students to critique them and extract important lessons. The second week can be offered directly after the first week, or after a break of a month.
The next course in the sequence, "Architecture Frameworks," is offered in recognition of the key role played by the Department of Defense Architecture Framework and other government-specified frameworks in many programs on which Aerospace advises. The "Architecture Design and Development" course presents a set of tools and methods used by Aerospace in space system and system-of-system architecting projects. The "Communicating Complex Architectures" course teaches customers to document and explain complex architecture projects. The "Complexity, Uncertainty, and Decision Making in Architecting" course teaches classifications and methods for modeling uncertainty, defines three major strategies for dealing with uncertainty, and discusses the standard adaptations to utility calculation and decision theory in the presence of structural uncertainty.
The intent of the capstone courses is to bring the tools taught in the basic and core courses together in applications whose attributes extend beyond the foundational systems-architecting model. These classes also bring the domain specifics of a working environment into the discussion. Systems engineering and architecting are not domain independent. Many of the most important practices and elements of knowledge are tied to the domain of application (e.g., spacecraft, launch vehicles, computer networks, intelligence collection systems). Customizing capstone courses to these domains makes it possible to ground the case studies and exercises directly in the realities of those domains.
The Aerospace Concept Design Center is used to develop future space architecture concepts. |
Courses for executives and managers address different sets of concerns than in the practitioner courses. A course for executives focuses on the interplay between organizational roles and responsibilities, the structure of the programs, and the technical aspects of architecting. A course for management focuses on how managers initiate and structure architecture projects to maximize the likelihood of success. It includes complete capsule descriptions of the methods taught in the "Foundations of Systems Architecting" course and has the managers apply them to a set of exemplary case exercise materials. The executive course is structured as a two- or three-day seminar; the manager course is a five-day class.
General Lessons
The eight-year process of developing and refining Aerospace's courses has taught many lessons. The most important include:
- The case exercises are the most important part of the courses.
- Spiral processes are critical, and must be carefully taught to overcome student bias against them.
- Description must follow decisions—not the other way around.
- Half the battle is avoiding a few big mistakes.
- Domain knowledge matters, and is often in short supply.
In the beginning, it was not obvious how much learning would take place in the case exercises. Judging by the student evaluations, they are by far the most important part of the course. In fact, students really seem to internalize the concepts presented in lecture only after they are applied in the case exercises. Moreover, an evaluation of the case exercises shows that a significant amount of learning takes place between the first and second exercises (in the foundations and capstone courses). In recent offerings, roughly 50 percent of the total time in the courses was spent solely on case exercises.
A key learning theme in systems architecture training is that the "architecture" is conceptual: it is the set of decisions that define a system's essential characteristics. An architecture description represents those decisions in a set of views or models such as those depicted above. The set of views is determined by the architecture's purpose, and so varies from project to project. Other views may include cost, programmatic concerns, logical or data considerations, security needs, and others. (View larger image) |
Most course offerings reveal a built-in bias against spiral or iterative design processes. Teams try to get things right the first time, try to get the problem fully defined at the beginning, and are reluctant to go back and rethink their basic premises. This is a big mistake. A willingness to rethink from the beginning is central to good architecting. One method for overcoming this bias is the use of a spirally oriented overall architecting process. Students are required to perform the case exercises using this process model, on a schedule that forces a minimum of two complete design cycles during the course. Another useful vehicle for driving a spiral process, particularly on real projects, is the "blitz" or "charette." As taught in the class, this is a method for organizing all architectural design tasks on a strict timeline, normally two weeks or less. This forces the group to complete all the steps, no matter how complex and how limited the information available, within a limited amount of time. Of course, the resulting product is rough, but it typically reveals useful information about the process and the underlying problems that cannot be revealed except through an end-to-end design cycle.
Another lesson often noted, but rarely learned, is: Describing an architecture comes after understanding what the architecture is. Thus, the Aerospace program emphasizes the distinction between architecture as description and architecture as decisions. This distinction often seems academic to students at first, but understanding it is important for achieving success on real ill-structured problems. For example, many of the standards in the architecture world focus on producing a descriptive document; however, just as a civil architect can draw many sets of plans describing alternative buildings, so can the systems architect produce many alternatives using different framework standards. Which alternative is preferred? In the classroom, an early focus on the standards for the document drew attention from the more central and difficult issue of what should the architecture be. As a result, the course has been structured strongly in terms of architectural decisions rather than architectural description. This was, and remains, a controversial choice. The immediate pressures to fulfill the specific requirements of various projects are usually directed solely at producing framework-compliant documents, not dealing with difficult conceptual problems in the overall project. However, the difficult problems invariably return again and again over the life of a program. Failing to deal with them early will only increase the difficulty of dealing with them later. The class has a simple and effective mechanism for reinforcing this lesson: All teams are required to present their exercise results as large hand-drawn wall charts (rather than PowerPoint slides, for example) delineating all the various process steps and viewpoints. There are several advantages to this approach. First, the ability to simultaneously see all views of the system of interest emphasizes the need to consider the full system from end to end; it also makes it easy for an instructor to jump from view to view in discussing how the parts relate. Second, hand-drawn charts lessen the student's typical bias to package a briefing, and instead emphasize the need to provide particular information and decisions. Third, the relatively freeform nature of hand-drawn charts tends to emphasize content over form; instead of debates over which sort of diagram is being used, the emphasis is on the information being presented. The fourth principle lesson is that a key to good architecting is avoiding common big mistakes. The major mistakes are simple enough, and common enough, to be nicknamed the "seven deadly sins" (see sidebar, The Seven Deadly Sins of Systems Architecting). An important offshoot of the program was to catalog common counterproductive behaviors, develop examples of each, and present counterexamples of effective behavior.
A last hard lesson is that domain knowledge matters. From the beginning, it was understood that architecting space systems takes a great deal of domain knowledge—for example, in orbital science, space environments, remote sensing performance, and analysis of the supported operations. Some may have hoped that systems at higher levels were somehow more generic and did not depend on their own domain knowledge. Experience in the Aerospace course has been otherwise. Unfortunately, the domain knowledge for higher-level systems is hard to come by and not always available at all. For example, many participants in the courses will be working on network-centric systems that fundamentally depend on the provision and use of computer networks. Just as in space systems, network-centric systems architecting requires substantial domain knowledge—in areas such as computer networking, communications analysis, middleware protocols, security analysis, layered system definition, and other analytical techniques. This skill depth is rare in the space community, and entirely absent in some areas where systems are operating on the cutting edge of research. As the program has evolved, material about network-centric systems was added, but this material can do little more than provide an introduction.
Conclusion
Architectures of many sorts—system, enterprise, framework—are major concerns in the space industry today. Through its systems architecting program, Aerospace is responding to the need to improve its practices in this area as well as the practices of the broader national security space community. The development and teaching of this course has taught lessons that have potentially much broader applicability.
