Developing the Next Generation of Systems Engineers

Heidi Davidz

A recent study sheds light on what it takes to become a successful senior systems engineer—and suggests a means to accelerate that process in today's engineering population.

As aerospace systems grow in complexity and interdependence, there is an increasing need for engineering professionals who can successfully plan, develop, manage, and evolve these systems. Yet, the national security space community is facing a growing shortage of senior systems engineers, as the number of systems positions increase and older workers leave the workforce. Organizations commonly lure skilled systems engineers away from each other or try to fill these roles with junior personnel who lack the requisite skills and/or experience, but these efforts fail to address the underlying problem. The question is, how can the national security space community expedite the development of the next generation of senior systems engineers?

The type of thinking required by systems professionals is sometimes referred to as "systems thinking." While systems thinking may be found throughout organizations, systems engineers are specifically charged with applying it to engineering systems. A systems engineer must be proficient in systems thinking. Therefore, it follows that if one can accelerate the development of systems thinking, one can accelerate the development of a systems engineer. Industry, government, and academia are scurrying to establish programs to develop systems thinking; however, many of these programs are designed solely on heuristics or unproven assumptions of how systems thinking develops.

A recent doctoral research study supported in part by Aerospace sought to test these assumptions by examining existing literature and gathering a large sample of empirical data. The goal was to show which methods are most effective in developing systems thinking in engineers. Data were collected to understand how systems engineers develop—particularly in regard to enablers, barriers, and precursors to developing systems-thinking proficiency. It also documented how engineers define systems thinking and investigated company procedures for developing systems engineers.

Aerospace is now using and expanding on this work to help its customers build integrated approaches to developing systems professionals in their organizations. Aerospace is helping its customers build effective and efficient programs to grow systems capability by designing programs based on evidence of how system thinking actually develops.

Study Design

In the course of this research, 205 employees in 9 companies were interviewed, primarily in the U.S. aerospace sector. These included The Aerospace Corporation, BMW, Boeing, Booz Allen Hamilton, General Dynamics, MITRE, Northrop Grumman, Pratt and Whitney, and Sikorsky.

The project began with an extensive literature review and a series of pilot interviews to gather information on the development of systems thinking and ascertain how best to structure the study. Next, a point of contact was designated at each of the participating organizations; this person helped gather a panel of experts highly familiar with the policies and practices of how that company developed senior systems engineers. The qualifications of these experts varied by company, but a typical panel consisted of a vice president of engineering, two senior systems engineers with approximately 30 years each of experience, and a human resources representative who designed systems engineering training.

The expert panelists in each company were asked to identify interview subjects in three groups: senior systems engineers, junior systems engineers, and senior technical specialists. The primary interest was in the characteristics, development histories, and opinions of the senior systems professionals; the junior systems engineers and the senior technical specialists were control groups. The premise was that if certain types of people are drawn to systems roles, the responses of junior and senior systems engineers would be similar. Likewise, if systems expertise develops through experience, senior systems engineers and senior technical specialists would have similar responses to the interview questions.

Before the interviews, participants completed a survey designed to gather information on demographics, educational background, assigned work roles, and past training history. During the interview, participants were asked about definitions of systems thinking, enablers and barriers to systems thinking development, individual characteristics that predict the development of systems thinking, and key steps to systems-thinking development. Expert panelists were also asked about formal company procedures for developing systems engineers. Interviews were also conducted with "blue-chip" professionals—experts who are well recognized throughout the aerospace sector for their systems-thinking expertise. These interviews were used for additional validation of the field study results.

The survey data were analyzed using statistical data processing software. The interview data were analyzed using content analysis techniques, whereby key ideas and thoughts were categorized and aggregated. Approximately 1000 pages of transcripts were coded, yielding 908 categories of information. The coded data were then exported to a statistical data analysis tool.

Christina Smith discusses with Denny Pidhayny a model of the satellite tracking function of a three-gimbaled ground station. Smith is pointing to the changes in the gimbal angle while the antenna tracks.

Systems Thinking Definitions

The interview subjects were asked to define systems thinking. The 205 interviews conducted yielded 205 different definitions of systems thinking. Additionally, subjects were asked to consider a given definition of systems thinking and this too generated a variety of responses. Evidently, when people refer to systems thinking, they are often not articulating the same concept.

The variety of definitions that emerged were distilled to identify common elements or descriptors of a systems-thinking framework. Five foundational elements were identified: componential, relational, contextual, dynamic, and modal. The componential element addresses what types of things are considered in systems thinking, for example, system objectives, the system elements, and the system domain. The relational element addresses the interconnections, interactions, and interdependencies both within the system of interest and between the system of interest and other systems. The contextual element addresses the nested and embedded nature of systems. The dynamic element links systems in time to the future and past, to include important aspects such as feedback, uncertainty, risk, and what are referred to as the "ilities." Examples of these include flexibility, agility, reliability, and sustainability. The modal element aids with understanding and comprehension of the system and is the "how" of systems thinking; it includes the variety of aids engineers use to comprehend the complexity of a system, such as frameworks, processes, models, simulations, tools, methods, and different types of thinking. The definition that resulted from combining these five basic elements became: "Systems thinking is utilizing modal elements to consider the componential, relational, contextual, and dynamic elements of the system of interest."

There are disadvantages to having numerous definitions of systems thinking within an organization. First, this divergence may lead to imprecise goal definitions. Second, it can also lead to inconsistent measures of the strength of systems thinking within an organization. For example, when subjects were asked, "How does your company determine if an employee displays strong systems thinking?" the number one response was "do not know," while the other top-ranked responses were related to observation and subjective measures. When senior systems engineers within the same company have different definitions of systems thinking, and when quality of systems thinking is determined by observation and subjective measures based on a variety of definitions, diverse behaviors emerge. It should then come as no surprise that 71 percent of the junior systems engineers surveyed had difficulty understanding how their company determined strong systems thinking. This raises an important question: How can an organization expedite the development of junior systems engineers when these engineers do not know what they are supposed to develop into and do not understand how their progress is being measured?

Enablers to Systems Thinking

Although the study found that definitions of systems thinking diverge, it found substantial agreement on the mechanisms that enable, as well as obstruct, systems-thinking development. The mechanisms deemed most effective at fostering systems thinking included experiential learning (which encompassed both work and life experiences), development of certain personality traits, and a supportive work environment. Many organizations facing the challenge of developing a systems engineering workforce immediately jump to implementing training classes. However, the data from this study show that this may be a flawed strategy, because systems engineering skills develop primarily through experiential learning—not through a traditional classroom setting. Thus, systems engineering development programs should emphasize experiential learning as an effective approach to developing systems thinking.

For example, study participants were asked, "What were key steps in your life that developed your systems-thinking abilities?" The top-ranked response was "work experiences," cited by 139 participants (69 percent). In each participant category, more respondents cited work experiences than anything else. Likewise, 95 percent of the expert panelists noted work experiences as a key step to the development of systems thinking—remarkable consensus for this data-solicitation format. Interview subjects were asked, "In your experience, what enablers or barriers have you seen to the development of systems thinking in engineers?" The responses again highlighted the importance of experiential learning, with "experience" noted as the top-ranked category for enablers to systems thinking for all interview participants.

The blue-chip interviews also emphasized experiential learning. One interviewee stated that during the mid-1960s, programs went from concept to operation in three to five years. He said, "In a period of 15 years, an engineer would work on three to five programs, progressively working up to greater responsibilities. There was a whittling down process so that we could pick the systems engineer. There would be three to five programs with four to five segments each, so we could pick the systems engineers for the new programs from this pool. We might have eight people to choose from and we could pick the best engineer. We never had a problem with training because it was provided on the job. We never thought about setting up training until the 2001 timeframe, when we thought about how to fix the problems in space acquisition."

Specific character traits also seem to foster systems thinking in engineers. Interview subjects were asked, "Are there certain individual characteristics or innate traits that seem to predict the development of systems thinking? If so, what are they?" Respondents noted thinking broadly, curiosity, questioning, open-mindedness, communication, tolerance for uncertainty, strong interpersonal skills, and an ability to "think outside of the box" as top-ranked characteristics.

Top-cited barriers to the development of systems thinking included schedule and cost constraints, organizational confinements, and a narrow job definition. For example, after employees receive systems training, they may return to an environment where the "tyranny of the urgent" prohibits thinking of the larger system, or where organizational boundaries obstruct further development of systems understanding, or where a narrow job focus deters investment in the rest of the system.

A supportive environment enables the development of systems thinking in engineers, and systems training should coordinate with organizational incentives. Otherwise, a misaligned work environment may invalidate investments in systems training. An organization might heavily invest in systems engineering training but use organizational incentives that only reward skill in technical depth, rather than skill in systems integration. Here, the misaligned work environment is counterproductive to the investment in systems training. Data from the blue-chip interviews also substantiated the need for a supportive environment to develop systems thinking.

Expert panelists were asked to explain how their companies currently develop systems engineers. The maturity of the systems engineering development programs varied considerably across the companies interviewed. In some cases, although the company might have a systems development program in place, the research revealed unclear objectives of what was wanted in its systems professionals. Often, systems training classes were noted as ineffective, experiential learning was not being emphasized, and the organizational environment was not supportive of this development. The findings also revealed that many companies lack a meaningful feedback structure to ascertain whether the training program in place is even working.

A series of tables show the data analysis of the field study. Five classifications were used: (1) All Participants, (2) Expert Panelists, (3) Senior Systems Engineers, (4) Senior Technical Specialists, and (5) Junior Systems Engineers. The number in parentheses is the number of participants who responded to a question. "Rank" is the rank of the category in comparison to the other categories for that question. "Number" is the number of respondents in each classification who cited that category. Categories cited by 10 percent or more of a classification are shaded. Though some of the percentages may seem small, this was a semistructured interview format that allowed for open-ended responses. It was not a structured tool where more convergent ideas might be identified. Thus, when convergence does appear, it is notable.

Applying the Findings

Organizations that want to develop their systems engineering workforce need to set up integrated learning programs to build these skills in employees. First, well defined goals of what is wanted in systems professionals must be established. Next, there needs to be a clear understanding of why the development of systems capability is necessary, what types of systems capability are needed, and what the current state of systems practice is within a given organization. Data and literature collected from inside and outside the group should be used to make this assessment.

Some organizations need to improve classical systems engineering skills, while others need skills for system-of-systems environments. One organization might need to develop systems engineers who are requirements owners and process engineers, while another might need to enhance the capabilities of systems engineers who are system designers and integrators. Each organization should clearly identify its workforce development requirements.

The strategy an organization formulates and communicates about how to develop these systems skills in employees should be based on proven studies of how systems skills actually develop. This study revealed that systems thinking develops primarily through experiential learning. A strategy for developing systems professionals should therefore include experiential learning opportunities. This study also demonstrated how specific individual characteristics play a role in the successful development of systems thinking. Though more research is needed to determine the full impact of individual characteristics on organizational systems performance, organizations could eventually foster and filter for the identified characteristics in their systems groups.

The workforce development strategy should integrate the individual, group, organizational, and sector levels of analysis. For example, as an individual is given systems training, group and organizational incentives should support that training. The systems capabilities of a team could be developed instead of focusing on just an individual. Systems training courses could be integrated with work assignments, job rotations, mentoring programs, systems engineering processes, and knowledge management tools to continuously develop systems competencies in individuals. Organizations might consider working with professional societies to study more effective mechanisms to developing systems engineers.

In an integrated development strategy, an organization should set up feedback and quality-control mechanisms to gather data on how well its development program is working. These feedback mechanisms should extend beyond how well a student liked a course or the number of students trained. Next, systems engineering competency at the individual, team, and organizational levels should be assessed. The organization should also sponsor research projects to better understand how systems engineers develop and how to improve the quality of systems engineering delivered by the organization. This type of support should enable innovation and continuous improvement to the organization.

Finally, the organization must plan to develop the systems engineering workforce efficiently and effectively. Too often, the development of systems professionals is needlessly driven by ambiguity, imprecision, and assumptions about how these skills develop. Clarity of purpose, multilevel integration, a critical examination of current methods, and assessments of results will accelerate the development of systems professionals. In addition, increased research funding and rigorous study in this area will lead to enhanced understanding and innovative methods to developing a systems engineering workforce.

Conclusion

The development of the next generation of space systems professionals can be expedited through a better understanding of the mechanisms that develop systems thinking. The results of this study show that the mechanisms currently being used are not the most effective. The drastic shortage of senior systems professionals demands the exploration of new and innovative ways of developing systems engineers. Many people talk about the aging aerospace workforce and the need to develop the next generation of space systems professionals, yet few seem willing to invest the research resources needed to critically, precisely, and creatively examine how the workforce can develop these requisite skills.

Editor's Note

Segments of this article were excerpted from Dr. Davidz's Ph.D. dissertation, "Enabling Systems Thinking to Accelerate the Development of Senior Systems Engineers," ©2006, Massachusetts Institute of Technology (MIT). Those segments were reprinted with the express permission of MIT. The research was sponsored by the Lean Aerospace Initiative at MIT.

To Spring 2007 Table of Contents