Chapter 25 Uncertainty in the Design of Non-prototypical Engineered Systems

Abstract

In this paper, the authors discuss the challenges engineers face when designing non-prototypical engineered systems, which are systems that cannot be tested using prototypes. Unlike prototypical systems where uncertainties can be reduced through testing, non-prototypical systems present greater levels of uncertainty due to the lack of prototype testing.

The engineers manage these uncertainties in several ways:

Codes of Practice: To ensure minimum safety levels, engineers follow established codes of practice. These codes provide guidelines and standards that help in making informed decisions despite the uncertainties.

Quality Control Measures: To minimize the risk of human error, which can introduce additional uncertainties in the design process, engineers implement stringent quality control measures.

Designing for Robustness: Considering the possibility of extreme and unpredictable events, engineers incorporate additional details into the system to make it more robust. These details are not necessarily required by minimum standards but are intended to enable the system to withstand conditions that are far beyond its anticipated design capacity.

In summary, the paper highlights the importance of managing uncertainties in engineering design, especially for non-prototypical systems, by adhering to standard practices, ensuring quality control, and designing for robustness to handle extreme conditions.

25.1 Introduction

This introduction outlines the challenges engineers face when designing non-prototypical or "one-off" engineered systems, which are systems that cannot be tested through prototypes. These challenges arise due to various uncertainties that are higher compared to systems where prototype testing is feasible. The uncertainties in engineering design are categorized into two types:

Aleatory Uncertainty: This type of uncertainty is related to luck or chance. It involves elements that are inherently random and unpredictable.

Epistemic Uncertainty: This uncertainty is linked to knowledge and information. It involves aspects that could be known in principle but are currently unknown or not fully understood.

The paper identifies five main sources of uncertainty in engineering design:

• Time: Uncertainties related to how systems will perform over time.
• Randomness: Uncertainties that arise due to random or unforeseeable events.
• Statistical Limits: Limitations in understanding or predicting behaviors due to insufficient statistical data.
• Modeling: Challenges in accurately modeling complex systems or phenomena.
• Human Error: Errors or mistakes made by engineers and designers.

Engineers manage these uncertainties in several ways:

Codes of Practice: These are established guidelines or standards that engineers follow to address certain types of uncertainties, particularly those related to safety.

Quality Control Measures: Procedures and practices implemented to minimize human errors and ensure the quality of the engineering work.

Implicit Methods: This includes using heuristics or rules of thumb, which are less formal but often effective ways of dealing with uncertainties.

In summary, the introduction sets the stage for a discussion on the complexities of engineering design under uncertainty, especially for systems where prototype testing is not feasible, and how engineers approach these challenges.

25.2 Aleatory Versus Epistemic Uncertainties

This section of the paper discusses the distinction between aleatory and epistemic uncertainties in engineering design, particularly for non-prototypical engineered systems, and how these uncertainties are managed. It categorizes the sources of uncertainty and explores how each type contributes to the overall uncertainty in engineering projects.

Aleatory vs. Epistemic Uncertainties:

Aleatory Uncertainty: Relates to chance or luck, like flipping a coin. It is inherent and unpredictable.

Epistemic Uncertainty: Relates to knowledge. It implies that uncertainty can be reduced with more information or understanding.

Sources of Uncertainty:

Time: Future predictions based on past data contain mostly aleatory uncertainty since future events cannot be completely known.

Randomness: Variability in material properties and loads are seen as aleatory, but some aspects might be epistemic, reducible with more knowledge.

Statistical Limits: Uncertainty due to limited data samples. Primarily epistemic, as more data could reduce uncertainty, though practical limitations exist.

Modeling: Involves creating predictive models of systems. Contains both aleatory and epistemic uncertainties due to inherent limits in models and the potential for improvement with better understanding.

Human Error: Errors in design and construction, primarily epistemic, as they can be reduced through increased knowledge, quality control, and checking processes.

Management of Uncertainties:

• Engineers use codes of practice to set safety standards and minimize aleatory uncertainties.
• Quality control measures are implemented to reduce human errors and other epistemic uncertainties.
• Additional engineering details, not necessarily required by standards, may be included to make systems more robust against extreme events.

The paper underscores the complex nature of engineering design, particularly for unique systems where prototypes aren't feasible. It highlights the blend of aleatory and epistemic uncertainties in such projects and the strategies engineers use to manage these uncertainties effectively.

25.3 Designing and Building Under Uncertainty

This section discusses how engineers manage uncertainties in the design and construction of non-prototypical engineered systems, which are unique and do not allow for prototype testing.

Use of Codes of Practice:

• Codes help manage uncertainties caused by randomness, time, and modeling.
• Complexity of a code affects design uncertainty; too complex can lead to interpretation errors, too simple may overlook crucial aspects.
• Engineers need to interpret and apply codes carefully to balance between complexity and simplicity.

Dealing with Randomness:

• Probability concepts like exclusion values, extreme values, and return periods help manage uncertainties in loads and environmental effects.
• These concepts are used in codes of practice with safety factors to address uncertainties in material properties and loads.

Managing Human Error:

• Quality control methods like peer reviews and construction inspection minimize human error in design and construction.
• Simple calculations alongside complex computer models help verify results and prevent severe input or model errors.

Addressing Contingency:

• Designs are contingent as they are based on a visualization of a system that doesn't yet exist.
• Techniques like blueprints, 3D computer models, and examining similar past systems aid in visualization and understanding.
• The goal is to minimize differences between the designed and the as-built system.

Differences Between Science and Engineering:

• Unlike science, engineering deals with systems that do not yet exist, leading to contingency.
• This difference causes greater uncertainty in unique engineered systems compared to those similar to existing ones.

In summary, engineers face various uncertainties in the design of non-prototypical systems. They use codes of practice, quality control measures, and visualization techniques to manage these uncertainties and make informed design decisions. The aim is to minimize the gap between the envisioned design and the actual constructed system, acknowledging that some level of uncertainty is inherent and unavoidable.

25.4 Time and Again

In this section, the impact of time on uncertainty in engineering design, especially for non-prototypical engineered systems, is discussed.

Time as a Source of Uncertainty:

• Time-related uncertainties include predicting future occurrences based on past data, societal changes affecting design requirements, changes in material properties over time, and evolving engineering knowledge.
• Most uncertainties related to time are aleatory, meaning they are largely due to chance and cannot be fully predicted or eliminated.

Feedback in Engineering Design:

• Over time, the performance of engineered systems provides valuable feedback, influencing the design of future systems.
• This feedback can be explicit, like learning from system failures, or implicit, through reflection on design decisions.
• For non-prototypical systems, feedback cycles can be lengthy, as these systems don’t allow for prototype testing.

Evolution of Design from Feedback:

• Engineers learn and evolve designs over time, often in response to failures. For example, the design of suspension bridges has evolved significantly due to past failures and successes.
• Suppressing failure information can delay this learning process, potentially leading to unsafe design directions.

Reflection as Internal Feedback:

• Engineers reflecting on their designs can identify potential issues and improve safety.
• The Citicorp Building case illustrates how reflection and subsequent action by the engineer William LeMessurier led to critical retrofitting to address design flaws.

Importance of Time in Design Process:

• The role of time in engineering design is crucial, especially for non-prototypical systems where feedback is slow and often comes from real-world performance or failures.
• Continuous reflection and learning from both successes and failures are essential for the evolution and safety of engineering designs.

In summary, the section emphasizes the significance of time and feedback in the engineering design process, particularly for unique, non-prototype systems. Engineers must navigate uncertainties and use both external feedback (from system performance and failures) and internal methods (like reflection) to continuously improve and ensure the safety and effectiveness of their designs.

25.5 Black Swan Events

This section discusses the concept of Black Swan events in the context of engineering design, particularly focusing on non-prototypical engineered systems.

Complex Systems and Black Swan Events:

• Complex systems, such as petrochemical plants, the power grid, and nuclear power plants, are often tightly coupled and have interactions that can lead to cascading failures.
• A Black Swan event is an unpredictable and highly impactful occurrence, which is outside the realm of normal expectations and past experiences. These events are rare and their consequences are significant.

Cascading Failures in Tightly Coupled Systems:

• Small failures in complex, tightly-coupled systems can trigger a chain reaction, leading to major system collapses.
• Safety devices in these systems, both human-operated and automatic, may inadvertently contribute to system failures, as seen in incidents like the Three Mile Island accident.

Management of Information and Small Failures:

• In complex systems, there is often a tendency to overlook or suppress information about minor failures, which can be dangerous as these small issues may escalate into major problems.
• The Challenger space shuttle disaster is an example where warnings about potential issues were not adequately addressed.

Black Swan Events in Complex Systems:

• Black Swan events can arise from complex, tightly-coupled systems and are difficult to predict. They encompass all the uncertainties discussed in the paper, along with the interactions that can lead to unforeseen consequences.
• Examples include natural disasters or unexpected socio-political events that can have profound impacts on engineered systems.

Designing for Robustness:

• To mitigate the impact of Black Swan events, systems are designed to be robust, meaning they can endure and function under unexpected stress or unusual events.
• Robust design involves incorporating engineering details that go beyond minimum standards, potentially allowing a system to withstand events well beyond its intended capacity.
• However, since Black Swan events are by nature unpredictable, designing specifically for them is not feasible. Instead, the focus is on enhancing overall system robustness.

Examples of Black Swan Events in Engineering:

• Consequences that are far outside the realm of experience are sometimes referred to as black swan events (Taleb 2007 ). ‘Black swan’ refers to the long held belief that all swans were white until black swans were discovered in Australia. Karl Popper ( 2002 ) used the black swan example when discussing the problem of induction. Taleb ( 2007 ) discussed black swan events from the perspective of his experience in the investment community, although the concept applies broadly. As far back as 1921, Knight (1921/ 1948 ) considered the types of uncertainty in the business environment. He divided them into measurable uncertainties and unmeasurable uncertainties. From a business standpoint, if the probability of an event can be determined, it is a measurable uncertainty and can be managed using insurance. An unmeasurable uncertainty cannot be managed using insurance because it is not possible to determine its probability due to its unpredictability. Knight’s unmeasurable uncertainties are black swan events.
• The bombing of the Alfred P. Murrah Federal Building and the September 11 attacks on the World Trade Center are cited as Black Swan events. These events were unforeseen and led to significant changes in building design and security protocols.

In summary, Black Swan events represent significant challenges in engineering design, especially for complex and tightly-coupled systems. While these events cannot be predicted, engineers aim to create robust systems that can endure a wide range of unexpected stresses, thereby mitigating potential catastrophic outcomes.

25.6 Conclusion

The conclusion of the paper on non-prototypical engineered systems discusses the various uncertainties engineers face and the strategies used to manage them:

Types of Uncertainties:

• Engineers encounter two main types of uncertainties: aleatory (related to chance) and epistemic (related to knowledge).
• Aleatory uncertainties arise from unpredictable factors like natural events, while epistemic uncertainties stem from limitations in knowledge or information.

Uncertainty in Non-Prototypical Systems:

• Non-prototypical systems, or unique, one-off systems, face greater uncertainty compared to systems where prototype testing is feasible.
• Prototype testing in other systems helps reduce uncertainty by providing immediate feedback during the design phase.

Managing Uncertainty:

• Engineers use codes of practice, which are standardized guidelines, to manage and mitigate uncertainties.
• They also employ methods to reduce contingency, or reliance on specific conditions, and conduct inspections during construction to ensure adherence to design specifications.
• Robustness, or the system's ability to withstand events beyond its design capacity, is a key focus. This involves building in extra safeguards and strength.

Complex Systems and Black Swan Events:

• Complex systems, especially those with significant component interactions, can exhibit behaviors that are highly unpredictable and beyond the designers' experience.
• Black Swan events are extreme, unpredictable occurrences that can have a profound impact on these systems. Preparing for such events is challenging due to their unforeseeable nature.

Importance of Robustness in Complex Systems:

• Efforts to enhance the robustness of complex systems are crucial for safety. However, the effectiveness of these efforts is often hard to measure due to the complexity involved.
• The true robustness of a system is often only revealed when it faces an extreme, unpredictable event.

Design Challenges in Non-Prototypical Systems:

• Designing non-prototypical systems requires a broader set of strategies for managing uncertainty compared to systems where prototyping is possible.
• Engineers must creatively address both aleatory and epistemic uncertainties to ensure the reliability and safety of these unique systems.

In summary, the paper emphasizes the heightened level of uncertainty in non-prototypical engineered systems and the necessity for engineers to employ a variety of methods to manage this uncertainty effectively. This includes focusing on robustness to prepare for extreme and unpredictable events.