Guidance for Modeling Crew Variability

1. Introduction: Why Crew Variability Matters in DDET

A core strength of the ADS-IDAC simulation approach is its ability to systematically model sources of crew-to-crew variability within the Dynamic Decision Event Tree (DDET) framework. Even when subject to similar personnel selection, training, and administrative requirements, nuclear plant control room operators can exhibit significant variability during non-routine events. ADS-IDAC captures this variability within three main categories:

• Procedure adherence and implementation — whether and how operators follow written steps
• Crew knowledge and experience — mental models, diagnostic capabilities, and memorized actions
• Individual preferences and tendencies — goal selection, use of memorized information, execution speed

Each source of variability is encoded as a branching event in the DDET. When a branch point is reached during simulation, ADS-IDAC generates separate event sequences representing each behavioral outcome. This allows the analyst to explore the full envelope of plausible crew responses to an accident scenario in a structured, probabilistic framework.

Scope Limitation

ADS-IDAC with RELAP5/MOD3.2 is currently limited to scenarios up to the onset of core damage. The crew model is not valid once the Technical Support Center (TSC) and Offsite Emergency Facility (EOF) are activated (approximately one hour post-accident). The DDET methodology is therefore applicable to the early-to-mid phase of accident progression.

---

2. General Guidance: Pre-Analysis Information Collection

Before defining branching rules, the analyst must gather information about operator behavior through four primary sources:

Written Procedures : Collect all procedures associated with the event(s) to be analyzed, including emergency operating procedures (EOPs), abnormal operating procedures (AOPs), and plant administrative procedures. Administrative procedures address communication norms, conduct of operations, self-checking behavior, skill-of-the-craft actions, and procedural adherence guidelines.

Training Materials : Review lesson plans, simplified plant drawings, and simulator training scenarios to identify key operator mental models and diagnostic processes. Training materials may contain "rules of thumb" or non-proceduralized shortcuts that operators employ in practice.

Operating Experience : Review plant-specific operating experience, NRC inspection reports, generic communications, licensee event reports (LERs), and NUREG reports for instances of unexpected or non-proceduralized operator actions, or cases where crews developed an incorrect mental model of plant conditions.

Expert Elicitation : Consult operations experts — training instructors, senior reactor operators, NRC inspectors — to identify areas where crews are known to exhibit difficulty responding to specific accident scenarios.

---

3. Five-Step Methodology (Specific Guidance)

Step i — Identify Potential Sources of Crew-to-Crew Variability

a) Operating Procedures

1. Identify major control actuations (stopping/starting significant pumps, opening/closing major valves, initiating/stopping depressurizations, cooldowns). These actions delineate major procedure phases. Crew briefings prior to significant plant actions can be captured with procedure hold points and action time branching rules.

2. Identify possible control manipulation variations. Where operators may apply too little or too much control input (e.g., throttle valve opened more or less than desired), model these with control value branching events.

3. Identify ambiguous criteria by searching procedure text for keywords such as "increasing," "steady," "decreasing," and "stable." These mark steps where crews may use different quantitative thresholds for qualitative descriptors. Model with conditional simulation runs to explore threshold ranges.

4. Assess procedure step objectives. Steps directly linked to main procedure objectives (e.g., safety injection actuation following LOCA) have lower likelihood of being skipped than supporting steps (e.g., shutdown of a feedwater heater). Use the step_type variable in the ZProcedure_procedure_name_step_name.txt file to categorize step objectives.

   !!! example "Steps with Safety Impact but Low Operator Salience" - Tripping reactor coolant pumps following loss of secondary heat sink reduces RCS heat input but does not directly recover feedwater flow. - Blocking automatic safety injection actuation prior to an intentional cooldown prevents unnecessary actuation but may not be salient during the accident.

5. Identify continuous action steps and fold-out page steps. These require periodic monitoring of a target parameter. Failure to monitor can delay required actions. Model using ActionTaker.txt, DecisionMaker.txt, and KB_OAT(ODM)_Scanned_Parameter.txt. Capture performance variability with mental beliefs in KB_OAT(ODM)_HardWired_Diagnosis.txt.

6. Identify steps involving failed components. These may trigger recovery or compensatory actions that can be modeled with mental beliefs in KB_OAT(ODM)_HardWired_Diagnosis.txt.

b) Mental Beliefs and Memorized Actions

1. Identify expected mental beliefs the crew will reach during the scenario (e.g., "uncontrolled steam generator level increase" for a tube rupture; "faulted steam generator" for a main steam line break). Crew variability may arise from failure to reach appropriate belief states, delays in acting, or higher/lower evidence thresholds. Encode in KB_OAT(ODM)_HardWired_Diagnosis.txt.

2. Identify non-proceduralized anticipatory actions to increase time available before automatic safety feature actuation (e.g., maximizing RCS makeup during a primary leak, reducing steam loads during partial feedwater loss). These arise from training and experience and are activated by mental beliefs, with the action described in ZProcedure_MPBG_procedure_name_step_name.txt.

3. Identify conditions for manual safety system actuation. A crew may manually initiate safety injection at low pressurizer level. Model with appropriate mental beliefs.

4. Identify control system failures with similar symptoms. The KB_OAT(ODM)_Event_Matrix.txt must include a wide spectrum of simple failures, abnormal conditions, and emergency events to realistically represent diagnostic behavior. Operators may confuse accident events with minor control system failures, distracting the crew from the actual event.

c) Crew Preferences and Tendencies

1. Use of memorized versus current information. Set crew tendency in ActionTaker.txt and DecisionMaker.txt. Using memorized information reduces activity execution time but may introduce stale or incorrect data.

2. Procedural adherence tendency. Each procedure step has three components: (1) initial action activity, (2) expectation evaluation, (3) non-response action. The skip probability for components (1) and (3) is calculated dynamically based on baseline skip probability, procedure type, step objectives, situational relevance, and PIFs. The skip threshold is set in ActionTaker.txt and DecisionMaker.txt.

3. Emergency procedure entry threshold. The ADS-IDAC diagnostic engine calculates a membership value for each possible accident event. When this value exceeds the abnormal_signal_threshold, the crew transitions to EOPs. This threshold captures crew variability in delaying EOP entry or requiring strong evidence before shutdown.

4. Knowledge-based troubleshooting versus written procedures. Some crews address emergencies through knowledge and experience rather than following written procedures. The troubleshooting_probability and procedure_use_probability in ActionTaker.txt and DecisionMaker.txt model this tendency.

5. Procedure execution speed. Use procedure hold points (e.g., at procedure starts, transfers, or major actions) with "dummy" procedure steps providing a time delay. Combine with action time branching rules to simulate fast, slow, or nominal crews.

Step ii — List All Branching Events

Compile a complete list of all applicable branching events, including:

• All crew-to-crew variability sources identified in Step i
• Hardware-related events: initial failures, conditional failures, and operator recovery actions

This forms the basis of the simulation branching event matrix.

Step iii — Define Branching Parameters

For each branching event, specify the appropriate parameter values:

Branching Probabilities : The Beta Distribution (parameters α and β) is used for hardware failure and recovery probabilities. Mental belief activation uses point probability estimates. The analyst must supply either the Beta Distribution parameter estimates or a point estimate for each event.

Timing Parameters : Procedure step execution times and certain decision-making activities use a three-parameter Weibull distribution. Crew communication and procedure transition times use point estimates.

$$F(t)=1-\exp [-{\left(\frac{t-u}{\alpha }\right)}^{\beta }]$$

Where: u = minimum time (location parameter), α = scale parameter, β = shape parameter.

Control Value Variability : Identify the control values (e.g., valve position fractions) and point probability estimates for control input variations.

Timing Variability : Identify procedure steps where action timing branches should be generated and determine the number of branches per step.

Mental Belief Activation Time Variability : Identify mental beliefs and the number of associated branching events for variabilities in initiating skill- and rule-based actions.

Goal and Problem-Solving Strategy Tendencies : Point probability estimates in ActionTaker.txt and DecisionMaker.txt.

Step iv — Define Sequence End States

Two methods for terminating a sequence when a critical parameter exceeds a threshold:

1. A VSTOP non-response procedure action
2. Activation of an end sequence alarm with an A_ENDSEQ_ prefix

Identify all critical parameters and their threshold values (e.g., low reactor vessel water level indicating fuel uncovery / onset of core damage).

Step v — Develop a Simulation Matrix

Due to the sequence explosion problem (exponential growth in event sequences with branch count), it is not currently practicable to exercise all branching points in a single simulation.

Sequence Explosion

As the number of branching events increases, the number of generated accident sequences grows exponentially. This can make simulation run times prohibitive. Use conditional runs with a reduced, carefully selected set of active branching rules.

Recommended approach:

• Select three or four high-likelihood branching events per simulation run
• Suppress all other branching rules in that run
• Ensure the full matrix of runs covers all significant branching rules (each rule included in at least one run)
• Capture a wide range of potential crew behaviors with minimal processing overhead

---

4. Table 1 — Mapping Crew Variability to ADS-IDAC Branching Rules

Type of Variability	Source of Crew-to-Crew Variability	ADS-IDAC Branching Rule(s)
Procedure Execution	Selection of component target control settings (e.g., throttle valve positions, manual controller set points)	Action control value branching rule
Procedure Execution	Time required to execute procedure actions	Action time branching rule
Procedure Execution	Recovery of failed equipment	Equipment failure and recovery branching rule
Procedure Execution	Omission or skipping of procedure steps	Procedure step skipping branching rule. Branching enabled by the step skipping probability threshold in the operator profile.
Procedure Execution	Interpretation of potentially ambiguous procedure criteria (e.g., translation of qualitative descriptors such as "decreasing," "increasing," or "steady" into context-specific quantitative criteria)	No branching rule available. Conditional simulation runs can be used to examine different quantitative thresholds for qualitative criteria.
Operator Mental Models	Training, experience, and diagnostic capabilities	Mental belief branch probability branching rule. The operator knowledge base should include a sufficient spectrum of mental beliefs to capture crew variabilities due to training and knowledge.
Operator Mental Models	Time required to initiate automatic memorized actions	Mental procedure activation time branching rule
Operator Mental Models	Selection of high level goals and problem solving strategies	Troubleshooting probability and procedure use probability branching rules
Operator Preferences	Tendency to rely on memorized information rather than control panel readings	Use of memorized information branching rule
Operator Preferences	Delays or breaks during procedure execution (e.g., crew briefings)	Action time branching rule. Appropriate procedure hold steps must be incorporated into ADS-IDAC procedures to model delays or breaks.
Operator Preferences	Tendency to manually perform anticipatory safety actions prior to automatic actions	Mental belief branch probability branching rule. The operator knowledge base must include appropriate mental belief to trigger the desired actions.

---

5. Table 2 — Sample Branching Rules: Complete Loss of Feedwater Analysis

Branch ID	Branching Event	Branching Options	Branching Values	Importance	Comments
TRIP	Action Taker Hardwired Diagnosis: "Complete Loss Of Feedwater"	Branch Probability (2 Branches)	Branch probability options: 0.0, 0.5, or 1.0	High	Enables early reactor trip upon loss of main feed pumps. Probability of 1.0 enables early trip; 0.0 blocks early trip; intermediate value generates both event sequences.
FRGTime	Decision Maker Hardwired Diagnosis: "CSF Loss of Secondary Heat Sink"	Mental Procedure Activation Time Branch (3 Branches)	Time Delay: α = 100., β = 1.5. Long, nominal, and short delay times.	High	Sets time delay for initiation of FRG H.1 (once entry conditions have been perceived). Represents crew delay associated with transition from ES 0.1 to FRG H.1.
Brief-1	FRG H.1 Briefing Hold 1	Action Time Branches (3 Branches)	Time Delay: α = 200., β = 1.75. Long, nominal, and short delay times.	High	Delay time prior to initiation of FRG H.1. Represents crew briefing prior to initiation of FRG H.1 actions.
SKIPRCP	Skip FRG H.1 Step 3 (trip reactor coolant pumps)	Skip step threshold value, base step skip probability (2 Branches per pump: perform step and skip step)	Decision Maker "skip action threshold": 1.0, 0.75 (skip). Base RCP Skip Probability: α = 400.0, β = 10,000.0	Low	Failure to trip reactor coolant pumps will increase net RCS heat input. RCPSkip base probability and skip step threshold were increased to avoid generation of excessive skipped step branches.
STMDUMP	FRG H.1, Step 7.c, excessive or insufficient steam dump rate during secondary depressurization	Procedure Control Value Branches for FRG H.1 Step 7.5 Action "X_Stm_Dump" (3 Branches)	Control Values: 0.25 – Excessive; 0.10 – Nominal; 0.05 – Insufficient	Medium	Establishes secondary depressurization using the steam dump. Excessive steam dump rate could actuate main steam isolation and delay recovery. Insufficient steam dump rate could delay recovery.
MDAFPA / MDAFPB	Hardware Reliability: Motor Driven AFW Pump A(B). Activated by actuation of "Reactor_Trip"	Hardware Reliability X_MD_AFW_Pump_A / X_MD_AFW_Pump_B (3 branches each: initial success, recovery, and failure)	Trigger Event: Reactor Trip. Failure: α = 1.0×10⁵, β = 1.0. Recovery: α = 1.0, β = 1.0×10⁵	High (established by scenario description)	Failure of motor driven AFW pump A(B) following Rx Trip. Low probability of recovery prevents recovery of pumps.
MFPA / MFPB	Initiating Event: Main Feedwater Pump A & B	Initiating Event X_MF_Pump_A / X_MF_Pump_B (2 branches each: permanent failure and failure/recovery)	Trigger Event: Reactor Trip. Failure: α = 1.0×10⁵, β = 1.0. Recovery: α = 1.0, β = 1.0×10⁵	High (established by scenario description)	Initiating event is trip of main feedwater pumps. The condensate pump head curve is modified in the RELAP input deck to model degradation. Low probability of recovery prevents recovery of pumps.

---

6. Branching Type Details

Timing Branches (Weibull)

Action time branches model variability in the time required to execute a procedure step or initiate a mental action. A three-parameter Weibull distribution is used:

$$F(t)=1-\exp [-{\left(\frac{t-u}{\alpha }\right)}^{\beta }]$$

Typically three branches are generated: long, nominal, and short delay times, corresponding to different quantiles of the Weibull distribution. The analyst specifies the α (scale) and β (shape) parameters. The location parameter u (minimum time) establishes a lower bound on action time.

Procedure Hold Points for Crew Briefings

Insert a "dummy" procedure step immediately before a major action to create a hold point. Apply an action time branching rule to this step to model the variability in time spent on crew briefings or pre-action discussions. This avoids creating a branching rule in the middle of a procedure step.

Control Value Branches (Discrete)

Control value branches model operator variability in the selection of a continuous control setting — for example, a throttle valve position or a manual controller setpoint. The analyst specifies:

• The name of the interactive control variable (e.g., X_Stm_Dump)
• A discrete set of control values representing the range of operator behavior (e.g., excessive, nominal, insufficient)
• A probability for each control value branch

These branches are encoded in the ProcedureControlValueBranches.txt input file and cross-referenced to the specific procedure step in ZProcedure_procedure_name_step_name.txt.

Mental Belief Activation Branches

Mental belief branches model variability in whether and when an operator reaches a particular situational assessment. The KB_OAT(ODM)_HardWired_Diagnosis.txt file encodes:

• The activation criteria (evidence threshold) for each mental belief
• The branch probability for belief activation
• The mental procedure triggered by belief activation (ZProcedure_MPBG_procedure_name_step_name.txt)
• The activation time Weibull parameters for mental procedure activation time branches (encoded in MentalProcedureActivationTimeBranches.txt)

Mental belief branches capture variability in crew diagnostic capability, training differences, and tendencies to take anticipatory actions.

Hardware Failure Branches (Beta Distribution)

Hardware failure and recovery events use the Beta Distribution to describe the probability:

• Failure probability: specified by Beta parameters α and β at the trigger event time
• Recovery probability: separate Beta parameters for the conditional recovery branch
• Trigger event: the plant event (e.g., "Reactor_Trip") that activates the hardware reliability evaluation

These are encoded in the SystemReliability.txt input file.

Beta Distribution Convention

For near-certain failure (α >> β, e.g., α = 1.0×10⁵, β = 1.0), the expected probability approaches 1.0. For near-certain success (α << β, e.g., α = 1.0, β = 1.0×10⁵), the expected probability approaches 0.0. The Beta mean is α/(α+β).