Drivers for safety culture in waste processing

Based on experience in management positions within both DOE and U.S. Army environments it is clear that although contractual standards and requirements can be different, expectations for safe performance are very consistent. A review of four case studies, one from the West Valley Demonstration Project (WVDP), a DOE high-level waste (HLW) management facility located in New York State, one from a groundwater treatment plant at a DOE Legacy Management facility located in Tuba City, Arizona, and two from the Tooele Chemical Agent Disposal Facility (TOCDF), a U.S. Army chemical demilitarization facility located in Utah, demonstrates significant commonalities in the primary causes of significant events. Conclusions can then be drawn for those elements most critical to safe performance in a high-hazard operational environment.

The four case studies examined in this analysis were chosen due to the impact they have had on the author, either from direct experience in terms of implementing post-event corrective actions, or in terms of involvement in the root cause analysis process, as well as the impact they had on overall project performance. Although having varying degrees of seriousness, all incidents resulted in consequences, and were defining moments for these facilities. Since the incidents, all of the facilities have demonstrated their commitment to continuous improvement processes by later achieving STAR status in the Voluntary Protection Programmes (VPP) on safety management administered by DOE, in the case of the WVDP and Tuba City facilities, and the Occupational Safety and Health Administration (OSHA) in the case of the TOCDF site.

The author’s experiences gained as a federal project manager within the DOE environmental management programme from the early 1990s through 2003, and as a senior manager within the U.S. Army’s chemical demilitarization programme contractor community from 2003 through 2010 support this analysis. The primary data sources for this examination are the investigation reports: in two cases the post-event investigations were independent investigations chartered by the government (the 1999 West Valley incident and the 2002 chemical agent exposure), and in the other two cases internal root cause analysis reports were used. It is also important to note that each of the post-event investigations utilized a slightly different cause analysis method or technique. For the purposes of comparison, the term ‘key contributing causes’ is used in the following discussion to present other primary factors, in addition to the root causes, that led to the events and that are important to consider as part of this analysis. Note that as is the case with such operational events, the investigation reports are very detailed and quite complex. The summaries below capture the events at a level sufficient for a comparison of the primary performance failures, but do not attempt to provide a complete description of all aspects of each event.

Vitrification facility HLW backflow event

The Department of Energy West Valley Demonstration Project, West Valley, New York began high-level waste (HLW) solidification (vitrification) operations in 1996. Operations included transfer of HLW from a tank farm to a shielded cell within the vitrification facility, where waste was mixed with chemicals to produce a ‘slurry’ as part of the vitrification feed preparation process. The process required routine sampling of the slurry, which was performed remotely from a sampling station located within an operating aisle outside the shielded cell. On Saturday 16 November 1996, during routine sampling operations, operators experienced lower-than-anticipated sample flow and consequent problems in filling the sample bottles. In accordance with their procedure they contacted engineering support and per engineering instructions, a back-flushing operation was attempted to clear a presumed case of blocked lines. Shortly following the back-flushing attempt, various radiation monitors in the area went into alarm, personnel in the operating aisles exited, and response actions were taken. Although the event resulted in HLW being transferred outside of engineering controls, no personnel exposures resulted.

Initial investigation determined that diluted HLW ‘slurry’ had exited the shielded cell through a demineralized water line and into piping in an operating aisle. The local DOE office commissioned an independent investigation into the root cause as well as an evaluation of the contractor’s corrective actions to verify that they would adequately prevent recurrence. Barrier analysis was used to support the accident investigation in accordance with the DOE ‘Root Cause Analysis Document’ (DOE-NE-STD-1004-92). The DOE investigation determined that HLW slurry exited engineering controls due to failure or misalignment of a three-way valve (Valve HV-0213), which was the last of a number of barriers that could have prevented the event. The investigation report describes how the plugging problem experienced was not entirely uncommon and back-flushing operations had been used successfully in the past (using a different valve line-up), however no back-flushing operation had been included in the operational procedure.

When the problem was encountered that particular Saturday, the most knowledgeable system (the so-called ‘cognizant’) engineer was not available on site, and the process for back-flushing was developed by a less knowledgeable on-site engineer with some guidance provided remotely by the cognizant engineer. No formal work control process was used that would have involved appropriate hazard analysis and reviews/approvals. The instructions created, which were informally written on a system drawing, did not use the same back-flushing process provided by the System Description, apparently due to the view on the part of the cognizant engineer that it was overly complex. Guidance provided by the cognizant engineer to the on-site engineer provided a method to back flush by arranging the sample station valves and slurry pump such that demineralized water would flow backwards through the slurry sample station valves/piping and through the slurry pump up to Valve HV-0213, which would be positioned to flush back to the slurry waste tank inside the shielded cell.

A reconstruction of the event confirmed that with alignment of the valves to back flush through the sample station piping and slurry pump to the slurry waste tank via Valve HV-0213, HV-0213 malfunctioned and resulted in an incorrect position that allowed flush water to pass up through the demineralized water line and out of the shielded cell into the operating aisle. Figure 1 provides a summary of the failure sequence.

The investigation identified the following as the root cause:

“An approved procedure was not used to perform the back-flushing procedure. The informal instructions provided did not consider the risk of failure of Valve HV-0213 nor were they properly reviewed by appropriate technical disciplines.”

The following key contributing factors were identified:

• There was a perception of management pressure to complete the sampling operation
• The engineer available on site for providing work instructions was not as familiar with the system as the cognizant engineer
• A decision was made not to base the back-flushing instructions on a method described in the existing System Description
• Uncontrolled drawings were used to verify valve line-up for back-flushing operations
• The System Description did not reflect the current configuration of the sampling station
• The location of Valve HV-0213 made verification of valve position very difficult.

The primary corrective action put in place following the event was the development of procedures for the sampling operation and other similar operations that included the appropriate hazard analyses to ensure that the necessary defence-in-depth and controls were established.

Chemical agent exposure

The U.S. Army Chemical Materials Agency Tooele Chemical Agent Disposal Facility in Tooele, Utah is a chemical agent munitions destruction facility that has processed both nerve and blister agents. The facility includes four incinerators for agent destruction: two liquid incinerators (LICs), a metal parts furnace (MPF) and a deactivation furnace (DFS). In short, the process involves draining the agent from the munitions bodies or items (for example, ton containers or spray tanks), transferring the liquid agent to one of two LICs for incineration, processing energetics (explosive components) through the DFS, and processing the remaining metal parts/items through the MPF. This particular event occurred during what is referred to as a changeover period, when the facility is non-operational following completion of an agent campaign, and preparing for the next campaign. During this period, the facility equipment has been decontaminated to a specified level (determined by air monitoring) that allows work to be conducted within certain areas in a lower level of personal protective equipment (PPE).

In July 2002, TOCDF was in a changeover period following completion of a campaign of the agent GB (the nerve agent also referred to as Sarin) and was making preparations for a future VX agent destruction campaign. On 15 July 2002, two workers were conducting maintenance on one of the LICs (referred to as LIC 2). (See Figure 2 for a simplified schematic of the LIC feed system). The maintenance involved installation of a modified air pressure regulator. TOCDF had performed a similar installation of a modified air pressure regulator on the other LIC (LIC 1) several months prior when the facility was still processing GB. To start work on LIC 2, the workers removed a section of pipe containing the existing air pressure regulator (using a wrench and by hand) and placed it on the floor. Immediately thereafter the GB air monitor went into alarm. The workers exited the immediate area and changed their masks to those that offered a higher protection factor. During the mask change-over, agent GB from the glove of the worker that handled the pipe was transferred to the area around the worker’s head, resulting in exposure which was confirmed via medical evaluation (the worker experienced symptoms, although the exposure was not fatal).

A board of investigation was established by the Army to determine the causes of the incident and recommend corrective actions to preclude future reoccurrence of this and similar incidents at TOCDF. The Board utilized an event and causal factor analysis process to investigate the incident in accordance with Department of Army (DA) Pamphlet 384-40, “Army Accident Investigation and Reporting.” The investigation determined that the prior air pressure regulator replacement work on LIC 1 had identified the failure of two check valves and a block valve that were intended to prevent backflow of agent into the air purge system associated with the incinerator feed system. (LIC 2 was configured with the same backflow isolation devices.) Since the maintenance work on LIC 1 was conducted during agent operations, the LIC 1 pressure regulator replacement work was conducted in a fully-encapsulated chemical protective suit with supplied air, referred to as a Demilitarization Protection Ensemble (DPE).

Unfortunately, the lessons that could have been learned from the LIC 1 work were not transferred to support planning for the similar maintenance work on LIC 2. In addition, the work planning for the LIC 2 job was done primarily by the maintenance department and did not include sufficient involvement or review by the engineering and safety organizations. Those planning the work made the poor assumption that an initial decontamination of the TOCDF meant that it was completely safe. In fact, only external surfaces had been cleaned. This particular maintenance job required breaking into the internal piping, and did not include any steps for confirming the safety of the internal air system. The PPE specified for the job, a full-face industrial respirator, overalls and leather boots and gloves, did not adequately mitigate the risk of the potential presence of agent.

The investigation revealed multiple shortcomings in the areas of: (1) worker safety, including work planning; (2) engineering, operations, and maintenance, including process safety and configuration management; (3) hazard communication and lessons learned; and (4) management involvement and oversight. The investigation concluded that the following root causes led to the incident:

• Failure to establish a TOCDF-specific lessons learned programme that disseminated this information to the workforce
• Poorly-defined roles and responsibilities for the safety and engineering groups that did not require active participation and oversight of non-routine work
• A lack of management involvement and oversight of non-routine activities during an outage involving many changes to TOCDF in preparation for a new agent campaign.

The following summarizes the key contributing factors (note that the investigation revealed many other contributors related to both the event and the response to the event; the list below highlights the primary contributors to the initiating event):

• Inadequate non-routine work planning and pre-entry planning
• Procedure non-compliance
• Inadequate consideration of process safety basis in engineering change proposals
• Failure to follow hierarchy of controls (an over-reliance on PPE instead of prior administrative/engineering factors)
• Failure to recognize potential agent hazards during a changeover period.

The Board of Investigation identified a common theme in the findings of the report regarding safety culture, which it defines as “a set of attitudes and attributes reflected in workers, supervisors, and managers that safety is the fundamental priority and prerequisite for doing work.” The report goes on to describe the attributes of a healthy safety culture: working in a structured, disciplined manner; observing the hierarchy of controls in work planning and execution; and providing an atmosphere that encourages the workforce to participate in near miss reporting, promotes technical inquisitiveness, and reinforces individual accountability for safety.

TOCDF was non-operational for approximately nine months following the chemical exposure event. Significant process improvements were put in place, including an entirely revised, team-based work planning process; revised engineering procedures for more robust configuration management; and a process for routine agent boundary verifications. In the spirit of continuous improvement, TOCDF continued to pursue many management initiatives that led to additional and significant performance improvement well after the event, to include more effective management engagement through a manager/supervisory leadership development programme; new condition reporting, cause analysis, and corrective action processes; and the development of a key performance indicator programme. As a result, the OSHA recordable incidence rate (number of accidents/100 employees/year) went from more than four to below one.

Caustic burn during toxic entry

Approximately six years after the agent exposure event, and following a period of full implementation of the many process and management-related improvements that were instituted in response, TOCDF experienced an event that reinforced the concept that once an organization believes it is approaching excellence, it is at risk of slipping backwards in performance. On 7 May 2008, two workers made an entry into an area within the TOCDF Munitions Demilitarization Building (MDB) to perform maintenance on a piece of demilitarization equipment, as well as several routine preventative maintenance items (PMs). The level of protection worn was DPE (a fully-encapsulated chemical protective suit with supplied air). Work within the MDB can involve tight spaces and the risk that contact with equipment might tear the suit. The entry procedure requires frequent ‘suit checks’ by entrants, including prior to personal decontamination activities. During entry, one of the workers experienced a wet sensation in the shoulder area and asked the other worker to assist in a suit check, which resulted in the discovery of a small tear in the area of the wet sensation. The personal decontamination process includes a wash with an 18% sodium hydroxide solution. The worker was performing this wash just prior to the identification of the suit tear, which resulted in a caustic burn to the left shoulder.

An internal management-led root cause investigation was chartered, with participation from subject-matter experts from the Umatilla Chemical Agent Disposal Facility (UMCDF), a similar US Army facility in Oregon, to provide an external perspective on entry control processes. An approach using event and causal factor analysis was taken to determine root, apparent, and contributing causes in accordance with the site-specific cause analysis procedure. DPE entries at TOCDF are videotaped and stored for a period of time, a procedure that provided an invaluable tool for the investigation team. When the videotape from the entry that resulted in the caustic burn was reviewed, it revealed that not only were entry procedures for performing suit checks not followed, but also other procedures were not followed. Videotapes from other maintenance entries were also reviewed. The entry videotape reviews, coupled with results of personnel interviews and document reviews, made it clear that the lack of procedure compliance that contributed to this event was not an isolated occurrence and that the lack of procedure adherence and poor work practices were known by supervision and not corrected.

The report was clear about the root cause of the event:

“High hazard work has become routine and allowed complacency and poor work habits to develop; management has failed to identify and correct the issues due to a lack of oversight.”

Key contributing factors to the event included:

• A continuing overall lack of procedure compliance culture
• Real or perceived pressure to complete DPE entry tasks
• Ineffective work planning; particularly with respect to the use of generic safety plans, inconsistency in the level of detail provided within work packages, and inconsistent implementation of the team-based integrated work planning concept
• Lack of demilitarization equipment reliability, which contributed to reactive, as opposed to proactive, work planning
• Lack of a clear and consistent entry control process.

Corrective actions implemented following this event substantially improved entry performance. Actions included:

• Institutionalizing senior management expectations for management assessments of entries
• A review and revision of PM instructions to ensure correct work steps, and that hazards are analyzed and appropriate controls established
• Establishing clear accountability within the work control procedure
• Developing a revised entry control process that included increased resources to manage toxic entries
• Establishing a team led by the engineering department to proactively manage demilitarization equipment maintenance and improve reliability
• Establishing an employee-based team focused on cultivating a questioning attitude during toxic entries and improving entry performance.

Adverse trends leading to operational shutdown

The DOE Tuba City Disposal Site, Tuba City, Arizona, is located within the Navajo Nation, in the area of a former uranium mill that was remediated under the Uranium Mill Tailings Radiation Control Act. DOE completed site remediation in 1990; all mill tailings and remediation debris were consolidated within an on-site engineered disposal cell. Following site remediation, long-term surveillance and maintenance activities have been primarily associated with maintaining and monitoring the disposal cell, monitoring groundwater, and operating a Water Treatment Plant (WTP) to reduce levels of uranium and other constituents in the groundwater. There are nearly 90 sites across the United States that have been remediated but require post-closure surveillance and maintenance; these sites are managed by Legacy Management Support (LMS) contractor S.M. Stoller Corporation.

The Tuba City water treatment process consists of three steps: an ion-exchange water softening stage, a sulphuric acid feed system to reduce pH, and an evaporator system that concentrates contaminants such as molybdenum, nitrates, selenium, sulfates and uranium. Over an approximately two-year period in 2009 and 2010 there were repeated occurrences of operational issues at Tuba City related to water chemistry, clogging of the evaporator pond drain line, acid management, and poor condition of system equipment and components. Following an acid tank overflow incident, the WTP was shut down in October 2010. Shortly thereafter two additional operational incidents occurred involving the acid tank system: (1) a minor acid burn to a worker’s face and (2) an acid tank drain line flange leak. These incidents prompted S.M. Stoller Corporation to charter a root cause analysis of the adverse trends and determine actions necessary prior to WTP restart.

An investigation approach using event and causal factor analysis, in accordance with LMS programme-specific procedures, determined the following root causes:

• Management failed to recognize system design flaws, understand process chemistry, identify the lack of maintenance and inspection activities in assessments, and recognize and control hazards associated with the design and with operational workarounds
• There was a lack of accountability regarding corrective action response, identification and control of operational and maintenance issues, and analysis of incident trends. In addition, sufficient management mentoring and support were not provided for the Tuba City staff and operations.

The root cause analysis also prompted two additional reviews, an independent third-party engineering evaluation of system safety and operability, and a LMS programme-wide conduct of operations assessment that included Tuba City and the other primary LMS operational facilities. A lessons-learned report was generated that captured the collective results from all of the reviews and provides the following lessons from the Tuba City experience:

• Conduct of operations must be graded appropriately to consider the wide range of facilities and operations within LMS; in the case of Tuba City there was a lack of understanding of the level of rigour required for operations and work planning considering hazards such as bulk storage and use of 93% sulphuric acid
• LMS processes must maintain the appropriate level of engineering and software configuration management across its varied locations
• Senior management must properly balance project management and line/operations management priorities. Essentially, there was too much emphasis by the project management organization on cost; this contributed to insufficient resources being applied to maintain equipment and ensure necessary procedures were in place.

Following the cause analysis, the safety and operability assessment, and the conduct of operations assessment, LMS’s list of corrective actions for restart included: (1) an enhanced programme for management oversight and support to include the establishment of new site operations manager and process engineer positions to provide the necessary on-site support to operators, (2) significant equipment modifications and repairs, (3) revision to the LMS work control process to increase rigour in corrective maintenance planning, (4) the development of a preventative maintenance programme (none had previously existed), (5) the development of an operator training and qualification programme (none had previously existed), and (6) significantly-revised operational procedures that included step-by-step instructions as opposed to general descriptive language. The Tuba City WTP was non-operational for nearly a year while plant repairs and process-related corrective actions were implemented.

Conclusions

There are many common themes in these disparate incidents. Three of the four root cause analyses cite lack of management engagement (oversight, involvement, ability to recognize issues, and so on) as a root cause to the events. Two of the four root cause analyses cite work planning failures as a root cause to the events. All cause analyses mention work planning failures as contributing factors to the events. All events except the Tuba City plant shutdown indicate procedure noncompliance as a key contributor; in the case of Tuba City the procedure issues were primarily related to a lack of procedures, or a lack of sufficiently-detailed procedures. All events included discussion or suggestion of a lack of a questioning attitude, either on the part of management/supervision, work planners, or workers.

This analysis suggests that the most critical drivers to safety culture are:

• Management engagement
• Effective work planning and procedures
• Procedure adherence with a questioning attitude to ensure procedural problems are identified and fixed.

In high-hazard operational environments the importance of robust work planning processes and procedure adherence cannot be overstated. However, having the processes by themselves is not enough. Management must actively engage in setting expectations and ensure that work planning meets expectations for hazard analysis and control. It must develop a culture that encourages incident reporting and a questioning attitude. It must routinely observe work performance to reinforce expectations for adherence to procedures/work control documents.

Elizabeth A. Lowes, corporate compliance manager, the S.M. Stoller Corporation, 105 Technology Dr., Suite 190, Broomfield, Colorado 80021. This article was based on a paper (no. 12382) presented at the WM2012 conference, February 28-March 1, 2012, Phoenix, Arizona, USA.

This article was published in the July 2012 issue of Nuclear Engineering International magazine