The simulator sell

By design and by law, full-scope training simulators closely replicate the experience of operating the reactor, so that students can learn the ropes without risking the ship. Although simulators of traditional nuclear power plant analogue I&C systems may closely match the behaviour of their reference plant, their architecture is completely different, and now wholly-digital.

Ironically, simulators are used much more intensively than the real plant I&C. In many countries, regular training on simulators is a condition of maintaining a reactor operator’s licence qualification. Full-scope simulator schedules can be booked solid for months in advance.

As a result, simulator instrumentation tends to age more quickly than plant equipment, so requiring upgrades every few years, according to Michael Chatlani, vice president, marketing & sales, power systems and simulation at Canadian simulation vendor L-3 MAPPS.

The frequency of incremental simulator upgrades depend on the specific type of componentry, according to Bob Felton, director of operations at US simulator vendor Western Services. Computers are typically replaced every 3-5 years for performance improvements and computer operating system upgrades. Input/output (I/O) systems (which mediate between the computers and the panels) are typically replaced every 10-15 years partly due the decreasing availability of spare parts. Panels are typically not replaced; first, they must remain faithful to the plant’s control room; second, the installation of panels is a major part of the total cost of full-scope simulator hardware. Still, they are likely to require maintenance (replacing bulbs, for example).

End-users’ simulator teams manage most small-scale simulator upgrades themselves, says Kevin Lagasse, GE’s senior vice president, nuclear services, since they are intimately involved with the simulator day-to-day. (Although GE does not supply simulators, as a reactor vendor it supplies control system hardware and software elements in simulators). Much of this kind of work would be contracted out to local specialist engineering firms who have built up familiarity with the simulators over the years, he says.

Ongoing maintenance of modern simulators is even less onerous. Jim Eberle, CEO of simulator vendor GSE Systems, said that hardware maintenance consists of replacement of computer mice, keyboards and monitors. End-users are also confident about making minor tweaks to the simulator software, says GE’s Lagasse; there are ways of introducing modifications, and there is a procedure for change requests.

Still, simulator vendors are involved in supporting these processes. GSE’s Eberle says that half of the company’s revenue comes from this kind of work, which he characterises as ‘installed base services’. L-3 MAPPS’ Chatlani says that although the company is ‘continuously’ involved in FSS upgrades, and that this work helps fund its own R&D, his company is mainly contacted for larger upgrades, which usually have a budget assigned to them.

Chatlani outlines what might make a utility go for a large uprate. “Once you start seeing numerous training interruptions due to hardware failures or modelling issues, you need to consider an upgrade. These training interruptions are tracked and evaluated. Perception of simulator performance can also be a driving force behind an upgrade. Benchmarking against other plant simulators in the industry and self-assessments are also used to evaluate if the simulator performance meets expectations. Over the years, simulator performance expectations by plant management and other organizations (such as the regulator, INPO, WANO, etc.) are increasing every year.”

Major simulator changes can take a year and cost up to several millions of dollars, Eberle says, depending on the system’s complexity (as measured by the number of inputs and outputs simulated). He estimates that most of the simulators still operating with hard-wired electrical connections will be replaced in the next ten years.

Felton at Western says that the biggest factor of success in a simulator upgrade is a thorough and concise description of user requirements. The vendor then collaborates with the customer in refining that list. Once technical requirements are developed, the project moves on to negotiation of commercial and financial aspects.

GSE’s Eberle says that personal relationships between vendors and station simulator staff play a big role in the business of upgrades, and in selling them the advantages of an upgrade, be it human-machine interface (HMI) improvements, improving scenario-based testing, or new features in the instructor stations, for example. Despite this informal contact, larger-scale upgrades are usually procured via a formal bidding process that requires technical pre-qualification, says Eberle. In such a bidding process, the original simulator vendor would have a natural advantage, since every simulator toolset is slightly different (although none are tied to a reactor vendor). Despite this, simulator vendor switch-overs do happen.

Simulators are also upgraded to match upgrades or changes in the plant itself; in some countries, simulators are required to match plant changes within a specified period. Most utilities’ Plant Design Change Modification engineering process alerts the simulator team of a potential reference unit change that needs to be evaluated with reference to the need to upgrade the simulator, according to Felton. He adds: “The benefit of an upgrade should always be measured against the improved fidelity of the simulation and hence the more lifelike representation of the reference unit presented to the student.”

Thanks to the increasing spread and popularity of microcomputers worldwide, fidelity is cheaper than ever to achieve. In the space of a generation, commercially-available computers have become very powerful. Michael Chatlani of L-3 MAPPS says, “Today, there is absolutely no reason, if the information is available, not to simulate every pipe, every wire, valve and component in the plant.””

Simulating a digital control system

Over the past 10 years, new nuclear power plants have incorporated digital control systems (DCS), which fundamentally rely on microcomputers to drive operational and safety instrumentation. In the same period, operational plants with pre-digital instrumentation and control systems have begun to examine the possibility of switching these systems from analogue to digital, partly as a result of station life extension. Last year, Duke’s Oconee Nuclear Station was the first in the USA to go fully-digital (based on an Areva Teleperm XS I&C system, and with an L-3 MAPPS simulator). Other recent analogue-to-digital I&C upgrades include Hungary’s Paks (Areva Teleperm XSt), OKG’s Oskarshamn 2 and EnBW’s Philippsburg 2. In France, a tender has been published to partly upgrade the instrumentation and control system to digital at the eight stations with twenty 1300 MW nuclear reactors, according to Pascal Gain, vice president, power simulation, at France-based simulator vendor corys tess.

This wide-ranging change affects many aspects of the plant; making sure the new system will be safe and reliable requires years of planning and analysis before final execution. Similarly, upgrading a full-scope simulator to reflect a plant change from analogue to digital I&C is a huge undertaking, says Pascal Gain, not so much for technological reasons as for the scale of the scope required. For example, an emergency backup diesel generator replacement that included a new digital controller would require a relatively minor simulator upgrade, he says, because the scope of the alteration is limited. However, a global I&C change affects everything, and so everything in the simulator must be changed, and then validated.

There are three ways to simulate a nuclear power plant’s digital control system: stimulation, emulation and simulation. Each has advantages and disadvantages. ‘Stimulation’ relies on a duplicate of the plant’s own hardware and software to react to an input created by a simulated process model. ‘Emulation’ reproduces the actual plant system software code on the simulator computer, without its (probably expensive) proprietary hardware. ‘Simulation’ incorporates a model of the plant within the simulator environment itself, based on data taken from the reference plant.

Schematic views of information architecture in three types of simulations of digital control systems. Top to bottom: Stimulation; Emulation; Simulation. Shaded area depicts extent of simulator environment.

NB: the model building tool is also in the simulation information architecture in the bottom figure.

Two non-computer examples of emulation come from GE. At one older US BWR plant, under certain circumstances, the control room clock would stop. So in the simulator, the clock was rigged to stop in those same circumstances, to help recreate the pressure on the operator in such scenarios. Also, the old hardwired control panels produce a lot of heat; they become hot to the touch, and make the operators sweat. So in the simulator, where the computers driving the simulator’s control panels do not generate as much heat, quartz heaters were added under the panels to provide a more realistic experience.

Gain of corys points out that the levels of plant I&C that can be simulated and the variety of simulation types muddy the waters. Systems to simulate include the plant’s control room environment, typically an HMI running on a computer, and the plant’s actual control and regulation functions, which are typically run on a programmeable logic controller (PLC). Dividing nuclear power plant I&C in another way, there are systems for normal plant operations and reactor protection systems. Each could be stimulated, emulated or simulated. Different vendors could be used for different systems, raising the possibility of conflict between them; someone within the simulator team would need to be responsible for integration. Later upgrades may need to coordinate multiple vendors.

The result of negotiations with the DCS supplier may determine which combination of solutions is the best fit for the overall budget, according to Chatlani. For example, he says, “if the DCS vendor of the plant has an emulation that we can use then we’ll contract with that DCS vendor, as opposed to reinventing the wheel.” Cooperation with the DCS vendor may be negotiated by the end customer or the simulation vendor, or sometimes by all three.

But he also says that the type of simulation carried out will affect the way that it is upgraded. “If the entire DCS is simulated then it will automatically take advantage of all of the features and tools inherent in the simulation provided by us. However, from an economic standpoint it doesn’t always make sense to go with a full simulation of DCS, which is labour-intensive, and there may be IP [intellectual property] issues where we can’t even get information on the DCS to do a high-fidelity simulation of it.”

A new DCS

Advanced simulators are of course supplied to new-builds as well. For example, Westinghouse took a key role in developing the full-scope simulator for its AP1000 reactor design. It was the integrator, and did design and processing of all of the component parts. “We use the GSE [simulator] platform, another vendor for the panels, and we integrate it ourselves. For an operating plant, the utility tends to do integration during the [simulator] upgrade, and we just provide our portion,” says Meena Mutyala, Westinghouse’s vice president, engineering and products, nuclear automation.

Gain at corys argues that one of the biggest issues with building a simulator of a new digital control system (whether new-build or upgrade) is essentially a scheduling issue. Most customers want simulations ready years before an actual plant upgrade is executed, so that their staff will have sufficient time to train and prepare. Moreover, since according to Gain a simulator upgrade takes a minimum of six months and usually a year for verification and validation, given their complexity (a simulator DCS algorithm is thousands of pages), there is a risk that the data received will be too preliminary to use. A related contractual issue is how to define the amount of modification required of the simulator vendor over the course of the upgrade project, he says.

Simulator vendors argue that given the drawn-out nature of the process, and the high fidelity of the simulation models, it would be efficient to use the simulator as an advanced verification and validation test of the basic reactor design. The simulator engine could run through scenarios based on early data and flag up bugs or other problems with the design, as seen through its data. Changes to the real I&C system would then be able to be changed in the planning stages, when doing so would be cheaper and easier.

Another recent trend extends simulators into the classroom. Modern digital simulators are possible to duplicate at a relatively low cost and can run on commercial-grade computers, so many vendors implement some of the functions of a full-scope simulator in the classroom to train students, engineering students for example, who would never normally rate high enough on the priority list to win a training slot on the FSS.

These so-called ‘classroom simulators’ may include virtual recreations of hard-panels and light-box displays on a computer screen-type interface (‘soft panels’). They may also reduce the number of monitors required by implementing some kind of navigation software on one or perhaps two monitors that switches between the displays normally shown simultaneously on multiple monitors. These classroom simulators do run at a lower fidelity than the full-scope simulator, perhaps by an order of magnitude, according to Pascal Gain of corys. They may also depict flows through translucent or cut-away components to help aid understanding. For example, L-3 MAPPS recently launched a product called Learning Simulators which offers 3D visualization of components and systems specifically designed for young learners.

The fringe benefit of a classroom simulator extension may be a reason for some utilities to consider upgrading their simulator, Gain says, since it is difficult to produce classroom versions of older simulators without a spare piece of the particular hardware that they run on, which may be hard to find.

The same desire, to maximize productivity, drives utilities in developing countries to move away from centralised national multipurpose simulators, according to Chatlani at L-3 MAPPS. Of two countries who never historically had simulators—Switzerland and Argentina—L-3 MAPPS has supplied (Beznau), and will supply (Embalse) a simulator. He says that in his view a 1:2 ratio, providing one full-scope training simulator for two reactor units, is industry best practice.

Utility Axpo used to send Beznau operators to train at Westinghouse’s Waltz Mill, Pennsylvania PWR simulator. Westinghouse also has a BWR simulator at Charlotte, North Carolina, and two AP1000 simulators at its headquarters in Cranberry Township, Pennsylvania. Mutyala at Westinghouse says she has noticed a trend for new-nuclear countries in the Middle East to send nuclear engineering students for training on its classroom simulators, because of a lack of availability of simulators there.

Severe accidents

Since the Fukushima accident last year, much attention has turned to severe accidents. Full-scope training simulators generally do not model severe accidents; a fundamental assumption of simulator models is that the fuel is not deformed. There are of course core neutronics and thermohydraulics codes that model reactor behaviour in severe accidents, such as the MAAP code commissioned by the US Electric Power Research Institute, but they are not designed to be used in real-time, as simulators are. Moreover, the scope of these models do not include core cooling, balance of plant and control systems, so they cannot be used on their own for full-scope operational modelling, according to L-3 MAPPS.

There is a lot of interest in the possibility of incorporating these analyses into a training simulator. Chatlani at L-3 MAPPS said that a severe accident retrofit is ‘absolutely do-able’ on any of its simulators. For example, L-3 MAPPS has integrated MAAP4 for severe accident simulation at the Krsko plant in Slovenia, in cooperation with MAAP developer Fauske & Associates.

Whether or not it is necessary to include severe accident simulations in full-scope simulations is a debate that is playing out at the moment, says Chatlani. GE’s Lagasse says that it has been starting to work with customers directly on severe accident management guidelines (SAMGs). But he says he is skeptical that SAMGs will be required in the control room (and control room simulator). Severe accident training is procedural and decision-based, and so is less likely to require full-scope simulators than classroom or partial-scope simulation. Westinghouse is also not working on codes to expand the scope of training simulators, according to Mutyala.

Notwithstanding reactor vendor reluctance, most of the simulator vendors are developing a severe accident module for their full-scale simulators, according to Gain at corys, although he adds that it would be wise to wait and see how exactly the US NRC phrases its post-Fukushima recommendations on severe accident training before launching a product. With 72 full-scope simulators in the USA alone, a large regulatory change could be tricky to implement for a short deadline.

This article was first published in the June 2012 issue of Nuclear Engineering International