EDF’s digital simulators18 May 2015
EDF has installed digital full-scope training simulators for operator training and licensing in five plants. These high-fidelity replicas of the control room are easily customised and updated. What is more, they will also significantly reduce the cost of simulators.
EDF's digital control room simulator, or SDCN (Salle De Commande Numérique) meets the requirements of the ANSI ANS 3.5 standard. It could become the standard for future nuclear training simulators if it meets the objectives of EDF: high-fidelity replica of the control room panels; easy customisation and flexibility; low upfront investment and maintenance costs. The SDCN has been in use for training for several months and it is being assessed for operator licensing.
The new simulator was developed following an EDF working group discussion in 2010, which discussed how to adapt simulators in the light of plant overhauls and changes in the man-machine interfaces. The project had to use standard, easily obtainable materials to keep costs below those of current simulators, shortening production times and allowing easy replication. This would also minimise obsolescence issues as well as maintenance and adaptation time. These requirements, and the need to provide early training for operators for the 1300MW series third ten-yearly overhaul, created the concept of a soft digital control room (SDCR), faithful to operator movement, customisable and reconfigurable and based on touch screens.
EDF is facing a large number of personnel retirements and development projects. It needs to renew the highly qualified staff in several of its core businesses -- particularly plant operators -- at a time when teams face a significant number of retirements. This shift is a potential safety issue for EDF and underscores the need for strengthened skills management to ensure the integration of newcomers as well as maintaining normal services. There will be more operators enrolled in initial training courses and there will probably be a training simulator deficit.
Meanwhile, EDF's nuclear fleet is regularly subject to change due to continuous improvement in safety, evolving regulatory requirements, and new plant performance improvement objectives. The policy of design standardisation on the same site allows several operator teams (six at some sites) to be trained on a single full-scope simulator. However, plant upgrades can last several years. At Tricastin the full scope simulator was updated when half of the units on the site had been upgraded, so training was only semi-representative. Under these conditions, it becomes increasingly difficult to have a true to life simulator that is timeline compliant to EDF requirements.
A new solution was needed before the large-scale third and fourth decennial upgrades on the 1300MW and 900MW reactor fleets. The changes will include improved instrumentation & control systems, recorders, man-operated control relay, and adaptations to new plant regulations, so operators will be faced with significant changes in the control room human-machine interface. Training drills must precede the control room upgrades.
EDF had two options: modify existing simulators, knowing that the final design of future control rooms will evolve over the project execution, or launch the concept of a digital control room, interfaced with the simulator and its process models.
In the latter case, the control room simulator becomes a series of screens. The change is to modify the images that appear on screens, rather than to replace hardware. The result is faster, less expensive modifications.
“Reviewing our simulator strategy has enable us to establish that we need customisable and reconfigurable resources," explains Philippe Clous, Head of the Skills programme at the EDF Nuclear Production division "The SDCN meets these goals far better than a traditional simulator does, even if there's a risk involved for us in adopting a new technology." So far, this has been a winning strategy: the lead times for integrating changes in the control room will be halved.
EDF established a working group in 2010 led by plant operations training experts, to rewrite its simulator guidelines. This led to four main principles that would shape the project's direction:
- Easily customisable and reconfigurable simulators, to represent different reference unit designs: 900MW first and second generation, 1300MW first and second generation as modified
- Full scale SDCN, faithful to the operator movements as a shift member and within the control room, with four or five operators in the control room SDCN must match the view of component views when close up and the overall control room view. Noises should faithfully recreate those heard in the control room
- Fully tactile and faithful handling of real equipment controls: there are no more buttons, recorders, or alarm tiles, only a screen representation of push buttons, recorders or alarm tiles which are used via a touch screen.
EDF has added to these four principles: exclusive use of off-the-shelf hardware, a significant time saving for the simulator control room manufacturing, duplication or upgrade, and lower cost than current full-scope simulators.
The high-fidelity challenge
The main challenge is to find ways to reproduce on-screen components, actuators, colours, layout space, and the different sounds of a real control room, and features that allow operators to use the same gestures they would perform while operating the plant.
EDF had the experience of "simulated control rooms" in mind. These were installed on some sites but they dictated the use of a mouse for all actions, so the instructors only used them as a supplementary tool -- not for operator training.
So in the case of SDCN, a partial or complementary solution of the conventional control room was then totally excluded. EDF wanted to train operators at the same level of competence as a through the use of full-scope simulator interfaced with a hardware replica control room.
High-definition 27in touch screens were chosen. Though the market offers larger monitors (up to 55in) these proved inadequate as the pixel size could lead to errors in reading values, particularly on some paper recorders. The 27-inch provides a 0.32mm pixel, allowing more precise displayed information.
Using 27in screens instead of 55in meant installing four times more screens, requiring extensive studies to optimise the assembly and settings and allow for maintenance.
Regarding the tactile slabs that are placed on these screens, extensive market research identified five different technologies. A technical and financial analysis concluded that optical infrared was better than technology-based resistive or capacitive sensors (which account for 95% of the market). It is a "multitouch" reliable technology (able to detect up to twelve simultaneous contact points on the monitor), the calibration is stable for the lifetime of the monitor, and the monitor can be acquired separately from the computer display.
Finally, haptic actuators were needed to vibrate the touch screen so the operator feels the button has its usual resistance. If the button is pressed while watching another part of the panel, the force feedback confirms that the operation has been performed.
These haptic actuators are glued and vibrate the tile monitor perpendicularly to its plane. Four sensors are placed on each tile of four screens. The technology chosen would have been able to generate a number of different levels of strength. In practice and after validation with users, three varying intensities were chosen to represent force feedback for all components.
Finally, the graphical editor toolset achieved a close replica of the different control panel components. A library of 60 objects was designed, each defined by its visual appearance, the sounds it produces, its operation and the touch or haptic feedback it generates, allowing the faithful reproduction of the 2300 components installed in the plant control room.
A user-centred design
One of the risks of this kind of project, with its new concept, is to leave the entire design responsibility to engineers without having them endorsed by experienced users. So a user working group with 20 members was established. It included 900MW and 1300MW plant operators and instructors, maintenance staff, human factor specialists, and Corys/Euriware experts and team leaders.
During the SDCN specification phase, the group met five times for three consecutive days to validate the mockups, solve critical issues, and suggestimprovements. The group had a decisive role in design validation. It led to a detailed understanding of each control room action -- is a button being operated with one or two fingers; should it be selected before activation, for example. This identified the most accurate way to reproduce different gestures. Whenever a new solution had passed the first development stage, the group was also involved in its validation.
The hardware components, for example the right timing of haptic feedback when a control is activated, must be represented accurately -- but that is also true of dynamic processes. For example, when operating a 900MW reactor, 40 arm-and-depress luminous buttons (ADLB) may have to be successively activated within a maximum defined allowed operation time to achieve pressure balancing inside the primary circuit. An experienced operator checked that, with the digital ADLB, the operation was achievable within the time required by the procedure.
This work on the representativeness has mobilised a lot of effort, sometimes with major difficulties because it is not always possible to digitally reproduce every component. The users in the working group themselves identified and proposed solutions in these complex cases.
One example applies to the reproduction of a push-button that is technically impossible. According to the operation rules on a French nuclear reactor, any push-button to be activated must first be designated by the operator finger, and its tag name loudly pronounced to the team shift supervisor. The challenge is to allow the finger designation without causing the manoeuvre. Group members spontaneously proposed operations very close to reality using a finger or two fingers depending whether it was for tag name designation or manoeuvring the actuator. Of course, each of these discrepancies with the reality has been validated. For each lack of fidelity, design work then targeted the 100% effective, then the best representativeness.
“The [user working group] played a decisive part in overcoming certain difficulties," agrees Philippe Bodin, SDCN project coordinator at the EDF Nuclear Production division, "in addition, its members quickly became ambassadors for the technology -- including those who had originally been opposed to it. They are publicising the SDCN on their EDF sites."
Given the challenges around training availability, EDF set a requirement for a maximum troubleshooting time for the SDCN: maintenance technicians must be able to remedy any failure within two hours.
This constraint, which does not allow repair of screens or tactile slabs onsite, is why we selected off-the-shelf products manufactured in large volumes and readily available. There is a replacement stock that can be easily mobilised. Two ergonomists reporting to the project team helped define, with the user working group, easy to use technical solutions.
This is not computing nor software programming, but mechanical actions. For screens, a system of hoists, runner slides, check prop, and hatches was developed to allow the screens to be removed, rotated and lowered into a storage box on wheels. The replacement screen is installed with the same equipment, following the reverse path.
For the tactile slab we use suction cups, identical to those used by glaziers. Two maintenance operators can rotate it 180°, lift and move, this move being guided by a ruler built into the panel structure.
In both cases, screen or slab, developers must be mindful of French labour laws, which prohibit the handling of objects over 15kg, as the complete set weight "screen + slab + haptic actuator" can vary from 12kg to 21kg.
Many other maintenance issues have been reviewed and improved from the first design: the type of castors for the screen storage box, the most suitable steel for the tactile slab runner slides, the way the wiring cables are embedded in the panel sides to facilitate screen replacement. Screens and tactile slabs have been selected, among other criteria, for their very low failure rate.
Assessment of operators for licensing
The ultimate goal of the project, alongside training, is to ensure this new tool can assess operators on whether they pass their nuclear operating licence.
The equipment meets necessary standards but is it accepted by the operators or by instructors? Is a software replica control room able to provide the fidelity needed in a skills assessment tool? What risk does the introduction of a new tool pose to its the operators? If their assessment results on the new tool are not as expected, who shall be blamed -- the assessment or the trainee?
These issues are particularly important in France, where the operator licensing only emerged about 10 years ago, under pressure from international peer reviews and the world nuclear industry. For teams, this is a sensitive subject which still raises some concerns.
EDF has identified this risk from the beginning and now mainly relies on the assessment organisation to remove any obstacle. This requires preparation time, training sequences, and additional tutorials to make sure that operators have succeeded in using this new tool.
Philippe Clous is well aware of the difficulties. "Right from the design phase, we've integrated everything that could make the qualification procedure easier. Representativeness, reliability, ease of observation, and so on. Now, we are working on the organisation: for example, by putting together HMI familiarisation sequences to give the operators time to get used to the new technology, using additional tutorials geared to the SDCN design to help them become familiar with certain actuators, and things of that nature.
“We also point out that the training and qualification drills are designed above all to ensure that transients and procedures are understood -- rather than to reproduce details such as how the touch of such control room instrument is perceived."
It is also worth highlighting the amount of thinking and development that has gone into the Instructor Station and its arrangement in the booth. It is higher than the simulator room, providing a better general view of the simulator and trainee behaviour. The new interactive monitoring station provides a precise image of the control room, and more efficient means for the instructor to know which instrument is being actuated. The simulator is equipped with a number of video cameras to see exactly what each operator can see, at the same distance and angle from the control room panel. All this equipment for monitoring trainee movements and behaviour allows the instructor to observe with accuracy.
A cost-effective solution
At the end of the current phase of design and manufacturing, we can predict that SDCN will be cheaper than the interface with a replica control room, and there are more gains when maintenance costs are taken into account. This is because the costs of screens and tactile slabs, like all electronic equipment, are falling while the hardware component costs rise as they become rarer. Corys, which manufactures simulators for the power and transport industries, is regularly confronted with this problem. Each component to be replaced in a full-scale replica of a nuclear control room or locomotive cabin has to be ordered separately, and some have been discontinued for years.
“The initial financial investment is much lower; the lower cost of maintenance and obsolescence management needs to be taken into account too, says Philippe Bodin "furthermore, the procurement and manufacturing time is much shorter."
The Bugey, Dampierre and Gravelines simulators, illustrate the problem. They were originally built in the 1980s and have been upgraded. But replacing obsolete components means recovering design and manufacture from scratch. Even when the components are still available, they can be very expensive -- a single paper recorder can cost up to 40,000 euros ($46,000).
The use of screens and tactile slabs will completely change the perspective: fewer parts to supply, spares easily and quickly available at controlled costs and predictable returns.
As with any project where innovation plays a major role and makes a significant break from the state of the art, the challenges are many and their treatment is further complicated by a very tight schedule. However, the project progress allows EDF and Corys to be confident about the ability to successfully achieve most goals.
But what is particularly noteworthy in the project has been the involvement and cooperation between many project team members belonging to various entities very different in size, culture, and organisation.
As with any emerging technology, difficulties with implementation and migration remain. The major challenge for EDF is still replacing hardware control room simulators with software based SDCN without compromising the quality of operator licensing assessments or their training.
But equally, the use of digital technology opens up new possibilities. A touchscreen image can be simply replaced with another one to alter the appearance of a panel, or even the entire panel. Faster upgrades are possible and there are smaller differences between the simulator and the control room.
Teaching can also be enhanced: we can include images or videos alongside actuators, or use multiple screens as "image walls", to present operational diagrams and teaching aids directly as needed during transients in the simulator control room.