Preventing AS Controller Shutdowns During Redundant PCS 7 OS Server Switchovers

An Operator Station (OS) Server redundancy switchover should never cause an AS controller (PLC) or plant shutdown because PCS 7 architectures strictly separate supervisory HMI tasks from deterministic controller logic. When an OS switchover causes a plant shutdown, the failure is caused by improper application design: specifically, AS controller logic implementing a shutdown interlock tied to an OS heartbeat or lifebit, or automation logic relying on WinCC Global Scripts on the server to write critical permissive tags. To eliminate this vulnerability, remove supervisory watchdog trips from safety-critical AS logic, ensure all process interlocks reside purely inside CFC in the AS, and isolate the Plant Bus from Terminal Bus communication storms.

Executive Summary

A cornerstone principle of distributed control system (DCS) engineering is the autonomous execution of the automation layer. Controllers (such as S7-400 and CPU 410-5H) execute safety, interlocking, and regulatory loops deterministically regardless of whether human-machine interface stations are online, offline, or transitioning. Nevertheless, commissioning and operations teams occasionally report catastrophic events where initiating an intentional OS Server redundancy failover (or experiencing an unexpected server crash) results in an immediate controller stop or process trip. This article analyzes the failure mechanisms linking supervisory server transitions to controller shutdowns, highlights common architectural design flaws, and provides engineering remedies to ensure complete AS operational autonomy.

Architecture and Fundamental Mechanics

Separation of Supervisory vs Automation Layers, Watchdog Interlocks, and Network Fault Vectors:

Autonomous Controller Operation in PCS 7:
S7-400 / CPU 410-5H automation stations execute cyclically in OB1, OB3x, and OB8x without requiring continuous acknowledgment from WinCC OS Servers.
If an OS Server disconnects, fails over, or reboots, the AS continues to execute PID algorithms, motor/valve interlocking, and safety sequences uninterrupted.
Root Cause 1: AS-to-OS Watchdog and Lifebit Trip Logic:
In poorly engineered systems, software developers sometimes program a 'Heartbeat' or 'Lifebit' check between the AS and OS (e.g. WinCC toggles a bit every 1 second, and the AS decrements a watchdog timer).
If the logic author configures this watchdog to trigger a plant trip or emergency stop whenever the OS communication stops for more than a few seconds, an OS failover will trigger a shutdown! During server failover, Server 1 halts its cyclic scripts, and Server 2 takes 10 to 30 seconds to assume active master role and begin executing Global Scripts. This momentary pause trips the fragile AS watchdog.
Root Cause 2: Control Logic Relied on WinCC Scripts Instead of CFC Logic:
Automation interlocks, recipe parameters, or sequence advancement triggers must NEVER reside inside WinCC Global Actions (C or VBS scripts).
If a script calculates a permissive condition and writes it to the PLC, taking down Server 1 halts the script, resetting the permissive to zero or failing to handle sequence handoffs.
Root Cause 3: Plant Bus Flooding and CP Communication Failure:
If redundant servers are incorrectly cabled such that the Plant Bus (CP 1623) and Terminal Bus share common unmanaged switches without VLAN separation, an OS failover generating broadcast synchronization storms can saturate CP communications.
Severe communication overloads can cause S7-400 cycle time overruns (exceeding configured watchdog time in OB80) or trip redundant S7-400H synchronization links if network cards fail to handle buffer exhaustion.

Step-by-Step Implementation Procedure

Audit AS Controller Logic for Supervisory Interlocks:
* In SIMATIC Manager, search the CFC charts for any logic monitoring @ServerState, @RedundancyState, or custom WinCC lifebits.
* Verify that NO process safety trips, motor stops, or valve interlocks depend on the state of supervisory SCADA tags.
* If a watchdog is required for remote operator station supervision, ensure it generates an operator alarm only, never an automatic plant trip.
Audit WinCC Global Scripts for Process Control Logic:
* Open WinCC Explorer on the OS Server.
* Open Global Script C and VBS Editors; review all cyclic and tag-triggered actions.
* Move all interlocking, timing, calculations, and permissive evaluations out of WinCC scripts and into CFC function blocks inside the S7-400 / CPU 410.
Verify Physical Network Separation:
* Ensure physical or VLAN-based segregation between the Plant Bus (connecting CPs to CPUs) and the Terminal Bus (connecting OS Servers to OS Clients and Engineering Stations).
* Verify that redundant server redundancy synchronization utilizes dedicated cross-connect cables or isolated network paths.
Analyze AS CPU Diagnostic Buffers:
* In SIMATIC Manager, select the PLC and click PLC > Module Information > Diagnostic Buffer.
* Check the exact timestamp of the shutdown event. Look for OB80 (Cycle time exceeded), OB86 (Rack failure / bus fault), or OB85 (Program organization unit not loaded) entries to verify whether the CPU experienced an internal hardware/software fault.
Conduct Controlled Failover Testing:
* During a scheduled maintenance window, deactivate WinCC Runtime on OS Server 1.
* Observe the AS CPU LEDs (RUN, REDF, IFM) and verify that the PLC continues cyclic execution with zero alarms or trips.

Verification

Initiate an OS Server failover by stopping WinCC Runtime on Server 1; verify that OS Server 2 transitions to Master role seamlessly.
Confirm that all running motors, control valves, and PID loops maintain their operating states without disturbance.
Verify that the AS CPU diagnostic buffer logs zero communication faults or execution errors during the switchover.

Limitations and Operational Precautions

Under international safety standards (IEC 61508 / IEC 61511), safety-instrumented functions (SIF) must execute autonomously within certified Safety Automation Systems (e.g. S7-400F/FH) and must never have trip conditions subordinated to non-safety HMI software or communication networks.

Engineer FAQ

Can a standard PCS 7 OS Server failover trip an AS controller?

No. By design, PCS 7 controllers execute autonomously. A trip occurs only if custom PLC logic was incorrectly programmed to shut down on HMI communication loss.