Diagnosing 'The connection to the master was interrupted' on Redundant WinCC OS Standby Servers

In SIMATIC PCS 7 V8.0 redundant OS server architectures, an active alarm 'The connection to the master was interrupted' persisting on the Standby server—while client redundancy status displays a green square—indicates that background archive synchronization (CCAlgRtServer or CCTlgServer) timed out during an abrupt server reboot. Although basic network ICMP pings and runtime process control remain functional, historical message and tag databases are not fully synchronized between the servers. To resolve, inspect the WinCC Redundancy diagnostics using RedundancyControlDiagnostics.exe, restart the Standby server runtime gracefully to trigger database resynchronization, verify that the internal system tag @RedundancyState equals 2, and confirm that both servers share identical terminal bus network adapter bindings in Simatic Shell.

Executive Summary

During virtualized server stress testing or unscheduled power interruptions on PCS 7 V8.0 redundant OS server pairs, operating teams often encounter an alarming paradox: the WinCC OS Client indicates healthy redundant operations (displaying a solid green square on the status bar), yet the Standby server's alarm control displays an active, un-cleared system message: 'The connection to the master was interrupted'. Furthermore, new alarms appear on the Master server but fail to populate the Standby server. This technical article analyzes how WinCC Redundancy monitors partner heartbeats, explains why abrupt shutdowns disrupt SQL Server replication queues, and provides the step-by-step procedure to restore complete database synchronization.

Architecture and Fundamental Mechanics

The fault-tolerant architecture of WinCC Redundancy divides station monitoring into two distinct mechanisms:
1. Runtime Heartbeat vs Database Synchronization:
- Runtime Heartbeat (Connection Monitoring): Servers exchange UDP/TCP heartbeat telegrams across the terminal bus and dedicated redundant network connection every 500 ms. If heartbeats succeed, client stations observe normal operations and report 'Green' redundancy status.
- Database Archive Synchronization (CCAlgRtServer & CCTlgServer): When a server restarts, the Standby server queries the Master server's Microsoft SQL Server instance to replicate missed alarms and process value logs across the downtime gap.
2. The Abrupt Shutdown Disconnection Fault:
- An abrupt virtual machine termination or power cut closes network sockets without sending formal TCP FIN packets or teardown tokens.
- When the Standby server powers back up, it attempts to reconnect to the Master's synchronization service. If the Master server still holds a stale, unreleased socket from the previous session, the connection handshake is rejected.
- The Standby server logs the system alarm: 'The connection to the master was interrupted'. In this state, archive synchronization halts, leaving the Standby server's historical tables frozen at the shutdown timestamp while active runtime tags continue updating.
3. Simatic Shell Binding and Network Time Drift:
- If the Standby server's network adapter order shifts upon reboot, Simatic Shell may bind to an alternate adapter, preventing SQL replication calls even though general LAN pings succeed.

Step-by-Step Implementation Procedure

Inspect Redundancy State Tags and Diagnostic Logs:
a. On the Standby server, open WinCC Explorer and inspect the Tag Management window.
b. Check the value of internal system tag @RedundancyState:
- Value 1: Master
- Value 2: Standby / Redundant
- Value 3 or 4: Fault / Desynchronized
  c. Open Windows File Explorer and navigate to C:\Program Files (x86)\Siemens\WinCC\Diagnostics.
  d. Open WinCC_Sys_01.log and CCRtsLoader.log: search for 'Redundancy partner connection lost' or 'SQL replication error'.
Run Redundancy Diagnostics Utility:
a. On both the Master and Standby servers, launch RedundancyControlDiagnostics.exe (located in the WinCC bin folder).
b. Check the 'Archive Synchronization' section:
- Look for 'Tag Logging Synchronization: Inactive / Failed'.
- Look for 'Alarm Logging Synchronization: Inactive / Failed'.
  c. Note the exact error code or connection endpoint reported.
Verify Simatic Shell Network Adapter Configuration:
a. On both servers, open Windows Explorer, right-click Simatic Shell, and select Settings....
b. Verify that both stations have selected the identical Terminal Bus network adapter.
c. Confirm that both stations see each other in the Simatic Shell station tree.
Execute Graceful Standby Runtime Restart:
a. On the Standby server, click the WinCC deactivation button (or close WinCC Explorer cleanly) to allow Windows services to release SQL sockets.
b. Wait for all WinCC processes (CCAlgRtServer.exe, pdlrt.exe, gscrt.exe) to terminate completely in Task Manager.
c. Restart the WinCC Runtime on the Standby server.
Monitor and Verify Archive Resynchronization:
a. Observe the Standby server system tray: confirm the icon shifts from yellow to green.
b. In RedundancyControlDiagnostics.exe, verify that 'Archive Synchronization' switches to 'Active' and progress percentages increment until reaching 100%.
c. Confirm that the alarm 'The connection to the master was interrupted' transitions from Active to Cleared/Gone.

Verification

In WinCC Alarm Control on the Standby server: verify that the alarm 'The connection to the master was interrupted' has cleared.
Trigger a test alarm in CFC: confirm the alarm appears simultaneously on both Master and Standby server alarm screens.
Open a historical process trend on an OS Client: scroll back across the reboot timeframe to verify no data gaps exist in the historical trace.

Limitations and Operational Precautions

Never reboot the Master server while the Standby server is in an un-synchronized or faulted state. Doing so will result in permanent archive data gaps for the period of the outage.

Engineer FAQ

Why did the client show a green redundancy square when an error was active?

The client indicates that at least one server is actively serving data; it does not indicate whether background SQL archive synchronization between servers is healthy.

What causes the alarm 'The connection to the master was interrupted'?

An abrupt shutdown or socket failure prevented the Standby server from establishing the archive synchronization link with the Master server's SQL database.