Troubleshooting 7-15 Second Faceplate Opening Delays on WinCC Redundant OS Clients

In SIMATIC PCS 7 redundant OS architectures, if faceplates open instantly (<1 second) when one OS server is stopped but suffer 7 to 15 second delays when both redundant servers are active, the root cause is network name resolution timeouts (DNS/NetBIOS) or asymmetric redundancy synchronization states. When both servers run, the OS client queries both Master and Standby partners for tag subscriptions and script validation. If reverse DNS lookups or NetBIOS broadcasts fail, RPC calls time out after 5–10 seconds. To resolve this: add static IP-to-hostname mappings in Windows 'hosts' and 'lmhosts' files on all clients and servers, prioritize the Terminal Bus network adapter in Windows binding order, verify Simatic Shell settings, and perform a full OS download with runtime deactivated to clear delta synchronization locks.

Executive Summary

In multi-client SIMATIC PCS 7 control rooms (such as systems with 2 redundant server pairs and 9 OS clients running PCS 7 V8.0 SP1 or V8.1), operators frequently report severe graphic latency. Clicking a block icon to open an APL faceplate (e.g., motor or valve standard view) freezes the operator interface for 7 to 15 seconds, often accompanied by temporary 'Error Read' gray banners. Crucially, disabling either OS Server A or OS Server B causes faceplates to open instantaneously. This paradox indicates that the issue is not AS controller overload, but rather network resolution latency or database synchronization contention between redundant partners. This guide provides the systematic troubleshooting workflow.

Architecture and Fundamental Mechanics

The WinCC Redundancy and Client-Server subsystem operates across multiple inter-process communication mechanisms:
1. Server Redundancy Engine: Master and Standby servers synchronize alarm archives, tag archives, and internal state tags cyclically over the dedicated Redundancy Partner Bus and Terminal Bus.
2. Client Tag Subscriptions: An OS Client establishes Microsoft Remote Procedure Call (RPC) and DCOM sessions with its preferred server. However, to maintain seamless bumpless failover, the client also maintains standby communication links with the redundant partner.

When an operator clicks a block icon to open a faceplate (@pg_*.pdl):
- WinCC executes faceplate initialization scripts that read multiple block status tags (Status1, Status2, Mode, Interlock).
- In PCS 7 V8.0/V8.1, faceplates invoke synchronous read functions (GetTagChar(), GetTagDWordWait()).
- If Windows cannot instantly resolve hostnames via DNS/NetBIOS, or if the client's network requests route through an unoptimized network adapter, RPC calls to the partner server stall, waiting for the default TCP/RPC timeout (typically 5 to 10 seconds).
- Once one server is turned off, the client immediately detects the offline socket and bypasses partner synchronization checks, eliminating the delay.

Secondary contributors include:
- Corrupted delta OS downloads leaving asymmetric picture trees or tag dictionaries.
- Overloaded WinCC communication queues (Channel Diagnosis request queue spikes).
- Antivirus software locking SQL Server archive files in the project's ArchiveManager directory.

Step-by-Step Implementation Procedure

Configure Static Hostname Resolution (hosts and lmhosts):
a. On all OS Clients and OS Servers, open Notepad as Administrator.
b. Open C:\Windows\System32\drivers\etc\hosts.
c. Add static IP addresses and exact computer names for all OS Servers (both Terminal Bus and Plant Bus interfaces) and all OS Clients.
d. Open C:\Windows\System32\drivers\etc\lmhosts.sam, save as lmhosts (no extension), and add identical mappings formatted with #PRE tags.
e. Flush the DNS cache in command prompt: ipconfig /flushdns.
Configure Network Adapter Binding and Simatic Shell:
a. Open Windows Network Connections -> Advanced -> Advanced Settings.
b. Ensure the network card connected to the Terminal Bus has top priority in the binding order.
c. Open Windows Explorer, right-click 'Simatic Shell', select 'Settings...', and confirm that the Terminal Bus network adapter is explicitly selected on all machines.
Inspect WinCC Channel Diagnosis and Request Queues:
a. On the active Master OS Server, open WinCC Channel Diagnosis from the Windows Start menu or WinCC Explorer.
b. Select the active channel (e.g., 'SIMATIC S7 Protocol Suite -> Named Connections').
c. Monitor the 'Request Queue' parameter while an OS Client opens a faceplate. If the queue spikes significantly, optimize AS communication cycle loads in HW Config.
Execute Full OS Compilation and Full Download:
a. Incremental delta downloads can introduce tag synchronization mismatches between redundant partners.
b. During a planned maintenance window, deactivate WinCC Runtime on OS Server B, then OS Server A.
c. In SIMATIC Manager on the ES, perform a full OS compilation ('Entire program').
d. Download the complete OS project ('Entire project') to OS Server A and OS Server B.
e. Activate Runtime on Server A, allow database synchronization to complete (@RedundantServerState = 1), then activate Server B (@RedundantServerState = 2).

Verification

On an OS Client, click several APL block icons (MotL, VlvL, MonAnL) while both redundant servers are active: verify that faceplates open in less than 1.5 seconds.
Monitor the active server banner: confirm no 'Error Read' messages appear on faceplate subwindows.
Open WinCC Channel Diagnosis: confirm that channel status remains 'OK' with minimal request queue latency.
Check the WinCC diagnostic directory (C:\Program Files (x86)\Siemens\WinCC\Diagnostics\): verify that WinCC_Sys_01.log and SIMATIC_S7_Protocol_Suite.log show zero RPC timeout warnings.

Limitations and Operational Precautions

Executing a full OS project download requires deactivating WinCC Runtime on the target server; always ensure one server remains active or schedule work during an authorized maintenance window to prevent loss of operator visualization. Verify that third-party antivirus suites exclude all WinCC project folders and SQL Server directories according to Siemens security guidelines.

Engineer FAQ

Why did disabling one server make faceplates open instantly?

When one server is offline, the client does not attempt redundant partner synchronization queries, thereby bypassing the 5–10 second network RPC lookup timeouts.

What does 'Error Read' indicate on a faceplate window?

It indicates that the WinCC graphic script attempted a synchronous tag read that timed out before receiving a response from the server tag database.

Can updating PCS 7 resolve faceplate opening delays?

Yes. Siemens released updates (such as PCS 7 V8.0 SP1 Update 5 and V8.2 Upd1) that optimize APL faceplate scripts to use asynchronous tag handling, substantially reducing opening times.

How do I check if redundant servers are properly synchronized?

Inspect internal tags @RedundantServerState (1 = Master, 2 = Standby) and @RM_SYNCHRONIZING (0 = fully synchronized).

Does the physical redundancy partner bus cable affect client performance?

Yes. If the direct cross-connect redundancy network cable experiences packet loss, servers spend excessive CPU resources in resynchronization loops, slowing client responses.