Troubleshooting WinCC Emergency Restart Errors on Redundant OS Servers in PCS 7

Troubleshooting WinCC Emergency Restart Errors on Redundant OS Servers in PCS 7

The critical modal error 'An error occurred that requires a restart of computer. Close all applications, if possible, and trigger a restart with OK button' in SIMATIC PCS 7 V9.0 SP3 is triggered by the WinCC core watchdog (CCProjectMgr.exe or CCEmergency.exe) when an unrecoverable system exception occurs. On redundant OS servers, the primary cause is a severe time synchronization failure (a time step discrepancy exceeding 1000 ms between redundant partners) or Microsoft SQL Server resource exhaustion in virtualized environments. To resolve the error, configure deterministic NTP time synchronization using an external master clock, allocate dedicated non-dynamic RAM and CPU cores to the virtual machines, and inspect WinCC_sys.log to identify the crashing subsystem.

Table of Contents

Executive Summary

Continuous availability is the primary design requirement for redundant Operator Station servers in SIMATIC PCS 7. During runtime operations—especially within virtualized hypervisor environments (such as VMware vSphere or Microsoft Hyper-V)—engineers may encounter an unexpected, fatal WinCC system dialogue stating that a computer restart is required immediately. When this occurs, WinCC Runtime halts its background communication engines, severing client connections and triggering an uncoordinated server failover. Because this error represents a protective failsafe shutdown triggered by the WinCC supervisory watchdog, resolving it requires identifying the underlying infrastructure violation. This article details the watchdog mechanics, analyzes time synchronization dependencies, and provides verified stabilization procedures.

Architecture and Fundamental Mechanics

WinCC Core Watchdog Mechanics, CCEmergency Execution, and Redundancy Time Synchronization:

  • The WinCC Emergency Watchdog Architecture:
  • WinCC incorporates background watchdog supervisors (CCProjectMgr.exe, CCEmergency.exe, and CCRedCom.exe) that continuously monitor the health of core subsystems: Microsoft SQL Server, Tag Logging, Alarm Logging, and S7 channel drivers.
  • If a critical subsystem encounters an unhandled memory corruption, deadlock, or communication failure, CCEmergency.exe is spawned. To prevent corrupted data from being written to plant archives or replicated to the redundant partner, the watchdog locks the interface and mandates an immediate computer reboot.

  • Root Cause 1: Time Synchronization Jumps (The NTP Desynchronization Hazard):

  • In redundant OS server configurations, archive synchronization relies strictly on chronological monotonicity.
  • If an OS Server experiences a time jump (e.g. jumping backward by several seconds or forward by more than 1000 ms due to conflicting Windows Time service and Siemens SIMATIC Time Sync configurations), the SQL database engine detects temporal contradictions.
  • To prevent split-brain archive corruption, CCRedCom.exe aborts and invokes the emergency restart sequence.

  • Root Cause 2: Virtualization Resource Starvation in Virtual Machines:

  • When PCS 7 runs inside virtual machines, configuring dynamic memory allocation (ballooning) or shared vCPUs causes CPU latency spikes.
  • If the hypervisor throttles CPU cycles or swaps RAM to disk, Microsoft SQL Server drops internal heartbeat timers, causing WinCC service timeouts.

  • Diagnostic Tracing via WinCC_sys.log:

  • The precise trigger is recorded chronologically in C:\Program Files (x86)\Siemens\WinCC\Diagnostics\WinCC_sys.log and CCRedundancy.log immediately preceding the shutdown timestamp.

Step-by-Step Implementation Procedure

  1. Inspect WinCC Diagnostic Logs for Exact Failure Cause:
    * Navigate to the WinCC diagnostics folder on the failing server:
    C:\Program Files (x86)\Siemens\WinCC\Diagnostics\.
    * Open WinCC_sys.log with a text editor.
    * Scroll to the exact timestamp of the restart error.
    * Look for fatal entries: CCProjectMgr: Application <App> crashed, Time difference between servers too high, or Database connection dropped.
    * Also examine CCRedundancy.log for redundancy state machine transitions.

  2. Configure Deterministic Time Synchronization:
    * Disable the standard Windows Time service (W32Time) if using Siemens Time Synchronization, or configure an external central NTP time master (such as a SICLOCK GPS transmitter or domain controller).
    * In WinCC Explorer, open Time Synchronization.
    * Ensure Synchronization via Terminal Bus is configured properly.
    * Verify that both redundant servers query the same time source and that time drift remains strictly under 100 milliseconds.

  3. Enforce Strict Virtual Machine Resource Reservations:
    * If running on VMware ESXi or Hyper-V, open the hypervisor management console.
    * Configure 100% CPU Reservation (zero CPU overcommit).
    * Configure 100% RAM Reservation (disable dynamic memory / memory ballooning).
    * Ensure VM virtual disks are provisioned as Thick Provision Eager Zeroed on high-performance storage.

  4. Verify Redundancy Network Separation:
    * Ensure redundant servers utilize a dedicated network adapter (or direct cross-connect cable) for server-to-server redundancy synchronization, separate from the Terminal Bus.

  5. Restart and Monitor Runtime Stability:
    * Perform a clean computer restart.
    * Launch WinCC Runtime; monitor WinCC_sys.log for 24 hours to confirm zero critical exception warnings.

Verification

  1. Open Windows Event Viewer (eventvwr.msc); confirm that Application and System logs record zero SQL Server or WinCC service crashes.
  2. In WinCC Explorer, open Channel Diagnosis; verify that all connections to controllers and the redundant partner report normal status.
  3. Monitor @RedundancyState in WinCC; confirm value 1 (Master) or 2 (Standby) with zero unplanned failovers.

Limitations and Operational Precautions

Never ignore repeated 'restart of computer' dialogues by simply clicking OK and restarting without log investigation; recurring watchdog crashes indicate underlying database corruption or hypervisor resource throttling that will eventually cause archive data loss.

Engineer FAQ

What causes 'An error occurred that requires a restart of computer' in WinCC?

The WinCC core watchdog (CCEmergency.exe) triggered a failsafe shutdown due to an unhandled system crash, severe time desynchronization, or SQL database failure.

Where is the root cause of the restart error logged?

In C:\Program Files (x86)\Siemens\WinCC\Diagnostics\WinCC_sys.log.

Why does time synchronization cause WinCC to crash?

Large time steps (over 1000 ms) corrupt chronological database sorting in redundant servers, forcing an emergency restart to protect archive integrity.

Can PCS 7 redundant OS servers use dynamic memory in virtual machines?

No! Virtual machines running PCS 7 OS Servers must have 100% dedicated RAM and CPU reservations without dynamic ballooning.

How do I check redundancy status in WinCC?

Inspect the internal system tags @RedundancyState (1 = Master, 2 = Standby) and @RedundancyServerState.

Technical References

Related Articles

Related Tags

  • WinCC Emergency Restart
  • CCEmergency.exe
  • WinCC_sys.log
  • Time Synchronization
  • Redundant OS Server
  • VMware ESXi
  • Virtualization Sizing
  • PCS 7 V9.0
20 respuestas