Troubleshooting Periodic WinCC Alarm Logging Freezes and Redundant Server Message Disconnections in SIMATIC PCS 7

In SIMATIC PCS 7 redundant OS architectures, when alarm messages periodically cease displaying on operator workstations every few weeks and recover only after an OS server switchover, the failure is caused by memory exhaustion, thread deadlocks, or archive write queue saturation in the Alarm Logging runtime processes (CCAlgRtServer.exe or CCWriteArchiveServer.exe). In redundant systems, unbacked-up SQL circular logs, transaction log (.ldf) bloat, or corrupted archive segments cause the active server's write buffer to stall, halting incoming alarm processing until an OS failover routes clients to the healthy partner.

Architecture of the WinCC Alarm Logging Runtime Engine

In SIMATIC PCS 7 (v7.1, v8.x, v9.x), alarms are managed by specialized background processes:
* The Message Server (CCAlgRtServer.exe):
Receives asynchronous Alarm_8P event frames from automation stations (S7-400 / CPU 410-5H), decodes timestamps and format parameters, and distributes active messages to connected OS clients.
* The Archive Writer (CCWriteArchiveServer.exe):
Responsible for committing arriving alarms and operator input messages into Microsoft SQL Server circular databases. Up to three distinct instances execute simultaneously (Alarm Logging, Tag Logging Fast, and Tag Logging Slow).
* Redundancy Synchronization (CCRedMon.exe):
In redundant server pairs, the active server continuously mirrors alarm state transitions and archive records across the redundant network link to the standby server.

Root Causes for Periodic Message Stalls in Redundant Systems

When alarms freeze after running for several weeks:
1. Archive Queue Saturation / Disk I/O Bottlenecks:
If high-frequency alarm chattering occurs, incoming alarms arrive faster than SQL Server can commit them to disk. The internal RAM write buffer fills up. Once exhausted, CCWriteArchiveServer.exe stops acknowledging new frames, causing CCAlgRtServer.exe to block incoming controller messages.
2. SQL Transaction Log (.ldf) Bloat:
If automated archive backup is disabled or points to an unreachable network drive, circular archive segment files cannot detach. The SQL transaction log grows until hard drive space is exhausted.
3. Internal Process Crash / Deadlock:
Corrupted database pages or memory-mapped file conflicts (e.g., caused by unexcluded antivirus scanners) can cause CCWriteArchiveServer.exe to crash silently. The server remains online in WinCC Redundancy, but its alarm archiving engine is dead.

Diagnostic Matrix: Alarm Logging Process States and Symptoms

The table below outlines process health indicators and failure signatures:

Process NameNormal Operational StateFailed / Frozen StateDiagnostic Indicator / LogCCAlgRtServer.exeMemory: 30 - 80 MB; Low CPUMemory continuously climbing or unresponsiveApplication Event Log: Event ID 1000CCWriteArchiveServer.exe3 instances active; Low CPU1 or more instances missing from Task ManagerWinCC SIMATIC_S7_PROTOCOL_SUITE.logsqlservr.exeMemory: 1 - 4 GB (Dynamic); Moderate CPU100% CPU or disk queue length > 2SQL Error Log: Autogrow of file failedCCRedMon.exeSync status: RedundantStatus: Fault or Partner DisconnectedWinCC Redundancy Alarm @RedundancyServerState

SQL Server Transaction Log Management and Circular Archiving Rules

In WinCC Explorer -> Alarm Logging -> Archive Configuration:
* Segment Sizing Rule:
Configure Maximum size of all segments to occupy no more than 50% of available hard drive capacity.
* Segment Time Limit:
Configure Time period covered by a single segment (e.g., 1 day or 1 week).
* Backup Destination:
Always configure a valid backup path (e.g., to SIMATIC Process Historian 6ES7652-8AX58-0YB0 or a dedicated NAS). When a segment reaches its time or size limit, WinCC backs it up and swaps to a new segment cleanly.

Step-by-Step Procedure to Stabilize Alarm Logging and Archive Processes

Inspect Active Windows Task Manager Processes:
When alarms freeze, open Task Manager on both OS Servers. Verify that CCAlgRtServer.exe and three instances of CCWriteArchiveServer.exe are running.
Inspect Windows Event Viewer:
Open Event Viewer -> Windows Logs -> Application. Search for error events originating from WinCC, CCAlgRtServer, or MSSQL$WINCC.
Verify SQL Database Health:
Open Windows Command Prompt as Administrator. Check disk space on all partitions. Ensure at least 20 GB of free space exists on the drive hosting WinCC projects.
Re-Initialize Alarm Logging Runtime Cache:
* If the archive is corrupted, stop WinCC Runtime on Server B during a scheduled maintenance window.
* Run Reset_WinCC.vbs to terminate hanging database handles.
* In the project directory, locate the Alarm Logging archive folder (ArchiveManager\Message). Delete or move corrupted .mdf segment files.
* Restart Server B and allow redundancy synchronization to rebuild archives from Server A.
Tune Alarm Flooding and Hysteresis:
In CFC, audit process tags generating thousands of alarms per hour. Implement alarm deadbands and delay timers to eliminate chatter.

Verification

To verify that alarm archiving is permanently stabilized:
1. Open the WinCC Alarm Logging runtime control on an operator workstation.
2. Trigger a test alarm from a controller. Confirm that the alarm appears instantly in the active alarm banner.
3. Acknowledge the alarm. Confirm that the state transitions to acknowledged and logs to the historical journal.
4. Monitor the OS Server over a 30-day period. Confirm that memory usage stabilizes and no server failovers are required to view alarms.

Limitations and Operational Precautions

Never Shrink Active SQL Databases in Runtime: Executing manual SQL shrinking commands (DBCC SHRINKDATABASE) while WinCC Runtime is active causes high disk fragmentation and can corrupt active alarm pointers.
Redundancy Synchronization Duration: When restarting an OS server whose archives were reset, redundancy synchronization across the terminal bus can take several hours depending on network bandwidth and database size.
Licensing Disclaimer: Redundant WinCC OS architectures require valid WinCC Redundancy licenses (6AV6371-1CF07-4AX0) on both server stations validated against the applicable Siemens catalog and authorized distributor quotation.