Industrial Control Challenge 3: Modbus Communication Delays

Posted by on

1. Introduction

During recent on-site commissioning, a Modbus communication delay issue emerged. Symptoms included a watchdog signal flashing with noticeable receive delays and prolonged periods without toggling. Alarm logs consistently reported VSD communication interruption alarms. This article details a thorough troubleshooting process, emphasizing the diagnostic approach rather than specific solutions. Modbus is widely used in IC for master-slave communications, where engineers often prioritize functionality over deep understanding.

2. Project Overview
This project involves a standard unit operation, divided into Unit Control System (UCS) and Emergency Shutdown System (ESD), with multiple subsystems acting as slaves: CCC, VSD, and Bently. These subsystems also serve as slaves to the Process Control System (PCS). Data exchange between systems relies entirely on Modbus.
  • Configuration:
    • PCS acts as the Modbus TCP/IP master.
    • CCC, VSD, and Bently function as Modbus TCP/IP slaves.
    • MCC operates as a Modbus 485 serial slave.
  • Network Topology: A star configuration

3. Communication Fault Symptoms
Typically, a 0-1 flashing heartbeat signal, toggling every second, is implemented to monitor inter-system communication status. These signals, detectable by the UCS, indicate normal subsystem communication when flashing consistently.
  • Observed Points:
    • bCCC214watchdog_3A1
    • bVSD214watchdog_3A1

  • Issue: During monitoring, these heartbeat signals frequently interrupted, exceeding 10 seconds and sometimes 30 seconds, indicating unstable Modbus communication.
4. Test Mechanism Setup
4.1 Test Program Development
To monitor and log watchdog flashing status, a dedicated program was developed to track the watchdog signal. The program measures:
  • TIME1: Duration when watchdog = 1.
  • TIME2: Duration when watchdog = 0.
  • Calculates the average time over five toggles.
4.2 Visual Recording
Monitoring via the HMI revealed that Modbus delays (TIME1, TIME2) repeatedly exceeded 30 seconds, failing to meet communication rate requirements.


5. Troubleshooting Process
5.1 PLC Load Capacity




The significant communication delay initially suggested that the subsystem’s communication blocks and data volume might exceed the PLC’s load capacity, given the unusually high number of communication nodes (49 master and slave stations).

A statistical breakdown is provided below:



  • Calculation:
    • 3 LCM cards: Time = 2 * 27 * 100 ms = 5.4 seconds.
    • 3 RCM cards: Time = 2 * 22 * 100 ms = 4.4 seconds.
    • Thus, T1 = T2 = 5.4 seconds.
  • Conclusion: Modbus delays should not exceed 5.4 seconds, confirming the PLC’s load capacity is sufficient for heartbeat polling within 10 seconds. With load capacity ruled out, troubleshooting proceeded using the checklist from the referenced foundational article, excluding issues like loose interfaces, unshielded twisted pair cables, length mismatches, mixed cable pulls, interface incompatibility, and equipment aging.
5.2 Ethernet Packet Loss
5.2.1 Network Ping Test
A network-wide ping test was conducted, exporting logs using ping 192.168.*.* -t >> D:\log.txt (as detailed in “Industrial Control Challenge 1: Network Conflicts”). Results showed acceptable ping delays without packet loss, confirming network connectivity. However, ping delays (based on ICMP) and Modbus delays operate on different levels. Unstable ping delays may indicate conflicts or anomalies, potentially causing packet loss and preventing PLCs from receiving slave responses. For a 5.4-second polling cycle, a missed response delays the next acknowledgment to 10.8 seconds, which aligns with fault detection timing.

5.2.2 Modscan Tool Scan
Modscan was used to scan and read packets, checking for packet loss. Both ping and Modscan tests were normal, yet no clear insights emerged.


5.3 Multiple Devices on a Single Bus
Given the high node count on the NET2 port, the following tests were performed:
  1. Disconnect CCC network cable and observe TIME1, TIME2.
  2. Disconnect Bently cable 1 and observe TIME1, TIME2.
  3. Disconnect Bently cable 2 and observe TIME1, TIME2.
  4. Disconnect VSD cable and observe TIME1, TIME2.
  • Finding: Removing the VSD cable improved all communications significantly.
6. Further VSD Troubleshooting
VSD-specific investigation remains incomplete and will be continued in the next article.
We welcome suggestions and alternative test proposals—please leave your comments!