[EX/QFX] FPC vulnerability after DDoS Protection is disabled

Summary:

This article briefly describes the behavior of the device when DDoS-Protection is disabled, which may eventually result in the FPCs becoming exposed to the risk of crashing.

Symptoms:

When DDoS protection is disabled, FXPC may crash and a core may be generated in the device.

root@jtac> show system core-dumps no-forwarding
-rw-rw----  1 root  field    5185493 Sep 11 14:44 /var/tmp/fxpc.core.0.gz
-rw-rw----  1 root  field    4654257 Sep 11 14:51 /var/tmp/fxpc.core.1.gz
-rw-rw----  1 root  field    4573888 Sep 11 14:56 /var/tmp/fxpc.core.2.gz
-rw-rw----  1 root  field    4538735 Sep 11 15:01 /var/tmp/fxpc.core.3.gz
total files: 4

Error Logs

----------snip--------
Sep 11 14:44:08  jtac dc-pfe: eip: 0x0a217258 eflags: 0x00000213    trapno: 0
Sep 11 14:44:08  jtac dc-pfe: eax: 0xaf97d75e    ebx: 0x94e89c68    ecx: 0x00000000    edx: 0x00000001
Sep 11 14:44:08  jtac dc-pfe: esi: 0xaf97d762    edi: 0x94e89c6c    esp: 0xaf97d724    ebp: 0xaf97d768
Sep 11 14:44:08  jtac dc-pfe: cs: 0x0033 ds: 0xaf97003b es: 0x003b fs: 0xccb003b gs: 0x001b ss: 0x003b
Sep 11 14:44:08  jtac dc-pfe:
Sep 11 14:44:08  jtac dc-pfe: PANIC PANIC PANIC PANIC PANIC PANIC
Sep 11 14:44:08  swQFXJonesboro dc-pfe: Watchdog fired delta=3 sig_watchdog_counter=221257 watchdog_counter=221257

Cause:

When DDoS protection is disabled on a standalone device/VC member, the device/VC may become unstable due to the risk of FPC crash when there is high-control traffic hitting the device.

Solution:

DDoS protection is enabled by default on devices. Although users can modify the default configuration for the rate-limiting policers that identify excess control traffic and drop packets before the switch is adversely affected, it is not a recommended practice to disable DDoS-protection since a high amount of control traffic can overwhelm the system, causing system instability.

Disabling DDoS protection will disable rate limiting for all host-bound traffic. By doing so, we are allowing all control traffic to be punted to the host path at a high rate, which eventually hogs the CPU. This causes vulnerability on FPCs and they may crash, generating FXPC core dumps or a watchdog scenario. This is expected behavior when one manually disables the default setting.

To prevent this, the below configuration should be removed from the device:

show system ddos-protection
global {
    disable-fpc;
}

Knowledge Base