Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[EX/QFX] How to calculate and to optimize TCAM usage in firewall filters

0

0

Article ID: KB30953 KB Last Updated: 07 May 2019Version: 2.0
Summary:

This article explains how to calculate the number of TCAM (Ternary Content Addressable Memory) entries a particular firewall filter term will take up. Based on this, one can optimize the matching conditions to use less TCAM space.

Symptoms:

To understand how firewall filters are programmed into the TCAM and therefore be better able to optimize them to reduce overall TCAM usage.

Solution:

The TCAM needs to create a separate entry for every possible combination of matching items within a firewall filter term. Below are examples of how the TCAM entries are calculated, how to check your TCAM usage, and related TCAM KB articles.

Examples showing how TCAM entries are calculated

Example 1

In the following configuration, this term will only take one rule in the TCAM because there is only one of each type of matching condition. When they are multiplied together, this makes for on TCAM entry: 1 x 1 x 1 x 1 = 1 TCAM entry.

{master:0}[edit firewall family inet filter FOO]
user@switch# show
term one {
    from {
        source-address {
            10.10.10.0/24;
        }
        destination-address {
            20.20.20.0/24;
        }
        source-port 65000;
        destination-port ssh;
    }
    then accept;
}

 

Example 2

In the following configuration, multiple types of terms yields a different result. Here, there are two options for source-address and two options for destination-address. This creates 4 possible ways to match on this term: 2 x 2 x 1 x 1 = 4 TCAM entries.

{master:0}[edit firewall family inet filter FOO]  
user@switch# show 
term one {
      from {
            source-address {
                  10.10.10.0/24;
                  30.30.30.0/24;
            }
            destination-address {
                  20.20.20.0/24;
                  40.40.40.0/24;
            }
           source-port 65000;
           destination-port ssh;
      }
      then accept;
}

The 4 TCAM entries are:

term1.1: match [Source IP A, Destination IP C, Source Port E, Destination Port F]
term1.2: match [Source IP A, Destination IP D, Source Port E, Destination Port F]
term1.3: match [Source IP B, Destination IP C, Source Port E, Destination Port F]
term1.4: match [Source IP B, Destination IP D, Source Port E, Destination Port F]

Notice that since there is only one source-port and one destination-port, these two matching conditions do not increase the total number of TCAM entries needed.

Example 3

There are some things to keep in mind. Most switches will not automatically summarize continuous network address prefixes, so the following configuration will result in 4 TCAM entries.

{master:0}[edit firewall family inet filter FOO]
user@switch# show
term one {
    from {
        source-address {
            10.0.0.0/24;
            10.0.1.0/24;
            10.0.2.0/24;
            10.0.3.0/24;
        }
    }
    then accept;
} 

However, these 4 subnets can be summarized, so if you configure them the following way, then you will have just 1 TCAM entry used for this term.

{master:0}[edit firewall family inet filter FOO]
user@switch# show
term one {
    from {
        source-address {
            10.0.0.0/22;
        }
    }
    then accept;
}

This can make a huge difference when you have many entries for both source and destination prefixes.

Caution: Care should be taken when summarizing network prefixes, as depending on the summarization done, you may be matching against a larger set of IP addresses than you were when using a series of smaller, more specific subnets. In that case, you will need to weigh what is more important: security (with permitting more addresses) or TCAM space reduction.

Example 4

The following example has 8 source-address subnets, 8 destination-address subnets, 2 source-ports and 2 destination-ports. This makes for a total of 8 x 8 x 2 x 2 = 256 TCAM entries.

{master:0}[edit firewall family inet filter FOO]
user@switch# show
term one {
    from {
        source-address {
            10.0.0.0/24;
            10.0.1.0/24;
            10.0.2.0/24;
            10.0.3.0/24;
            10.0.4.0/24;
            10.0.5.0/24;
            10.0.6.0/24;
            10.0.7.0/24;
        }
        destination-address {
            20.0.0.0/24;
            20.0.1.0/24;
            20.0.2.0/24;
            20.0.3.0/24;
            20.0.4.0/24;
            20.0.5.0/24;
            20.0.6.0/24;
            20.0.7.0/24;
        }
        source-port [ 100 200 ];
        destination-port [ ssh bgp ];
    }
    then accept;
}


If the configuration is changed to summarize the source and destination prefixes, we can produce a term that only uses 4 TCAM entries:

{master:0}[edit firewall family inet filter FOO]
user@switch# show
term one {
    from {
        source-address {
            10.0.0.0/21;
        }
        destination-address {
            20.0.0.0/21;
        }
        source-port [ 100 200 ];
        destination-port [ ssh bgp ];
    }
    then accept;


This is an extreme example but illustrates an important point.  If you summarize source and destination prefixes it can have a large impact on TCAM space needed.

Caution: Care should be taken when summarizing network prefixes, as depending on the summarization done, you may be matching against a larger set of IP addresses than you were when using a series of smaller, more specific subnets. In that case, you will need to weigh what is more important: security (with permitting more addresses) or TCAM space reduction.

Note: Support for automatically summarizing contiguous prefixes varies between switches. Some switches support doing this automatically upon programming the term conditions in the TCAM, whereas many will require manual optimization for most efficient TCAM usage.

Example 5

The behavior is different with source-port and destination-port. The switches will optimize contiguous port numbers as much as is possible by using bit mask ranges. The same principle as network address summarization.

In the following configuration, one would expect 100 TCAM entries, but the optimization manages to reduce the number of TCAM entries to only 7.

{master:0}[edit firewall family inet filter FOO]
user@switch# show
term one {
    from {
        source-port 101-200;
    }
    then accept;
}


When you compare this to the following term, which uses the same number of port numbers, but they are not contiguous so the switch cannot optimize using bit masks. This one takes 100 TCAM Entries.

{master:0}[edit firewall family inet filter FOO]
user@switch# show
term one {
    from {
        source-port [ 000 010 020 030 040 050 060 070 080 090 100 110 120 130 140 150 160 170 180 190 200
210 220 230 240 250 260 270 280 290 300 310 320 330 340 350 360 370 380 390 400 410 420 430 440 450 460 470
480 490 500 510 520 530 540 550 560 570 580 590 600 610 620 630 640 650 660 670 680 690 700 710 720 730 740
750 760 770 780 790 800 810 820 830 840 850 860 870 880 890 900 910 920 930 940 950 960 970 980 990 ];
    then accept;
}



Tip: Keep in mind that that if your firewall filter does not contain an explicit default term, then the implicit default term that discards all traffic will add one extra TCAM entry to your firewall filter.

 

Checking your TCAM usage

Checking PFE usage requires going into the PFE and the commands used depend on the model:

For EX4300, EX4600, QFX3500, QFX5100

On QFX5100, run the following CLI command to get an overview of TCAM usage:

{master:0}
user@QFX5100> show pfe filter hw summary

Slot 0

Group                    Group-ID       Allocated      Used           Free
---------------------------------------------------------------------------
> Ingress filter groups:
  iRACL group            14             512            21             491
> Egress filter groups:


On broadcom chipsets in general you can check the TCAM usage directly in the PFE:

{master:0}
user@EX4300> start shell
user@EX4300:RE:0% vty fpc0
 

BSD platform (QorIQ P202H processor, 0MB memory, 0KB flash)

(vty)# show filter
Program Filters:
---------------
   Index     Dir     Cnt    Text     Bss  Name
--------  ------  ------  ------  ------  --------

Term Filters:
------------
   Index    Semantic    Name
--------  ----------------
       1  Classic   test --> A firewall filter with name 'test' has hw index 1
   17000  Classic   __default_arp_policer__
   57008  Classic   __cfm_filter_shared_lc__
16777216  Classic   fnp-filter-level-all
46137360  Classic   pfe-cos-cl-557-5-2
46137361  Classic   pfe-cos-cl-558-5-2
46137362  Classic   pfe-cos-cl-559-5-2

Resolve Filters:
---------------
   Index
--------

(vty)# show filter hw 1 show_term_info --> The number used here is the hw index number discovered via the previous command
======================
Filter index   : 1
======================

- Filter name  : test

+ Hardware Instance : 1
  + Hardware key (struct pfe_bcm_dfw_hw_key_t):
    - Type          : IRACL
    - Vlan id       : 0
    - Direction     : ingress
    - Protocol      : 2 (IPv4)
    - Port class id : 0
    - Class id      : 1
    - Loopback      : 0
    - Vlan tag      : 0
  + FP usage info (struct pfe_bcm_dfw_fp_t):
    - Group                           : IFP iRACL group (3)
    - List of tcam entries            : [ total: 152; 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86
 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116
 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143
 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170
 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197
 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 ]
    - List of ranges                  : [ total: 0; ]
  + Misc info (struct pfe_bcm_dfw_misc_info_t):
  + Bind point info (union pfe_bcm_dfw_bind_point_info_t):
    + Class id      : 1
      - Vlans       : [4093 (total:1/4096)]
  + AE intf match list:
  + Programmed: YES
  + Total TCAM entries available: 7016
  + Total TCAM entries installed  : 152
  + Term Expansion:
    - Term    1: will expand to   151 terms: Name "term-one"
    - Term    2: will expand to     1 term : Name "final"
  + Term TCAM entry requirements:
    - Term    1: needs   151 TCAM entries: Name "term-one"     --> Total number of TCAM entries needed per term     
    - Term    2: needs     1 TCAM entry  : Name "final"
  + Total TCAM entries available: 7016        --> Total number of TCAM entries available   
  + Total TCAM entries installed  : 152       --> Total number of TCAM entries used 
 Total hardware instances: 1

(vty)# exit

% exit

{master:0}
user@EX4300>


For EX2200, EX3200, EX3300, EX4200, EX4500, EX4550, EX8200:

{master:0}
user@EX4200> start shell
% vty fpc0
BSD platform (MPC 8544 processor, 48MB memory, 0KB flash)
PFEM0(vty)# show tcam vendor 1 rules
Number of rules as Ingress PACL: 0
Number of rules as Ingress VACL: 0
Number of rules as Ingress RACL: 152  ---> This particular switch has one layer 3 firewall filter using a total of 152 terms.
Number of rules as   Egress PCL: 0
152 Ingress RACL rules
HW-index    Page_id    Entry_id    rule_size         fw_id    fmt    Rule
--------------------------------------------------------------------------------
     472        118           0            2             5     4    term-one.ext.0  --> each line in TCAM used shows the corresponding term name
     474        118           2            2             5     4    term-one.ext.1
.
.
.
   13894       3473           2            2             5     4    term-one.ext.149
   14112       3528           0            2             5     4    term-one.ext.150
   14114       3528           2            2             5     4    final.ext.0
TCAM utilization: 152(used), 6886(free), 7038(total)  --> Here it shows TCAM space used, free and total available.
PFEM0(vty)# exit
% exit
{master:0}
user@EX4200>

For QFX5200:

To calculate the TCAM entries available and Total TCAM entries needed in QFX5200, do the following:

1. Run 'show filter hw groups' on the QFX5200.

2. Look for the IFP iRACL group. See the blue highlighted output below. Entries used are 520, and Max entries are 768. The remaining entries are 248 (768 - 520 = 248).

FPC0(FRA vty)# show filter hw groups
Unit:0 Group Information:
VFP groups:
IFP groups:
BA classifier dynamic group id: 21. Pipe:  0 Entries:   76 Max Entries(total_available):  256( 256) Pri:  2 Slice: 1 Def Entries:  0
BA classifier dynamic group id: 22. Pipe:  1 Entries:   76 Max Entries(total_available):  256( 256) Pri:  2 Slice: 1 Def Entries:  0
BA classifier dynamic group id: 23. Pipe:  2 Entries:   76 Max Entries(total_available):  256( 256) Pri:  2 Slice: 1 Def Entries:  0
BA classifier dynamic group id: 24. Pipe:  3 Entries:   76 Max Entries(total_available):  256( 256) Pri:  2 Slice: 1 Def Entries:  0

   iRACL group id: 33. Pipe:  0 Entries:  520 Max Entries(total_available):  768( 768) Pri:  6 Slice:  9 Def Entries:  0 <<<<<<
   iRACL group id: 34. Pipe:  1 Entries:  520 Max Entries(total_available):  768( 768) Pri:  6 Slice:  9 Def Entries:  0 <<<<<<<
   iRACL group id: 35. Pipe:  2 Entries:  520 Max Entries(total_available):  768( 768) Pri:  6 Slice:  9 Def Entries:  0 <<<<<<<
   iRACL group id: 36. Pipe:  3 Entries:  520 Max Entries(total_available):  768( 768) Pri:  6 Slice:  9 Def Entries:  0 <<<<<<<

  Dynamic group id: 17. Pipe:  0 Entries:  146 Max Entries(total_available):  256( 256) Pri:  4 Slice:  2 Def Entries:  0
  Dynamic group id: 18. Pipe:  1 Entries:  146 Max Entries(total_available):  256( 256) Pri:  4 Slice:  2 Def Entries:  0
  Dynamic group id: 19. Pipe:  2 Entries:  146 Max Entries(total_available):  256( 256) Pri:  4 Slice:  2 Def Entries:  0
  Dynamic group id: 20. Pipe:  3 Entries:  146 Max Entries(total_available):  256( 256) Pri:  4 Slice:  2 Def Entries:  0

3. Run the command 'show filter hw 1 show_term_info'. The "Total TCAM entries available" listed is actually the remaining entries.

FPC0(FRA vty)# show filter hw 1 show_term_info   
======================
Filter index   : 1
======================

- Filter name  : control-plane-protection-v4

+ Hardware Instance : 1
  + Hardware key (struct brcm_dfw_hw_key_t):
    - Type          : IRACL
    - Vlan id       : 0
    - Direction     : ingress
    - Protocol      : 2 (IPv4)
    - Port class id : 0
    - Class id      : 0
    - Loopback      : 1
    - Port          : 0(xe-1)
    - Vlan tag      : 0
    - Non-overflow  : 1
  + FP usage info (struct brcm_dfw_fp_t):
    - Group                           : IFP iRACL group (33)
    - My Mac                          : 00:00:00:00:00:00
    - Loopback Reference Count        : 00000001
    - IFL Type                        : unknown (0)
    + List of tcam entries            : [ total: 520; ]
        - Pipe: 0; [1033 1037 1041 1045 1049 1053 1057 1061 1065 1069 1073 1077 1081 1085 1089 1093 1097 1101
 1105 1109 1113 1117 1121 1125 1129 1133 1137 1141 1145 1149 1153 1157 1161 1165 1169 1173 1177 1181 1185 1189
 1193 1197 1201 1205 1209 1213 1217 1221 1225 1229 <<SNIP>> 2964 2968 2972 2976 2980 2984 2988 2992 2996 3000
 3004 3008 3012 3016 3020 3024 3028 3032 3036 3040 3044 3048 3052 3056 3060 3064 3068 3072 3076 3080 3084 3088 
 3092 3096 3100 3104 3108 3112 ]
    + List of ranges                  : [ total: 0; ]
        - Pipe: 0 []
        - Pipe: 1 []
        - Pipe: 2 []
        - Pipe: 3 []
    + List of interface match entries : [ total: 0; ]
        - Pipe: 0 []
        - Pipe: 1 []
        - Pipe: 2 []
        - Pipe: 3 []
    + List of dot1q-tag match entries : [ total: 0; ]
        - Pipe: 0 []
        - Pipe: 1 []
        - Pipe: 2 []
        - Pipe: 3 []
    - List of l3 ifl index entries    : [ total: 0; ]
    + List of vfp tcam entries        : [ total: 0; ]
        - Pipe: 0 []
        - Pipe: 1 []
        - Pipe: 2 []
        - Pipe: 3 []
  + Misc info (struct brcm_dfw_misc_info_t):
    - List of <anlz_id, entry_id> : [ total: 0; ]
  + Bind point info (union brcm_dfw_bind_point_info_t):
    + Loopback      : CPU Traffic
  + Programmed: YES
  + BD ID     : 223
  + Total TCAM entries available: 248
  + Total TCAM entries needed   : 520
  + Term Expansion:
    - Term    1: will expand to    15 terms: Name "snmp"
    - Term    2: will expand to    15 terms: Name "snmp-frags"
    - Term    3: will expand to    15 terms: Name "http"
    - Term    4: will expand to    15 terms: Name "ssh"
    - Term    5: will expand to     1 term : Name "icmp"
    - Term    6: will expand to    11 terms: Name "ntp"
    - Term    7: will expand to    11 terms: Name "ntp-back"
    - Term    8: will expand to     3 terms: Name "dns"
    - Term    9: will expand to     7 terms: Name "gre"
    - Term   10: will expand to     7 terms: Name "bgp"
    - Term   11: will expand to     7 terms: Name "bgp-back"
    - Term   12: will expand to     1 term : Name "rsvp"
    - Term   13: will expand to     1 term : Name "vrrp"
    - Term   14: will expand to     1 term : Name "ospf"
    - Term   15: will expand to     3 terms: Name "bfd"
    - Term   16: will expand to     3 terms: Name "bfd-back"
    - Term   17: will expand to     2 terms: Name "dhcp"
    - Term   18: will expand to     2 terms: Name "dhcp-back"
    - Term   19: will expand to     9 terms: Name "traceroute"
    - Term   20: will expand to     1 term : Name "default-term"
  + Term TCAM entry requirements:
    - Term    1: needs    60 TCAM entries: Name "snmp"
    - Term    2: needs    60 TCAM entries: Name "snmp-frags"
    - Term    3: needs    60 TCAM entries: Name "http"
    - Term    4: needs    60 TCAM entries: Name "ssh"
    - Term    5: needs     4 TCAM entries: Name "icmp"
    - Term    6: needs    44 TCAM entries: Name "ntp"
    - Term    7: needs    44 TCAM entries: Name "ntp-back"
    - Term    8: needs    12 TCAM entries: Name "dns"
    - Term    9: needs    28 TCAM entries: Name "gre"
    - Term   10: needs    28 TCAM entries: Name "bgp"
    - Term   11: needs    28 TCAM entries: Name "bgp-back"
    - Term   12: needs     4 TCAM entries: Name "rsvp"
    - Term   13: needs     4 TCAM entries: Name "vrrp"
    - Term   14: needs     4 TCAM entries: Name "ospf"
    - Term   15: needs    12 TCAM entries: Name "bfd"
    - Term   16: needs    12 TCAM entries: Name "bfd-back"
    - Term   17: needs     8 TCAM entries: Name "dhcp"
    - Term   18: needs     8 TCAM entries: Name "dhcp-back"
    - Term   19: needs    36 TCAM entries: Name "traceroute"
    - Term   20: needs     4 TCAM entries: Name "default-term"
  +
Total TCAM entries available: 248       <<<<<<<<<<<<< Remaining Entries.
  + Total TCAM entries needed   : 520

Related KB articles

For more TCAM information refer to the following KB articles:

KB28925 - TCAM filter space allocation and verification in QFX devices from Junos OS 12.2X50-D20 onward
     -> Information on how memory slices are reserved in the TCAM when using PACLs, RACLs and VACLs

KB25927 - [QFX] How to check the TCAM Utilization on QFX-3500
     -> Shows how to check TCAM utilization on a QFX3500

KB25106 - Calculate the TCAM utilization by loopback Firewall Filter on the QFX3500 switch
     -> Caveats regarding the application of firewall filters to the loopback interface (i.e. they use more TCAM space)

KB30804 - QFX5100 failed to program firewall filters with multiple port range options.
     -> Proper use of the source-port-range-optimize and destination-port-range-optimize options for QFX5100

 

 
Modification History:
2019-05-07: QFX5200 information added.
Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search