Knowledge Search
SRX Getting Started -- Troubleshooting Traffic Flows and Session Establishment
|
|
[KB16110] Show KB Properties
[KB16110] Hide KB Properties
Summary:
Symptoms:
Cause:
Solution:
Symptoms:
Cause:
Solution:
This article contains the following sections for troubleshooting traffic through SRX devices. Click the section title below to jump directly to that section:
Troubleshoot Traffic Flows
The traffic flow for Junos flow-based processing is depicted in the following figure:
One of the most important troubleshooting features is the ability to troubleshoot the decisions that the SRX platform makes on the traffic itself. Often just looking at the firewall logs will provide enough detail to understand what the firewall is doing to a packet (permit, deny, the firewall policy applied, NAT, and VPN), but sometimes there is not enough information about why the firewall is making a decision on a packet itself. This is when you may need to debug a packet flow. In ScreenOS, this is accomplished using “debug flow basic”, which records the decisions that the firewall makes on a packet. In the SRX, the primary method of capturing this information is through the “set security flow traceoptions basic-datapath”, and there is also the ability to filter only certain packets for advanced debugging using the “set security flow traceoptions packet-filter”.
In the SRX, only one expression can be configured per packet filter, so if you want to capture more than one flow direction, then you would need to define multiple packet filters per capture (one for each direction). You will always want to make sure that you enable packet-filters when tracing traffic, not only because there is often too much information in the file if you don’t, but also because the SRX will take a performance hit if you are trying to trace ALL traffic going through it. If you set the appropriate filter and you are not passing lots of data through the filter, then it shouldn’t have any noticeable impact.
In this example, a client with the IP address 1.1.1.100 is initiating a Microsoft RDP connection (TCP port 3389) to a server with the IP address 1.1.1.30 (which is NAT’d to the real server address 192.168.224.30).
192.168.224.30 does not actually have knowledge of the 1.1.1.0/24 network, so the client IP addresses in the Untrust zone are source NAT'd and this must be accounted for in our Packet Filters.
Comments will precede the statements starting with ##.
## Packet from source address 1.1.1.100 (source port 57650) captured by the filter, destined to 1.1.1.30 port 3389, with the protocol # 6 for TCP
The same test is performed below, after a policy was added to permit the traffic.
## Packet arrives which is matched by the filter MatchTraffic. It has a source address of 1.1.1.100 (source port 51303) with a destination address 1.1.1.30, destination port 3389 with IP Protocol number 6, which is TCP.
The reverse direction capture is as follows:
## Here the traffic arrives from 192.168.224.30 (the server's internal IP address) source port 3389 , and is destined to our NAT’d interface 192.168.224.3 destination port 48810 (client)
Troubleshoot SRX Session Establishment
Perform Packet Capture on SRX Branch Devices
The SRX Branch Platforms have the capability to perform packet capture for transit and self-traffic using the Packet Capture Feature. This Packet Capture Feature is not supported for the High-End SRX devices. However, packet capture on High-End SRX devices can be performed with the
Additionally, a monitored packet capture of 'self-traffic' only (e.g. Dynamic Routing Protocol messages, ARP, management traffic, ICMP to Routing Engine) can be done using the '
The Packet Capture Feature setup is done as follows in configuration mode, in a similar way to how traces are configured:
Packet Capture Analysis on SRX Branch Devices
Once the packet capture has been complete, the packet capture information can either be viewed locally, or transferred to an external machine to view in a PCAP viewer such as Wireshark. All packet captures are stored in /var/tmp directory under the name of the file configured (in the above example it was pcap) followed by the interface name. So in this example it would be pcap.fe-0.0.7 “/” cannot be used because it is a special character. To view the contents on the SRX itself perform the following steps.
Purpose:- Troubleshoot Traffic Flows
- Troubleshoot SRX Session Establishment
- Perform Packet Capture & Analysis on SRX Branch devices
Troubleshoot Traffic Flows
The traffic flow for Junos flow-based processing is depicted in the following figure:
For detailed information regarding the packet flow and session management, refer to the technical documentation: Flow-Based Processing Feature Guide for Security Devices.
One of the most important troubleshooting features is the ability to troubleshoot the decisions that the SRX platform makes on the traffic itself. Often just looking at the firewall logs will provide enough detail to understand what the firewall is doing to a packet (permit, deny, the firewall policy applied, NAT, and VPN), but sometimes there is not enough information about why the firewall is making a decision on a packet itself. This is when you may need to debug a packet flow. In ScreenOS, this is accomplished using “debug flow basic”, which records the decisions that the firewall makes on a packet. In the SRX, the primary method of capturing this information is through the “set security flow traceoptions basic-datapath”, and there is also the ability to filter only certain packets for advanced debugging using the “set security flow traceoptions packet-filter”.
In the SRX, only one expression can be configured per packet filter, so if you want to capture more than one flow direction, then you would need to define multiple packet filters per capture (one for each direction). You will always want to make sure that you enable packet-filters when tracing traffic, not only because there is often too much information in the file if you don’t, but also because the SRX will take a performance hit if you are trying to trace ALL traffic going through it. If you set the appropriate filter and you are not passing lots of data through the filter, then it shouldn’t have any noticeable impact.
In this example, a client with the IP address 1.1.1.100 is initiating a Microsoft RDP connection (TCP port 3389) to a server with the IP address 1.1.1.30 (which is NAT’d to the real server address 192.168.224.30).
192.168.224.30 does not actually have knowledge of the 1.1.1.0/24 network, so the client IP addresses in the Untrust zone are source NAT'd and this must be accounted for in our Packet Filters.
The trace configuration is:192.168.224.30 (Server in Trust zone) ---------SRX-------- 1.1.1.100 (client in Untrust zone)SRX interfaces:
ge-0/0/0.0: IP Address 192.168.224.3/24 (Trust)
fe-0/0/7.0: IP Address 1.1.1.50/24 (Untrust)
Static NAT on SRX for server:
1.1.1.30 (public IP address for server) -> 192.168.224.30 (internal Server IP address)
Source NAT on SRX for incoming clients: (for this scenario because 192.168.244.30 network does not have knowledge of 1.1.1.0/24 network)
1.1.1.0/24 -> 192.168.224.3
Note that the packet filters are configured with the addresses that the packets will arrive at the SRX as, not as the translated IP Addresses in NAT. Within each packet-filter the arguments serve to match as a logical AND (e.g. for MatchTraffic statement the traffic will match source IP’s within 1.1.1.100/32 AND destination IP 1.1.1.30/32.) Adding additional Traffic Filters will function as a logical OR between the filters so each filter will operate independently.set security flow traceoptions file DebugTraffic# define filter to capture traffic from client to Server's public IP address 1.1.1.30
set security flow traceoptions flag basic-datapath
(Note: You may also specify a subnet for the prefix, 1.1.1.0/24, instead of the client's /32 IP address)
set security flow traceoptions packet-filter MatchTraffic source-prefix 1.1.1.100/32 destination-prefix 1.1.1.30/32
#define filter to capture traffic from Server in Trust to Client's NAT'd address
set security flow traceoptions packet-filter MatchTrafficReverse source-prefix 192.168.224.30/32 destination-prefix 192.168.224.3/32
Comments will precede the statements starting with ##.
## Packet from source address 1.1.1.100 (source port 57650) captured by the filter, destined to 1.1.1.30 port 3389, with the protocol # 6 for TCP
##IPID of the packet shows its 885, which is very useful for comparing packets across multiple capture points (e.g. viewing packet on originating machine, on FW, and on destination to make sure the packet arrivesAug 12 16:22:22 16:22:21.1100315:CID-0:RT:<1.1.1.100/57650->1.1.1.30/3389;6> matched filter MatchTraffic:
)
## This packet arrived on port fe-0/0/7.0Aug 12 16:22:22 16:22:21.1100315:CID-0:RT:packet [48] ipid = 885, @422ec91e
Aug 12 16:22:22 16:22:21.1100315:CID-0:RT:---- flow_process_pkt: (thd 1): flow_ctxt type 13, common flag 0x0, mbuf 0x422ec780
Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: flow process pak fast ifl 69 in_ifp fe-0/0/7.0
## Platform knows the 5 tuple of the packet and does a flow lookupAug 12 16:22:22 16:22:21.1100315:CID-0:RT: fe-0/0/7.0:1.1.1.100/57650->1.1.1.30/3389, tcp, flag 2 syn
## No existing session found in the table lookup, a policy lookup must be performedAug 12 16:22:22 16:22:21.1100315:CID-0:RT: find flow: table 0x4dd31658, hash 32955(0xffff), sa 1.1.1.100, da 1.1.1.30, sp 57650, dp 3389, proto 6, tok 448
## First, check to see if destination address should be NAT’d (static or destination NAT)Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: no session found, start first path. in_tunnel - 0, from_cp_flag - 0
Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: flow_first_create_session
## The destination should be NAT’d from 1.1.1.30 to 192.168.224.30Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: flow_first_in_dst_nat: in <fe-0/0/7.0>, out <N/A> dst_adr 1.1.1.30, sp 57650, dp 3389
Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: chose interface fe-0/0/7.0 as incoming nat if.
## Do a route lookup to determine what the destination interface and zone should be so the SRX can do a policy lookup.Aug 12 16:22:22 16:22:21.1100315:CID-0:RT:flow_first_rule_dst_xlate: packet 1.1.1.100->1.1.1.30 nsp2 0.0.0.0->192.168.224.30.
## The route lookup that points out the interface ge-0/0/0.0 with a next hop which is the destination on the attached 192.168.224.0/24 network.Aug 12 16:22:22 16:22:21.1100315:CID-0:RT:flow_first_routing: call flow_route_lookup(): src_ip 1.1.1.100, x_dst_ip 192.168.224.30, in ifp fe-0/0/7.0, out ifp N/A sp 57650, dp 3389, ip_proto 6, tos 0
Aug 12 16:22:22 16:22:21.1100315:CID-0:RT:Doing DESTINATION addr route-lookup
## Now a policy lookup is performed that determines that the destination zone on ge-0/0/0.0 is trust, and the packet arrived on fe-0/0/7.0 which is untrust.Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: routed (x_dst_ip 192.168.224.30) from untrust (fe-0/0/7.0 in 0) to ge-0/0/0.0, Next-hop: 192.168.224.30
## Policy lookup shows that the SRX does not find a match and the traffic is dropped due to no matching policy.Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: policy search from zone untrust-> zone trust
Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: app 0, timeout 1800s, curr ageout 20s
Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: packet dropped, denied by policy
Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: packet dropped, policy deny.
Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: flow find session returns error.
Aug 12 16:22:22 16:22:21.1100315:CID-0:RT: ----- flow_process_pkt rc 0x7 (fp rc -1)
The same test is performed below, after a policy was added to permit the traffic.
## Packet arrives which is matched by the filter MatchTraffic. It has a source address of 1.1.1.100 (source port 51303) with a destination address 1.1.1.30, destination port 3389 with IP Protocol number 6, which is TCP.
## Packet has the IPID 5015. The IP ID field is very useful when correlating this debug trace to packets in a simultaneous PCAP packet capture. Comparing the debug trace and PCAP packet captures on a source or destination machine help to identify packets that did or did not arrive.Aug 4 16:21:02 16:21:00.1872004:CID-0:RT:<1.1.1.100/51303->1.1.1.30/3389;6> matched filter MatchTraffic:
## Packet Arrived on the fe-0/0/7.0 interfaceAug 4 16:21:02 16:21:00.1872004:CID-0:RT:packet [48] ipid = 5015, @423d7e9eAug 4 16:21:02 16:21:00.1872004:CID-0:RT:---- flow_process_pkt: (thd 1): flow_ctxt type 13, common flag 0x0, mbuf 0x423d7d00
## Deeper look at the packet shows that it is the syn packet of the connection, indicating the first packet of the flowAug 4 16:21:02 16:21:00.1872004:CID-0:RT: flow process pak fast ifl 72 in_ifp fe-0/0/7.0
## Packet Lookup with the information above, where sa=source address, da=destination address, sp=source port, dp=destination port, proto=protocolAug 4 16:21:02 16:21:00.1872004:CID-0:RT: fe-0/0/7.0:1.1.1.100/51303->1.1.1.30/3389, tcp, flag 2 syn
## First packet in flow, so no session is foundAug 4 16:21:02 16:21:00.1872004:CID-0:RT: find flow: table 0x5258d7b0, hash 17008(0xffff), sa 1.1.1.100, da 1.1.1.30, sp 51303, dp 3389, proto 6, tok 448
## Create new sessionAug 4 16:21:02 16:21:00.1872004:CID-0:RT: no session found, start first path. in_tunnel - 0, from_cp_flag – 0
## Before a policy lookup, a check must be performed for destination NAT; in this case it is static NAT, which NAT’s the 1.1.1.30 address to 192.168.224.30. This lookup must be done BEFORE route lookup, since if there is a destination NAT change, then the SRX needs to do a route lookup on the translated destination address.Aug 4 16:21:02 16:21:00.1872004:CID-0:RT: flow_first_create_session
Aug 4 16:21:02 16:21:00.1872004:CID-0:RT: flow_first_in_dst_nat: in <fe-0/0/7.0>, out <N/A> dst_adr 1.1.1.30, sp 51303, dp 3389
## Begin Routing LookupAug 4 16:21:02 16:21:00.1872004:CID-0:RT: chose interface fe-0/0/7.0 as incoming nat if.
Aug 4 16:21:02 16:21:00.1872004:CID-0:RT:flow_first_rule_dst_xlate: packet 1.1.1.100->1.1.1.30 nsp2 0.0.0.0->192.168.224.30.
## Perform Route LookupAug 4 16:21:02 16:21:00.1872004:CID-0:RT:flow_first_routing: call flow_route_lookup(): src_ip 1.1.1.100, x_dst_ip 192.168.224.30, in ifp fe-0/0/7.0, out ifp N/A sp 51303, dp 3389, ip_proto 6, tos 0
## Performing the route lookup on destination IP address 192.168.224.30 shows that it should go out interface ge-0/0/0.0, since ge-0/0/0.0 IP address is 192.168.224.3/24, the network is directly attached, and therefore the next-hop is the destination itself 192.168.224.30. If the destination was not directly attached, then our Next-Hop would be the Next-Hop of the router.Aug 4 16:21:02 16:21:00.1872004:CID-0:RT:Doing DESTINATION addr route-lookup
## Since the ingress interface is fe-0/0/7.0 which is zone Untrust, a route lookup determines that the outgoing interface is going to be ge-0/0/0.0, which is zone Trust. Therefore a policy lookup can be performed.Aug 4 16:21:02 16:21:00.1872004:CID-0:RT: routed (x_dst_ip 192.168.224.30) from untrust (fe-0/0/7.0 in 0) to ge-0/0/0.0, Next-hop: 192.168.224.30
## The session timeout is set to 1800 seconds, which is based on the standard TCP application timeout of 30 minutes. (Note that the SRX does not use Ticks like ScreenOS; it uses seconds.) Also the curr ageout of 20 seconds is the initial TCP timeout, if the session isn’t established within 20 seconds, then the session will be dropped.Aug 4 16:21:02 16:21:00.1872004:CID-0:RT: policy search from zone untrust-> zone trust
Aug 4 16:21:02 16:21:00.1872004:CID-0:RT: policy has timeout 900
## Here the output shows the SRX NAT pool for source translation is used.Aug 4 16:21:02 16:21:00.1872004:CID-0:RT: app 0, timeout 1800s, curr ageout 20s
## DIP id 2/0 is used, where the term DIP is a legacy term from ScreenOS, but it shows that the SRX is translating the source from 1.1.1.100 (source port 51303) to 192.168.224.3 (source port 48810). In this example, translation is needed because the destination host 192.168.224.30 does not know about our 1.1.1.0/24 network.Aug 4 16:21:02 16:21:00.1872004:CID-0:RT:flow_first_src_xlate: src nat 0.0.0.0(51303) to 192.168.224.30(3389) returns status 1, rule/pool id 1/2.
## The SRX selects ge-0/0/0.0 as our outgoing interfaceAug 4 16:21:02 16:21:00.1872004:CID-0:RT: dip id = 2/0, 1.1.1.100/51303->192.168.224.3/48810
## Message references other service processing not applicable for this ruleAug 4 16:21:02 16:21:00.1872004:CID-0:RT: choose interface ge-0/0/0.0 as outgoing phy if
Aug 4 16:21:02 16:21:00.1872004:CID-0:RT:is_loop_pak: No loop: on ifp: ge-0/0/0.0, addr: 192.168.224.30, rtt_idx:0
## The traffic is matched to Policy 9Aug 4 16:21:02 16:21:00.1872004:CID-0:RT:policy is NULL (wx/pim scenario)
Aug 4 16:21:02 16:21:00.1872004:CID-0:RT:sm_flow_interest_check: app_id 0, policy 9, app_svc_en 0, flags 0x2. not interested
Aug 4 16:21:02 16:21:00.1872004:CID-0:RT:sm_flow_interest_check: app_id 1, policy 9, app_svc_en 0, flags 0x2. not interested
## TIP: The policy 9 content can be checked by looking through the policy list with the command “show security policies” or with the command “show security policies | find "Index\: 9"” which uses the regular expression to search for Index: 9 in the policy and will show the first instance. Visually make sure that the traffic is matching the correct rule
.
root@SRX210# run show security policies | find "Index\: 9"
Policy: Allow-RDP-Server, State: enabled, Index: 9, Sequence number: 1
Source addresses: 1.1.1.0/24
Destination addresses: 192.168.224.30
Applications: RDP
Action: permit, log
## Packet is sent out interface ge-0/0/0.0Aug 4 16:21:02 16:21:00.1872004:CID-0:RT:flow_first_service_lookup(): natp(0x51ee4680): app_id, 0(0).
Aug 4 16:21:02 16:21:00.1872004:CID-0:RT: service lookup identified service 0.
Aug 4 16:21:02 16:21:00.1872004:CID-0:RT: flow_first_final_check: in <fe-0/0/7.0>, out <ge-0/0/0.0>
The reverse direction capture is as follows:
## Here the traffic arrives from 192.168.224.30 (the server's internal IP address) source port 3389 , and is destined to our NAT’d interface 192.168.224.3 destination port 48810 (client)
## Do a hash lookup on the session to find it in the tableAug 14 19:11:40 19:11:40.192283:CID-0:RT:<192.168.224.30/3389->192.168.224.3/48810;6> matched filter MatchTrafficReverse:Aug 14 19:11:40 19:11:40.192283:CID-0:RT:packet [40] ipid = 8174, @42308b9e
Aug 14 19:11:40 19:11:40.192283:CID-0:RT:---- flow_process_pkt: (thd 1): flow_ctxt type 13, common flag 0x0, mbuf 0x42308a00
Aug 14 19:11:40 19:11:40.192283:CID-0:RT: flow process pak fast ifl 67 in_ifp ge-0/0/0.0
Aug 14 19:11:40 19:11:40.192283:CID-0:RT: ge-0/0/0.0:192.168.224.30/3389->192.168.224.3/48810, tcp, flag 10
## Since this is return traffic, it is found it in the session table, as id 12535Aug 14 19:11:40 19:11:40.192283:CID-0:RT: find flow: table 0x4dd31658, hash 44831(0xffff), sa 192.168.224.30, da 192.168.224.3, sp 3389, dp 48810, proto 6, tok 384
Aug 14 19:11:40 19:11:40.192283:CID-0:RT: flow got session.
## Translate the packet to the appropriate translation for outbound flowAug 14 19:11:40 19:11:40.192283:CID-0:RT: flow session id 12535Aug 14 19:11:40 19:11:40.192283:CID-0:RT: tcp seq check.
## Place packet in buffer to be sent outAug 14 19:11:40 19:11:40.192283:CID-0:RT: post addr xlation: 1.1.1.30->1.1.1.100.
Aug 14 19:11:40 19:11:40.192283:CID-0:RT:mbuf 0x42308a00, exit nh 0x80010
Aug 14 19:11:40 19:11:40.192283:CID-0:RT: ----- flow_process_pkt rc 0x0 (fp rc 0)
Troubleshoot SRX Session Establishment
- Is the packet arriving on the SRX?
The first thing that should be checked to make sure that the traffic is arriving on the SRX itself. The SRX should be able to show the traffic processed by just viewing the output of the debug itself. This may not be the case if Stateless Firewall Filters are being used, which may drop the packets before they are processed by the flow engine. If stateless packet filtering is used, ensure that the filters are not blocking the traffic by reviewing the configuration, or adding the “log” option to the then statement of the deny filter to log the packets on the SRX. If you don’t see the traffic arriving on the SRX, you should check the routing table of the peer device, as well as checking to see if the peer device has an ARP entry for the IP address that is to be reached. This is particularly important if you are using static NAT, as Proxy ARP MAY be required. - Is the packet being blocked by a Screen?
If a packet is an invalid packet, and screens are enabled on the source zone, it may be dropped. An example would be a packet that has a SYN, ACK, and FIN bits set. Check the output of 'show log messages' for screen errors. - Is the packet destined for the firewall itself?
This is most common with management traffic such as Telnet, SSH, and SNMP, but also for protocol traffic such as BGP, OSPF, RIP, and PIM. In this case, the SRX must make sure that these respective services are running on that interface. In Junos, this is configured at either the zone or interface level under [edit security zones security-zone <zone> host-inbound-traffic]. If this is not configured then the firewall will drop the packet. - Is the session asymmetric?
Stateful SYN checking is on by default, and if the firewall does not see the SYN packet, it will drop the subsequent packets and the session will not be established. This should be clear from the debug. In this situation you must either ensure that the SRX is in the initial packet path so that it sees the SYN packet (recommended!) or turn off SYN checking. If the SRX cannot see both directions of the flow then some features such as IDP may not be able to work at the full capacity. Therefore it is recommended to design your network so that asymmetric flow does not occur. - Is the packet from an existing session?
If the packet matches an existing session, then the SRX will not include all debugging information in that output. It will display the basics and refer to the session ID to cross reference. You can still examine the session table to track this session with the “show security flow session” command. - Is the packet arriving on the correct interface/zone?
When looking at the debug output you should be able to tell what interface the packet has arrived on, and subsequently, what the source zone is. This is important for policy lookup, and if it is not matched properly, then most likely a problem will occur. - Is the appropriate route / L2 forwarding path being selected?
Depending on if you are using Layer 3 mode or Transparent mode, the SRX will need to ensure that the appropriate forwarding decision is chosen. With layer 3 mode, the firewall will do a route lookup based on the destination of the route, which will determine what the egress interface will be (which also determines the to-zone). If the device is in Transparent mode, the device will either do an ARP, or will broadcast the packet out all interfaces (except that from which it came) in the same bridging domain. If the wrong route is set (in Transparent mode, the forwarding decision should happen without intervention, unless other mechanisms such as Dynamic ARP inspection or L2 Broadcast filtering are in place to block this), then the SRX may chose the wrong outbound interface to send the traffic out, as well as the wrong NAT decision. Currently, a route-id is not shown in the output of the debug like it is in ScreenOS. This will probably change in the future. You may need to do a route lookup with the “show route <prefix>” command to make sure that the correct route is chosen and that the appropriate outbound interface and next-hop is selected. In Transparent mode, you can check the L2 forwarding table with the “show arp” and “show ethernet-switching table” if using the branch SRX, while the High End SRX uses a different command, “show l2-learning interface” to see what entries are known by the system. - Is the correct policy being selected by the firewall?
Now that the appropriate egress interface / zone has been chosen, assuming that those are correct, the firewall policy matching the traffic is chosen. Today that number is referenced in the debug logs as “policy <#>” and as shown in the debug above, you can run the “show security policies | find "Index\: <#>" command to make sure that the policy is correct. If it is not the correct policy, you will need to take a look at the source/destination address and service. It is often prudent to look into the address and service objects themselves to make sure that they are really matching what you are looking for, and not just the names of the address objects themselves as shown in the policy. For instance, an object can be named 1.1.1.0/24, but the actual address that it matches if the subnet mask is omitted then it would be 1.1.1.0/32. Just by looking at the object name , it may still be wrong. Also, see the next step about NAT configuration, as it may impact your policy. - Is the correct NAT being applied by the policy?
Today there are some improvements that are going to be put into NAT debugging, but for now, the best thing to check is to look at the output of the debug and see what translation takes place. Is it what you expect (e.g. is the source translated or not translated as expected? Is the destination translated or not translated as expected?) Remember that static and destination address translations happen before policy lookup, while source address translations occur AFTER policy lookup, so you need to make sure that you have factored this in when configuring your policy. If the wrong action is being taken on your policy, you may want to review the rulebase configuration to visually inspect the match and action criteria. There can be many NAT rulebases depending on the configuration, so be sure to take that into account. - Does the debug show that the packet is being processed by “Application Services” or is destined to a VPN tunnel?
If the packet is being processed by application services then further debugging may be necessary, for instance, if you have IDP enabled on that policy, and the IDP detects that packet as malicious, it may drop it based upon policy (so even though the SRX flow looked OK, it could still be dropped based upon policy). The same is true for other UTM features, so you need to keep this in mind. If the packet is destined to a tunnel, then you should see this output. If a VPN is referenced, the VPN must be active, or else the packet will not have anywhere to go.
Perform Packet Capture on SRX Branch Devices
The SRX Branch Platforms have the capability to perform packet capture for transit and self-traffic using the Packet Capture Feature. This Packet Capture Feature is not supported for the High-End SRX devices. However, packet capture on High-End SRX devices can be performed with the
datapath-debug method. For datapath-debug instructions, refer to KB21563- How to capture packets on High-End SRX devices. Additionally, a monitored packet capture of 'self-traffic' only (e.g. Dynamic Routing Protocol messages, ARP, management traffic, ICMP to Routing Engine) can be done using the '
monitor traffic interface' command. This is supported on both SRX Branch and High-end SRX devices. For more information, see KB15779 - SRX Getting Started - Troubleshooting Commands. The Packet Capture Feature setup is done as follows in configuration mode, in a similar way to how traces are configured:
- First, configure the packet capture file information, including the file name, size, copies, and the amount of traffic to capture per packet. The minimum amount of traffic that must be captured will be 68 Bytes (Layer 2-4) but a maximum of 1500 Bytes can be captured.
- Next, a packet capture filter will be applied to select what packets should be captured in this configuration. Technically the SRX does not need to filter the traffic, however it is highly recommended to filter only the desired traffic, which is what is shown in this example. This is done by configuring the firewall filter which defines what to match. You can select the source-address, destination-address, source-port, destination-port, and protocol. The following packet capture is going to collect traffic that arrives on the interface with a source address in the prefix 192.168.224.0/24 AND destination-address 192.168.224.3 AND protocol OSPF AND protocol ESP. Also the action allow-all-else is used to make sure that the SRX does not drop any other traffic, but do not sample it either. It is important to remember that the packet capture will match the traffic as it leaves or arrives on the interface based on the Layer 3 or Layer 4 properties in the packet itself. So you need to take this into account when NAT is being performed on the packet, and match the traffic as is.
- Lastly, apply the filter to the interface to define that it should capture the traffic. Then commit the configuration. In this example, only the traffic arriving on the input direction is captured, but the outbound direction can be set as well in another statement.
root@SRX210# set forwarding-options packet-capture file filename pcap files 10 size 10000
root@SRX210# set forwarding-options packet-capture maximum-capture-size 1500
root@SRX210# set firewall filter PCAP term capture from source-address 192.168.224.0/24
root@SRX210# set firewall filter PCAP term capture from destination-address 192.168.224.3/32
root@SRX210# set firewall filter PCAP term capture from protocol ospf
root@SRX210# set firewall filter PCAP term capture from protocol esp
root@SRX210# set firewall filter PCAP term capture then sample
root@SRX210# set firewall filter PCAP term capture then accept
root@SRX210# set firewall filter PCAP term allow-all-else then accept
root@SRX210# set interfaces fe-0/0/7 unit 0 family inet filter input PCAP
root@SRX210# commit and-quit
Disabling Packet Captures: To stop the packet capture, either deactivate or delete the configuration “interfaces fe-0/0/7 unit 0 family inet filter input PCAP” and optionally, remove the configuration “firewall filter PCAP” and “forwarding-options packet-capture” as shown below:
For additional information on the Packet Capture feature, refer to http://www.juniper.net/techpubs/en_US/junos11.4/information-products/topic-collections/security/software-all/monitoring-and-troubleshooting/index.html?config-pcap-chapter.html.root@SRX210# delete interfaces fe-0/0/7 unit 0 family inet filter input PCAP(deactivates packet capture)(optional: delete filter if not using it again)
root@SRX210# delete firewall filter PCAP
root@SRX210# delete forward-options packet-capture
Packet Capture Analysis on SRX Branch Devices
Once the packet capture has been complete, the packet capture information can either be viewed locally, or transferred to an external machine to view in a PCAP viewer such as Wireshark. All packet captures are stored in /var/tmp directory under the name of the file configured (in the above example it was pcap) followed by the interface name. So in this example it would be pcap.fe-0.0.7 “/” cannot be used because it is a special character. To view the contents on the SRX itself perform the following steps.
- Either login to the SRX with the root account, or use the “start shell” command with a super-user class account permission.
- Change directories to the /var/tmp directory
- Issue the tcpdump command with the –r option to read the pcap formatted file (in this case the name pcap is used but that name is arbitrary). The –v option gives additional verboseness to the output and shows some additional info, and finally pcap.fe-0.0.0 is the name of the capture. There are other options that can be explored by viewing the tcpdump manual page at http://www.tcpdump.org/tcpdump_man.html.
root@SRX210> start shell
root@SRX210% cd /var/tmp
Below the output can be dissected on a packet by packet basis including the applicable protocol information (particularly for dynamic routing info and VPN information.
root@SRX210% tcpdump –r –v pcap.fe-0.0.7
21:05:19.974301 In IP (tos 0xc0, ttl 64, id 3464, offset 0, flags [none], proto: ESP (50), length: 120) 192.168.224.1 > 192.168.224.3: ESP(spi=493530462,seq=0x324)
21:05:24.979969 In IP (tos 0xc0, ttl 1, id 31089, offset 0, flags [none], proto: OSPF (89), length: 72) 192.168.224.6 > ospf-all.mcast.net: OSPFv2, Hello, length 52
Router-ID 192.168.225.1, Backbone Area, Authentication Type: none (0)
Options [External]
Hello Timer 10s, Dead Timer 40s, Mask 255.255.255.0, Priority 1
Designated Router 192.168.224.3, Backup Designated Router 192.168.224.1
Neighbor List:
Juniper2350
192.168.230.1
21:05:24.980113 In IP (tos 0xc0, ttl 64, id 3500, offset 0, flags [none], proto: ESP (50), length: 120) 192.168.224.1 > 192.168.224.3: ESP(spi=493530462,seq=0x325)
21:05:26.977403 In IP (tos 0x0, ttl 64, id 3513, offset 0, flags [none], proto: ESP (50), length: 104) 192.168.224.1 > 192.168.224.3: ESP(spi=493530462,seq=0x326)
21:05:27.985879 In IP (tos 0xc0, ttl 64, id 3515, offset 0, flags [none], proto: ESP (50), length: 136) 192.168.224.1 > 192.168.224.3: ESP(spi=493530462,seq=0x327)
21:05:28.976092 In IP (tos 0xc0, ttl 1, id 3521, offset 0, flags [none], proto: OSPF (89), length: 72) 192.168.224.1 > ospf-all.mcast.net: OSPFv2, Hello, length 52
Router-ID 192.168.230.1, Backbone Area, Authentication Type: none (0)
Options [External]
Hello Timer 10s, Dead Timer 40s, Mask 255.255.255.0, Priority 1
Designated Router 192.168.224.3, Backup Designated Router 192.168.224.1
Neighbor List:
Juniper2350
192.168.225.1
Implementation
Related Links:
Copyright© 1999-2012 Juniper Networks, Inc. All rights reserved.
