Data Collection and Troubleshooting Guides can help with issue investigation as well as reduce time to resolve. Each problem/issue could require a different set of data to collect. This article contains a list of data to collect as well as pointers to Resolution Guides and references on how to collect the data.

• What information should I collect to assist in troubleshooting prior to opening a case?
• The goal of this document is to reduce the time spent on initial data collection and reduce time to resolve by providing a comprehensive list of what to collect or gather to troubleshoot an issue.

This section contains the following:

• Data to Collect for all configurations
• Additional Data to Collect
• References

Caveats and tips:

• Traceoptions require additional system resources to gather and store data:

  • Ensure you have enough disk space when enabling traceoptions
  • Gauge current system utilization before enabling traceoptions
    • “show chassis routing-engine”
    • “show security monitoring fpc <spc-slot>” (use slot 0 for branch)
  • Instead of using “flag all” you can flag specific areas of interest
  • Delete all traceoptions that are not needed for immediate debugging
  • Don’t forget to remove traceoptions after data collection is completed. This can be done by deactivating or deleting the traceoptions configuration stanza that you previously added to activate traceoptions.

    For example, let's say that traceoptions that you have enabled was configured as follows:

    set chassis cluster traceoptions file cluster.tr size 5m files 5 world-readable
set chassis cluster traceoptions flag all

    To disable this traceoptions configuration, you can issue one of the following two commands (not both) and commit the changes:

    deactivate chassis cluster traceoptions
    OR
    delete chassis cluster traceoptions

• To deactivate paging (output stopping each page requiring you to press space bar) you can:

  • Run “set cli screen-length 0” for this to apply for all commands for your sessions
  • Add the “| no-more” option at the end of a command

  Data to Collect for all configurations:

Regardless of configuration, all cases will benefit by attaching the session captures, request information output, and logs when initially opening the case. If you need to investigate an intermittent concern (for example slow transfers at peak hours) please be sure to collect this data at the time of the problem.

All Configurations	Background information	Provide all SSH / Telnet session captures Provide any available topology information Summary of how the device is being used (production, lab system, co-location, etc) Summary of device history (new install, production for X months/years, other recent cases, etc) Summary of any recent changes in the network or on the device
	Request support info	Enter: request support information | save /var/log/rsi1.log Once step #1 completes, wait enough time to ensure that the condition you wish to address continues/appears before proceeding to the next step. Enter: request support information | save /var/log/rsi2.log
	Logs	Archive the /var/log/ contents:   file archive compress source /var/log/* destination /var/tmp/CURRENT-DATE.tgz To ensure the /var/log/ directory was properly archived, check the file size using the command: file list /var/tmp/CURRENT-DATE.tgz detail

See References section for the following:

• How to gather the data
• Resolution Guides and Troubleshooting Checklists
• Technical Bulletins

  Additional Data to Collect:

In addition, collect the data shown below for the following issues:

Jump to:

Chassis Cluster
Traffic failing for a specific host / application
High CPU
OSPF
BGP
Multicast
ALG
UTM - Anti-Virus
UTM - Anti-Spam
UTM - Web Filtering
UTM - Content Filtering
IPSec - Route Based
IPSec - Policy Based
IPSec - Dynamic VPNM
IPSec- NCP Exclusive Remove Access Client Connections
IDP - Security Package Update
IDP - Policy update after commit
IDP - Attack Detection
ATP - Advanced-Anti-Malware File Inspection
ATP - Security-Intelligence

Chassis Cluster	Show commands:	set cli timestamp show chassis fpc pic-status show chassis cluster status show chassis cluster interfaces show chassis cluster statistics show chassis cluster information show chassis cluster ip-monitoring status
	Logs -Each node:	/var/log/messages /var/log/jsrpd /var/log/chassisd
	Traceoptions: Caveats	set chassis cluster traceoptions file cluster.tr size 5m files 5 world-readable set chassis cluster traceoptions flag all
	Known Issues:	List PRs

[Back to Top]

Traffic failing for a specific host / application:	Show commands:	show security flow session summary show security flow session {source-prefix | destination-prefix | source-port | destination-port} <ip-prefix> extensive show security flow session session-identifier <session-id> (same output as above) show security flow cp-session summary show interface extensive show arp no-resolve (for locally connected hosts) traceroute <ip-prefix> (for failing host)
	Logs:	· None by default. · If security policy logs are enabled, check the configured log file for policy RT_FLOW events
	Traceoptions: Caveats	set security flow traceoptions file flow.trace set security flow traceoptions file size 5m set security flow traceoptions file files 5 set security flow traceoptions flag basic-datapath set security flow traceoptions flag packet-drops set security flow traceoptions packet-filter hostinit source-prefix a.a.a.a/32 set security flow traceoptions packet-filter hostinit destination-prefix b.b.b.b/32 set security flow traceoptions packet-filter hostresp source-prefix b.b.b.b/32 set security flow traceoptions packet-filter hostresp destination-prefix c.c.c.c/32 a.a.a.a - source address for initial traffic (use "inside/local" address if using source NAT) b.b.b.b - destination address for initial traffic and source address for return traffic c.c.c.c - destination address for return traffic (use "outside/global" address if using SRC NAT)
	Notes:	See the ALG sections for more application-specific details.

[Back to Top]

High CPU	CLI commands:	set cli timestamp show chassis routing-engine show system processes extensive show system users show system connections show system statistics show chassis forwarding show security monitor fpc pic <SPC-slot> (use 0 for Branch platforms) show security monitor performance spu show security monitor performance sess
	Logs:	None
	Traceoptions:	None

[Back to Top]

OSPF
	Show commands:	(If OSPF is running in a routing instance, specify which instance where applicable) set cli timestamp show ospf overview show ospf database show ospf neighbor detail show ospf route show ospf statistics show ospf interface show ospf log show route protocol ospf show route <x.x.x.x> extensive show ospf database extensive
	Logs:	/var/log/messages
	Traceoptions: Caveats	(use below for inet.0 default instance) set protocols ospf traceoptions file ospf.tr set protocols ospf traceoptions file size 5m set protocols ospf traceoptions file files 5 set protocols ospf traceoptions flag all (use below for routing instances) set routing-instances ospf-vr protocols ospf traceoptions file ospf-vr.tr set routing-instances ospf-vr protocols ospf traceoptions file size 5m set routing-instances ospf-vr protocols ospf traceoptions file files 5 set routing-instances ospf-vr protocols ospf traceoptions flag all
	Known Issues:	List PRs

[Back to Top]

BGP	Show commands:	set cli timestamp show bgp summary show bgp neighbor show route advertising-protocol bgp <neighbor-address> extensive show route receive-protocol bgp <neighbor-address> show route forwarding-table show route resolution unresolved
	Logs:	/var/log/messages
	Traceoptions: Caveats	set protocols bgp traceoptions file bgp.tr set protocols bgp traceoptions file size 5m set protocols bgp traceoptions file files 5 set protocols bgp traceoptions flag all set routing-instances bgp-vr protocols bgp traceoptions file bgp.tr set routing-instances bgp-vr protocols bgp traceoptions file size 5m set routing-instances bgp-vr protocols bgp traceoptions file files 5 set routing-instances bgp-vr protocols bgp traceoptions flag all
	Known Issues:	List PRs

[Back to Top]

Multicast	Show commands:	show multicast route show multicast statistics show multicast sessions show multicast usage show multicast interface show multicast next-hops show multicast rpf summary show interface <if-name> extensive show igmp group detail show igmp statistics show igmp interface detail show pim statistics show pim neighbors show pim rps detail show pim join extensive show pim bootstrap show msdp source-active show msdp detail show msdp statistics show route
	Logs:	/var/log/messages
	Traceoptions: Caveats	set routing-options multicast traceoptions file mcast.tr set routing-options multicast traceoptions file size 5m set routing-options multicast traceoptions file files 5 set routing-options multicast traceoptions flag all
	Known Issues:	List PRs

[Back to Top]

ALG	CLI commands:	set cli timestamp show security alg status show security alg <alg-name>    [obtain all sub-commands for the ALG in question,    use “?” to view options] show security resource-manager summary show security resource-manager resource active show security resource-manager resource active <number> show security resource-manager group active show security resource-manager group active <number> show security flow gate
	Logs:	none
	Traceoptions: Caveats	set security traceoptions file alg-sec.tr set security traceoptions file size 5m set security traceoptions file files 5 set security traceoptions file world-readable set security traceoptions flag all set security alg <alg-type> traceoptions flag all set security flow traceoptions file alg-flow.tr set security flow traceoptions file size 5m set security flow traceoptions file files 5 set security flow traceoptions file world-readable set security flow traceoptions flag basic-datapath set security flow traceoptions flag packet-drops set security flow traceoptions packet-filter alginit source-prefix a.a.a.a/32 set security flow traceoptions packet-filter alginit destination-prefix b.b.b.b/32 set security flow traceoptions packet-filter algresp source-prefix b.b.b.b/32 set security flow traceoptions packet-filter algresp destination-prefix c.c.c.c/32
	Notes:	See "Traffic failing for a specific host/application" for an example of packet filters for flow traceoptions.
	Known Issues:	List PRs

[Back to Top]

UTM - Anti-Virus	CLI commands:	set cli timestamp show system licenses show security utm status show security utm session show security utm anti-virus status detail show security utm anti-virus statistics show chassis routing-engine show system processes extensive Updating Full AV database: request security utm anti-virus kaspersky-lab-engine pattern-update Updating Express AV database: request security utm anti-virus juniper-express-engine pattern-update Updating Sophos AV database: request security utm anti-virus sophos-engine pattern update
	Logs:	/var/log/utmd /var/log/utmd-av
	Traceoptions: Caveats	set security utm traceoptions flag all set security utm application-proxy traceoptions flag all set security utm feature-profile anti-virus traceoptions flag all set security traceoptions file av.tr set security traceoptions file size 5m set security traceoptions file files 5 set security traceoptions file world-readable set security traceoptions flag all
	Known Issues:	List PRs

[Back to Top]

UTM - Anti-Spam	CLI commands:	set cli timestamp show system licenses show security utm status show security utm session show security utm anti-spam status show security utm anti-spam statistics show chassis routing-engine show system processes extensive
	Logs:	/var/log/utmd /var/log/utmd-as
	Traceoptions: Caveats	set security utm traceoptions flag all set security utm application-proxy traceoptions flag all set security utm feature-profile anti-spam traceoptions flag all set security traceoptions file as.tr set security traceoptions file size 5m set security traceoptions file files 5 set security traceoptions file world-readable set security traceoptions flag all
	Known Issues:	List PRs

[Back to Top]

UTM - Web Filtering	CLI commands:	set cli timestamp show system licenses show security utm status show security utm session show security utm web-filtering status show security utm web-filtering statistics show chassis routing-engine show system processes extensive
	Logs:	/var/log/utmd /var/log/utmd-wf
	Traceoptions: Caveats	set security utm traceoptions flag all set security utm application-proxy traceoptions flag all set security utm feature-profile web-filtering traceoptions flag all set security traceoptions file wf.tr set security traceoptions file size 5m set security traceoptions file files 5 set security traceoptions file world-readable set security traceoptions flag all
	Known Issues:	List PRs

[Back to Top]

UTM - Content Filtering	CLI commands:	set cli timestamp show system licenses show security utm status show security utm session show security utm content-filtering statistics
	Logs:	/var/log/utmd
	Traceoptions: Caveats	set security utm traceoptions flag all set security utm application-proxy traceoptions flag all set security utm feature-profile content-filtering traceoptions flag all set security traceoptions file cf.tr set security traceoptions file size 5m set security traceoptions file files 5 set security traceoptions file world-readable set security traceoptions flag all
	Known Issues:	List PRs

[Back to Top]

IPsec VPN - Route-Based	Show commands:	show security ike security-association show security ike security-association index <#> detail show security ipsec security-association show security ipsec security-association index <#> detail show security ipsec statistics show security ipsec statistics index <#> show security ipsec next-hop-tunnels monitor interface st0.x show interfaces extensive st0.x show security flow session tunnel show route show security pki local-cert detail show security pki ca-cert detail show security pki crl detail
	Logs:	/var/log/kmd* /var/tmp/kmd* (SRX 1400 and higher)
	Traceoptions: Caveats	set security ike traceoptions file vpn.tr size 5m files 5 world-readable set security ike traceoptions flag ike set security ike traceoptions flag general set security ipsec traceoptions flag security-associations set security ipsec traceoptions flag packet-drops set security ipsec traceoptions flag packet-processing
	Notes:	If tunnels are up but traffic not passing, see section “Traffic failing for a specific host/application” and setup packet filters for outer ESP traffic as well as inner application/host traffic.

[Back to Top]

IPsec VPN - Policy-Based	Show commands:	show security ike security-association show security ike security-association index <#> detail show security ipsec security-association show security ipsec security-association index <#> detail show security ipsec statistics show security ipsec statistics index <#> show security ipsec next-hop-tunnels show security flow session tunnel IF PKI certs are used: show security pki local-cert detail show security pki ca-cert detail show security pki crl detail show security policies detail show log /var/etc/policy.id
	Logs:	/var/log/kmd* /var/tmp/kmd* (SRX 1400 and higher)
	Traceoptions: Caveats	set security ike traceoptions file vpn.tr size 5m files 5 world-readable set security ike traceoptions flag ike set security ike traceoptions flag general set security ipsec traceoptions flag security-associations set security ipsec traceoptions flag packet-drops set security ipsec traceoptions flag packet-processing
	Notes:	If tunnels are up but traffic not passing, see section “Traffic failing for a specific host/application” and setup packet filters for outer ESP traffic as well as inner application/host traffic.

[Back to Top]

IPsec - Dynamic VPN	Show commands:	show security ike security-association show security ike security-association index <number> detail show security ike active-peer show security ipsec security-association show security ipsec security-association index <id> show security ipsec statistics show security dynamic-vpn client version show security dynamic-vpn users detail show system license
	Logs:	SRX: /var/log/kmd /var/log/httpd.log /var/log/authd Pulse client:   File > Logs > Log level > (detailed / normal) File > Logs > Save as > <filename>
	Traceoptions: Caveats	set system processes general-authentication-servic traceoptions file dynvpn-auth.tr set system processes general-authentication-servic traceoptions file size 5m set system processes general-authentication-servic traceoptions file files 5 set system processes general-authentication-servic traceoptions file world-readable set system processes general-authentication-servic traceoptions flag all set security ike traceoptions file dynvpn.tr size 5m files 5 world-readable set security ike traceoptions flag ike set security ike traceoptions flag general set security ipsec traceoptions flag security-associations set security ipsec traceoptions flag packet-drops set security ipsec traceoptions flag packet-processing
	Notes:	If tunnels are up but traffic not passing, see section “Traffic failing for a specific host/application” and setup packet filters for outer ESP traffic as well as inner application/host traffic. The Pulse client version is also helpful for troubleshooting. For more info on how to get it, see KB22857 - How to: Find the Junos Pulse version in the SRX and in the Pulse client itself.
	Known Issues:	List PRs

[Back to Top]

IPSec VPN - NCP Exclusive Remote Access Client Connections	Show commands:	show security ike active-peer show security ike security-association show security ike security-association index <#> detail show security ipsec security-association show security ipsec security-association index <#> detail show security ipsec tunnel-events-statistics show security ipsec statistics show security ipsec statistics index <#> show interfaces extensive st0.x show security flow session tunnel show route show security pki local-cert detail show security pki ca-cert detail show security pki crl detail show network-access requests statistics show system license
	Logs:	/var/log/kmd*
	Traceoptions: Caveats	set security ike traceoptions file vpn.tr size 5m files 5 world-readable set security ike traceoptions flag ike set security ike traceoptions flag general set security ipsec traceoptions flag security-associations set security ipsec traceoptions flag packet-drops set security ipsec traceoptions flag packet-processing
	Notes:	If tunnels are up but traffic not passing, see section “Traffic failing for a specific host/application” and setup packet filters for outer ESP traffic as well as inner application/host traffic.

[Back to Top]

IDP - Security Package Update	Show commands:	show security idp security-package-version show security idp status show security idp memory request security idp security-package download request security idp security-package download status request security idp security-package install request security idp security-package install status
	Logs:	show log messages show log idpd show log idp-traceoptions
	Traceoptions: Caveats	edit security idp traceoptions set file idp-traceoptions set flag all set level all edit security flow traceoptions set file flow-trace set flag basic-datapath set flag packet-drops set packet-filter 1 … ALWAYS CONFIGURE PACKET-FILTERS

[Back to Top
 

IDP - Attack Detection	Show commands:	show security idp security-package-version show security idp status show security idp counters flow show security idp counters application-identification show security idp counters flow show security idp counters log show security idp counters packet show security idp memory show security idp application-statistics show security idp attack table
	Notes:	Latency/Performance: Change IDP policy to one of the predefined template IDP policies, like Recommended Policy, to verify if this is a customer IDP policy issue. Datasheet benchmarks are based on IDP Recommended Policy. False Positives/Negatives: Gather: 1) Packet capture of the False Positive/Negative 2) IDP signature that is causing issue 3) show security idp security-package-version 4) show configuration security idp | display set Contact signatures@juniper.net with above info

[Back to Top]

ATP - Advanced-Anti-Malware File Inspection	CLI commands:	set cli timestamp show services advanced-anti-malware status request services advanced-anti-malware diagnostic <url> detail request services advanced-anti-malware data-connection test start <packet-size> request services advanced-anti-malware data-connection test status show security pki local-certificate detail show security pki ca-certificate detail show services advanced-anti-malware statistics show services advanced-anti-malware profile show services advanced-anti-malware policy show services ssl proxy statistics
	Logs:	/var/log/messages /var/log/aamw_traceoptions
	Traceoptions: Caveats	edit services advanced-anti-malware traceoptions set file size 20m set file files 10 set file aamw_traceoptions set flag all set level all
	Notes:	Ensure the clock is correct or configure NTP vSRX requires a Sky ATP license installed on device for ATP Cloud enrollment

[Back to Top]

ATP - Security-Intelligence	CLI commands:	set cli timestamp request services security-intelligence download status show configuration services security-intelligence url show services advanced-anti-malware status show services security-intelligence update status show services security-intelligence statistics show services security-intelligence category summary show services security-intelligence statistics show security pki local-certificate detail show security pki ca-certificate detail show services ssl proxy statistics
	Logs:	/var/log/messages /var/log/secintel_traceoptions
	Traceoptions: Caveats	edit services security-intelligence traceoptions set file size 20m set file files 10 set file secintel_traceoptions set flag all set level all
	Notes:	Ensure the clock is correct or configure NTP vSRX requires a Sky ATP license installed on device for ATP Cloud enrollment

[Back to Top]
 

References:

How to:

• Tar / Zip / Archive data collection - KB17228 - [Junos] How to collect /var/log files from Routing Engine (RE)
• Upload collected data:
  • Files smaller than 10MB can be attached to the case
  • Files greater than 10MB, refer to KB23337 - How to upload large files to a JTAC Case   

Resolution Guides and Troubleshooting Checklists:
JTAC Certified step-by-step troubleshooting flowcharts and articles

Resolution Guides and Articles - SRX - NAT

Resolution Guides and Articles - SRX - High Availability (Chassis Cluster)
Resolution Guides and Articles - SRX - VPN
Resolution Guide - SRX - Troubleshooting IDP
Troubleshoot Dynamic VPN or Access Manager client that is not working (Start here)

Troubleshooting Checklist - SRX - UTM
Troubleshooting Checklist - SRX - DHCP
Troubleshooting Checklist - SRX - RADIUS

Technical Bulletins:  SRX Series (login to see more)
For more information on Technical Bulletins, see KB9890 - How do I subscribe to a technical bulletin so I can I get email alerts regarding product issues, new product release announcements and security or safety issues.

2020-07-02: Added ATP Advanced Anti Malware File Inspection and Sec-Intel
2020-03-30: Removed reference to outdated J-Series article.
2017-05-01: Added-IPsec VPN -NCP Exclusive Remote Access Client Connection.