Data Collection and Troubleshooting Guides can help with issue investigation as well as reducing resolution time. Each issue might require a different set of data to collect. This article contains a list of data to collect, as well as pointers to Resolution Guides and references on how to collect the data.
On a EX Series Switch or a QFX Series Switch, what information should I collect to assist in troubleshooting, prior to opening a case?
The goal of this document is to reduce the time spent on initial data collection and reduce resolution time by providing a comprehensive list of what to collect or gather to troubleshoot an issue.
Note: Your feedback is welcome on the format of the Data Collection Checklist. Use the feedback form on the right.
This section contains the following:
Caveats and tips: - Traceoptions require additional system resources to gather and store data:
- Please ensure you have enough disk space when enabling traceoptions
- Gauge current system utilization before enabling traceoptions
"show chassis routing-engine"
- Instead of using "flag all" you can flag specific areas of interest
- Delete all traceoptions that are not needed for immediate debugging
- Don't forget to remove traceoptions after data collection is completed
- To deactivate paging (output stopping each page requiring you to press space bar) you can:
- Run "set cli screen-length 0" for this to apply for all commands for your sessions
- Add the "| no-more" option at the end of a command
Data to Collect for all configurations:
Regardless of configuration, all cases will benefit by attaching the session captures, request information output, and logs when initially opening the case. If you need to investigate an intermittent concern (for example slow transfers at peak hours) please be sure to collect this data at the time of the problem.
| All Configurations | Background information | - Provide all SSH / Telnet session captures
- Provide any available topology information
- Summary of how the device is being used (production, lab system, co-location, etc)
- Summary of device history (new install, production for X months/years, other recent cases, etc)
- Summary of any recent changes in the network or on the device
|
| Request support info | - Enter:
request support information | save /var/log/rsi1.log - Once step #1 completes, wait enough time to ensure that the condition you wish to address continues/appears before proceeding to the next step.
- Enter:
request support information | save /var/log/rsi2.log |
| Logs | Archive the /var/log/ contents:
file archive compress source /var/log/* destination /var/tmp/CURRENT-DATE.tgz
To ensure the /var/log/ directory was properly archived, check the file size using the command: file list /var/tmp/CURRENT-DATE.tgz detail
|
See
References section for the following:
Additional Data to Collect:
In addition, collect the data shown below for the following issues:
Jump to:
Aggregated - LACP
BFD
BGP
CoS
DHCP
dot1X
ERP (protection-group)
EX8200 Fabric Plane
Firewall Filters
High CPU
IPv6
ISIS
L2 VPN
L3 VPN
MPLS
Multicast
NTP
OAM
OSPF
PoE
RIP
SNMP
Spanning Tree
Virtual Chassis
VLAN
VRRP
Aggregated link
(LACP)
| CLI commands: | show interfaces terse | match ae
show interfaces statistics
show ethernet-switching table | match xx-x/x/x
show lacp interfaces
show lacp statistics interfaces xx-x/x/x
show interface extensive
|
| Logs: | /var/log/messages |
| Traceoptions: | set protocols lacp traceoptions file lacp.tr
set protocols lacp traceoptions file size 5m
set protocols lacp traceoptions file files 5
set protocols lacp traceoptions flag all |
| Questions: | - What device is connected at the other end?
- Is link protection enabled?
- What are the troubleshooting steps so far?
- Did the issue start after any kind of device or config change? |
[
Back to Top]
BFD
| CLI commands: | show bfd session brief
show bfd session
show bfd session summary
show bfd session detail
show bfd session discriminator brief
show bfd session discriminator detail address |
| Logs: | /var/log/messages |
| Traceoptions: | set protocols bfd traceoptions file bfd.tr
set protocols bfd traceoptions file size 5m
set protocols bfd traceoptions file files 5
set protocols bfd traceoptions flag all |
[
Back to Top]
| BGP | Show commands: | set cli timestamp
show bgp summary
show bgp group
show bgp neighbor
show route next-hop
show route protocol bgp all
show route receive-protocol bgp < peer address> extensive
show route advertising-protocol bgp extensive
show route forwarding-table
show route resolution unresolved
show route community
show route damping [history|decayed|surpressed]
show route aspath-regex
monitor traffic interface XX-X/X/X size 4096 detail
|
| Logs: | /var/log/messages |
Traceoptions:
Caveats | set protocols bgp traceoptions file bgp.tr
set protocols bgp traceoptions file size 5m
set protocols bgp traceoptions file files 5
set protocols bgp traceoptions flag all
set routing-instances bgp-ri protocols bgp traceoptions file bgp.tr
set routing-instances bgp-ri protocols bgp traceoptions file size 5m
set routing-instances bgp-ri protocols bgp traceoptions file files 5
set routing-instances bgp-ri protocols bgp traceoptions flag all |
[
Back to Top]
| CoS | CLI commands: | show interface xx-x/x/x detail
show interface queue xx-x/x/x both-ingress-egress
show interfaces XX-X/X/X extensive
show class-of-service interfaces XX-X/X/X
show class-of-service code-point-aliases
show class-of-service code-point-aliases dscp
show class-of-service code-point-aliases inet-prec
show class-of-service code-point-aliases exp
show class-of-service forwarding-class
show class-of-service classifier [name]
show class-of-service scheduler-map
show class-of-service rewrite-rule
show class-of-service drop-profile
show class-of-service forwarding-table
show class-of-service forwarding-table classifier mapping
show class-of-service forwarding-table scheduler-map
|
| Logs: | /var/log/messages |
| Traceoptions: | set class-of-service traceoptions file class-of-service.tr
set class-of-service traceoptions file size 5m
set class-of-service traceoptions file files 5
set class-of-service traceoptions flag all |
[
Back to Top]
DHCP
| CLI commands: | show configuration | display set | match dhcp
show system services dhcp binding detail
show system services dhcp client
show system services dhcp pool detail
show interfaces terse
show dhcp snooping binding vlan
show dhcp snooping binding interface xx-x/x/x
monitor traffic interface xx-x/x/x
|
| Logs: | /var/log/messages |
| Traceoptions: | none |
| Questions: | - Any troubleshooting steps so far?
- Confirm whether the discover packet is reaching the switch?
- Is the switch replying with an offer packet?
- Any packet captures taken at the interface ends? |
[
Back to Top]
dot1X
| CLI commands: | show dot1x authentication-bypassed-users
show dot1x authentication-failed-users
show dot1x static-mac-address
show dot1x statistics
show dot1x interface brief
show dot1x interface detail
|
| Logs: | /var/log/messages |
| Traceoptions: | set protocols dot1x traceoptions file dot1x.tr
set protocols dot1x traceoptions file size 5m
set protocols dot1x traceoptions file files 5
set protocols dot1x traceoptions flag all |
[
Back to Top]
ERP (protection-group)
| CLI commands: | show configuration protocols protection-group
show ethernet-switching interfaces xx-x/x/x
show protection-group ethernet-ring interface brief
show protection-group ethernet-ring interface detail
show protection-group ethernet-ring node-state
show protection-group ethernet-ring statistics
show protection-group ethernet-ring configuration
|
| Logs: | /var/log/messages |
| Traceoptions: | set protocols protection-group traceoptions file protection-group.tr
set protocols protection-group traceoptions file size 5m
set protocols protection-group traceoptions file files 5
set protocols protection-group traceoptions flag all |
[
Back to Top]
EX 8200
Fabric
Plane
| CLI commands: | General:
show chassis environment cb
show chassis alarms
Fabric related:
show chassis hardware
show chassis fabric summary
show chassis fabric map
show chassis fabric fpc
show chassis fabric plane
show class-of-service fabric statistics (take multiple outputs to see counters incrementing)
show pfe statistics traffic (take multiple outputs to see counters incrementing)
|
| Logs: | show log messages
show log chassisd |
Hidden Commands:
Caveats | NOTE: These commands are not officially supported, and are hidden, albeit are good for troubleshooting Fabric Plane issues.
Please exercise caution when executing this commands, as incorrect syntax may lead to problems.
show chassis fabric statistics detail
show chassis fabric asic-configuration plane
|
| Questions: | - Have the SCB's for which the alarms have generated had any previous history of any issues?
- What kind of troubleshooting has already been done to mitigate the problem. Please provide detailed steps.
- Were there any recent changes to the configuration?
- Were there any changes to the hardware setup, or was new hardware introduced?
- Were there any events that occurred just before the log message that may have triggered it? This may include events such as routing engine switchover etc.
- Was a software upgrade/downgrade done recently? |
[
Back to Top]
| Firewall Filters | CLI commands: | show interfaces filters
show firewall
show firewall filter
show firewall log
show policy conditions
|
| Logs: | /var/log/messages |
| Traceoptions: | None |
[
Back to Top]
High CPU
| CLI commands: | set cli timestamp
show chassis routing-engine (multiple snapshots, atleast 5)
show system processes extensive (multiple snapshots atleast 5)
show system users
show system connections
show system statistics
show task memory detail
show task memory summary
show task io
show task history
show task statistics
show task job
show task jobs
show krt queue
show krt state
|
| Logs: | /var/log/messages |
| Traceoptions: | user@router# show routing-options
traceoptions {
file routing-trace size 10m files 20 world-readable;
flag task;
flag state;
flag timer;
} |
[
Back to Top]
| IPv6 | CLI commands: | show interface terse
show route table inet6
show ipv6 neighbor
show interface lo0 extensive | display xml |
| Logs: | /var/log/messages |
| Traceoptions: | None |
[
Back to Top]
| ISIS | Show commands: | set cli timestamp
show isis adjacency
show isis adjacency extensive
show isis interface
show isis hostname
show isis spflog
show isis statistics
show isis route
show route protocol isis
show isis database
show isis database detail
show bfd session
show route forwarding-table
|
| Logs: | /var/log/messages |
Traceoptions:
Caveats | set protocols isis traceoptions file rip.tr
set protocols isis traceoptions file size 5m
set protocols isis traceoptions file files 5
set protocols isis traceoptions flag all
set routing-instances isis-vr protocols isis traceoptions file isis.tr
set routing-instances isis-vr protocols isis traceoptions file size 5m
set routing-instances isis-vr protocols isis traceoptions file files 5
set routing-instances isis-vr protocols isis traceoptions flag all |
[
Back to Top]
| L2 VPN | CLI commands: | show l2vpn connections extensive
show route table
show route advertising-protocol bgp x.x.x.x
show route receive-protocol bgp x.x.x.x
show route table mpls.o
show route forwarding-table family mpls |
| Logs: | /var/log/messages |
| Traceoptions: | None |
[
Back to Top]
| L3 VPN | CLI commands: | show route table
show route table hidden
show route forwarding-table vpn
ping x.x.x.x routing-instance
traceroute x.x.x.x routing-instance
telnet x.x.x.x routing-instance
ping mpls l3vpn prefix
show route advertising-protocol bgp x.x.x.x
show route receive-protocol bgp x.x.x.x
show arp
OSPF VPN commands
show ospf interface instance
show ospf neighbor instance
show ospf database instance
|
| Logs: | /var/log/messages |
| Traceoptions: | None |
[
Back to Top]
| MPLS | Show commands: | show mpls interface
show mpls lsp
show mpls lsp extensive
show mpls lsp ingress
show mpls lsp transit
show ted database
RSVP
show rsvp interface
show rsvp neighbor
show rsvp session
show rsvp session ingress
show rsvp version
show route table mpls.0
show route table inet.3
LDP
show ldp neighbors
show ldp session
show ldp database [session peer]
|
| Logs: | /var/log/messages |
Traceoptions:
Caveats | set protocols mpls|ldp|rsvp traceoptions file mpls.tr
set protocols mpls|ldp|rsvp traceoptions file size 5m
set protocols mpls|ldp|rsvp traceoptions file files 5
set protocols mpls|ldp|rsvp traceoptions flag all
set protocols mpls label-switched-path traceoptions file mpls-lsp.tr
set protocols mpls label-switched-path traceoptions file size 5m
set protocols mpls label-switched-path traceoptions file files 5
set protocols mpls label-switched-path traceoptions flag all
|
[
Back to Top]
| Multicast | Show commands: | General
show multicast route
show multicast sessions
show multicast usage
show multicast interface
show multicast next-hops
show multicast rpf summary
show interface extensive
IGMP
show igmp group detail
show igmp statistics
show igmp interface detail
PIM
show pim statistics
show pim neighbors
show pim rps detail
show pim join extensive
show pim bootstrap
show pim source detail
MSDP
show msdp source-active
show msdp detail
show msdp statistics
|
| Logs: | /var/log/messages |
Traceoptions:
Caveats | set protocols pim|igmp|msdp traceoptions file multicast-proto.tr
set protocols pim|igmp|msdp traceoptions file size 5m
set protocols pim|igmp|msdp traceoptions file files 5
set protocols pim|igmp|msdp traceoptions flag all
enable traceoptions for either of PIM, IGMP, MSDP
|
[
Back to Top]
NTP
| CLI commands: | show ntp associations
show ntp status
show version
show system uptime
|
| Logs: | /var/log/messages |
| Traceoptions: | none |
[
Back to Top]
OAM
| CLI commands: | show configuration protocols oam
show oam ethernet link-fault-management brief
show oam ethernet link-fault-management detail
show oam ethernet link-fault-management vlan detail
show oam ethernet connectivity-fault-management adjacencies
show oam ethernet connectivity-fault-management interfaces brief
show oam ethernet connectivity-fault-management interfaces detail
show oam ethernet connectivity-fault-management interfaces vlan
|
| Logs: | /var/log/messages |
| Traceoptions: | set protocol oam ethernet link-fault-management traceoptions file link-fault.tr
set protocol oam ethernet link-fault-management traceoptions file size 5m
set protocol oam ethernet link-fault-management traceoptions file files 5
set protocol oam ethernet link-fault-management traceoptions flag all
set protocol oam ethernet connectivity-fault-management traceoptions file connect-fault.tr
set protocol oam ethernet connectivity-fault-management traceoptions file size 5m
set protocol oam ethernet connectivity-fault-management traceoptions file files 5
set protocol oam ethernet connectivity-fault-management traceoptions flag all |
[
Back to Top]
| OSPF |
| Show commands: | (If OSPF is running in a routing instance, specify which instance where applicable) set cli timestamp
show ospf overview
show ospf database
show ospf neighbor detail
show ospf route
show ospf statistics
show ospf interface (brief | detail | extensive)
show ospf log
show route protocol ospf
show route extensive
show ospf database summary
show ospf database extensive
show bfd session
|
| Logs: | /var/log/messages |
Traceoptions:
Caveats | (use below for inet.0 default instance)
set protocols ospf traceoptions file ospf.tr
set protocols ospf traceoptions file size 5m
set protocols ospf traceoptions file files 5
set protocols ospf traceoptions flag all
(use below for routing instances)
set routing-instances ospf-vr protocols ospf traceoptions file ospf-vr.tr
set routing-instances ospf-vr protocols ospf traceoptions file size 5m
set routing-instances ospf-vr protocols ospf traceoptions file files 5
set routing-instances ospf-vr protocols ospf traceoptions flag all |
| Known Issues: | NA |
[
Back to Top]
PoE
| CLI commands: | show configuration poe
show poe controller
show poe interface
show interfaces terse
show poe notification-control
show poe telemetries interface xx-x/x/x all
|
| Logs: | /var/log/messages |
| Questions: | - Did the device face any power outage?
- Are the poe devices currently connected? If so, which all ports?
- Details of the POE devices connected? |
[
Back to Top]
| RIP | Show commands: | set cli timestamp
show rip neighbor
show route protocol rip
show route advertising-protocol rip extensive
show route receive-protocol rip extensive
show rip statistics
show route forwarding-table
show route resolution unresolved
|
| Logs: | /var/log/messages |
Traceoptions:
Caveats | set protocols rip traceoptions file rip.tr
set protocols rip traceoptions file size 5m
set protocols rip traceoptions file files 5
set protocols rip traceoptions flag all
set routing-instances rip-vr protocols rip traceoptions file rip.tr
set routing-instances rip-vr protocols rip traceoptions file size 5m
set routing-instances rip-vr protocols rip traceoptions file files 5
set routing-instances rip-vr protocols rip traceoptions flag all |
[
Back to Top]
| SNMP | CLI commands: | set cli timestamp
show configuration snmp
show snmp statistics # Multiple time with polling activity.
show snmp mib walk
show interface lo0 extensive
show system processes extensive | no-more # Multiple time with polling activity.
show chassis routing-engine | no-more # Multiple time with polling activity.
show system commit | no-more
show system statistics # Multiple time with polling activity.
show system buffers # Multiple time with polling activity.
show system core-dumps
show route
ping source count 500 rapid
show pfe statistics traffic # Multiple time with polling activity.
show interface extensive | no-more
|
| Logs: | /var/log/messages |
| Traceoptions: | set snmp traceoptions file snmp files 10 size 50m
set snmp traceoptions flag all |
[
Back to Top]
STP
(Spanning
Tree
Protocol)
| CLI commands: | show configuration protocols
show spanning-tree bridge brief
show spanning-tree bridge detail
show spanning-tree statistics
show spanning-tree interface
show ethernet-switching mac-learning-log
|
| Logs: | /var/log/messages |
| Traceoptions: | set protocols stp traceoptions file stp.tr
set protocols stp traceoptions file size 5m
set protocols stp traceoptions file files 5
set protocols stp traceoptions flag all |
| Questions: | - What kind of spanning tree flavour is running?
- Is the customer using other vendor device with in the same spanning tree ?
- Is the customer using same spanning tree flavour on all the switches?
- Any recent configuration changes related to cost, bridge priority, storm control, BPDU block?
- Is there any hub connected to any of the switches? |
[
Back to Top]
VLAN
| CLI commands: | show configuration vlans
show vlans brief
show vlans
show vlans summary
show interfaces terse | match in
|
| Logs: | /var/log/messages |
| Traceoptions: | none |
| Questions: | Did the issue start after any configuration/ device change?
What are the devices connected? |
[
Back to Top]
VRRP
| CLI commands: | show configuration protocols vrrp
show vrrp brief
show vrrp detail
show vrrp interface xx-x/x/x
show vrrp track summary
show vrrp track detail
|
| Logs: | /var/log/messages |
| Traceoptions: | set protocol vrrp traceoptions file vrrp.tr
set protocol vrrp traceoptions file size 5m
set protocol vrrp traceoptions file files 5
set protocol vrrp traceoptions flag all |
| Questions: | - Any new device was added in to the network?
- What are the troubleshooting steps so far?
- Any recent configuration changes?
- Any spannning tree relates issues ?
- Are the hello packets are being recieved on the other end? |
[
Back to Top]
Virtual Chassis
| CLI commands: | show version
show configuration virtual-chassis
show virtual-chassis status
show virtual-chassis vc-port all-members
show virtual-chassis mode
show virtual-chassis protocol interface
show virtual-chassis protocol statistics
|
| Logs: | /var/log/messages |
| Traceoptions: | set virtual-chassis traceoptions file virtual-chassis.tr
set virtual-chassis traceoptions file size 5m
set virtual-chassis traceoptions file files 5
set virtual-chassis traceoptions flag all |
| Questions: | - Any troubleshooting steps so far?
- Did the issue begin after any device or config change?
- Is the issue happening again and again? |
[
Back to Top]
References:
How to: Resolution Guides:
KB9936 - JTAC Certified step-by-step troubleshooting flowcharts and articles Technical Bulletins: EX Series (login to see more)
For more information on Technical Bulletins, see
KB9890 - How do I subscribe to a technical bulletin so I can I get email alerts regarding product issues, new product release announcements and security or safety issues?.