Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[M/MX/PTX/T] Data Collection Checklist - Logs/data to collect for troubleshooting

0

0

Article ID: KB22637 KB Last Updated: 16 Jul 2020Version: 18.0
Summary:

Data Collection and Troubleshooting Guides can help with issue investigation as well as reduce resolution time. Each issue might require a different set of data to collect. This article contains a list of data to collect, as well as pointers to Resolution Guides and references on how to collect the data. The goal is to reduce the time spent on initial data collection and reduce resolution time by providing a comprehensive list of what to collect or gather to troubleshoot an issue.

Solution:

This section contains the following:

Caveats and tips:

  • Traceoptions require additional system resources to gather and store data:

    • Please ensure you have enough disk space when enabling traceoptions.
    • Gauge current system utilization before enabling traceoptions:  show chassis routing-engine
    • Instead of using “flag all” you can flag specific areas of interest.
    • Delete all traceoptions that are not needed for immediate debugging.
    • Don’t forget to remove traceoptions after data collection is completed.
  • To deactivate paging (output stopping each page requiring you to press space bar) you can:

    • Run “set cli screen-length 0” for this to apply for all commands for your sessions.
    • Add the “| no-more” option at the end of a command.

 

Step 1. Data to Collect for all configurations:

Regardless of configuration, all cases will benefit by attaching the session captures, request information output, and logs when initially opening the case. If you need to investigate an intermittent concern (for example slow transfers at peak hours) please be sure to collect this data at the time of the problem.

All Configurations

Background Information

  1. Provide all SSH / Telnet session captures.
  2. Provide any available topology information.
  3. Summary of how the device is being used (production, lab system, co-location, etc).
  4. Summary of device history (new install, production for X months/years, other recent cases, etc).
  5. Summary of any recent changes in the network or on the device.

Request Support Information

  1. Enter: request support information | save /var/log/rsi1.log

  2. After step #1 completes, wait enough time to ensure that the condition you wish to address continues/appears before proceeding to the next step.

  3. Enter: request support information | save /var/log/rsi2.log

Logs


Archive the /var/log/ contents:

file archive compress source /var/log/* destination /var/tmp/CURRENT-DATE.tgz

To ensure the /var/log/ directory was properly archived, check the file size using the command: file list /var/tmp/CURRENT-DATE.tgz detail
 


See References section for the following:


Step 2. Additional Data to Collect:

In addition, collect the data shown below for the following issues:

Jump to:

BGP
OSPF
RIP
ISIS
Multicast
MPLS
VPLS
L3 VPN
L2 VPN
CoS
Firewall Filters
IPv6
SNMP
High CPU
Fabric Plane (SBC, FChip, Plane)
NG-MPVN
CGNAT
Spanning Tree
BFD

BGP

Show Commands


set cli timestamp
show bgp summary
show bgp group 
show bgp neighbor
show route next-hop database
show route protocol bgp
show route protocol bgp terse
show route receive-protocol bgp
show route advertising-protocol bgp
show route advertising-protocol bgp <neighbor-address> extensive
show route receive-protocol bgp <neighbor-address>
show route forwarding-table
show route resolution unresolved
show route hidden
show route community <reg-ex>
show route terse community-name <name>
show route damping
show route aspath-regex "regex"
show route damping [history|decayed|surpressed]
monitor traffic interface XX-X/X/X size 4096 detail

 

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

Caveats


set protocols bgp traceoptions file bgp.tr
set protocols bgp traceoptions file size 5m
set protocols bgp traceoptions file files 5
set protocols bgp traceoptions flag all
set routing-instances bgp-ri protocols bgp traceoptions file bgp.tr
set routing-instances bgp-ri protocols bgp traceoptions file size 5m
set routing-instances bgp-ri protocols bgp traceoptions file files 5
set routing-instances bgp-ri protocols bgp traceoptions flag all

 

[Back to Top]

OSPF

Show Commands


If OSPF is running in a routing instance, specify which instance, where applicable.

set cli timestamp
show ospf overview
show ospf database
show ospf neighbor detail
show ospf route
show ospf statistics
show ospf interface
show ospf log
show route protocol ospf
show route <x.x.x.x> extensive
show ospf database summary 
show ospf database extensive
show bfd session

 

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

Caveats


Use below for inet.0 default instance:
set protocols ospf traceoptions file ospf.tr
set protocols ospf traceoptions file size 5m
set protocols ospf traceoptions file files 5
set protocols ospf traceoptions flag all

Use below for routing instances:
set routing-instances ospf-vr protocols ospf traceoptions file ospf-vr.tr

set routing-instances ospf-vr protocols ospf traceoptions file size 5m

set routing-instances ospf-vr protocols ospf traceoptions file files 5

set routing-instances ospf-vr protocols ospf traceoptions flag all

Known Issues

N/A

[Back to Top]

RIP

Show Commands


set cli timestamp
show rip overview
show rip neighbor
show route protocol rip
show route advertising-protocol rip <neighbor-address> extensive
show route receive-protocol rip <neighbor-address>
show rip statistics
show route forwarding-table 
show route resolution unresolved

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

Caveats


set protocols rip traceoptions file rip.tr
set protocols rip traceoptions file size 5m
set protocols rip traceoptions file files 5
set protocols rip traceoptions flag all

set routing-instances rip-vr protocols rip traceoptions file rip.tr
set routing-instances rip-vr protocols rip traceoptions file size 5m
set routing-instances rip-vr protocols rip traceoptions file files 5
set routing-instances rip-vr protocols rip traceoptions flag all

[Back to Top]

ISIS

Show Commands


set cli timestamp
show isis adjacency
show isis adjacency extensive
show isis interface
show isis interface details
show isis hostname
show isis spflog
show isis statistics 
show isis route 
show route protocol isis
show isis database
show isis database detail
show bdf session
show route forwarding-table

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

Caveats


set protocols isis traceoptions file rip.tr  
set protocols isis traceoptions file size 5m  
set protocols isis traceoptions file files 5 
set protocols isis traceoptions flag all 

set routing-instances isis-vr protocols isis traceoptions file isis.tr
set routing-instances isis-vr protocols isis traceoptions file size 5m
set routing-instances isis-vr protocols isis traceoptions file files 5
set routing-instances isis-vr protocols isis traceoptions flag all

[Back to Top]

Multicast

Show Commands


General
show multicast route   
show multicast statistics
show multicast sessions
show multicast usage
show multicast interface
show multicast next-hops
show multicast rpf summary
show interface <if-name> extensive


IGMP
show igmp group detail
show igmp statistics
show igmp interface detail


PIM
show pim statistics
show pim neighbors
show pim rps detail
show pim join extensive
show pim bootstrap
show pim source detail


MSDP
show msdp source-active
show msdp detail
show msdp statistics

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

Caveats


set protocols pim|igmp|msdp traceoptions file multicast-proto.tr
set protocols pim|igmp|msdp traceoptions file size 5m
set protocols pim|igmp|msdp traceoptions file files 5
set protocols pim|igmp|msdp traceoptions flag all

Enable traceoptions for PIM, IGMP, or MSDP.

[Back to Top]

MPLS

Show Commands


show mpls interface 
show mpls lsp
show mpls lsp extensive
show mpls lsp ingress
show mpls lsp transit
show ted database


RSVP
show rsvp interface
show rsvp neighbor
show rsvp session
show rsvp session ingress
show rsvp version
show route table mpls.0
show route table inet.3


LDP
show ldp neighbors
show ldp session
show ldp database [session peer]

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

Caveats


set protocols mpls|ldp|rsvp traceoptions file mpls.tr
set protocols mpls|ldp|rsvp traceoptions file size 5m
set protocols mpls|ldp|rsvp traceoptions file files 5
set protocols mpls|ldp|rsvp traceoptions flag all

set protocols mpls label-switched-path <lsp-name> traceoptions file mpls-lsp.tr

set protocols mpls label-switched-path <lsp-name> traceoptions file size 5m

set protocols mpls label-switched-path <lsp-name> traceoptions file files 5

set protocols mpls label-switched-path <lsp-name> traceoptions flag all

 

[Back to Top]

VPLS

Show Commands


show vpls connections extensive
show bgp summary
show route table <instance-name>.l2vpn.0 extensive
show route table bgp.l2vpn.0 extensive
show route table mpls.0 extensive
show route table inet.3 extensive
show vpls flood extensive
show vpls mac-table
show interfaces terse
show interfaces routing
show route forwarding table family mpls extensive
show route forwarding table family vpls extensive
show interface ge-x/y/z extensive | no-more
show l2-learning interface
show l2-learning mac-move-buffer extensive
show l2-learning l2alm-peers
show l2-learning debug-statistics events
show l2-learning debug-statistics ipc
show l2-learning debug-statistics mac-events
show l2-learning debug-statistics mac-messages
show l2-learning debug-statistics mac-processing
show l2-learning debug-statistics performance-counters
show l2-learning debug-statistics rtsock
show bridge statistics
show vpls statistics
show bridge mac-table instance <instance-name> extensive
show vpls mac-table instance <instance-name> extensive


Determine if traffic is getting the VPLS interface, apply the following Firewall filter in the VPLS interface as input and output:

set firewall family vpls filter jtac-test term t1 from destination-mac-address <>
set firewall family vpls filter jtac-test term t1 from source-mac-address <>
set firewall family vpls filter jtac-test term t1 then count counter_in
set firewall family vpls filter jtac-test term t1 then accept
set firewall family vpls filter jtac-test term t2 then accept

root@root> show firewall filter jtac-test
 
Filter: jtac-test
Counters:
Name               Bytes              Packets
counter_in         0                  0
Test PE to PE connectivity over the VPLS PW:

user@PE1> ping vpls instance red destination-mac <*> source-ip <*>
For the source-ip, configure IRB under the VPLS instance within the same address range within CPEs. Remember to have loopback interface open for the source address.

If M/T Series
show route forwarding-table family vpls vpn <instance-name> extensive

This command is used for the M/T series. It is equivalent to the previous command.

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

Caveats

If allowed, turn on the following traceoptions for approximately five minutes while the traffic is on. Then, turn it back off and collect the log files. See Caveats.

set routing-instances <instance-name> protocols vpls traceoptions flag all

This command will collect all vpls-related traces.

set protocols l2-learning traceoptions file l2ald-trace size 100m;

set protocols l2-learning traceoptions file flag mac-learning;
set protocols l2-learning traceoptions file flag ipc;
set protocols l2-learning traceoptions file flag routing-socket;

[Back to Top]

L3 VPN

CLI Commands


show route table <vpn-a>  
show route table <vpn-a> hidden
show route forwarding-table vpn <vpn-a>

ping x.x.x.x routing-instance <vpn-a>
traceroute x.x.x.x routing-instance <vpn-a>
telnet x.x.x.x routing-instance <vpn-a>
ping mpls l3vpn <vpn-a> prefix <prefix>

show route table bgp.l3vpn.0 
show route advertising-protocol bgp x.x.x.x
show route receive-protocol bgp x.x.x.x
show arp


OSPF VPN commands
show ospf interface instance <vpn-a>
show ospf neighbor instance <vpn-a>
show ospf database instance <vpn-a>

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

None

[Back to Top]

L2 VPN

CLI Commands


Kompella
show l2vpn connections extensive     
show route table <vpn-a>
show route table bgp.l2vpn.0 
show route advertising-protocol bgp x.x.x.x
show route receive-protocol bgp x.x.x.x
show route table mpls.o
show route forwarding-table family mpls
show rsvp session extensive


Martini
show l2circuit connections extensive
show ldp neighbor detail
show ldp database detail
show route table mpls.o

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

Kompella
set routing-instances l2vpn protocols l2vpn traceoptions file l2vpn_trace
set routing-instances l2vpn protocols l2vpn traceoptions file size 50m
set routing-instances l2vpn protocols l2vpn traceoptions file files 5
set routing-instances l2vpn protocols l2vpn traceoptions flag all

Martini
set protocols l2circuit traceoptions file l2circuit_trace
set protocols l2circuit traceoptions file size 50m
set protocols l2circuit traceoptions file files 5
set protocols l2circuit traceoptions flag all

set protocols ldp traceoptions file ldp_trace
set protocols ldp traceoptions file size 50m
set protocols ldp traceoptions file files 5
set protocols ldp traceoptions flag all

[Back to Top]

CoS

CLI Commands


show interface xx-x/x/x detail   
show interface queue xx-x/x/x both-ingress-egress
show interfaces XX-X/X/X extensive

show class-of-service interfaces XX-X/X/X 
show class-of-service code-point-aliases
show class-of-service code-point-aliases dscp
show class-of-service code-point-aliases inet-prec
show class-of-service code-point-aliases exp
show class-of-service forwarding-class
show class-of-service classifier [name]
show class-of-service scheduler-map
show class-of-service rewrite-rule
show class-of-service drop-profile
show class-of-service forwarding-table
show class-of-service forwarding-table classifier mapping
show class-of-service forwarding-table scheduler-map

 

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

None

[Back to Top]

Firewall Filters

CLI Commands


show interfaces filters
show firewall
show firewall filter <filter-name>
show firewall filter <name> prefix-action <psa-name> from 1 to 8
show firewall log
show log <log-file-name>
show policer
show interface policer XX-X/X/X

Logs

Archive /var/log as specified in Step 1 above
show log /var/tmp/<sample file>
show log sampled


 

Traceoptions

None

[Back to Top]

IPv6

CLI Commands


set show interface terse
show route table inet6
show ipv6 neighbor
show interface lo0 extensive | display xml

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

None

[Back to Top]

SNMP

CLI Commands


set cli timestamp
show configuration snmp

show snmp statistics extensive # Multiple time with polling activity.

show snmp mib walk <oid>
show snmp mib walk ipfragfails
show interface lo0 extensive

show system processes extensive | no-more # Multiple time with polling activity.

show chassis routing-engine | no-more # Multiple time with polling activity.

show system commit | no-more
show system statistics # Multiple time with polling activity.
show system buffers # Multiple time with polling activity.
show system core-dumps
show route <ip of NMS server>
ping <ip of NMS server> source <snmp source ip> count 500 rapid

show pfe statistics traffic # Multiple time with polling activity.

show interface extensive | no-more
show system process extensive | no-more
show snmp statistics subagents | no-more
show snmp stats-response-statistics

Logs

Archive /var/log as specified in Step 1 above

Traceoptions


set snmp traceoptions file snmp files 10 size 50m
set snmp traceoptions flag all

 

[Back to Top]

High CPU

CLI Commands


set cli timestamp
show chassis routing-engine (multiple snapshots, atleast 5)
show system processes extensive (multiple snapshots atleast 5)
show system users
show system connections
show system statistics


Turn on task accounting and collect the task accounting detail output (three times with a gap of 30 seconds). Don't forget to turn it off after finished.

set task accounting on
show task accounting detail
set task accounting off

show task memory detail
show task memeory summary
show task io
show task history
show task statistics
show task job
show task jobs
show krt queue
show krt state

 

Logs

Archive /var/log as specified in Step 1 above

Traceoptions


user@router# show routing-options
traceoptions {
file routing-trace size 10m files 20 world-readable;
flag task;
flag state;
flag timer;
}

 

[Back to Top]

MX
Fabric
Plane

(SBC,
FChip,
Plane)

 

Show Commands


General
show chassis environment cb
show chassis alarms


Fabric-related
show chassis hardware
show chassis fabric summary
show chassis fabric map
show chassis fabric fpc
show chassis fabric plane

show class-of-service fabric statistics (take multiple outputs to see counters incrementing)

show pfe statistics traffic (take multiple outputs to see counters incrementing)

FPC Commands (from CLI)
request pfe execute target <fpc_no> command "show hsl2 statistics detail"

request pfe execute target <fpc_no> command "show syslog messages"

 

Logs

Archive /var/log as specified in Step 1 above
show log messages
show log chassisd
 

Hidden Commands

Caveats


Note: These commands are not officially supported, and are hidden; however, they are good for troubleshooting Fabric Plane issues.

Exercise caution when executing these commands, as incorrect syntax may lead to problems. 15.1R7 may reset the FPC, but 18.4R2 is stable.

show chassis fabric asic-configuration

show chassis fabric statistics detail <plane_no> totals

Take multiple outputs to see counters incrementing and from all the planes.

show chassis hsl channel asic-name CBXFY slot Z extensive

X is the CB slot.
Y is the Fabric Plane number.
Z is the FPC slot no.
CBXFY is in upper case.

For example:

show chassis hsl channel asic-name CB0F1 slot 3 extensive

Run this command for all the SCB's and for all the planes multiple times to see incrementing counters.

 

Questions

  • Have the SCB's for which the alarms have generated had any previous history of any issues?
  • What kind of troubleshooting has already been done to mitigate the problem. Please provide detailed steps.
  • Were there any recent changes to the configuration?
  • Were there any changes to the hardware setup, or was new hardware introduced?
  • Were there any events that occurred just before the log message that may have triggered it? This may include events such as routing engine switchover etc.
  • Was a software upgrade/downgrade done recently?

[Back to Top]

NG-MPVN

Show Commands


show route table<instance-name>.inet.0 | no-more
show route table<instance-name>.mvpn.0 | no-more
show route table<instance-name>.inet.0 extensive | no-more
show route table<instance-name>.mvpn.0 extensive | no-more
show pim joininstance <instance-name> extensive | no-more
show multicast route instance <instance-name> extensive | no-more
show interface extensive |no-more
show pim rps instance <instance-name> extensive | no-more
show mpls lsp p2mp ingress |no-more
show mpls lsp p2mp ingress extensive |no-more
show mpls lsp p2mp | no-more
show mpls lsp | no-more
show bgp summary |no-more


 

Logs
(each node)

Archive /var/log as specified in Step 1 above

Traceoptions


PIM detailed Traceoptions
RSVP detailed Traceoptions
BGP detailed Traceoptions
MVPN detailed Traceoptions
RTSOCKMON
 

Known Issues

List Open PR issues [change href values]

[Back to Top]

CGNAT

Show Commands 


show chassis pic fpc-slot x pic-slot x
show services service-sets cpu-usage
show services service-sets summary
show services service-sets statistics syslog
show services service-sets statistics packet-drops
show services service-sets memory-usage
show services service-sets memory-usage zone
show services stateful-firewall conversations
show services stateful-firewall conversations destination-prefix
show services stateful-firewall statistics extensive
show services stateful-firewall flow-analysis

show services stateful-firewall flows extensive
show services nat mappings detail
show services nat pool detail
show services nat mappings endpoint-independent
show services nat mappings address-pooling-paired

 

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

None

[Back to Top]

Spanning
Tree

Show Commands 


show spanning-tree bridge detail
show spanning-tree bridge msti x detail
show spanning-tree interface detail
show spanning-tree mstp configuration detail
show spanning-tree statistics bridge
show spanning-tree statistics interface detail
show spanning-tree statistics message-queues
show spanning-tree statistics routing-instance detail
show spanning-tree stp-buffer see-all

 

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

set protocols rstp|mstp|vstp traceoptions file spanning-tree.trace
set protocols rstp|mstp|vstp traceoptions file size 20m
set protocols rstp|mstp|vstp traceoptions file files 5
set protocols rstp|mstp|vstp traceoptions flag all

[Back to Top]

BFD

Show Commands 


( take 4 outputs at 30 seconds apart)
show ppm connections detail
show bfd session extensive | no-more
show ppm interfaces detail | no-more
show ppm adjacencies detail | no-more
show ppm transmissions detail | no-more
show pfe statistics traffic | no-more


 

Logs

Archive /var/log as specified in Step 1 above

Traceoptions

set routing-options ppm traceoptions file ppm
set routing-options ppm traceoptions file size 30m
set routing-options ppm traceoptions flag all
set protocols bfd traceoptions file bfd
set protocols bfd traceoptions file size 30m
set protocols bfd traceoptions flag all

 

[Back to Top]


References:

How To

Technical Bulletins

Modification History:

2020-07-16: Command added to VPLS (test PE to PE connectivity over VPLS PW)
2020-05-12: Updated VPLS CLI Commands
2020-04-30: Updated SNMP CLI Commands
2017-04-24: Added PTX product series, and did minor edits.

Related Links

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search