Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

Service Automation User Guide Addendum for the TX Platforms

0

0

Article ID: TN287 TECHNOTES Last Updated: 30 Jun 2016Version: 4.0
Description:


Introduction

The Service Automation User Guide describes the typical behaviors of Service Now, Service Insight and the AI-Scripts, when used with most of Juniper’s products. However, there are several nuances, limitations and other variations when using Service Automation with Juniper’s TX platforms: TX-Matrix, TX-Matrix Plus and TX-Matrix Plus 3D. This article explains these variations. 

Topics covered:  


Service Automation TX Platform Differences Summary

The multi-chassis design of the TX platform results in some unique operational differences with respect to Service Now, Service Insight and AI-Scripts. The following table summarizes the differences between standard Service Automation behavior for a typical device and the Service Automation behavior for the TX Platform

Service Automation Area Standard Behavior TX Platform Behavior

Service Automation Data flow
Event JMBs generated by AI-Scripts on backup REs are sent to the master RE on the same device for retrieval by Service Now. Event JMBs generated by AI-Scripts on backup REs of LCC nodes are sent to the master RE on the SCC/SFC node for retrieval by Service Now. (See sections titled “Service Automation Data Flow” and “JMBs” for more details.)
Junos Space / Service Now Management Interface A single IP address management interface will result in a single managed device in Junos Space / Service Now. A single IP address management interface will result in a multi-chassis device representation in Junos Space / Service Now. (See sections titled “Physical (Layer 2) Connections” and “Service Now Representation of the TXP”.)
AI-Scripts installation and activation
Service Now will initiate and manage AI-Scripts installation and activation on the master RE of a device and Junos will initiate the installation and activation of AI-Scripts on the backup RE.
Service Now will initiate and manage AI-Scripts installation and activation on the master RE of the SCC/SFC and Junos will initiate the installation and activation of AI-Scripts on the backup RE of the SCC/SFC node as well as all of the REs on the LCC nodes. (See section titled “Script Installation”.)
RSI Collection For event JMBs, RSI will contain information on the entire device. For event JMBs, RSI will contain information on all of the TX Platform nodes if the event JMB was executed on the SCC/SFC node. If an event JMB is executed on an LCC node, RSI will only contain information about that LCC node. (See section titled “RSI Collection.)

Important Note:

Although it is always recommended to use the most recently released versions of Service Now, Service Insight and AI-Scripts, it is especially important that Service Now 13.3R1 (or later) and AI-Scripts 4.0R1 (or later) be used to provide Service Automation support to the TX platforms. Earlier releases of both provide very limited support for the TX platforms and dramatically reduce the value of Service Automation.


Overview

The TX platforms (TX-Matrix, TX-Matrix Plus and the TX-Matrix Plus 3D) are multi-chassis products comprised of one controller node (Switch-Card Chassis (SCC) on TX-Matrix, Switch-Fabric Chassis (SFC) on TX-Matrix Plus / TX-Matrix Plus 3D) and multiple Line-Card Chassis (LCC) nodes (up to four for TX-Matrix / TX-Matrix Plus and up to eight for TX-Matrix Plus 3D).

Each node (SFC/SCC and LCC) has one or two Routing Engines (RE). Each RE on each node is running an independent instance of Junos. The master REs on each node must all be running the same version of Junos. The same is true of the backup REs on each node – they must all run the same version of Junos, but not necessarily the same version as the master REs. The SFC/SCC distributes and initiates the installation of a Junos release to the LCCs.


Service Automation Data Flow

Event data generated by AI-Scripts on LCC nodes is passed to Service Now via the SxC node of the TX platform (Refer to sections JMBs, Log File Collection, RSI Collection, and Core File Collection for more details on the type of data generated and specific collection instances). The following diagram illustrates the difference between TX Platform Control Plane data path and the Service Automation data communication path:

The remainder of this document provides additional details of the nuances of Service Automation support for the TX platforms.


Physical (Layer 2) Connections

Junos Space and Service Now communicate with the TX Platform through a single IP address for managing the platform and sending and receiving Service Automation data. The single IP management address on the TX Platform could be any reachable IP address on the SCC/SFC node of the TX Platform. This IP should be used to discover the device and create a management session with Junos Space and Service Now.

The management addresses of the LCCs should not be used in Junos Space device discovery as that would create an overlap of Space and Service Now management of the same device component. For example, if a T1600 that is configured as an LCC node in a TX Matrix Plus system is managed by Junos Space / Service Now, it would appear as both a T1600 and an LCC node in the TX Matrix Plus system, which will cause problems collecting Service Automation data from the device.

Junos Space and Service Now will maintain a management session with the TX Platform if the master RE of the SCC/SFC goes down, as long as the backup RE is configured to share the management IP address. There is no node redundancy, however, if the SCC/SFC node goes down, in which case the SN device representing the TX platform will have a status of “out of sync”.


Service Now Representation of the TXP

Junos Space and Service Now represent the chassis inventory of a TX platform as a set of interconnected chassis nodes. In the sample picture below, the TX-Matrix Plus (TXP) system, hostname host, represents the entire system of three nodes which are listed as chassis elements under the Multi Routing Engine object. The TXP controller node, sfc0-re0, is shown as the bottom chassis element in the list and above it are the two T1600 LCC nodes, chassis elements lcc0-re- and lcc1-re0).

Below is an example of similar chassis inventory data obtained through Junos CLI commands:

user@host> show chassis hardware
sfc0-re0:
--------------------------------------------------------------------------
Hardware inventory:
Item Version Part number Serial number Description
Chassis JN119A171AHB TXP
...

lcc0-re0:
-------------------------------------------------------------------------
Hardware inventory:
Item Version Part number Serial number Description
Chassis JN116D09BAHA T1600
...

lcc1-re0:
--------------------------------------------------------------------------
Hardware inventory:
Item Version Part number Serial number Description
Chassis JN11739A7AHA T1600
...

user@host> show chassis hardware models
sfc0-re0:
--------------------------------------------------------------------------
Hardware inventory:
Item Version Part number Serial number FRU model number
Midplane REV 05 710-022574 ABAB2596 CHAS-BP-TXP-S
...

lcc0-re0:
--------------------------------------------------------------------------
Hardware inventory:
Item Version Part number Serial number FRU model number
Midplane REV 01 710-027486 RC8197 CHAS-BP-T1600-S
...

lcc1-re0:
--------------------------------------------------------------------------
Hardware inventory:
Item Version Part number Serial number FRU model number
Midplane REV 01 710-027486 RC8202 CHAS-BP-T1600-
S


Script Installation

Service Now manages the installation and activation of the AI-Scripts bundle on the master and backup REs of the SCC/SFC node of the TX Platform. When Service Now initiates the installation and activation of the AI-Scripts bundle on the master RE of the SCC/SFC node, Junos will automatically install and activate the the AI-Scripts bundle on the master RE of all of the LCC nodes. Similarly, when Service Now initiates the installation and activation of the AI-Scripts bundle on the backup RE of the SCC/SFC node, Junos will automatically install and activate the AI-Scripts bundle on the backup RE of all of the LCC nodes.


Script Uninstallation

Service Now manages the uninstallation of the AI-Scripts bundle on the master RE of the SCC/SFC node of the TX Platform. . When Service Now initiates the uninstallation of the AI-Scripts bundle on the master RE of the SCC/SFC node, Junos will automatically uninstall the the AI-Scripts bundle on the master RE of all of the LCC nodes.

Due to a known Junos bug, Service Now cannot uninstall the AI-Scripts bundle on the backup RE of any device, TX Platform included. Users can manually uninstall the AI-Scripts bundle on the backup RE of the SCC/SFC node per KB29073 - How to uninstall AI-Scripts from a Backup Routing Engine and Junos will automatically uninstall the AI-Scripts bundle on the backup REs of all of the LCC nodes.


Event Profiles

Service Now associates an Event Profile with every AI-Scripts bundle that it deploys on Junos devices. An Event Profile is a customized set of event triggers that users can select for a given AI-Scripts bundle. Part of the Service Now feature for deploying AI-Scripts on a TX platform includes:

  • Associating customer-generated Event Profiles for each AI-Scripts release,
  • Pushing an Event Profile to every node in the TX platform system, and
  • Activating the Event Profile on every node.


Dynamic Hardware Reconfiguration

The following sections describe the impact of various hardware reconfiguration scenarios on the installation of AI-Scripts. For any hardware reconfiguration scenario, if there is any question regarding the status of AI-Scripts bundle installation on any RE of any TX Platform node, use the “show version invoke-on all-routing-engines” CLI command to verify the AI-scripts bundle is installed on all master and backup REs on all nodes of a TX Platform. For example:

MASTER
user@host> show version invoke-on all-routing-engines
sfc0-re0:
--------------------------------------------------------------------------
...

lcc0-re1:
--------------------------------------------------------------------------
Hostname: crest1-em0
Model: t1600
JUNOS Base OS boot [12.3R3]
JUNOS Base OS Software Suite [12.3R3]
...
JUNOS AIS Script Suite [4.0R1.3] <<<
...

RE Removal / Addition

If an RE is removed or added, Junos Space will automatically re-adjust the TX platform chassis node inventory during the periodic re-sync operation. If an RE is added, AI-Scripts installation will not occur automatically. AI-Scripts will have to be installed on the RE if they have not been installed already on that RE.

For example, if a backup RE on an LCC node is replaced, the AI-Scripts bundle should be re-installed on the backup RE on the SCC/SFC node so that Junos can automatically install the AI-Scripts bundle on the replacement backup RE of the LCC node.

LCC Removal / Addition

If an LCC is removed or added, Junos Space will automatically re-adjust the TX platform chassis node inventory during the periodic re-sync operation. If an LCC node is added, AI-Scripts installation will not occur automatically. AI-Scripts will have to be re-installed on the SxC controller node so that the AI-Scripts installation can occur on the new LCC node.


Redundancy

RE Redundancy

Every node in a TX platform system can operate with RE redundancy. The backup RE will assume control of the node (or the entire system in the case of the backup RE on the controller node) should the master RE become non-functional. While AI-Scripts are installed and activated on backup REs on all TX platform nodes, only a small subset of events will generate JMBs and those JMBs will be sent to the master RE on the SCC/SFC.

Node Redundancy

TX platforms do not provide node redundancy. If an entire LCC node goes down, the SCC/SFC node will likely generate an alert that will appear in the Service Now Service Central Incident Manager regarding the loss of the LCC node. If the entire SCC/SFC node goes down, the device will appear as “out of sync” in Service Now after the next device discovery polling cycle takes place.


JMBs

AI-Scripts create JMBs (Juniper Message Bundles) to record information at the instant of an Event, when requested (On-Demand) or periodically for trend data capture. The format of JMBs and associated data, collected as JMB attachments, has changed significantly in AI-Scripts release 4.0+. The following sections describe this new format and behavior.

Event JMBs (and attachments)

In a TX platform, an Event JMB is created on the node on which the Event is detected. Event JMBs generated on controller (SCC, SFC) nodes, contain information about the entire system, including all LCC nodes. Event JMBs generated on LCC nodes only contain information about that specific LCC node. Event JMBs that occur on LCC nodes are funneled to Service Now via the controller node.

Event JMBs will contain some very basic information about the chassis node and the detected Event. In addition, AI-Scripts will generate a set of attachment files that can be used to diagnose, troubleshoot and debug problems with the associated Event. One of these attachments, the Event Support Information file, contains Event-related and platform-specific data. The collection and handling of other Event attachments files is described in the later sections, Log File Collection, RSI Collection, and Core File Collection.

Informational JMBs (and attachments)

Informational JMBs are executed once a week on the controller node of a TX platform system. This type of JMB will contain some data and information from the entire system. Specifically, the type of data collected is as follows:


On-Demand requests

On-Demand requests (JMBs) can only be executed on the controller node of a TX platform system. This type of JMB will contain some data and information from the master REs of all nodes of the TX Platform. Specifically, the type of data collected is as follows:
On Box On-Demand

  1. Chassis and software data collected is from all master REs of all nodes of the TX Platform.
  2. Log files are primarily controller node information, except for messages file, which contains all events for all nodes.
  3. Trend data info is only controller node information.

Off Box On-Demand

  1. Chassis and software data collected is from all master REs of all nodes of the TX Platform.
  2. Log files are not collected in current version.
  3. Trend data info is only controller node information.

On Box Informational JMB

  1. Chassis and software data collected is from all master REs of all nodes of the TX Platform.
  2. Log files are primarily controller node information, except for messages file, which contains all events for all nodes.
  3. Trend data info is only controller node information.

Off Box Informational JMB

  1. Chassis and software data collected is from all master REs of all nodes of the TX Platform.
  2. Log files are not collected at this time. This feature may be planned for future releases.
  3. Trend data info is only controller node information.


Log Collection

When an Event JMB or On-Demand JMB is created on the master RE of the SCC/SFC, AIS checks the amount of disk space available at /var, and if sufficient, will generate a compressed tar file of the /var/log/ directory. Otherwise, if not enough space is available on the SCC/SFC for a tar file of the /var/log directory, the files in that directory will be listed in the JMB, so that Service Now can retrieve them directly.

When an Event JMB or On-Demand JMB is created on the backup RE of the SCC/SFC, or on any LCC RE, AIS checks the amount of disk space available at /var, and if sufficient, will generate a compressed tar file of the /var/log/ directory. However if there is not sufficient disk space available on both the LCC and the SCC/SFC, then no log data collection will be done.

Service Now will depict the files in the /var/log directory in the same manner as they were retrieved from the TX platform. If a tar file was retrieved, the /var/log files will be shown as a single .tgz file. If there was insufficient disk space available on an LCC or SCC/SFC node for generating a compressed tar file of the /var/log directory, individual log files will be retrieved, and Service Now will list each individual log file as an attachment in the incident record in Service Central Incident Manager. Should users see these individual log files associated with a TX Platform incident alert, they should consider managing the disk space on the affected TX Platform node REs.


RSI Collection

The Junos CLI command ‘request support information’ (RSI) is a collection of other Junos CLI commands. These commands collect data related to Junos versions, hardware models, core dump files, last boot messages, physical and logical interface statistics, and so forth. This information is the baseline troubleshooting data that may be needed to investigate an issue. However, in some cases, additional information may be needed to further shed light on more complex problems.

By default, RSI will be executed, and the text output included as part of JMBs generated on LCC or SxC nodes. RSI will contain information from the TX platform node it is run on. For example, RSI run on an SCC or SFC collects information from the master REs of all TX Platform nodes, containing data related to all the LCC nodes as well as the SCC/SFC node. RSI run on the LCC will only generate information associated with that LCC node. The node that RSI runs on will be determined by the node where the event JMB is generated.


Core File Collection

As with any other dual-RE or multi-chassis platform, Service Now can only retrieve core files if they have occurred on the master RE. In the case of TX Platform, Service Now can only retrieve core files if they have occurred on the master RE of the SCC/SFC node. Service Now does not currently have a mechanism for retrieving files directly from the other REs. In order to retrieve core files from the SCC/SFC backup RE or from any RE on LCCs, users will have to directly connect to those REs and manually retrieve those files.


Additional Information

For additional information on the TX Platforms, refer to the following links:

Technical Documentation:
http://kb.juniper.net/InfoCenter/index?page=answers&type=search&searchid=1397149629591&question_box=TX+Matrix&cntnt=Technical_Documentation


Technical Notes:
http://kb.juniper.net/InfoCenter/index?page=answers&type=search&searchid=1397149629591&question_box=TX+Matrix&cntnt=Technotes

Technical Bulletins:
http://kb.juniper.net/InfoCenter/index?page=answers&type=search&searchid=1397149629591&question_box=TX+Matrix&cntnt=Technical_Bulletins


Knowledge Base Articles:
http://kb.juniper.net/InfoCenter/index?page=answers&type=search&searchid=1397149629591&question_box=TX+Matrix&cntnt=Knowledge_Base



For additional Service Automation User Guide Addenda, refer to the master list:

Service Automation User Guide Addenda - Master List:
http://kb.juniper.net/KB29188

Source:
JTAC
Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search