Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

Service Automation User Guide Addendum for MX Virtual Chassis

0

0

Article ID: TN294 TECHNOTES Last Updated: 30 Jun 2016Version: 2.0
Description:


Introduction

The Service Automation User Guide describes the typical behaviors of Service Now, Service Insight and the AI-Scripts, when used with most of Juniper’s products. However, there are several nuances, limitations and other variations when using Service Automation with Juniper’s MX Virtual Chassis (MX-VC) platform. This article explains these variations. 

Topics covered:  


Service Automation MX-VC Platform Differences Summary

The multi-chassis design of the MX-VC platform results in some unique operational differences with respect to Service Now, Service Insight and AI-Scripts. The following table summarizes the differences between standard Service Automation behavior for a typical device and the Service Automation behavior for the MX-VC Platform

Service Automation Area Standard Behavior MX-VC Platform Behavior

Service Automation Data flow
Only the Master Routing Engine (RE) communicates with Service Now. The backup RE, which does not have a management connection with Service Now, sends data to Service Now via the Master RE. MX-VC systems communicate with Service Now via the Master Routing Engine (RE) of the Master Node Chassis. The Backup REs on both the Master and Backup Node Chassis, as well as the Master RE on the Backup Node, send data to Service Now via the Master RE on the Master Node. (See sections titled “Service Automation Data Flow” and “JMBs”.)
Junos Space / Service Now Management Interface Only the Master RE requires a management IP address interface with Junos Space / Service Now. On the MX-VC platform, only the Master RE on the Master Node requires a management IP address interface with Junos Space / Service Now. (See sections titled “Physical (Layer 2) Connections” and “Service Now Representation of the MX-VC”.)
AI-Scripts installation and activation
Users direct Service Now to initiate and manage AI-Scripts installation and activation on the Master RE of a device and Junos will automatically initiate the installation and activation of AI-Scripts on the backup RE.
Users direct Service Now to initiate and manage AI-Scripts installation and activation on the MX-VC Master RE on the Master Node and Service Now will automatically initiate the installation and activation of AI-Scripts on the Master RE on the Backup Node as well as the Backup REs on both Master and Backup Nodes. (See section titled “Script Installation”.)
RSI Collection For event JMBs, RSI will contain information from the entire device. For event JMBs, RSI information from all MX-VC nodes will be concatenated. RSI will contain information primarily from all MX-VC nodes. (See section titled “RSI Collection.)


Overview

An MX Series Virtual Chassis (MX-VC) interconnects two MX Series routers into a logical system that can be managed as a single network element. The MX-VC configuration allows for just two MX chassis to be connected in Virtual Chassis mode. Each MX chassis in VC mode is required to have two routing engines (Master and Backup), and, in addition, graceful Routing Engine switchover (GRES) and nonstop active routing (NSR) must be enabled on both member routers in the Virtual Chassis.

The Virtual Chassis configuration option is available on the following MX platforms:

  • MX240 3D Universal Edge Router
  • MX480 3D Universal Edge Router
  • MX960 3D Universal Edge Router

One of the two MX-VC node pair operates as the Master Node for the Virtual Chassis. The Virtual Chassis Master Node maintains the global configuration and state information for both member nodes, and runs the chassis management processes and control protocols. The Master RE that resides in the Virtual Chassis
Master Node is the global Master for the Virtual Chassis.

The Virtual Chassis Backup node takes over mastership of the Virtual Chassis if the Master Node is unavailable, and synchronizes routing and state information with the Master RE of the new Backup node.

Each RE on each node is running an independent instance of Junos. All REs on all nodes must all be running the same version of Junos OS.

Important Note:

Although it is always recommended to use the most recently released versions of Service Now, Service Insight and AI-Scripts, it is especially important that Service Now 14.1R1 (or later) and AI-Scripts 4.1R1 (or later) be used to provide Service Automation support to the MX-VC platforms. Earlier releases of both provide very limited support for the MX-VC platforms and dramatically reduce the value of Service Automation.

The remainder of this document provides additional details of the nuances of Service Automation support for the MX-VC platforms.


Service Automation Data Flow

Event data generated by AI-Scripts on MX-VC nodes is passed to Service Now via the Master Node of the MX-VC platform (refer to sections JMBs, Log File Collection, RSI Collection, and Core File Collection for more details on the type of data generated and specific collection instances). The following diagram illustrates the difference between MX-VC Platform Control Plane data path and the Service Automation data communication path:


Note: See PR945340 for a known issue with AIS not installing on the Backup Routing Engine.  The fixed versions are listed in the PR.


Physical (Layer 2) Connections

Junos Space and Service Now communicate with the MX-VC system through an IP management session with the MX-VC Master Node in order to manage the MX-VC system and send and receive Service Automation data. The single IP management address on the MX-VC platform could be any reachable IP address on the platform. This IP should be used to discover the device and create a management session with Junos Space and Service Now.

The management address of the Backup node should not be used in Junos Space device discovery as that would create an overlap of Junos Space and Service Now management of the same device component, both as a member of the MX-VC system as well as a separate MX chassis.


Service Now Representation of the MX-VC

Junos Space and Service Now represent the chassis inventory of an MX-VC platform as a set of two interconnected chassis nodes. In the sample picture below, the MX-VC system, hostname host, represents the entire system of two nodes which are listed as chassis elements under the Multi Routing Engine object.

Below is an example of similar chassis inventory data obtained through Junos CLI commands:

{master:member0-re0}
user@host> show chassis hardware models
member0:
--------------------------------------------------------------------------
Hardware inventory:
Item Version Part number Serial number FRU model number
Midplane REV 03 750-047849 ACAX3159 CHAS-BP3-MX960-S
...

member1:
--------------------------------------------------------------------------
Hardware inventory:
Item Version Part number Serial number FRU model number
Midplane REV 01 750-047865 ACRA1471 CHAS-BP3-MX240-S
...


{master:member0-re0}
user@host> show chassis hardware
member0:
--------------------------------------------------------------------------
Hardware inventory:
Item Version Part number Serial number Description
Chassis JN121376FAFA MX960
...

member1:
--------------------------------------------------------------------------
Hardware inventory:
Item Version Part number Serial number Description
Chassis JN122198BAFC MX240
...


Script Installation

The user must manage the installation and activation of the AI-Scripts bundle on the MX-VC system through Service Now. When Service Now initiates the installation and activation of the AI-Scripts bundle on the Master RE on the Master Node, Service Now will automatically install and activate the AI-Scripts bundle on all MX-VC system REs:

  • Master RE on the Master Node
  • Backup RE on the Master Node
  • Master RE on the Backup Node
  • Backup RE on the Backup Node

Due to a known Junos bug (PR/945340), Service Now may not be able to install the AI-Scripts bundle on the backup RE of any node in an MX-VC system. Users can manually install the AI-Scripts bundle using the procedure described in the AI-Scripts Release Notes section, “Manually Configuring and Installing AI-Scripts on Devices”.


Script Uninstallation

The user must manage the uninstallation of the AI-Scripts bundle on the MX-VC system through Service Now. When Service Now initiates the uninstallation of the AI-Scripts bundle on the Master RE on the Master Node, Service Now will automatically uninstall the AI-Scripts bundle on all MX-VC system REs:

  • Master RE on the Master Node
  • Backup RE on the Master Node
  • Master RE on the Backup Node
  • Backup RE on the Backup Node

Due to a known Junos bug (PR/945340), Service Now may not be able to uninstall the AI-Scripts bundle on the backup RE of any node in an MX-VC system. Users can manually uninstall the AI-Scripts bundle using the procedure described in the AI-Scripts Release Notes section, “Deleting an AI-Scripts Package”.


Event Profiles

Service Now associates an Event Profile with every AI-Scripts bundle that it deploys on Junos OS devices. An Event Profile is a customized set of event triggers that users can select for a given AI-Scripts bundle. When a user deploys an Event Profile, Service Now will propagate that Event Profile to all of the REs in both nodes of the MX-VC system.


Dynamic Hardware Reconfiguration

The following sections describe the impact of various hardware reconfiguration scenarios on the installation of AI-Scripts. For any hardware reconfiguration scenario, if there is any question regarding the status of AI-Scripts bundle installation on any RE of any MX-VC Platform node, use the “show version” CLI command to verify the AI-scripts bundle is installed on all Master and Backup REs on all Nodes of an MX-VC Platform. For example,

MASTER
{master:member0-re0}
user@host> show version invoke-on all-routing-engines
member0-re0:
--------------------------------------------------------------------------
Hostname: host
Model: mx960
Junos: 13.3R4.6
JUNOS Base OS boot [13.3R4.6]
JUNOS Base OS Software Suite [13.3R4.6]
...
JUNOS AIS Script Suite [4.0R2.1] <<<<<<
...

member0-re1:
--------------------------------------------------------------------------
Hostname: host
Model: mx960
Junos: 13.3R4.6
JUNOS Base OS boot [13.3R4.6]
JUNOS Base OS Software Suite [13.3R4.6]
...
JUNOS AIS Script Suite [4.0R2.1] <<<<<<
...

member1-re0:
--------------------------------------------------------------------------
Hostname: host1
Model: mx960
Junos: 13.3R4.6
JUNOS Base OS boot [13.3R4.6]
JUNOS Base OS Software Suite [13.3R4.6]
...
JUNOS AIS Script Suite [4.0R2.1] <<<<<<
...

member1-re1:
--------------------------------------------------------------------------
Hostname: host1
Model: mx960
Junos: 13.3R4.6
JUNOS Base OS boot [13.3R4.6]
JUNOS Base OS Software Suite [13.3R4.6]
...
JUNOS AIS Script Suite [4.0R2.1] <<<<<<
...

RE Removal / Addition

If any MX-VC node RE is removed or replaced, Junos Space will automatically re-adjust the MX-VC platform chassis node inventory during the periodic re-sync operation. If an RE is replaced, AI-Scripts installation will not occur automatically. AI-Scripts will have to be installed on the new RE if they have not been installed already on that RE.

For example, if the MX-VC Backup RE on the Backup Node is replaced, the AI-Scripts bundle should be re-installed on MX-VC system so that Service Now can reinstall the AI-Scripts bundle on all of the nodes including the Backup RE on the Backup Node.

Node Removal / Addition

If an MX-VC node is removed or added, Junos Space will automatically re-adjust the MX-VC chassis node inventory during the periodic re-sync operation. If an MX-VC node is added, AI-Scripts installation will not occur automatically. AI-Scripts will have to be re-installed on the MX-VC system so that the AI-Scripts installation can occur on the new MX-VC node.


Redundancy

RE Redundancy

Both MX-VC nodes can operate with RE redundancy. Should the Master RE on the Master Node become non-functional, the Master RE on the Backup Node will assume control of the MX-VC system. The Backup RE on what was formerly the Master Node will then become the Master RE on the new Backup node. While AI-Scripts are installed and activated on both Backup REs on both MX-VC nodes, only a small subset of events will generate JMBs and those JMBs will be sent to the Master RE on the Master Node of the MX-VC system.

Node Redundancy

MX-VC platforms provide Master to Backup Node redundancy. An MX-VC system will not switchover system mastership between the REs on the Master Node. If the Master RE on the Master Node goes down, the MX-VC system will switch mastership to the Master RE of the Backup node as that is where all of the chassis and protocol management state is being mirrored.

Master to Backup node switchover will result in Junos Space reestablishing the management session with the new Master Node. If the Master Node or the Master RE on the Master Node goes down, the Master RE on the Backup Node will likely generate an alert that will appear in the Service Now Service Central Incident Manager regarding the loss of the Master Node.


JMBs

AI-Scripts create JMBs (Juniper Message Bundles) to record information at the instant of an Event, when requested (On-Demand) or periodically for trend data capture. The format of JMBs and associated data, collected as JMB attachments, has changed significantly in AI-Scripts release 4.0+. The following sections describe this new format and behavior.

Event JMBs (and attachments)

In an MX-VC platform, an Event JMB can be created on both REs of both nodes of the MX-VC system. Event JMBs generated on each MX-VC node primarily contain information about that node. Some of the troubleshooting data generated by Event JMBs, such as the output of “request support information (RSI)” will contain some information about the other node. See the section, “RSI Collection”, below for more details on the information collected by the RSI command. Event JMBs that occur on the Backup node are funneled to Service Now via the Master Node.

Event JMBs will contain some very basic information about the chassis node and the detected Event. In addition, AI-Scripts will generate a set of attachment files that can be used to diagnose, troubleshoot and debug problems with the associated Event. One of these attachments, the Event Support Information file, contains Event-related and platform-specific data. The collection and handling of other Event attachments files is described in the later sections, Log File Collection, RSI Collection, and Core File Collection.

Informational JMBs (and attachments)

Informational JMBs are executed once a week on the Master Node of an MX-VC system. This type of JMB will contain some data and information from the entire system. Specifically, the type of data collected is as follows:


On-Demand requests

On-Demand requests (JMBs) can only be executed on the Master RE on the Master Node of an MX-VC system. This type of JMB will contain some data and information from the entire system. Specifically, the type of data collected is as follows:

On Box On-Demand

  1. Chassis and software data collected is from all Master REs of the MX-VC system.
  2. Log files are primarily Master Node information, except for messages file, which contains all events for all nodes.
  3. Trend data contains information specific to the Master Node.

Off Box On-Demand

  1. Chassis and software data collected is from all Master REs of the MX-VC system.
  2. Log files are not collected in current version.
  3. Trend data contains information specific to the Master Node.

On Box Informational JMB

  1. Chassis and software data collected is from all Master REs of the MX-VC system.
  2. Log files are not collected in current version.
  3. Trend data contains information specific to the Master Node.

Off Box Informational JMB

  1. Chassis and software data collected is from all Master REs of the MX-VC system.
  2. Log files are not collected in current version.
  3. Trend data contains information specific to the Master Node.


Log Collection

When an Event JMB or On-Demand JMB is created on the Master RE on the Master Node of the MX-VC system, AI-Scripts checks the amount of disk space available at /var, and if sufficient, will generate a compressed tar file of the /var/log/ directory. Otherwise, if not enough space is available on the Master RE on the Master Node for a compressed tar file of the /var/log directory, the files in that directory will be listed in the JMB, so that Service Now can retrieve them directly.

When an Event JMB or On-Demand JMB is created on the Backup RE on the Master Node of the MX-VC system, or on any RE of the Backup node, AI-Scripts checks the amount of disk space available at /var, and if sufficient, will generate a compressed tar file of the /var/log/ directory. However if there is not sufficient disk space available on both the Master RE on the Master Node and the RE that generated the JMB, then no log data collection will be done.

Service Now will depict the files in the /var/log directory in the same manner as they were retrieved from the MX-VC system. If a compressed tar file was retrieved, the /var/log files will be shown as a single .tgz file. If there was insufficient disk space available on an RE for generating a compressed tar file of the /var/log directory, individual log files will be retrieved, and Service Now will list each individual log file as an attachment in the incident record in Service Central Incident Manager. Should users see these individual log files associated with an MX-VC system incident alert, they should consider managing the disk space on the affected MX-VC system node RE.


RSI Collection

The Junos CLI command ‘request support information’ (RSI) is a collection of other Junos CLI commands. These commands collect data related to Junos versions, hardware models, core dump files, last boot messages, physical and logical interface statistics, and so forth. This information is the baseline troubleshooting data that may be needed to investigate an issue. However, in some cases, additional information may be needed to further shed light on more complex problems.

By default, an event JMB will execute RSI on all MX-VC nodes and concatenate the information. The RSI JMB attachment file will contain RSI information from all of the MX-VC nodes.


Core File Collection

Service Now can only retrieve core files if they have occurred on the Master RE on the Master Node of the MX-VC system. Service Now does not currently have a mechanism for retrieving files directly from the other REs. In order to retrieve core files from the other REs in the MX-VC system, users will have to directly connect to that RE and manually retrieve those files.


Additional Information

For additional information on the MX-VC Platform, refer to the following links:

Technical Documentation:
http://kb.juniper.net/InfoCenter/index?page=answers&type=search&searchid=1397149629591&question_box=MX-VC&cntnt=Technical_Documentation


Technical Notes:
http://kb.juniper.net/InfoCenter/index?page=answers&type=search&searchid=1397149629591&question_box=MX-VC&cntnt=Technotes

Technical Bulletins:
http://kb.juniper.net/InfoCenter/index?page=answers&type=search&searchid=1397149629591&question_box=MX-VC&cntnt=Technical_Bulletins


Knowledge Base Articles:
http://kb.juniper.net/InfoCenter/index?page=answers&type=search&searchid=1397149629591&question_box=MX-VC&cntnt=Knowledge_Base



For additional Service Automation User Guide Addenda, refer to the master list:

Service Automation User Guide Addenda - Master List:
http://kb.juniper.net/KB29188

Source:
JTAC
Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search