Search our Knowledge Base sites to find answers to your questions.
Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles[MX] Bringing up in-chassis node slicing on MX960
Starting from Junos OS Release 19.1R1, Junos Node Slicing supports an in-chassis model, which allows all Junos Node Slicing components, such as Juniper Device Manager (JDM), base system (BSYS), as well as Guest Network Functions (GNFs), to run within the Routing Engine of MX Series routers.
This article provides step-by-step information to bring up in-chassis node slicing.
Note: To configure in-chassis Junos Node Slicing, the MX Series router would require one of the following REs to be installed (see Upgraded SSD size and RAM size):
RE-S-2X00x6-128 (used in MX480 and MX960 routers)
RE-MX200X8-128G (used in MX2010 and MX2020 routers)
Acronyms Used
JDM or Juniper Device Manager: A Linux container with an Ubuntu root file system. It provides an isolated “Junos-like” environment for the orchestration of VMs.
GNF or Guest Network Function: A GNF, or slice, which consists of the forwarding plane (one or more line cards from the physical router) and the control plane (the Junos VM running on an external x86 server)
BSYS or Base System: The RE on the physical router which runs Junos, creates slices, and manages the chassis
In this example, the following packages are used:
junos-vmhost-install-mx-x86-64-19.1R1-S2.2.tgzjns-jdm-vmhost-19.1-R1.6.x86_64.rpmjunos-install-ns-mx-x86-64-19.1R1.6.tgzPerform the following steps:
-rw-r--r-- 1 labroot wheel 3731607357 Aug 15 22:19 junos-vmhost-install-mx-x86-64-19.1R1-S2.2.tgz
Apply the following configuration, and then reboot vmhost on both the Routing Engines (re0 and re1).
labroot@ns-re0# set vmhost resize vjunos compact
labroot@ns-re0# set system commit synchronize
labroot@ns-re0# commit
re0:
configuration check succeeds
re1:
[edit]
'chassis'
warning: Chassis configuration for network services has been changed. A system reboot is mandatory. Please reboot *ALL* routing engines NOW. Continuing without a reboot might result in unexpected system behavior.
[edit]
'vmhost'
warning: VMHOST configuration for 'resize vjunos compact' has been changed. A vmhost reboot is mandatory. Please reboot *ALL* routing engines NOW using 'request vmhost reboot'. Continuing without a reboot might result in unexpected system behavior.
commit complete
re0:
commit complete
user@ns-re0> request vmhost reboot re1
user@ns-re0> request vmhost reboot re0
Install and configure the JDM in In-Chassis Model after copying the jns-jdm-vmhost-19.1-R1.6.x86_64.rpm image to both REs.
labroot@ns-re0> request vmhost jdm add /var/tmp/jns-jdm-vmhost-19.1-R1.6.x86_64.rpm Starting to validate the Package Finished validating the Package Starting to validate the Environment Finished validating the Environment Starting to copy the RPM package from Admin Junos to vmhost Finished Copying the RPM package from Admin Junos to vmhost Starting to install the JDM RPM package Preparing... ################################################## Detailed log of jdm setup saved in /var/log/jns-jdm-setup.log jns-jdm-vmhost ################################################## Setup host for jdm... Done Setup host for jdm Installing /vm/vm/iapps/jdm/install/juniper/.tmp-jdm-install/juniper_ubuntu_rootfs.tgz... Configure /vm/vm/iapps/jdm/install/juniper/lxc/jdm/jdm1/rootfs... Configure /vm/vm/iapps/jdm/install/juniper/lxc/jdm/jdm1/rootfs DONE Setup Junos cgroups...Done Done Setup jdm stopping rsyslogd ... done starting rsyslogd ... done Finished installing the JDM RPM package: jns-jdm-vmhost-19.1-R1.6.x86_64.rpm Starting to generate the host public keys at Admin Junos Finished generating the host public keys at Admin Junos Starting to copy the host public keys from Admin Junos to vmhost Finished copying the host public keys from Admin Junos to vmhost Starting to copy the public keys of Admin junos from vmhost to JDM Finished copying the public keys of Admin junos from vmhost to JDM Starting to cleanup the temporary file from Vmhost containing host keys of Admin Junos Finished cleaning the temporary file from Vmhost containing host keys of Admin Junos Installation Successful ! labroot@ns-re0> request vmhost jdm start Starting JDM Starting jdm: Domain jdm defined from /vm/vm/iapps/jdm//install/juniper/lxc/jdm/current/config/jdm.xml Domain jdm started labroot@ns-re0> show vmhost jdm status JDM Information --------------------------- Package : jns-jdm-vmhost-19.1-R1.6.x86_64 Status : Running PID : 8403 Free Space : 112432 (MiB)
In configuration mode, apply the configurations shown in the following example in the JDM:
labroot@ns-re0> request vmhost jdm login
Warning: Permanently added '192.168.1.3' (ECDSA) to the list of known hosts.
****************************************************************************
* The Juniper Device Manager (JDM) must only be used for orchestrating the *
* Virtual Machines for Junos Node Slicing *
* *
* Host Linux Distro: Wind River Linux *
* JDM Version: jns-jdm-vmhost-19.1-R1.6.x86_64 *
* Free Disk Space on JDM's root-fs ("/"): 129628(MiB) *
****************************************************************************
root@jdm:~# cli
root@jdm>
OR
labroot@ns-re0> start shell user root
Password:
root@ns-re0:/var/home/labroot # vhclient -s
Last login: Fri Aug 16 03:48:28 PDT 2019 from ns-re0 on pts/4
root@ns-re0-node:~# ps -aux | grep jdm
Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
root 8403 0.0 0.0 145308 1944 ? Ssl Aug15 0:10 /usr/lib64/libvirt/libvirt_lxc --name jdm --console 25 --security=none --handshake 28 --background --veth vnet1 --veth vnet3
root 9312 0.0 0.0 9536 1312 pts/4 S+ 00:01 0:00 /bin/bash /usr/sbin/jdm_gnf_console.sh qemu+tcp://hypervisor/system gnf-cgnat --force
root 10092 0.0 0.0 4412 500 pts/6 S+ 04:23 0:00 grep jdm
root 12507 0.0 0.0 61600 3088 ? S Aug15 0:00 /usr/sbin/sshd -D -f /etc/ssh/sshd_config_jdm_nv_ns
root 12525 0.0 0.0 1062656 25160 ? S Aug15 0:02 /usr/sbin/jdmd -N
root 12568 0.0 0.0 175540 668 ? Sl Aug15 0:00 /usr/sbin/jdmmon
root 20110 0.0 0.0 73164 2020 ? Ss Aug15 0:00 /vm/vm/iapps/jdm/install/usr/sbin/jlinkmon
root@ns-re0-node:~# jdm status
JDM (pid 8403) is running as server0
root@ns-re0-node:~# jdm console
Connected to domain jdm
Escape character is ^]
root@jdm:~#
set groups server0 system host-name ns-jdm0
set groups server0 interfaces jmgmt0 unit 0 family inet address 10.85.17.100/25
set groups server1 system host-name ns-jdm1
set groups server1 interfaces jmgmt0 unit 0 family inet address 10.85.17.101/25
set apply-groups server0
set apply-groups server1
set system root-authentication encrypted-password $ABC123"
set system services ssh root-login allow
set system services ssh client-alive-count-max 255
set system services ssh client-alive-interval 60
set system services netconf ssh
set system services netconf rfc-compliant
set routing-options static route 0.0.0.0/0 next-hop 10.85.17.1
Copy the SSH public key to the peer JDM.
root@jdm:~# cli root@ns-jdm0> request server authenticate-peer-server The authenticity of host '192.168.2.245 (192.168.2.245)' can't be established. ECDSA key fingerprint is 26:00:00:00:60:41:38:79:df:b0:26:f2:74:a0:e3:eb. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@192.168.2.245's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@192.168.2.245'" and check to make sure that only the key(s) you wanted were added. The authenticity of host '192.168.2.249 (192.168.2.249)' can't be established. ECDSA key fingerprint is 26:00:00:00:60:41:38:79:df:b0:26:f2:74:a0:e3:eb. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: WARNING: All keys were skipped because they already exist on the remote system.
Backup
root@jdm> request server authenticate-peer-server The authenticity of host '192.168.2.246 (192.168.2.246)' can't be established. ECDSA key fingerprint is 0d:00:00:00:92:72:3a:9d:3a:95:12:62:c7:b7:6f:67. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@192.168.2.246's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@192.168.2.246'" and check to make sure that only the key(s) you wanted were added. The authenticity of host '192.168.2.250 (192.168.2.250)' can't be established. ECDSA key fingerprint is 0d:00:00:00:92:72:3a:9d:3a:95:12:62:c7:b7:6f:67. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: WARNING: All keys were skipped because they already exist on the remote system root@ns-jdm0> edit Entering configuration mode root@ns-jdm0# commit synchronize server0: configuration check succeeds server1: commit complete server0: commit complete
ns-jdm0:~# cat /etc/hosts
127.0.0.1 localhost jdm ns-jdm0
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
127.0.0.1 redis-server
192.168.1.2 hypervisor
192.168.1.3 jdm
192.168.2.246 jdm-server0
192.168.2.245 jdm-server1
root@ns-jdm0> show server connections
Component Interface Status Comments
Host to JDM port jnpr-int-br up
Physical CB0 port eth1 up
Physical CB1 port eth2 up
Physical JDM mgmt port eth0 up
JDM-GNF bridge bridge_jdm_vm up
JDM mgmt port jmgmt0 up
JDM to HOST port bme1 up
JDM to GNF port bme2 up
JDM to JDM link0* cb0 up StrictKey peer SSH - OK
JDM to JDM link1 cb1 up StrictKey peer SSH – OK
root@ns-jdm0> show bridges
Bridge name Bridge id STP enabled Interfaces
------------------------------------------------------------------
bridge_jdm_vm 8000.ea2fbfd956ac no
vnet1
vnet2
vnet3
Bridge name Bridge id STP enabled Interfaces
------------------------------------------------------------------
jnpr-int-br 8000.32ea67f31b63 no
tap1
vnet0
Bridge name Bridge id STP enabled Interfaces
------------------------------------------------------------------
virbr0 8000.525400229690 yes
virbr0-nic
ns-jdm0:~# ifconfig
bme1 Link encap:Ethernet HWaddr 52:54:00:a5:3c:63
inet addr:192.168.1.3 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::5054:ff:fea5:3c63/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:27735 errors:0 dropped:0 overruns:0 frame:0
TX packets:18482 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2717516 (2.7 MB) TX bytes:1692391 (1.6 MB)
jmgmt0 Link encap:Ethernet HWaddr 20:d8:0b:b9:95:83
inet addr:10.85.17.100 Bcast:10.85.17.127 Mask:255.255.255.128
inet6 addr: fe80::22d8:bff:feb9:9583/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1983757 errors:0 dropped:0 overruns:0 frame:0
TX packets:954049 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2940613986 (2.9 GB) TX bytes:65717839 (65.7 MB)
Configure the Guest Network Functions (GNFs).
set chassis network-slices guest-network-functions gnf 1 description gnf-bng
set chassis network-slices guest-network-functions gnf 1 fpcs 2
set chassis network-slices guest-network-functions gnf 1 af2 description "af to gnf cgnat"
set chassis network-slices guest-network-functions gnf 1 af2 peer-gnf id 2
set chassis network-slices guest-network-functions gnf 1 af2 peer-gnf af1
set chassis network-slices guest-network-functions gnf 2 description gnf-cgnat
set chassis network-slices guest-network-functions gnf 2 fpcs 3
set chassis network-slices guest-network-functions gnf 2 af1 description "af to gnf bng"
set chassis network-slices guest-network-functions gnf 2 af1 peer-gnf id 1
set chassis network-slices guest-network-functions gnf 2 af1 peer-gnf af2
labroot@ns-re0# show chassis
network-services enhanced-ip;
network-slices {
guest-network-functions {
gnf 1 {
description gnf-bng;
fpcs 2;
af2 {
description "af to gnf cgnat";
peer-gnf id 2 af1;
}
}
gnf 2 {
description gnf-cgnat;
fpcs 3;
af1 {
description "af to gnf bng";
peer-gnf id 1 af2;
}
}
}
}
[edit]
labroot@ns-re0# commit
re0:
configuration check succeeds
re1:
commit complete
re0:
commit complete
[edit]
labroot@ns-re0#
[edit]
labroot@ns-re0#
[edit]
labroot@ns-re0#
[edit]
labroot@ns-re0# run show chassis fpc
Temp CPU Utilization (%) CPU Utilization (%) Memory Utilization (%)
Slot State (C) Total Interrupt 1min 5min 15min DRAM (MB) Heap Buffer GNF
0 Empty
1 Empty
2 Offline ---GNF initiated Restart--- 1
3 Offline ---GNF initiated Restart--- 2
4 Empty
5 Empty
6 Empty
7 Empty
8 Empty
9 Empty
10 Empty
11 Empty
Make sure that the JDM is pingable from outside. Copy the GNF image file to the JDM.
root@ns-jdm0> start shell ns-jdm0:~# ns-jdm0:~# cd /var/tmp/ ns-jdm0:/var/tmp# ls -tlr total 2732868 -rw-r--r--. 1 root root 2798451080 Aug 15 23:26 junos-install-ns-mx-x86-64-19.1R1.6.tgz ns-jdm0:/var/tmp# mv junos-install-ns-mx-x86-64-19.1R1.6.tgz /var/jdm-usr/gnf-images/
Assign this image to a GNF by using the JDM CLI command as shown in the following example:
root@ns-jdm0# run request virtual-network-functions gnf-cgnat add-image /var/jdm-usr/gnf-images/junos-install-ns-mx-x86-64-19.1R1.6.tgz all-servers server0: -------------------------------------------------------------------------- Added image: /vm-primary/gnf-cgnat/gnf-cgnat.img server1: -------------------------------------------------------------------------- Added image: /vm-primary/gnf-cgnat/gnf-cgnat.img root@ns-jdm0# run request virtual-network-functions gnf-bng add-image /var/jdm-usr/gnf-images/junos-install-ns-mx-x86-64-19.1R1.6.tgz all-servers server0: -------------------------------------------------------------------------- Added image: /vm-primary/gnf-bng/gnf-bng.img server1: -------------------------------------------------------------------------- Added image: /vm-primary/gnf-bng/gnf-bng.img [edit] root@ns-jdm0#
Configure a VNF by applying the configuration statements shown in the following example:
set virtual-network-functions gnf-bng id 2 set virtual-network-functions gnf-bng resource-template 2core-32g set virtual-network-functions gnf-cgnat id 1 set virtual-network-functions gnf-cgnat resource-template 1core-16g root@ns-jdm0# commit synchronize server0: configuration check succeeds server1: commit complete server0: commit complete
Verify that all the GNFs are up.
labroot@ns-re0> show chassis network-slices
guest-network-functions:
GNF Description State Uptime
1 gnf-bng Online 6 hours, 32 minutes, 31 seconds
2 gnf-cgnat Online 6 hours, 32 minutes, 35 seconds
labroot@ns-re0> show chassis network-slices gnf
GNF ID 1
GNF description gnf-bng
GNF state Online
FPCs assigned 2
FPCs online 2
BSYS ns-re0
BSYS sw version 19.1R1-S2.2
GNF sw version 19.1R1.6
Chassis mx960
BSYS master RE 0
GNF uptime 5 hours, 22 minutes, 56 seconds
GNF Routing Engine Status:
Slot 0:
Current state Master
Model RE-GNF-2000x1
GNF host name gnf-cgnat-re0
Slot 1:
Current state Backup
Model RE-GNF-1999x1
GNF host name gnf-cgnat-re1
GNF ID 2
GNF description gnf-cgnat
GNF state Online
FPCs assigned 3
FPCs online 3
BSYS ns-re0
BSYS sw version 19.1R1-S2.2
GNF sw version 19.1R1.6
Chassis mx960
BSYS master RE 0
GNF uptime 5 hours, 23 minutes
GNF Routing Engine Status:
Slot 0:
Current state Master
Model RE-GNF-2000x2
GNF host name gnf-bng-re0
Slot 1:
Current state Backup
Model RE-GNF-2000x2
GNF host name gnf-bng-re1
root@ns-jdm0> show virtual-network-functions all-servers
server0:
--------------------------------------------------------------------------
ID Name State Liveness
--------------------------------------------------------------------------------
1 gnf-cgnat Running up
2 gnf-bng Running up
server1:
--------------------------------------------------------------------------
ID Name State Liveness
--------------------------------------------------------------------------------
1 gnf-cgnat Running up
2 gnf-bng Running up
BNG GNF
root@ns-jdm0> request virtual-network-functions console gnf-bng
Connected to domain gnf-bng
Escape character is ^]
root@:~ # cli
{MASTER}
root@gnf-bng-re0>
root@gnf-bng-re0> show chassis hardware
bsys-re0:
--------------------------------------------------------------------------
Hardware inventory:
Item Version Part number Serial number Description
Chassis JN0000003AFA MX960
Midplane REV 03 710-013698 TR0807 MX960 Backplane
FPM Board REV 03 710-014974 KC2923 Front Panel Display
PDM Rev 03 740-013110 QC00000009F Power Distribution Module
PEM 2 Rev 01 740-063047 QC0000000D1 PS 4.1kW; 200-240V AC in
PEM 3 Rev 07 740-027760 QC0000000SJ PS 4.1kW; 200-240V AC in
Routing Engine 0 REV 05 750-072925 CALF7264 RE-S-2X00x6
Routing Engine 1 REV 05 750-072925 CALF7250 RE-S-2X00x6
CB 0 REV 03 750-055976 CADV4588 Enhanced MX SCB 2
CB 1 REV 07 750-062572 CAHR7198 Enhanced MX SCB 2
FPC 3 REV 30 750-031089 CABX5642 MPC Type 2 3D
CPU REV 09 711-030884 CABY6984 MPC PMB 2G
MIC 0 REV 31 750-028387 CAEP2018 3D 4x 10GE XFP
PIC 0 BUILTIN BUILTIN 2x 10GE XFP
PIC 1 BUILTIN BUILTIN 2x 10GE XFP
Fan Tray 0 REV 03 740-014971 TP1091 Fan Tray
Fan Tray 1 REV 03 740-014971 TP1407 Fan Tray
gnf2-re0:
--------------------------------------------------------------------------
Chassis GN0000006AA1 MX960-GNF
Routing Engine 0 RE-GNF-2000x2
Routing Engine 1 RE-GNF-2000x2
root@gnf-bng-re0> show chassis routing-engine
Routing Engine status:
Slot 0:
Current state Master
Election priority Master
DRAM 32722 MB (32768 MB installed)
Memory utilization 5 percent
5 sec CPU utilization:
User 0 percent
Background 0 percent
Kernel 0 percent
Interrupt 0 percent
Idle 99 percent
1 min CPU utilization:
User 1 percent
Background 0 percent
Kernel 0 percent
Interrupt 0 percent
Idle 99 percent
5 min CPU utilization:
User 1 percent
Background 0 percent
Kernel 0 percent
Interrupt 0 percent
Idle 99 percent
15 min CPU utilization:
User 1 percent
Background 0 percent
Kernel 0 percent
Interrupt 0 percent
Idle 99 percent
Model RE-GNF-2000x2
Start time 2019-08-16 06:39:40 UTC
Uptime 6 hours, 56 minutes, 22 seconds
Last reboot reason Router rebooted after a normal shutdown.
Load averages: 1 minute 5 minute 15 minute
0.42 0.28 0.21
Routing Engine status:
Slot 1:
Current state Backup
Election priority Backup
DRAM 32722 MB (32768 MB installed)
Memory utilization 4 percent
5 sec CPU utilization:
User 0 percent
Background 0 percent
Kernel 0 percent
Interrupt 0 percent
Idle 99 percent
Model RE-GNF-2000x2
Start time 2019-08-16 06:39:46 UTC
Uptime 6 hours, 56 minutes, 5 seconds
Last reboot reason Router rebooted after a normal shutdown.
Load averages: 1 minute 5 minute 15 minute
0.29 0.19 0.17
{MASTER}
root@gnf-bng-re0> show chassis fpc
bsys-re0:
--------------------------------------------------------------------------
Temp CPU Utilization (%) CPU Utilization (%) Memory Utilization (%)
Slot State (C) Total Interrupt 1min 5min 15min DRAM (MB) Heap Buffer GNF
3 Online 30 13 0 14 14 14 2048 10 20 2
CGNAT GNF
root@ns-jdm0> request virtual-network-functions console gnf-cgnat force
Connected to domain gnf-cgnat
Escape character is ^]
[edit]
root@gnf-cgnat-re0#
labroot@ns-re0> show chassis network-slices
guest-network-functions:
GNF Description State Uptime
1 gnf-bng Online 6 hours, 32 minutes, 31 seconds
2 gnf-cgnat Online 6 hours, 32 minutes, 35 seconds
labroot@ns-re0> show chassis network-slices gnf
GNF ID 1
GNF description gnf-bng
GNF state Online
FPCs assigned 2
FPCs online 2
BSYS ns-re0
BSYS sw version 19.1R1-S2.2
GNF sw version 19.1R1.6
Chassis mx960
BSYS master RE 0
GNF uptime 5 hours, 22 minutes, 56 seconds
GNF Routing Engine Status:
Slot 0:
Current state Master
Model RE-GNF-2000x1
GNF host name gnf-cgnat-re0
Slot 1:
Current state Backup
Model RE-GNF-1999x1
GNF host name gnf-cgnat-re1
GNF ID 2
GNF description gnf-cgnat
GNF state Online
FPCs assigned 3
FPCs online 3
BSYS ns-re0
BSYS sw version 19.1R1-S2.2
GNF sw version 19.1R1.6
Chassis mx960
BSYS master RE 0
GNF uptime 5 hours, 23 minutes
GNF Routing Engine Status:
Slot 0:
Current state Master
Model RE-GNF-2000x2
GNF host name gnf-bng-re0
Slot 1:
Current state Backup
Model RE-GNF-2000x2
GNF host name gnf-bng-re1
root@ns-jdm0> show virtual-network-functions all-servers
server0:
--------------------------------------------------------------------------
ID Name State Liveness
--------------------------------------------------------------------------------
1 gnf-cgnat Running up
2 gnf-bng Running up
server1:
--------------------------------------------------------------------------
ID Name State Liveness
--------------------------------------------------------------------------------
1 gnf-cgnat Running up
2 gnf-bng Running up
Getting Up and Running with Junos
Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search