When users upgrade an MX Series router that is running Junos OS release 15.1F5 (upgraded freeBSD) to Junos OS release 17.4 (upgraded freeBSD), they may not be able to create a recovery snapshot.
This article explains the reason that a recovery snapshot cannot be created on these routers and what must be done to successfully create one.
Note: The issue is not seen if the upgrade is from Junos OS release 15.1 (legacy) to Junos OS release 17.4.
The following error is seen when attempting to create a snapshot on the new image:
admin@RE>request system snapshot recovery
Creating image ...
Compressing image ...
Image size is 3533MB
ERROR: The OAM volume is too small to store a snapshot
This problem is caused by old packages still existing in the /packages/db
folder and the new package also being saved under the same image directory in the OAM volume. RE1800 has a 3.9G CF drive and the upgrade is causing the disk capacity to be exceeded.
Routing Engine 1 REV 08 740-041821 123456 RE-S-1800x4
ada0 28496 MB StorFly - VSFA18PI032G P1T12003591504100188 Disk 1
ada1 3919 MB 604784 00006009xxxxx000B7 Compact Flash >>>>>>>
When the Compact Flash is mounted on the router to check storage with the show system storage
command, the following details are displayed.
Note: Without mounting, the storage details are not shown in the show system storage
command.
/dev/gpt/oam 3.7G 80M 3.3G 2% /.mount/oam
To recover from this problem, you would need to delete the old packages from the Routing Engine and free up space.
To do so, perform the following steps:
- Check whether the following command succeeds before deleting any packages. If there is any need to roll back to the old version due to some failure, these packages would be needed. If these packages have already been deleted, we would need to execute the
request system software add
command and load the packages again.
admin@Good-RE> request system snapshot recovery
Jan 30 20:51:53
Creating image ...
Compressing image ...
Image size is 1541MB
Recovery snapshot created successfully
################################################
admin@failed-RE> request system snapshot recovery
Creating image ...
Compressing image ...
Image size is 3533MB
ERROR: The OAM volume is too small to store a snapshot
-
If the recovery snapshot fails, check the following directory for any unwanted packages that may be existing. In this example, the Junos OS release 15.1F5 packages still exist.
% cd /packages/sets/active/optional/
% ls -alth
total 16
drwxr-xr-x 2 root wheel 512B Jan 30 20:27 .
lrwxr-xr-x 1 root wheel 47B Jan 30 20:27 junos-openconfig -> /packages/db/junos-openconfig-x86-32-0.0.0.10-1
drwxr-xr-x 4 root wheel 2.0K Jan 30 20:25 ..
lrwxr-xr-x 1 root wheel 72B Jan 30 20:24 na-telemetry -> /packages/db/na-telemetry-x86-32-20181003.235536_builder_junos_174_r2_s1
lrwxr-xr-x 1 root wheel 72B Jan 30 19:11 jpfe-wrlinux -> /packages/db/jpfe-wrlinux-x86-32-20181003.235426_builder_junos_174_r2_s1
lrwxr-xr-x 1 root wheel 66B Jan 30 19:10 jpfe-X -> /packages/db/jpfe-X-x86-32-20181003.235426_builder_junos_174_r2_s1
lrwxr-xr-x 1 root wheel 67B Jan 30 19:10 jail-runtime -> /packages/db/jail-runtime-x86-32-20180814.8630ec5_builder_stable_11
lrwxr-xr-x 1 root wheel 46B Jan 22 20:24 junos-jam-x86-64 -> /packages/db/junos-jam-x86-64-15.1F5-S4.6-N1.9 . >>>>>>>>>>>
lrwxr-xr-x 1 root wheel 49B Jan 22 20:23 jam-mpc7-8-9 -> /packages/db/jam-mpc7-8-9-x86-64-15.1F5-S4.6-C1.9 . >>>>>>>>>>>>
lrwxrwxrwx 1 root wheel 35B Mar 24 2018 junos-x86-64 -> /packages/db/junos-x86-64-15.1F5.15
-
Remove these unwanted packages to free up space by logging in as the root
user.
su
<>
rm -rf /packages/db/jam-mpc7-8-9-x86-64-15.1F5-S4.6-C1.9
rm -rf /packages/db/junos-jam-x86-64-15.1F5-S4.6-N1.9
-
After deleting the unwanted packages, try to create a recovery snapshot again and check whether it is successful.
user@RE> show system snapshot
No non-recovery snapshots available on the Junos volume
Recovery Snapshots:
Snapshots available on the OAM volume:
recovery.ufs
Date created: Thu Jan 31 16:24:43 PST 2019
Junos version: 17.4R2-S1.2
Total recovery snapshots: 1
If the recovery snapshot creation fails even now, contact Support to troubleshoot further.