[MX] "ERROR: The OAM volume is too small to store a snapshot" message while creating a recovery snapshot on RE1800

Summary:

When users upgrade an MX Series router that is running Junos OS release 15.1F5 (upgraded freeBSD) to Junos OS release 17.4 (upgraded freeBSD), they may not be able to create a recovery snapshot.

This article explains the reason that a recovery snapshot cannot be created on these routers and what must be done to successfully create one.

Note: The issue is not seen if the upgrade is from Junos OS release 15.1 (legacy) to Junos OS release 17.4.

Symptoms:

The following error is seen when attempting to create a snapshot on the new image:

admin@RE>request system snapshot recovery
Creating image ...
Compressing image ...
Image size is 3533MB
ERROR: The OAM volume is too small to store a snapshot

Cause:

This problem is caused by old packages still existing in the /packages/db folder and the new package also being saved under the same image directory in the OAM volume. RE1800 has a 3.9G CF drive and the upgrade is causing the disk capacity to be exceeded.

Routing Engine 1 REV 08   740-041821   123456       RE-S-1800x4
  ada0  28496 MB  StorFly - VSFA18PI032G P1T12003591504100188 Disk 1
  ada1   3919 MB  604784               00006009xxxxx000B7 Compact Flash >>>>>>>

When the Compact Flash is mounted on the router to check storage with the show system storage command, the following details are displayed.

Note: Without mounting, the storage details are not shown in the show system storage command.

/dev/gpt/oam           3.7G     80M    3.3G     2%    /.mount/oam

Solution:

To recover from this problem, you would need to delete the old packages from the Routing Engine and free up space.

To do so, perform the following steps:

Check whether the following command succeeds before deleting any packages. If there is any need to roll back to the old version due to some failure, these packages would be needed. If these packages have already been deleted, we would need to execute the request system software add command and load the packages again.

admin@Good-RE> request system snapshot recovery
Jan 30 20:51:53
Creating image ...

Compressing image ...
Image size is 1541MB

Recovery snapshot created successfully

################################################

admin@failed-RE> request system snapshot recovery
Creating image ...

Compressing image ...
Image size is 3533MB

ERROR: The OAM volume is too small to store a snapshot

If the recovery snapshot fails, check the following directory for any unwanted packages that may be existing. In this example, the Junos OS release 15.1F5 packages still exist.

% cd /packages/sets/active/optional/

% ls -alth
total 16
drwxr-xr-x  2 root  wheel   512B Jan 30 20:27 .
lrwxr-xr-x  1 root  wheel    47B Jan 30 20:27 junos-openconfig -> /packages/db/junos-openconfig-x86-32-0.0.0.10-1
drwxr-xr-x  4 root  wheel   2.0K Jan 30 20:25 ..
lrwxr-xr-x  1 root  wheel    72B Jan 30 20:24 na-telemetry   -> /packages/db/na-telemetry-x86-32-20181003.235536_builder_junos_174_r2_s1
lrwxr-xr-x  1 root  wheel    72B Jan 30 19:11 jpfe-wrlinux   -> /packages/db/jpfe-wrlinux-x86-32-20181003.235426_builder_junos_174_r2_s1
lrwxr-xr-x  1 root  wheel    66B Jan 30 19:10 jpfe-X         -> /packages/db/jpfe-X-x86-32-20181003.235426_builder_junos_174_r2_s1
lrwxr-xr-x  1 root  wheel    67B Jan 30 19:10 jail-runtime   -> /packages/db/jail-runtime-x86-32-20180814.8630ec5_builder_stable_11
lrwxr-xr-x  1 root  wheel    46B Jan 22 20:24 junos-jam-x86-64 -> /packages/db/junos-jam-x86-64-15.1F5-S4.6-N1.9 . >>>>>>>>>>>
lrwxr-xr-x  1 root  wheel    49B Jan 22 20:23 jam-mpc7-8-9     -> /packages/db/jam-mpc7-8-9-x86-64-15.1F5-S4.6-C1.9 . >>>>>>>>>>>>
lrwxrwxrwx  1 root  wheel    35B Mar 24  2018 junos-x86-64    -> /packages/db/junos-x86-64-15.1F5.15

Remove these unwanted packages to free up space by logging in as the root user.

su 
<>

rm -rf  /packages/db/jam-mpc7-8-9-x86-64-15.1F5-S4.6-C1.9
rm -rf  /packages/db/junos-jam-x86-64-15.1F5-S4.6-N1.9

After deleting the unwanted packages, try to create a recovery snapshot again and check whether it is successful.

user@RE> show system snapshot 
No non-recovery snapshots available on the Junos volume

Recovery Snapshots:
Snapshots available on the OAM volume:
recovery.ufs
Date created: Thu Jan 31 16:24:43 PST 2019
Junos version: 17.4R2-S1.2

Total recovery snapshots: 1

If the recovery snapshot creation fails even now, contact Support to troubleshoot further.

Knowledge Base