Support Support Downloads Knowledge Base Service Request Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

Not enough storage while upgrading Junos EX2300 and EX3400

0

0

Article ID: KB31198 KB Last Updated: 15 May 2019Version: 10.0
Summary:

Users running into this problem should first run the command, "request system software add <image> force unlink" before attempting anything else. The command might need to be run twice after erroring out the first time. A previous version of this article referred to manually deleting specific files, which in the vast majority of cases should not be needed.

EX2300 and EX3400 comes with 2GB of storage space. This article addresses some important aspects of storage space and how it is managed by Junos running on the system.

In some cases, software update on EX2300 and EX3400 can fail due to lack of adequate storage space. EX2300 and EX3400 switches come with 2GB of storage space which may fill up if images, system snapshots and logs are not properly maintained. This article addresses some important aspects of managing this storage space and solutions to address the software update issue. If this article does not help address your issue, please open a JTAC case.

Symptoms:

During a software upgrade, if there is not enough storage space, the following error will be observed:

root@juniper> request system software add /var/tmp/junos-arm-32-18.4R1.8.tgz no-copy novalidate
reboot
Removing /var/log/dcd.0.gz
Removing /var/log/wtmp.0.gz
Removing /var/log/install.0.gz
Removing /var/log/chassisd.1.gz
Removing /var/log/chassisd.3.gz
Removing /var/log/dhcp_logfile.0.gz
Removing /var/log/chassisd.0.gz
Removing /var/log/chassisd.2.gz
Removing /var/log/chassisd.8.gz
Removing /var/log/chassisd.7.gz
Removing /var/log/chassisd.6.gz
Removing /var/log/chassisd.5.gz
Removing /var/log/chassisd.4.gz
Removing /var/preserve/jdhcp_client_data
Removing /var/preserve/jdhcp_client_data_bkp
Error: not enough space to unpack /var/tmp/junos-arm-32-18.4R1.8.tgz
ERROR: Either use 'force' or consider deleting snapshots using 'request system snapshot
delete <snap>'

root@juniper> show system storage
fpc0:
--------------------------------------------------------------------------
Filesystem      Size  Used  Avail  Capacity  Mounted on
/dev/gpt/junos  1.4G  532M  790M   40%       /.mount <-- Not enough space
tmpfs           651M  48K   651M   0%        /.mount/tmp
tmpfs           324M  436K  324M   0%        /.mount/mfs
Cause:

EX2300 and EX3400 switches come with 2GB of storage space.
Storage partitioning of EX2300 and EX3400:

  • A separate Operation, Administration and Management (OAM) volume is created in the storage for recovery operations. About ~500MB of space is pre-allocated for this.
  • A Junos recovery snapshot containing full copies of the packages and configuration taken at the time of the snapshot is stored in the OAM volume.
  • The remaining storage space of ~1.5GB is used in Junos partition.

Junos install package size (compressed) is typically around 300MB. In this example, the package size of "junos-arm-32-18.4R1.8.tgz" is around 330MB. When a new image needs to be installed on the system, the Junos for EX2300/3400 installation process typically works as follows:

  • The image is copied to the switch.

  • In order to ‘unpack and install’ the image, the installation process ensures that enough free space is available on the system for the process to complete.

  • Since the process involves unzipping the image and packages inside the .tgz file, as a heuristic, it looks for free space of 3 times the package size, for the whole operation to be complete.

  • In effect (assuming the image size is ~330MB), the install operation would require 330MB x 3 = ~990MB of free space to be available.

  • While installing on a Virtual-Chassis system, each member of the virtual-chassis needs to have 3 x image size (~990 MB from example above) of free space available during the installation process.

  • Prior to initiating the upgrade, verify that all members have sufficient storage space (roughly 3 x image size) prior to initiating the upgrade to avoid failures. If a member does not have enough available space then the image installation would fail. You can verify the available storage space using the following command.

root@juniper> show system storage
fpc0:
--------------------------------------------------------------------------
Filesystem      Size  Used  Avail  Capacity  Mounted on
/dev/gpt/junos  1.4G  532M  790M   40%       /.mount
tmpfs           651M  48K   651M   0%        /.mount/tmp
tmpfs           324M  436K  324M   0%        /.mount/mfs


In this output, you can see that there is only 790M available in the Junos volume. This will not be sufficient to perform an upgrade which in this example (18.4R1) will need at least 990M free space.

Solution:

You can use some of the following methods to free storage space to complete a successful software update on EX2300/EX3400.

Method 1

Use this method if you are currently running image 15.1X53-Dxx. For installed images 18.1R2 or later, refer to Method 2.

  1. Run the "request system storage cleanup" command (won’t clean under /root).

    root@juniper> request system storage cleanup
  2. Run the " request system snapshot delete snap*" command (delete any existing snapshots).

    root@juniper> request system snapshot delete snap*
  3. Use the no-copy and force options while performing a software update. For example:

    root@juniper> request system software add /var/tmp/junos-arm-32-18.4R1.8.tgz no-copy force

    The force option frees up storage space by removing the non-recovery snapshots (if any).

  4. If step 3 fails, run the software update with the force option again (it will work the second time in some situations.)

Method 2

If you are currently running image is 18.1R2 or later.

  1. Run the "request system storage cleanup" command (won’t clean under /root).

    root@juniper> request system storage cleanup
  2.  Run the " request system snapshot delete snap*" command (delete any existing snapshots).

    root@juniper> request system snapshot delete snap*
  3. Use the unlink and no-copy option along with force option while performing a software update. For example,

    root@juniper> request system software add /var/tmp/junos-arm-32-18.4R1.8.tgz force unlink no-copy

    The force option frees up storage space by removing the non-recovery snapshots, if any. The unlink option is available starting 18.1R2. The unlink option will remove the .tgz file immediately after un-packing so that the available image size calculation takes into account the free 300M. Without using unlink, this space is not considered.

  4. If step 3 fails, run the software update with the force and unlink option again (it will work the second time in some situations).

Format Install

If the above methods do not work, you can recover the switch using the method identified in this article “ KB31265 - How to format install on EX2300 / EX3400”. Please note that this procedure will format the switch and all configurations will be lost.

Note: NSSU will not work with these options

Modification History:
2019-05-14: Removed old commands and added new data

2019-05-03: Removed sentences that might cause problems to device

2018-11-02: Minor edit. Non-technical

2018-10-29: Updated with 'force unlink'

2018-02-01: Added 'Please raise a JTAC case if the problem persists'

2017-08-07: Added example for checking snapshots in the Solution section.

2017-05-19: Added information about 3x image size requirement for the upgrade and not 1.5 times. Also added Junos 15.1X53-D56 option to use "force" command while upgrading.

2017-03-21: Removed references to OCCAM.

Related Links

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Security Alerts and Vulnerabilities

Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search