Support Support Downloads Knowledge Base Juniper Support Portal Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[Junos Space] Data is inconsistant between sites after DR Failover



Article ID: KB33789 KB Last Updated: 30 Aug 2019Version: 1.0

When Junos Space Platform is configured for DR operations, the command 'jmp-dr start'  backs up the space database on the active site, and restores it on the second.

Under certain conditions, the backup is incomplete and the partial backup is restored on the standby DR site.  The commands do not report any error occurred to the user.

This issue is seen in Junos Space Platform 17.2R1, fixed in version 18.3R1, and likely impacts all versions before 18.3R1.


When failing over to DR site, data is incorrect or missing.  [Do NOT start DR with the bad site as the active, failover is ok]

Check if the backup finished:

  1. Look at the log, /var/log/jmp-geo/dr-cli-start.log on the (Original) Active/VIP for lines similar to:

    Tue Aug  7 07:23:51 2018: start MySQL dump ------ begin
    <Other Line removed>
    mysqldump: Error 2020: Got packet bigger than 'max_allowed_packet' bytes when dumping table `IdpVersionsEntity` at row: 0
    Tue Aug  7 07:24:44 2018: start MySQL dump ------ end: 0

    If similar error is seen, you have encountered the issue, the DB backup is incomplete.

  2. To manually see if the DB backup finished, from the VIP node of the Active DR site:

    zcat /var/cache/jmp-geo/data/db.gz | tail -n 5

    Successful DB backup for DR start will end with a line like below:

    -- Dump completed on 2018-08-07 21:13:52

    Failed backup, will be ANY other line as the last


DR Start backup script has no error checking for the DB backup command. Due to the size of some of the data, the backup may be incomplete.

This is the most likely cause and workaround.  After applying this workaround, if the database backup continues to be an incomplete file, contact your JTAC representative.


This issue can be resolved by increasing the max_allowed_packet value for the mysqldump utility command used by the DR start command.

  1. Stop DR (One node):

    jmp-dr stop
  2. Increase the max_allowed_packet size for DR utility scripts (Both sites, All Space JBoss and DB nodes):

    1. Check whether the utility already has max_allowed_packet option for mysqldump command as below on all the nodes on both sites:

      grep -oP '(?<=max_allowed_packet=)[0-9]*G'   /var/cache/jmp-geo/backup/script/ /usr/nma/lib/
    2. If there is NO output, the config needs to be added with the 2 commands:

      sed -i.bak -e 's/mysqldump/mysqldump --max_allowed_packet=2G/g' /usr/nma/lib/
      sed -i.bak -e 's/mysqldump/mysqldump --max_allowed_packet=2G/g' /var/cache/jmp-geo/backup/script/
    3. After these 2 commands, use the show command again:

      grep -oP '(?<=max_allowed_packet=)[0-9]*G'    /var/cache/jmp-geo/backup/script/ /usr/nma/lib/

    4. You should see output similar to:

  3. Start DR

    jmp-dr start

Related Links

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search