Support Support Downloads Knowledge Base Juniper Support Portal Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[QFX] Corrupted Qfabric MySQL DB replacement



Article ID: KB37048 KB Last Updated: 04 Jun 2021Version: 1.0

When MYSQL DB is corrupted on Qfabric, it should be replaced with a backed up MYSQL DB.

This article explains how to replace a corrupted MYSQL DB using a backup DB on Qfabric.


If MySQL does not start due to DB corruption, perform the steps provided in the solution on the primary DG to restore the backup.

[root@dg0~]# cli
DBI connect('sfcdb;host=db','root',....) failed: can't connect to MySQL server on 'db' (111) at /opt/demo/mgd/bin/ line 13 sfc-cli: CLI can not start. Could not retrieve LF id 
  1. Rename MySQL script to disable auto-start:

    [root@dg0 ~]# ls -lah /opt/dcf/scripts/*mysql-5.5.28
    -rwxr-xr-x 1 root 13K Aug 28  2020 /opt/dcf/scripts/mysql-5.5.28*
    [root@dg0]# date;mv /opt/dcf/scripts/mysql-5.5.28 /opt/dcf/scripts/0mysql-5.5.28 
    Wed Apr  7 16:43:39 UTC 2021
    [root@dg0 ~]# ls -lah /opt/dcf/scripts/*mysql-5.5.28
    -rwxr-xr-x 1 root 13K Aug 28  2020 /opt/dcf/scripts/0mysql-5.5.28*
  2. Stop MySQL service:

    [root@dg0 ~]# date; /opt/dcf/scripts/0mysql-5.5.28 stop                       
    Wed Apr  7 16:44:50 UTC 2021
    Shutting down MySQL.... SUCCESS!  <-- MySQL service may not be running in an error situation so this may return an error.
  3. Remove MySQL data directory:

    [root@dg0 ~]# date; rm -rf /pbdata/usr/mysql-5.5.28/data/*
    Wed Apr  7 16:46:17 UTC 2021
  4. Restore MySQL data directory from backup:

    [root@dg0]# date; scp -rp root@dg1:/mysql-backup/data.tgz /var/tmp/
    Wed Apr  7 19:30:27 UTC 2021
    root@dg1's password: 
    [root@dg0]# date; tar -xzvf /var/tmp/data.tgz -C /var/tmp 
    [root@dg0]# cp -a -R /var/tmp/pbdata/usr/mysql-5.5.28/data/* /pbdata/usr/mysql-5.5.28/data/
    Note: Wait 5 minutes after DB restore before going to the next step.
  5. Start MySQL service.

    [root@dg0]# date; /opt/dcf/scripts/0mysql-5.5.28 start
    Wed Apr  7 20:00:38 UTC 2021
    Starting MySQL.. SUCCESS! 
  6.  Rename MySQL script.

    [root@dg0 ]# date;mv /opt/dcf/scripts/0mysql-5.5.28 /opt/dcf/scripts/mysql-5.5.28
    Wed Apr  7 20:02:31 UTC 2021
  7. Check MySQL and Cluster services are running.

    [root@dg0 ~]# date; service mysql status
    Wed Apr  7 20:02:51 UTC 2021
    SUCCESS! MySQL running (9964)
    [root@dg0 ~]# date;clustat
    Wed Apr  7 20:03:24 UTC 2021
    Member Status: Quorate
    Member Name                             ID   Status
    ------ ----                             ---- ------
    dg0                                         1 Online, Local, rgmanager
    dg1                                         2 Online, rgmanager
    Service Name                   Owner (Last)                   State         
    ------- ----                   ----- ------                   -----         
    dcfservice:dcf_svc             dg0                            started  <-- Indicates MySQL is started      
    service:pbccif_svc0            dg0                            started       
    service:pbccif_svc1            dg1                            started       
    service:pbgfs_svc0             dg0                            started       
    service:pbgfs_svc1             dg1                            started       
    service:pblb_dhcp_svc0         dg0                            started       
    service:pblb_dhcp_svc1         dg1                            started       
    service:pbnfs_svc0             dg0                            started       
    service:pbnfs_svc1             dg1                            started     
  8. Delete MySQL backup data to conserve space:

    [root@dg0 ~]# date; rm -rf /var/tmp/pbdata*
    [root@dg0 ~]# rm  /var/tmp/data.tgz 
    rm: remove regular file `/var/tmp/data.tgz'? y
    [root@dg1 ~]# date; rm -rf /mysql-backup

Related Links

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search