CRE processor threads can die when the MAXMIND database is updated via Auto update. As a result, the JSA server would stop processing Offenses.

This could be verified by checking the event details where the "Custom Rule" and "Custom Rules Partially Matched" tables' content would be empty.

Offenses and Rules stop working.

It has been identified that CRE thread dies when the Maxmind database (used for geolocation updates) is updated via Auto Update. JSA  processing issues with the Custom Rule Engine (CRE), including Offense generation can occur due to an un-caught thread exception.

Messages similar to the following might be visible in /var/log/qradar.error on affected appliances when this issue is occurs after the Auto Update is performed:

com.q1labs.frameworks.core.ThreadExceptionHandler: [ERROR]
[NOT:0000003000][/- -] [-/- -]Exception was uncaught in thread: Preprocessor(events)_9 java.lang.InternalError: SIGBUS
    at com.maxmind.db.Reader.readNode(Reader.java:219)
    at com.maxmind.db.Reader.findAddressInTree(Reader.java:174)
    at com.maxmind.db.Reader.get(Reader.java:146)
    at com.maxmind.geoip2.DatabaseReader.get(DatabaseReader.java:151)
    at com.maxmind.geoip2.DatabaseReader.city(DatabaseReader.java:202)
    at com.q1labs.core.shared.location.LocationUtils.lookup(LocationUtils.java:531)
    at com.q1labs.core.shared.location.LocationUtils.lookup(LocationUtils.java:384)
    at com.q1labs.core.shared.location.LocationUtils.lookup(LocationUtils.java:336)
    at com.q1labs.core.types.event.NormalizedEventProperties$SourceGeographicLocation.createKey(NormalizedEventProperties.java:73)
    at com.q1labs.core.types.event.NormalizedEventProperties$SourceGeographicLocation.createKey(NormalizedEventProperties.java:65)
    at com.q1labs.cve.accumulation.ObjectArrayAccessors$ObjectArrayAccessor.getKey(ObjectArrayAccessors.java:355)
    at com.q1labs.cve.accumulation.ObjectArrayAccessors.getKey(ObjectArrayAccessors.java:265)
    at com.q1labs.cve.accumulation.ObjectArrayAccessors.buildRecord(ObjectArrayAccessors.java:233)
    at com.q1labs.cve.accumulation.Preprocessor$PreprocessTask.run(Preprocessor.java:26)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.lang.Thread.run(Thread.java:811)

When you have verified that a JSA server is experiencing the CRE issue as defined above, a restart of the ecs-ep service via the command line (SSH) on the affected appliance(s) can be used to correct the issue:

# systemctl restart ecs-ep

To prevent this from reoccurring until a patch fix is applied, you can disable updates of the maxmind/geographic data file using the following steps:

1. Go to Admin tab > System Settings / Geographic Settings

2. Set 'Disable Automatic content Updates' to 'True' (default is False)

Note: This issue is fixed in JSA 7.3.1 patch 9 (mentioned under resolved issues in release notes)