Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[STRM] Receiving dropped events message while still below the EPS license limits

0

0

Article ID: KB21340 KB Last Updated: 15 Sep 2011Version: 9.0
Summary:
This article explains the cause of the STRM dropping events over a specified time frame, where the STRM log (qradar.error) shows the eps is below the license limit.
Symptoms:
For the first 4 minutes there are messages similar to:

[SyslogSource] has detected a total of 2112341 dropped event(s). 149544 events(s) were dropped in the last 60 seconds. Queue is at 100 percent capacity.
A total of 2112341 dropped raw event(s) have been detected. 149544 raw event(s) have been dropped in the last 60 seconds. The current incoming raw event rate: 4218.87 eps is currently exceeding the license set on the system.


During the 5th minute the messages show:

[SyslogSource] has detected a total of 3016824 dropped event(s). 76741 events(s) were dropped in the last 60 seconds. Queue is at 90 percent capacity.
A total of 3016824 dropped raw event(s) have been detected. 76741 raw event(s) have been dropped in the last 60 seconds. The current incoming raw event rate: 2150.40 eps is currently exceeding the license set on the system.

Cause:

Solution:
The reason why the events continued to drop in the example above, even though the messages state the average for the minute was 2150 events per second (which is below the 2500 eps license limt), is because at the start of the minute the events were still over the 2500 eps, and only fell below the license limit about 30 seconds into the minute.

So for the first 30 seconds the events were still dropping.

The STRM allows for bursts of events without dropping events (overflow of up to 100,000 raw events), however once you continually exceed the license limit and the buffer of 100,000 events, the STRM begins dropping events. The STRM will continue to drop events until the incoming rate drops below the license limit. At this point the STRM will begin to process up to the license limit from both the log sources and the overflow queue.

Below is a table with sample data of events processing, then dropping and then cleaning up after the license is no longer being exceeded. This example is based on a 5000 eps license limit and the totals are based on 10 second intervals:

Per 10 seconds
Incoming rate
Overflow queue
Dropped Events
Average EPS for the minute
Message in Qradar.log
12:00:00
4500
0
0


12:00:10
4500
0
0


12:00:20
4500
0
0


12:00:30
5500
5000
0


12:00:40
5500
10000
0


12:00:50
6000
20000
0
5083.333333
Here the STRM reports no dropped events for previous minute, even though the license was exceeded they did not exceed the 100,000 overflow.
Next minute
12:01:00 2000
0
0


12:01:10 2000
0
0


12:01:20 2000
0
0


12:01:30 10000
50000
0


12:01:40 10000
100000
0


12:01:50 10000
100000
50000
6000
Here the STRM reports dropped 50000 events exceeding license. Average of 6000eps for the last minute
Next minute
12:02:00
5000
100000
0


12:02:10
6500
100000
15000


12:02:20
5000
100000
0


12:02:30
4500
95000
0


12:02:40
4500
95000
0


12:02:50
5000
95000
0
5083.333333
Here the STRM reports dropped 50000 events exceeding license. Average of 5083eps for the last minute
Next minute
12:03:00
10000
100000
40000


12:03:10
2000
70000
0


12:03:20
2000
40000
0


12:03:30
2000
10000
0


12:03:40
2000
0
0


12:03:50
2000
0
0
3333.333333
Here the STRM reports dropped 40000 events exceeding license. Average of 3333eps for the last minute
Next minute
12:04:00
2000
0
0


12:04:10
2000
0
0


12:04:20
2000
0
0


12:04:30
2000
0
0


12:04:40
2000
0
0


12:04:50
2000
0
0
2000
Here the STRM reports no dropped events for previous minute

At the end of the 12:03 minute, one can see how the STRM can report a drop of events because the license limit was exceeded yet the average for the minute was below the license. This is because at the start of the minute the STRM already had 90000 events sitting in the overflow buffer, and a 10 second burst of 10000 eps pushes 50000 into the overflow, but there is only room for 10000, so the STRM drops 40000.

For additional information regarding dropped events messages, see KB13554

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search