¶ What is data filtering?
All data sent to SOC.OS will be subject to filtering, with the aim of only allowing security alerts (not unwanted logging) to be processed and clustered by the system.
The filter is configured using a set of filter rules. A rule has a type, configuration, and action. Each rule is executed in order from top to bottom against the raw data. The first rule that matches the incoming data will take the corresponding action. Any further rules will not be executed.
¶ Filter Types
These are the types of rules which will be executed against the raw alert.
¶ 1. Regex
A Regex filter tests the incoming data for a match against the regular expression given in the configuration field. If the data is a positive match, the corresponding action is taken for the alert.
¶ 2. Default
A Default filter always matches the incoming data, requiring no configuration. It is the action taken against any data which does not match any other filters in the list.
¶ 3. Actions
These are the actions that can be taken when data is a positive match for a given rule.
- Drop - Dropped data will be completely ignored – it will not be processed or appear in SOC.OS clusters. Use this for data you do not consider a security alert.
- Deny - Denied alerts will not be processed or appear in SOC.OS clusters, but will be visible in the incoming alert feed. This can be useful when setting up new systems to discover what type of data is being sent to SOC.OS before it is processed.
- Allow - Allowed alerts will be processed by SOC.OS and correlated with other alerts into clusters.
¶ Examples
Example 1
The following example has 4 rules, and we’ve put a specific alert we want to drop into the test alert field. The following occurs:
-
The raw alert does not match the first rule (X), so no action is taken and it moves on to the second rule.
-
The raw alert matches the regex of the second rule (✓), so the action stated for this rule is taken - in this case Drop. This is confirmed by the red "Drop" result in the top right, next to the test alert.
-
The third and fourth rule are not executed (-) because an action has already been taken for this alert.
Note that any data not matching the three Regex rules would be allowed by the Default rule.
Example 2
In this scenario we’re expecting to receive a lot of unwanted logging data, and want to select specific alerts to be processed by SOC.OS. The Default rule is set to Drop anything not matching the prior two Regex rules. These Regex rules are configured to allow only the specific alerts we are interested in.
