Scriptable log filtering

The SSH Server's baseline logging settings in Advanced settings > Logging allow configuring which events are logged to the Application section of the Windows Event Log, and to the SSH Server's textual log files.

Selecting which event types should be logged does not require scriptable log filtering. Filtering is an advanced feature for administrators who wish to include or exclude events not only based on the event type, but also based on detailed content.

One example is when the SSH Server handles frequent connections from an internal process, as well as connections from external users. The administrator may wish to log activity from external users, but not the many events produced by the internal process.

This feature requires SSH Server version 9.51 or newer.

Log filter processing

For an event to be filtered, it first has to be enabled for logging using baseline logging settings in Advanced settings > Logging.

Events are enabled for logging separately for the Windows Event Log, and for the SSH Server's textual log files. Filtering is applied separately for each destination.

If any filters are configured, each filter is applied to a configurable subset of events. If an event type is enabled to be logged, but no filter is configured to process that event type, events of that type are logged as if no filtering was configured.

Scripting filter conditions

Each configured log filter defines a scripted condition. The SSH Server applies the scripted conditions to events that are about to be logged. The results of one or more configured filters determine if the event will be logged.

To discover the layout of a log event for which you wish to script a condition:

Reproduce the event which you want your condition to process. For example, perform an action which causes the event to be logged.
Find the event in the textual log files recorded by the SSH Server. By default, textual log files can be found in the Logs subdirectory of the SSH Server installation directory.

Example. The event I_SFS_TRANSFER_FILE is an Info-level event which is logged by the SSH Server when a client reads or writes a file, and then closes a file using any file transfer method - SFTP, SCP, FTPS, or BvShell. In the SSH Server's textual log files, events of this type appear like this:

This event contains the following member variables. All of these can be used in a scripted condition:

seq, time, app, type, name, desc conn.id, conn.winSesId, conn.service, conn.remoteAddress, conn.virtualAccount, conn.windowsAccount, conn.winSes channel.type, channel.id sfs.moduleName, sfs.mountPath, sfs.code, sfs.desc sfs.parameters.path, sfs.parameters.timeMs, sfs.parameters.bytesRead, sfs.parameters.bytesWritten, ..., sfs.parameters.startSize, sfs.parameters.finalSize, sfs.parameters.endedBy, sfs.parameters.download, sfs.parameters.upload sfs.help.message

When a scripted condition is applied to a log event, any sub-expression which attempts to access an element or attribute which is not present will evaluate to false. As long as other sub-expressions access attributes which are present, the condition as a whole can still produce a meaningful result. To control the result when an attribute could be absent, use the operators any and not any.

Log filter conditions use the SSH Server's scriptable expression syntax:

Scriptable expression syntax

[...] means the inside of square brackets is optional [...]* means zero or more repetitions of an optional part "..." means the quoted text appears literally ... | ... means one of the options { ... } means the inside of curly brackets appears together expression := exprElement | { ["not"] ["("] expression [combiningOp expression]* [")"] } combiningOp := "and" | "or" exprElement := ["not"] { anyExpr | basicExpr | compositeExpr } anyExpr := "any" memberName basicExpr := memberName basicOp value basicOp := "eq" | "ne" | "ge" | "gt" | "le" | "lt" | "startsWith" | "endsWith" compositeExpr := memberName ["not"] compositeOp "(" expression ")" compositeOp := "contains" memberName := path to the member variable, e.g. variable.name value := decimal number, double-quoted string, double-quoted time, or double-quoted date-time

Boolean operators

The SSH Server supports the boolean operators and, or and not. These are used as follows:

(expression) and (expression) (expression) or (expression) not (expression)

Parentheses are optional, but recommended for clarity. The following also works:

expression and expression expression or expression not expression

Comparison operators

The SSH Server supports comparison operators, all of which take the form:

variable.name basicOp value

The value can be a decimal integer; a string in single or double quotes; a boolean literal (true, false); or a date/time.

Comparison operators:

eq: true if the variable equals the value.
ne: true if the variable does not equal the value.
ge: true if the variable is greater than or equal to the value.
gt: true if the variable is greater than the value.
le: true if the variable is less than or equal to the value.
lt: true if the variable is less than the value.
startsWith: true if the variable starts with a specified string prefix.
endsWith: true if the variable ends with a specified string suffix.
contains: true if the variable contains a specified substring.

Operations on strings are case-insensitive using NTFS-like rules. Strings that contain spaces must be quoted. If a string is single-quoted, special characters are not escaped, and the single quote cannot appear in the string. If a string is double-quoted, special characters (", `) must be escaped using the grave accent (`).

Date/time values can be specified in any of the following ISO 8601 or ISO-like formats:

"HH:MM"	13:00
"YYYY-MM-DD HH:MM"	"2020-01-01 13:00"
"YYYY-MM-DDTHH:MM:SS"	"2020-01-01T13:00:30"
"YYYY-MM-DD HH:MM +/-ZONE"	"2020-01-01 13:00 +1000"
"YYYY-MM-DDTHH:MM:SS+/-ZONE"	"2020-01-01T13:00:30-2000"

If a time zone is not specified, local time is assumed.

"any" and "contains"

The any operator tests if a variable is defined:

any variable.name not any variable.name

The contains operator tests if a structure or list contains any member that matches an expression:

variable.name contains (expression) variable.name not contains (expression)

When using the contains operator in this way, parentheses around the expression are required.

Verifying filter operation

If any events were filtered since the last event was logged, the next event will include a final group element named filteredEvents. This contains the number of events of each type that were filtered since the previous event was logged. The counts are specific for the logging destination: textual log files keep a separate count from the Windows Event Log.

To check if configured log filters are being applied as intended, you can configure your log level for textual log files to Custom events, and enable the following Debug-level event:

D_SERVICE_LOG_EVENT_FILTERED

If enabled, this event is logged instead of every filtered event. This defeats the purpose of log filtering, so it should only be enabled for diagnostics. It is a Debug event so that it won't be automatically enabled if you set the log level to Trace.

If one of your filter conditions contains an error which prevents it from being evaluated, the SSH Server will log the first occurrence using the Warning-level event:

W_SERVICE_LOG_EVENT_FILTER_NOT_EVALUATED

Further occurrences of this event will be suppressed to Info severity. Check the SSH Server's textual log files for both Warning and Info versions of this event.