ait.core.bsc module

AIT Binary Stream Capturer

The ait.bsc module handles logging of network data to PCAP files along with the server definition for RESTful manipulation of running loggers.

class ait.core.bsc.SocketStreamCapturer(capture_handlers, address, conn_type)

Bases: object

Class for logging socket data to a PCAP file.

__init__(capture_handlers, address, conn_type)

Args:

capture_handlers:

A list of handler configuration dictionaries that contains the following values

name

A unique name for this handler

log_dir

The directory path into which log files will be written. This path may include format strings which reference handler metadata (E.g., {name}) as well as strftime format characters <https://docs.python.org/2/library/time.html#time.strftime>

Example:

'/tmp/additional_dir/test/%j'

rotate_log

True or False flag specifying whether logs should be rotated at a regular interval.

rotate_log_index

If rotate_log is True this controls the time frame of log rotations. The below values are all the valid options. Each row’s values are equivalent:

'year',    'years',    'tm_year',
'month',   'months',   'tm_mon',
'day',     'days',     'tm_mday',
'hour',    'hours',    'tm_hour',
'minute',  'minutes',  'tm_min',
'second',  'seconds',  'tm_sec',

Default:

'day'

rotate_log_delta

If rotate_log is True this controls the rotate_log_index delta between the current time at log rotation check versus the time the log file was open necessary to trigger a rotation.

Default:

1

file_name_pattern (optional)

The pattern to use for the log file name. This will be joined with the log_dir option to generate the full log file path. This may also include format strings like log_dir.

Example:

'%Y-%m-%d-randomUDPtestData-{name}.pcap'

Default:

'%Y-%m-%d-%H-%M-%S-{name}.pcap'

pre_write_transforms (optional)

A list of callables to be run prior to data output for this handler. The currently captured data is passed through each transformation in order supplied with the output of the previous being used as the input for the next.

address:

The address to which a socket connection should be made. What is considered a valid address depends on the conn_type value.

udp:

[host, port number]

E.g., ['', 8500]

ethernet:

['interface name', protocol number]

E.g., ['p2p2', 0]

tcp:

[host, port]

E.g., ['127.0.0.1', 8125]

conn_type:

A string identifying the connection type. Valid options are udp, ethernet, and tcp.

add_handler(handler)

Add an additional handler

Args:

handler:

A dictionary of handler configuration for the handler that should be added. See __init__() for details on valid parameters.

capture_packet()

Write packet data to the logger’s log file.

clean_up()

Clean up the socket and log file handles.

dump_all_handler_stats()

Return handler capture statistics

Return a dictionary of capture handler statistics of the form:

[{
    'name': The handler's name,

    'reads': The number of packet reads this handler has received

    'data_read_length': The total length of the data received

    'approx_data_rate': The approximate data rate for this handler
}, ...]

dump_handler_config_data()

Return capture handler configuration data.

Return a dictionary of capture handler configuration data of the form:

[{
    'handler': <handler configuration dictionary>,

    'log_file_path': <Path to the current log file that the logger
        is writing. Note that if rotation is used it's possible
        this data will be stale eventually.>,

    'conn_type': <The string defining the connection type of the
        logger.>,

    'address': <The list containing address info that the logger is
        using for its connection.>
}, ...]

remove_handler(name)

Remove a handler given a name

Note, if multiple handlers have the same name the last matching instance in the handler list will be removed.

Args:

name:

The name of the handler to remove

socket_monitor_loop()

Monitor the socket and log captured data.

handler_count

Return the number of active capture handlers.

class ait.core.bsc.StreamCaptureManager(mngr_conf, lgr_conf)

Bases: object

Manage handlers for binary data capture and logging

__init__(mngr_conf, lgr_conf)

Args:

mngr_conf:

Configuration dictionary for the manager. At the minimum this should contain the following:

{
    'root_log_directory': '<Root directory for log data>'
}

lgr_conf:

Configuration data for all the logger instances that should be created by default. Additional information on parameters that are required for logger initialization can be found in add_logger(). Data should be of the form:

[
   name, address, conn_type, log_dir_path, misc_conf_dict),
   name, address, conn_type, log_dir_path, misc_conf_dict),
]

add_logger(name, address, conn_type, log_dir_path=None, **kwargs)

Add a new stream capturer to the manager.

Add a new stream capturer to the manager with the provided configuration details. If an existing capturer is monitoring the same address the new handler will be added to it.

Args:

name:

A string defining the new capturer’s name.

address:

A tuple containing address data for the capturer. Check the SocketStreamCapturer documentation for what is required.

conn_type:

A string defining the connection type. Check the SocketStreamCapturer documentation for a list of valid options.

log_dir_path:

An optional path defining the directory where the capturer should write its files. If this isn’t provided the root log directory from the manager configuration is used.

get_capture_handler_config_by_name(name)

Return data for handlers of a given name.

Args:

name:

Name of the capture handler(s) to return config data for.

Returns:

Dictionary dump from the named capture handler as given by the SocketStreamCapturer.dump_handler_config_data() method.

get_handler_stats()

Return handler read statistics

Returns a dictionary of managed handler data read statistics. The format is primarily controlled by the SocketStreamCapturer.dump_all_handler_stats() function:

{
    <capture address>: <list of handler capture statistics>
}

get_logger_data()

Return data on managed loggers.

Returns a dictionary of managed logger configuration data. The format is primarily controlled by the SocketStreamCapturer.dump_handler_config_data() function:

{
    <capture address>: <list of handler config for data capturers>
}

rotate_capture_handler_log(name)

Force a rotation of a handler’s log file

Args:

name:

The name of the handler who’s log file should be rotated.

run_socket_event_loop()

Start monitoring managed loggers.

stop_capture_handler(name)

Remove all handlers with a given name

Args:

name:

The name of the handler(s) to remove.

stop_stream_capturer(address)

Stop a capturer that the manager controls.

Args:

address:

An address array of the form [‘host’, ‘port’] or similar depending on the connection type of the stream capturer being terminated. The capturer for the address will be terminated along with all handlers for that capturer if the address is that of a managed capturer.

Raises:

ValueError:

The provided address doesn’t match a capturer that is currently managed.

class ait.core.bsc.StreamCaptureManagerServer(logger_manager, host, port)

Bases: bottle.Bottle

Webserver for management of Binary Stream Capturers.

__init__(logger_manager, host, port)

Args:

logger_manager:

Instance of StreamCaptureManager which the server will use to manage logger instances.

host:

The host for webserver configuration.

port:

The port for webserver configuration.

start()

Starts the server.

ait.core.bsc.identity_transform(data)

Example data transformation function for a capture handler.