Sunday, August 10, 2008

Snort 3.0 Architecture Series Part 3: The command shell

One of the biggest user-facing changes in the Snort 3 architecture is the inclusion of a user shell interface to interact with the system. Up until now everything has been controlled strictly via the command line interface at startup time and via signals sent to the process. There are many reasons for going with a command shell that I detailed in the first part of this series but basically since Snort is designed to be able to run continuously now it was essential to have a way to interact with it.

Upon starting SnortSP you are sent directly to the command shell prompt.

[+] Loaded pcap DAQ
[+] Loaded file DAQ
[+] Loaded afpacket DAQ
[*] DAQ Modules Loaded...
[*] Loading decoder modules
[+] Loaded ethernet
[+] Loaded null
[+] Loaded arp
[+] Loaded ip
[+] Loaded tcp
[+] Loaded udp
[+] Loaded icmp
[+] Loaded icmp6
[+] Loaded gre
[+] Loaded mpls
[+] Loaded 8021q
[+] Loaded ipv6
[+] Loaded ppp
[+] Loaded pppoe
[+] Loaded gtp
[+] Loaded raw
[*] Decoder initialized...
[*] Flow manager initialized...
[*] Data source subsystem loaded
[*] Engine manager initialized
Control thread running - 3082939280 (18555)
[*] Loading command interface
[!] Loading SnortSP command metatable
[!] Loading data source command metatable
[!] Loading engine command metatable
[!] Loading output command metatable
[!] Loading analyzer command metatable
Executing etc/snort.lua
,,_ -*> SnortSP! <*-
o" )~ Version 3.0.0b2 (Build 9) [BETA]
'''' By Martin Roesch & The Snort Team:
(C) Copyright 2008 Sourcefire Inc.

If this is your first time running Snort you'll probably want to get help. Every subsystem in the Snort 3 architecture has its own help function available as do the engine modules, if you ever get lost working with a module just invoke its object help function. For example:

[*] SnortSP Commands:
set_log_level( [debug|info|notice|warn|error|critical] )
Available subsystems within SnortSP have their own help() methods:
dsrc - Data Source
eng - Dispatcher/Engine
analyzer - Analytics Modules
output - Output Modules
For example: will call the Data Source help function

As you can see, the top level module for SnortSP is called "ssp" and you can invoke its help function by calling it with a "" function. If you want to find out the data source subsystem's available functions simply call "" and so on.

One cool thing: if your system has Readline support available the Lua interpreter will pick it up automatically and you'll have standard shell functionality available within SnortSP such as command history and command line editing.

Under the covers a few things are happening. When a command is invoked in SnortSP there are a series of lookups performed by the code in the src/platform/lua_interface.c file. Lua is really handy for wrapping C function calls, it's one of the initial reasons I went with it for Snort. Let's take a look at some of the simple functionality in the lua_interface.c file for the ssp.* commands.

static int set_log_level_wrap(lua_State *L) {
int level;

if (lua_isnumber(L, 1))
level = lua_tointeger(L, 1);
const char *name = (char *) luaL_checkstring(L, 1);
if (name ==NULL) return 0;

if (strcasecmp(name, "debug") == 0) level = S_LOG_DEBUG;
else if (strcasecmp(name, "info") == 0) level = S_LOG_INFO;
else if (strcasecmp(name, "notice") == 0) level = S_LOG_NOTICE;
else if (strcasecmp(name, "warn") == 0) level = S_LOG_WARN;
else if (strcasecmp(name, "error") == 0) level = S_LOG_ERROR;
else if (strcasecmp(name, "critical") == 0) level = S_LOG_CRITICAL;
else return 0;
level = log_get_current_log_level();
S_INFO("Log level set to %s", log_level_to_string(level));
return 0;

static int shutdown_wrap(lua_State *L) {
stop_processing = 1;
return 0;

static int platform_help_wrap(lua_State *L) {
printf("[*] "PLATFORM_NAME" Commands:\n"
" help()\n"
" set_log_level( [debug|info|notice|warn|error|critical] )\n"
" shutdown()\n"
" Available subsystems within "PLATFORM_NAME" have their own help() methods:\n"
" dsrc - Data Source\n"
" eng - Dispatcher/Engine\n"
" analyzer - Analytics Modules\n"
" output - Output Modules\n"
" For example: will call the Data Source help function\n");
return 0;

static void platform_set_info(lua_State *L) {
lua_pushliteral (L, "_COPYRIGHT");
lua_pushliteral (L, "Copyright (C) 2008 Sourcefire Inc.");
lua_settable (L, -3);
lua_pushliteral (L, "_DESCRIPTION");
lua_pushliteral (L, "Network Intrusion Prevention System command interface");
lua_settable (L, -3);
lua_pushliteral (L, "_VERSION");
lua_pushliteral (L, PLATFORM_NAME" 0.1");
lua_settable (L, -3);

static const struct luaL_reg platformlib[] = {
{"help", platform_help_wrap},
{"shutdown", shutdown_wrap},
{"set_log_level", set_log_level_wrap},

static int platform_dir_create_meta (lua_State *L) {
luaL_newmetatable (L, METATABLE);
/* set its __gc field */
lua_pushstring (L, "__gc");
lua_settable (L, -2);
return 1;

static int luaopen_platform(lua_State *L) {
S_INFO("[!] Loading "PLATFORM_NAME" command metatable");
luaL_openlib(L, PLATFORM_PROMPT, platformlib, 0);
return 1;

This code sets up a "metatable" in Lua which allows us to call functions from its root with dot notation. The call to bind the root name is a little hidden in this case, it's the luaL_openlib() call in the last function above. We ended up using the PLATFORM_PROMPT substitution due to our habit of renaming the whole project periodically in its earlier days....

The array definition at the top of the code listing shows how the keywords are being mapped to the function calls themselves. When you put it all together, if you call "" it maps that back to calling the platform_help_wrap() function. If you're familiar with the internals of Snort 1.x-2x you'll know that this mapping of keywords in the interpreter to function calls is very similar to how Snort's parser worked so it was a very comfortable transition for me. Add in the validation and scripting that you can get from lua and you can do some really cool stuff!

Let's take a look at a sample lua function scripting the SnortSP architecture to demonstrate the power of this arrangement. One of the things I wanted to do with SnortSP in its early days is test out the decoders to make sure there were no huge flaws in any of them that would lead to crash, as you know Snort's decoders are "critical infrastructure" that must be fast and crash free, the rest of the system is counting on them. The function I wrote would allow a user to point Snort at a directory of pcap files and process each one sequentially until every file in the directory had been processed.

In the old days we would have had to start a new instance of Snort (2.x) for each file we wanted to process and take all the time to load and shutdown. In SnortSP you could (in theory) load a detection configuration and then process each file, all without having to restart. The way the function is setup right now it just processes the file and prints the packet dump to the screen. Here's a listing.

require"lfs"     # load filesystem functions

function rundir (path, mask) # args are path to pcap files, filename mask
once = 0 # initialize the data source once only
for file in lfs.dir(path) do # grab the path
if string.match(file, mask) ~= nil then # find files that match the mask
print("Processing File "..path..'/'..file)
if(once == 0) then # setup the dsrc params once only
once = 1
dsrc1 = {name="src1",
display="max"} # instantiate the dsrc once only"e1") # instantiate the dispatcher once only{engine="e1", source="src1"}) # link dsrc to dispatcher
print("Starting engine")
eng.start("e1") # run it
eng.run_file("e1", path..'/'..file) # just fun the file now
if(once == 1) then # process the last file too
eng.run_file("e1", "")

The first thing happening here is the inclusion of a Lua library called "lfs". That's necessary for the filesystem interface, Lua doesn't ship with one natively. (I know.) After that it's the function definition which takes two arguments, the path to the directory containing the pcap files and the filename mask to use to select the right files from the directory. The first iteration through the config data structures and engine objects are instantiated and configured and the first file is processed, after that it just processes all the rest of the files in the directory.

This is pretty cool! Practically speaking you can script the startup and shutdown of Snort and script the operation of all the different system objects as well. For example, you can make external system calls to lookup runtime parameters like interface configuration or available memory and use that information to tune your runtime configuration parameters of your engine. You can also script testing new software modules or even get into more crazy stuff like automatically turning engine modules on and off at certain times of day or pretty much anything else you can imagine.

This is just a brief tour of the functionality of the command line, there's more that can be done! I'll leave it to interested readers to explore the SnortSP command shell and I'd love to hear about interesting things you do with it.

In the next part of the series I'll start writing about the data source subsystem and its components.

Labels: , , , ,

Thursday, August 07, 2008

Snort 3.0 Architecture Series Part 2: Changes and Betas

Things have changed a bit in the Snort 3.0 world since my last post so I thought I'd provide an update as a foundation for moving forward with this "series". I promise it'll be more than one article!

In Part 1 I discussed the architecture of the Snort 3.0 technology and since then there have been some changes. The largest change has been organizational in nature. We've decided to name the core system framework apart from the overall project since you can do more than just Snort-style intrusion detection with it. So, as a result from now on we'll be calling the software framework SnortSP (the Snort Security Platform) and then the engines will be named separately. The overall architectural umbrella that this all lives under is still going to be called the "Snort 3 Architecture" and it will consist of different software components, chief among them will be SnortSP and the engine modules that utilize it.

Here's a handy reference diagram:

SnortSP engine block diagram.jpg

Ok, now that that's out of the way, let's talk about the beta. On June 30th we released the initial beta of SnortSP & the Snort 2.8.2 Engine to open source beta. It's located at To date we have done three releases of the code base with progressive versions nailing down loose ends and fixing compilation issues and the like.

We would love any feedback that people have on the betas, if you're a Snort fan you should definitely check it out and start getting your feet wet, this is the future of Snort!

For my next post I'll be spending some time talking about the SnortSP command shell and some neat stuff you can do with it!

Technorati Tags:
, , , ,

Labels: , , , ,

Daemonlogger 1.1 Released!

Daemonlogger 1.1 is available on my personal site for those of you interested in packet logging and network tapping. New features include:

  • Rollover size command line shortcuts (e.g. "-s 1M" vs "-s 1048576")

  • Disk utilization-based ringbuffer rollovers. For example, you can now tell Daemonlogger to write pcap files until the disk is 90% full and then "eat its tail" by deleting the oldest pcap file in the logging directory.

    I also fixed a bug that was found by Wesley Shields to prevent Daemonlogger from pruning files collected by previous runs of the software that were in its logging directory. This should also make it safe to have multiple instances of Daemonlogger write to the same logging directory without interfering with one another.


    Technorati Tags:
    , , ,

    Labels: , , ,