http://www.secuobs.com Observatoire de la securite Internet fr webmaster@secuobs.com 2009-06-08 20:06:40 - Security Sauce : I found myself with 9 hours to kill on an airplane ride this weekendso I coded up the two features I've been hearing the most for theoriginal IP Blacklisting patch I wrote The first new feature was tobe able to associate a name with a blacklist and have that nameproduced in the event that Snort outputs The second feature was to beable to load blacklists from external files so that very largeblacklists could be maintained without having to modify the snortconffileBoth of these features are now available in version 2 of the patchDirect loading of the IP address lists from the snortconfpreprocessor directive is no longer supported, you have to use theexternal filesHere is a sample directive for snortconf:preprocessor iplist: blacklist dshield /etc/snort/dshieldblacklist blacklist sourcefire /etc/snort/sourcefireblacklist whitelist /etc/snort/defaultwhitelistAnd here is a sample blacklist file:# This is a blacklist file, there are many like it but this one is mine# Comments are supported10110/24 19216800/16 # I can do inline comments too and put# multiple CIDR blocks on one line172161617/32172161514/32 # Whatever you likeAs per usual, bug reports and feature requests can be sent directly tome I still haven't done any performance testing of this code so yourmileage may vary I'd be interested to hear of any comparisons of theperformance of this code vs the Emerging Threats blacklistTested on Ubuntu, Fedora and OS X only so farYou can get the patch here:http://wwwsnortorg/users/roesch/code/iplistpatchv2tgzTechnorati Tags:cybersecurity, open source, snort, sourcefire, toolsIMAGEhttp://www.secuobs.com/revue/news/107064.shtmlhttp://www.secuobs.com/revue/news/107064.shtml Secuobs.com : 2009-05-14 00:32:49 - Security Sauce - After a discussion on the Snort-users mailing list last week regardingusing standard Snort rules to implement Reputation-based IP blockingin Snort and how badly the performance sucked I decided to writesome code to do it the "right way" The result is the "iplist"preprocessor, a module that supports IP Blacklisting and whitelistingvia user-provided lists of known hostile IP addressesThe internals of the system use the Patricia Trie code from the Snort30 code tree to provide the primary address lookup mechanismCurrently I'm only supporting IPv4 addresses although the P-Trie codesupports IPv6 addressing tooThis patch has been applied against Snort 2841 only I've testedbuilds on OS X, Ubuntu and Fedora so far It requires libdnet ordumbnet-dev for those of you on Debian-based distros to buildproperly Check the README file that comes with it for instructionson patching it into your codebase It supports inline blocking andalerting but not Flexresp-style TCP reset session shootdownsHave a look and let me know what features you'd like or bugs you findThis code is purely EXPERIMENTAL, this is just me spending some of myspare time doing a fun coding project so if your machine sprouts legsand refuses to work until it receives part of the TARP bailout it'snot my faultHere's the link:http://wwwsnortorg/users/roesch/code/iplistpatchtgzTechnorati Tags:open source, sourcefire, tools, snort, cybersecurityIMAGEhttp://www.secuobs.com/revue/news/96010.shtmlhttp://www.secuobs.com/revue/news/96010.shtml Secuobs.com : 2009-04-19 20:50:15 - Security Sauce - I'll be out at RSA in San Francisco all this week I'm speaking at theAmerica's Growth Capital conference tomorrow at 2:30 on "Snort and TheFuture of Real-Time Adaptive Network Security" Wednesday I'll bedoing a Peer2Peer talk at RSA on "The Future of Snort", primarilyfocusing on Snort 30 and upcoming changes to snortorgFor those of you attending RSA I hope to meet you out thereIMAGEhttp://www.secuobs.com/revue/news/85343.shtmlhttp://www.secuobs.com/revue/news/85343.shtml Secuobs.com : 2009-04-02 23:48:13 - Security Sauce - It's been quite a while since the last Snort 30 beta and yesterday wereleased Beta 3 The reason that it's taken so long to get out thedoor is that we decided to start doing performance analysis of theSnort 28x analytic engine that was ported over to run on top ofSnortSP and the results were interestingWhen I started developing and designing the Snort 30 architecture oneof the assumptions that I based my design around was that multi-corecomputing environments were going to be the norm rather than theexception in the platforms we'd target moving forward With that inmind and knowing the typical packet analysis load we place on machineswith Sourcefire's applications I was looking to utilize CPU cyclesmore efficiently by performing common processing that happened forevery packet acquire/decode/flow/stream/etc once and then spreadingthe analysis Snort/RNA/etc across the cores in separate threadsThis would allow us to perform parallel processing across analyticengines while only having to perform the common processing tasks theyall have to do onceSnort_3_Arch_ThreadedpngThis is great in theory and seemed like we'd see some real performancegains but when we started looking at the Snort 283x analytic enginethat we put out in the original Beta release we saw that performancewas not where we wanted it to be relative to the monolithic Snort283x architecture Initially we thought that the performancediscrepancies were due to inefficiencies that were a byproduct ofporting the monolithic Snort 2 packet processing engine to run in athreaded environment due to the incorporation of things likethread-local storage TLS Lots and lots of analysis was done overseveral months, tons of optimizations were made by Russ Combs and RonDempster, Snort's run-time processing cycle was studied at length andat the end of the day the performance still wasn't where we wanted itEventually we arrived at the conclusion that the performance issues wewere seeing were stemming from the way that modern Intel CPUs use andsynchronize cache memory If you'd like to see some more in-depthdiscussion on the Intel CPU caching architecture you should take alook at Gustavo's site and check out the articles on caching Theperformance hangup we ran into really manifested itself as we tried todistribute traffic across cores on multiple physical dies, theoverhead incurred by the data transfers and cache coherency operationsrequired by the cores was costing us lots of CPU cyclesWhat has become apparent from performing our analysis and extensiveexperimentation is that data spreading across the current generationof multi-core Intel CPUs is not something that works well forreal-time applications like IPS Intel CPUs really seem to favor themore traditional load balancing approach that's been used successfullywith the Snort 2x code base for years where independent processes arelocked to separate cores and flows are unique to each respectiveprocessWhile we were exploring the performance envelope of the Snort 3 codebase we looked at a number of different data distributionarchitectures to move data from the data source subsystem of SnortSPto the analytic engines ie the Snort 283 engine module A modelthat we've found to work well is what we've come to call the "stacksmodel" The stacks model works a lot like the Snort 2x preprocessorsubsystem but on a somewhat larger scale Instead of running severalanalytic engines in separate threads with each thread locked to a CPU,the stacked model runs the engines in the one thread and calls themsequentially, passing the packet stream from engine to engine Thestacks model is included in Beta 3 and is a compile-time option forrunning the systemSnort_3_Arch_StackedpngOne model that we haven't benchmarked extensively is what I'll call a"single core multithreaded model" where we run one thread per analyticengine but lock them all to the same CPU core, eliminating the cachecoherency and sync overhead while paying the price of heavier loadingon the individual CPU cores This will be an area of further researchdown the roadWe are planning on standing up a public CVS server in the near futureto host the Snort 3 code so that we can foster better interaction withthe open source communityThe next big hurdles to get over with Snort 3 are development of theTCP stream management subsystem and the Snort 30 analytic enginemodule Stay tuned for more and more regular releases as we getrolling on these subsystemsIMAGEhttp://www.secuobs.com/revue/news/78623.shtmlhttp://www.secuobs.com/revue/news/78623.shtml Secuobs.com : 2009-02-17 05:00:41 - Security Sauce - I'm posting this because I couldn't find this information succinctlyproduced in any corner of the web and I wanted people who had the sameproblem to have a place to go to find answersHere's the scenario A loved one drops their 1st gen iPhone into abody of water sink/tub/toilet/lake and, effectively, bricks it Whenit dries out enough to boot back up it goes into Recovery Mode andasks you to plug it into iTunes so it can be reimaged and reset to thefactory defaults erasing all the data on the phone in the processNormally this is no big deal, assuming of course the phone has beensync'd recentlyWhat if it hasn't What if there's precious data on there likepictures of your children that your wife took and loves dearly thatshe never backed up off the phone for whatever reasonYou, my friend, have a problemIf you search the web for "iPhone recovery mode save data" or whateveryou're going to get back a pile of results that basically say stufflike "take it to the Apple Store and let a Genius look at it" Whenyou do that, they'll tell you you're hosed, but of course they'll becool about it and you'll have the opportunity to buy some stuff whileyou wait in line to talk to themYou can also call a data recovery service They can bust open youriPhone and try to extract the media from its memory for the low lowprice of ~$900+ Generally speaking, that's not going to be an optionfor almost anyoneShouldn't a hacker have had this problem and solved it by nowAs a matter of fact, at least one has If you find yourself in thissituation and you want to be all heroic and stuff then go get yourselfZiPhone Among the tricks it can perform on your iPhone is theextremely handy trick of taking it out of Recovery Mode and back toNormal Mode Just launch ZiPhone and select the "Reboot in NormalMode" option from the menu bar Once you've done that you can justsync the phone as normal and save off all your data Once you've donethat you can then turn to your wife and receive your accolades andjust dessertsAnyway, I hope someone finds this post more helpful than the usualpile of useless results Google returns on the topichttp://www.secuobs.com/revue/news/62378.shtmlhttp://www.secuobs.com/revue/news/62378.shtml Secuobs.com : 2009-02-17 03:27:52 - Security Sauce - beachball_of_deathjpgI've been using 15" Mac laptops since about 2002and I love them dearly As general purpose computing platforms for allof the various things I have to do coding, presentations,communications, etc they are by far the best blend of power andfunctionality I've ever had Having said that, I've been pulling myhair out lately because my latest machine, a ~1 year old 24GHzmachine with 4GB of RAM and a hand upgraded hard drive, has beencrashingGenerally what's happening is the machine will operate fine for someperiod of time, several hours to several days At some point I'll have2-7 apps open and one of them, usually Firefox, will beachball Whenthis happens I can switch away to another app but then within 30seconds or so that app will go down, then the menu bar becomesunresponsive, I can't launch any new apps and that's all she wroteIt's reboot timeThis has been going on for about a month now and I've tried a varietyof solutions in terms of running repairs and all that jazz with noluck At this point I'm ready to throw in the towel In addition tobeing a machine that I write code on this is also a machine that I useto do presentations in front of rooms full of people A lockup duringa presentation would be an unacceptable embarrassment so tomorrow I'llbe taking this machine back to Sourcefire IT and getting a new MBPHopefully they can figure out what the problem is or send it to Appleand let them fix ithttp://www.secuobs.com/revue/news/62332.shtmlhttp://www.secuobs.com/revue/news/62332.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - Blogging I've been on the fence for quite a while regarding startingmy own blog but I decided it was about time to jump in since I findmyself with quite a few thoughts about goings on in the circles inwhich I travel So, hopefully this will be a pretty active blog andI'll keep it up to date and all of the other good things If nothingelse, it'll give me something to do when I'm on airplaneshttp://www.secuobs.com/revue/news/52029.shtmlhttp://www.secuobs.com/revue/news/52029.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - I was supposed to attend the BlackHat Briefings this year for thefirst time since 2001, but I've been called away to Londonunexpectedly I haven't been able to attend since then due to itsproximity to my first daughter Molly's birthday and, finally, thisyear the conference was moved a week and I was going to attendOh well, maybe next year I can try to speak, Snort 30 should be worthshowing off by then Hm, maybe I should try to go to WWDC next weekinsteadTechnorati Tags: Conferenceshttp://www.secuobs.com/revue/news/52028.shtmlhttp://www.secuobs.com/revue/news/52028.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - I wrote my first Cocoa app for OS X a few months back when everyoneincluding me thought Check Point was buying my company Silly meThe result is this nifty little program CtrlAltItsNeat What's it doPretty simple, when you hit opt-cmd-delete on the keyboard, it pops upa little dialog that looks like this:Look familiar It should, it's essentially the same dialog thatWindows pops up when you hit the ctrl-alt-delete key combo What's thepoint I hated having to move the mouse pointer down into the screencorner to do screen locks, I'm a UNIX guy and I like to use keyboardchords when possible to do simple tasks fast Tasks like locking thescreen, which is what this program does if you follow the initialchord up with the Enter keyThere are a few caveats with this program First, it's a Universalapp, so it should work on PPC and x86 Macs just fine Second, Ihaven't implemented preferences yet, so you're stuck with the defaulthotkey chord to activate the dialog Yes I know, lameI've thought of some obvious improvements that could made, such asassigning the buttons to any app or possibly system function, butfor what it does it works well I'm also thinking about open sourcingit, although I'd have to talk to Sourcefire legal before I could doanything like that For now I'm just going to post it up where peoplecan get at it and if I get some requests for improvements, I'll behappy to continue to release them for free for the time beingEnjoyDownloadTechnorati Tags: Cocoa, CtrlAltItsNeat, OS X, Programminghttp://www.secuobs.com/revue/news/52027.shtmlhttp://www.secuobs.com/revue/news/52027.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - It's 100+ degrees here in the DC area today so getting on a plane toLondon with highs in the high 60's this week doesn't seem like sucha bad thing all of a suddenTechnorati Tags: Travelhttp://www.secuobs.com/revue/news/52026.shtmlhttp://www.secuobs.com/revue/news/52026.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - I flew last week for the first time since the "Hair-gel Bombers" newsbroke and greatly enjoyed the new additions to the security theaterthat TSA has so lovingly rolled out for the fall season BWI had newsigns up all over declaring that the threat of a terrorist attack wascurrently high HIGH You had a good chance of having to fightterrorists to the death by getting on a plane last week, according toTSA Enjoy your flightImagine my surprise when I got home yesterday and found my 1/6th ozbottle of contact lens rewetting drops in one of the pockets on mylaptop bag I sailed through 2 security check points with this highlydangerous contraband too, this wasn't a one time event Imagine thedamage I could have done if that was filled with binary explosiveliquids, I probably could have blown off a few fingers with it andsuffered some temporary hearing lossLet's face facts people, airport security screening lines are thephysical world equivalent of a classic intrusion detection system Wehave traffic people and luggage analyzers xray machines, metaldetectors, explosive residue detectors which use deterministic setsbad things: guns, knives, gel bras to produce informationcontinuously about things that may or may not be bad which then mustbe contextualized by humans Humans are notoriously lousy at doingthis unless you can constrain the data set heavily or can somehowautomate the contextualization process like we do at SourcefireGuess what The dedicated individual manning the xray machine has alife span of attentiveness when they're looking at bag after bag goingbuy on the screen Has anyone else notice the ergonomic purgatory thatxray screeners have to endure Sit on an armless, backless stool andlook up at a 20-45 degree angle at item after item sailing through themachine How long before neck, back or shoulder pain affects thescreener's attentivenessWhat are we to suppose is the duty cycle of a baggage screener in atypical American airport with a rate of data flow of 172 passengersper hour with two shoes and two carry-ons per passenger The screenergets a whopping 52 seconds to pattern match for the entire set of badthings per item per passenger I'm not taking account of time spent inspool up/spool down periods for starting/stopping the belt either, sowe're probably talking about half that effectively 26 seconds Mylaptop bag currently contains a PowerBook, Airport Express, digitalcamera, airplane power adapter, iPod, EVDO PCMCIA card, cell phone,two laptop batteries, 80GB portable firewire hard drive, laptop poweradapter and a bag filled with various wires and other widgetry to makeit all plug together Not to mention books, pens and other businessstuff 26 seconds to positively identify all of that asnon-dangerous Let's be generous and call it 3 seconds If the set ofthings that need to be detected signatures is constrained to guns,knives and bomb materials, I'd say grudgingly that a motivatedscreener could maintain alertness through their entire period manningthe machine to have a reasonable probability of detection of thethings in the set of threats Once you extend that signature set to,well, pretty much everything that's not paper or cloth you're going tohave an analysts nightmare because you just did the equivalent of"alert ip any any - any any msg: "Something bad may havehappened";" in SnortIn an environment like this with no automated analysis tools, theanalyst is quickly overwhelmed by the level of noise false positivesthey have to deal with Once the analyst is saturated, their abilityto differentiate an iPod from a brick of C4 rapidly goes to zero, thebrain gives up or goes on to think about other things If the TSAwants screeners spending time looking for every tweezer, nail clipperand bottle of contact lens cleaner, they're asking for overload oftheir staff and reducing its ability to be effective In the intrusiondetection world we would call what they're doing "de-tuning thesensor", they're making it detect things much more frequently byincreasing the number of things that will set it off with no regardfor the effect that it will have on the human analysts and no tools tohelp them cope with the increased information flooding themThis problem is well known in the network security world as the "boywho cried wolf" syndrome and its the problem that led to networkintrusion detection as a technology being discredited back in2002-2003 The root cause of this problem was insufficientlysophisticated sensing technology that relied completely onbored/untrained/distractable humans to deal with staggering amountsof data to make the technology useful We've wised up since then andbuilt technology to address this root cause by automating thecontextualization of data sets and giving the intrusion detectionengines the capability to be self-tuning without the need for humanintervention well, I'm still working on that partIf we're going to increase the "number of signatures" in meatspace toinclude everything electronic, metal or liquid, we'd better find a wayto have smart, automated contextualization of the data, otherwisewe're doomed to failure and someday we'll see a report out of anewspaper or GAO office declaring "Airport Security Screening isDead"Technorati Tags: Flying, Futility, Meatspace, Security, IntrusionDetection, Travelhttp://www.secuobs.com/revue/news/52025.shtmlhttp://www.secuobs.com/revue/news/52025.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - I had an "entertaining" time getting a handle on OS X's firewall lastnight on my newest Mac Mini For those of you unfamiliar with the OS Xfirewall's inner workings, I suggest you take a look at Jay Beale'sDefCon presentationHere's the basic problem with OS X's firewall and it's configuration:It's not transparent to the user The OS X firewall configurationinterface is highly simplified and accessible but omits criticalinformation about ports that it is leaving open regardless of thestate indicated by the configuration GUI For security conscious userslike myself, it is unacceptable for the system to lead me to believethat I've blocked all ports but SSH for example and "stealthed" themachine when in fact it has left several other service ports wide openfor access by anyone on the network and not stealthed the machine inany credible mannerApple needs to take this seriously I can understand hiding complexityin UNIX systems from the user in a consumer OS , but when I ask forAdvanced configuration what I'd like to see is an honest summary ofwhich ports are not being blocked on my network interfaces It doesn'tseem like too much to ask but for some reason you just can't get thatinformation from your Mac unless you're willing to go out to thecommand line and punch in some ipfw commands yourself As has beenillustrated time after time, the cost of doing security right thefirst time is far less than fixing it laterOnce I achieved frustration with the transparency of Apple's firewallpreferences panel, I figured that someone else must have had thisproblem in the past and solved it with a good and completelytransparent firewall configuration GUI for OS X I checked out FlyingButtress and it was pretty good, but it seemed to have issues withhanging from time to time and it was difficult to get the actualfirewall status and configuration after a policy apply It had furtherproblems once I accidentally tried to explore Apple's FirewallTool andit reestablished the default configuration in addition to FlyingButress's, the worst of both worlds I then tried out Firewall Builderbut there was all sorts of flakiness with it in terms of the GUIdisplay as well as its intuitiveness Then it crashed on me, and sincethey wanted money for that version I decided to pass on working withit any further I checked out a couple other interfaces but they allhad issues with transparency, utility and functionality FlyingButtress was the closest, but it was still hiding configurationoptions from me while not displaying all of the ports it was leavingopen It does have an "expert" display mode that shows the exactconfiguration used, but it seems to be impossible to reflect anychanges made to the expert mode in the regular GUI modeThis is the exactly the sort of thing that gets people to start OpenSource Projects and normally I'd use this as the perfect lead-in tostarting to do some work with Cocoa which I am deeply interested inbut I have another project on the front burner right now more aboutthat laterThis is exactly the kind of thing that Apple excels at, makingcomplexity accessible to users If they can make real-time videoediting, movie creation, music creation and the like as easy andaccessible to users as they have, they certainly can take on a tasklike configuration of the firewall for both casual and advanced usersin a way that will be useful for both the casual and pro crowdsFor the record, at the end of the day I located this post which wasderived from this article and used the info to create my finalfirewall rule set manually with vi and some scripting Not exactlyApple-simple, but it got the job doneSo, anyone want to make a real firewall GUI for OS X Is this anyoneelse's Dream AppTechnorati Tags: OS X, Firewalls, Securityhttp://www.secuobs.com/revue/news/52024.shtmlhttp://www.secuobs.com/revue/news/52024.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - Guy Kawasaki has an interesting interview with the CEO of MySQL wherethey touch on one of my pet theories regarding the value of opensource development From the article:Question: How do you make money with an Open Source productAnswer: We start by not making money at all— but by making usersThe vast community of MySQL users and developers is what drivesour businessThen we sell an enterprise offering to those who need to scale andcannot afford to fail The enterprise offering consists ofcertified binaries, updates and upgrades, automated DBA services,7x24 error resolution, etc You pay by service level and thenumber of servers No nonsense, no special math Enterprisesoftware buyers are tired of complex pricing models per core, percpu, per power unit, per user, per whatever the vendor feels likethat day—models that are still in use by the incumbentsAt MySQL we LOVE users who never pay us money They are ourevangelists No marketing could do for us what a passionate MySQLuser does when he tells his friends and colleagues about MySQLOur success is based on having millions of evangelists around theworld Of course, they also help us develop the product and fixbugsAnd the few times that they say that they hate MySQL, that helpsus too because complaints usually contain some good suggestion forimprovementMaking money by creating users, some small percentage of which willeventually pay you money to solve the hard problems that they run intousing your open source technologyBack around the time that Nessus went to its new licensing model I gotquite a few questions from people regarding the open source businessmodel that Sourcefire operates under and whether they could expect usto follow suit and go to some non-OSI approved license My responsewas always "No, we'd be crazy to do that"The reason I say that is because I believe that the value of an opensource technology is not the technology that it implements beyond acertain point, it has to do something interesting and do it well Thevalue of an open source technology to the company that develops andsupports it is the community that grows around it It's pretty obviousthat the community that grows around your project is your potentialcustomer base, the thing that may not be obvious is that they are alsoa strong part of your marketing team My observation is that opensource users have a tendency to be evangelistic and that evangelismcan go a long way towards getting your company in the door at theircompany, as well as at the companies of the friends that they haveAdditionally, the guys who use open source tools when they're eitheryoung with no money eg proverbial college students or tasked withinvestigating a technology before getting into a formal deploymenteg proverbial IT security guys with hot tasking from on high startwith open source products and will stick with them if they have a goodexperience Guys who learned Snort in college in 1999-2000 are ITdirectors/managers/VPs now and having them familiar with andpossibly fond of the technology is a big deal for us at SourcefireBack in 2002 when Sourcefire was ~10 people and we'd lose a deal toopen source Snort my philosophy was always that it was not a big deal,the customer would be back when the problems got sufficiently hard andthey'd think of us first as the place to go for a solution if wecontinued to deal in an even handed fashion with the user communityand continued to advance the productAdvancing the product is a big deal too Some have theorized thatdoing things like adding a new detection engine to Snort that could dogigabit speeds and then giving it away was a Bad Idea because itallowed our Snort-based competitors to have a more level playing fieldwith which to compete against us My opinion is that it keeps the ballmoving forward and keeps people's eyes on what we're doing instead ofletting them get bored and going off to check out some other morerapidly developing OSS technology or a commercial solution Lettingyour technology get stagnant is almost as bad as closing thetechnology, once the community is bored they'll be looking elsewherefor something exciting One important point to note in this regard ina product company is that just because you're releasing advances tothe open source community at large doesn't mean that you are requiredto drive your differentiation from that technology to zero If youwant to be able to get people to want to pay for what you do, thenhaving some sort of key differentiation is a must At Sourcefire wedid things like developing a complementary technology that allowed usto address one of the toughest problems in the intrusion detectionworld, false positives If you can't maintain differentiation againstyour open source product or your competitors that use your open sourcetechnology, then you've got a problem that you need to get creativearound, closing the technology isn't an acceptable answer in myopinionOnce you've open sourced your technology then you have to approach itscontinued development as a community building exercise that works bestby advancing the technology and trying to maintain community-friendlypolicies and programs If you do this and try to be clueful aboutinteracting with the open source users as the company grows a wholedifferent topic then you have the foundation necessary to build abusiness of substance That's the principle that I originally builtSourcefire on and so far it has worked pretty wellTechnorati Tags: Open Source, Philosophy, Snort, Sourcefirehttp://www.secuobs.com/revue/news/52023.shtmlhttp://www.secuobs.com/revue/news/52023.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - I've mentioned before that I spend a lot of time in airports and am abig fan of security theater, but today I had a revelation that was onpar with what Neo must have felt like when he woke up in thatslime-filled pod I read yesterday with no small amount of joy thatthe War on Liquid was being relaxed and that, unlike the day prior,certain liquids were no longer dangerous explosives but instead hadbeen declared innocuous and were super duper ok to bring on aircraftonce again I was relieved that I'd be able to bring saline solutionand toothpaste on board without checking my bags with the attendant30-60 minute baggage delay on landing or resulting to smuggling 1/2 ozbottles through security in my pockets or discretely in my carry-onsLittle did I know that there was a caveat to this major victory: theseliquids are only safe for travel if they're contained within a clearplastic 1-quart zip-loc baggie I discovered this the hard way, Ibrought my shaving creme too Now, I had seen the announcementsyesterday, including the caveat of putting things in a clear plastic1-quart zip-loc baggie, but I didn't understand that the containeraround the liquids was the thing that was actually ensuring the safetyof the aircraft and all those aboardWhen I arrived at the check point there was a TSA person theredeclaring loudly for everyone to hear that the key to salvation was infact the clear plastic 1-quart zip-log baggie and that anything insidewas ok to travel and anything that wasn't in the bag was contrabandand hazardous to the national security of the United States At thispoint, I knew that there was going to be trouble because I had broughta 2oz travel can of shaving creme but I DIDN'T HAVE A CLEAR PLASTIC1-QUART ZIP-LOC BAGGIE WITH ME Now, not wanting my 2/3oz bottle ofsaline solution or travel toothpaste I was smuggling in my bag to getconfiscated, I came up with a planWhen I got I up to the head of the line I took out my shaving cremeand said "I'm sorry, I don't have a clear plastic 1-quart zip-locbaggie with me to contain this deadly shaving creme, do you guys haveany" I may have left out the "deadly shaving creme" part Anyway,they said "No, that can't go" There were no trash cans handy so Iasked what I should do with it and they said to put it in one of thesmall trays and send it through the X-ray For a second I thought thatsanity had prevailed and that they were going to apply criticalthought to the situation, but I was relieved to see that mindlessobedience to the rules won the day Upon emerging from the machine,one of the TSA people grabbed the tray and said "who's shaving cremeis this" I indicated it was mine and he shook his head and said"that can't go" and promptly chucked it in the garbage As I wasputting my shoes back on and packing away my laptop, he wandered byand I thought to ask him a questionMe: "Just so I'm clear, if I put that shaving creme in a clear plastic1-quart zip-loc baggie that would have been fine"TSA guy: "Uh, yeah"I made eye contact with him, shaking my head and he looked back at mefor a couple seconds before he cracked a sheepish-grinI don't know about you but I'm selling my stock in Halliburton andLockheedMartin and buying SC Johnson as soon as this plane landsTechnorati Tags: Flying, Futility, Meatspace, Security, Travelhttp://www.secuobs.com/revue/news/52022.shtmlhttp://www.secuobs.com/revue/news/52022.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - so we're in what's called the "quiet period" What that means forme, practically speaking, is that I won't be doing a lot waxingprofound in public for some time So while there are many"interesting" threads on both Sourcefire and one of the technologieswe develop in progress on various mailing lists and blogs and as muchas I'd like to, I won't be making any comments on those discussionsNot even if you buy me beer I dare you to tryTechnorati Tags: Sourcefirehttp://www.secuobs.com/revue/news/52021.shtmlhttp://www.secuobs.com/revue/news/52021.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - As some of you know, I'm an amateur astronomer It's a great hobby,I've been involved with it since I was 14 years old and I really enjoyrelaxing under the night sky and commanding a big stack of advancedoptics and robotics, not to mention some really cool cameras to beable to see the heavens and enjoy them more fully than many people doWhen you first buy a telescope, it's a pretty daunting experienceThere are different types of optical systems, different types ofmount, automated pointing systems, CCD cameras, eyepieces, filmcameras, focusers, dew control electronics, counterweights, pointingaids, and a host of reading material, not to mention a bunch ofspecialized software to use all the gear together effectively Now, alot of time when people first get into astronomy, the first thing theythink to do is just go out and buy a telescope Seems like a naturalthing to do, ehWhen you go to the store to get your telescope, you're likely to beassaulted by a barrage of marketing about all the wonders you willobserve with your shiny new telescope The pictures on the side of the60mm achromatic refractor you're likely to end up with promise youthat you'll be able to observe the universe from your back yard andit'll look a lot like this When you get your shiny new scope home andset it up in the back yard for the first time, you'll soon discoverthat reality and marketing sometimes are at odds, because what you'rereally going to see will look a lot more like this if you're luckyand spent some money on a decent scope and you live under a reallydark sky Chances are that you're not even going to see that becauseyou haven't been versed in using averted vision to pick out detail infaint objects Sometimes when I let people look through my telescopesfor the first time and I show them something pretty incredible like agalaxy that's 50 million light years away, they barely see anythingIt's always easiest to show them something bright and obvious likeJupiter or Saturn, something that requires little skill to find orobserve and that will give quick gratificationIn fact, if you're going to get good use out of that telescope, you'regoing to have to become familiar with a whole host of topics that youmay not have thought a whole lot about prior to getting intoastronomy Things like celestial coordinate systems, ephemeris, polaralignment techniques, averted vision, calculating the magnificationand field of view of an optical system, different object catalogs andtheir contents, gauging the viewing quality of the sky, amateurmeteorology hauling out 200lbs of gear on a night when a cold frontis going to bring rain 2 hours after you get setup is a bummer, notto mention all the topics of astrophotography if you really do want tosee sights like those promised on the box in which the telescope cameWhen you get right down to it most people's entry into astronomy isguided by a lot of marketing hype followed shortly by disappointmentDisillusionment, you might call it Once you understand what the realcapabilities of the equipment are and you spend the time to learn howit works, what its limitations are and what the optimal setup is tosee the things you're really interested in AND you take the time tolearn all the systems and background data on what's up in the sky andwhen is the best time to see it based on where you are and the localterrain, THEN you will be finally understand what you can expect andbe happy when you get your gear to reveal the things you wereinterested in in the first placeA lot of people don't have the patience or time to devote the energyrequired to actually get good at this hobby and a lot of peopleabandon it for just this reason There are a lot of hobbies out therethat have much more immediate gratification After all, you can justgo to STScI and see pretty much everything an amateur would ever wantto try to see and more In fact, in a world where automatic telescopicsurveys of the night sky are happening continuously, why the hellwould anyone sign up for this hobby It's futile, isn't it Everythingthat can be seen will be seen by the professionals, right WrongThere is still valuable science being done by amateurs, new cometsbeing discovered, supernovas, Red Spot Jr In fact, amateurs stillcontribute valuable science using the modest tools they haveavailable The fact of the matter is that the gear is getting so goodthese days that you can achieve incredible results with pretty modestequipment if you know what you're doingSome people don't get the appeal of this hobby, some people do, somepeople really get sucked into it and spend huge amounts of time andeffort on it because it's a really interesting and it gets them intouch with the universe around them Some people see a lot of value inthat, some people think it's a waste of time and money At the end ofthe day it turns out that you get out of astronomy what you put intoitA lot like another field of endeavor that I knowTechnorati Tags: Astronomy, Philosophyhttp://www.secuobs.com/revue/news/52020.shtmlhttp://www.secuobs.com/revue/news/52020.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - I've acquired several new gadgets over the past four months and Ithought that people might be interested in my experiences1 New phone - HTC TyTNWith Sourcefire expanding like it is, I need a phone that I can useanywhere in the world I've been using a Sony z600 for a few years nowand buying prepaid SIM cards for whatever country I'm heading to Thedownside of this is that nobody can really call me when I'm abroad andif the SIM card runs out of money then I've got to jump through hoopsto get it working again I solved that problem by moving to Cingularand getting the TyTN The TyTN is a quad-band GSM phone with tri-bandHSDPA data and 80211, plus Bluetooth It runs Windows Mobile 5 but isstill pretty useful despite that Compared to the Treo 650 it'sreplacing it's been reliable and pretty straight forward to use TheWindows paradigm is so so on the mobile platform, it's certainly got alot of clicking required to perform complex tasks compared to PalmThat said, it's a good phone and runs the apps that I need on a mobiledevice Mail, IM, SSH and a web browser Overall I've been about ashappy with this phone as I am with any cell phone, the dataconnectivity is nice and it worked well when I was overseas inDecember2 New laptop - MacBook ProI passed on the initial MacBook Pro release and waited very patientlyfor the Core 2 Duo processors to make their debut in the Apple laptopline and was rewarded with this very nice machine The Core 2 Duo chiphas a few nice new features over the first generation Core Duo chipsincluding the EM64T instruction set, a larger L2 cache, higherperformance and lower power consumption It turns out it also runscooler than the Core DuoSince my laptop is my primarydevelopment/presentation/communications/everything machine, I got itmaxxed out with a 233GHz CPU, 3GB of RAM and the 200GB hard drive andthe glossy screen option It's a heck of a lot faster than thePowerBook it's replacing for pretty much everything I do except maybeMS Office since it's run in emulation via Rosetta It's also great forrunning Parallels, the OS X virtualization environment I've beenrunning XP under Parallels for various esoteric applications, likerunning my telescope and CCD cameras for astrophotography and it workslike a champThis is without a doubt the best laptop I've ever owned, it's fast,stable and like all Macs, it just works It's a great developmentplatform, a great travel machine and all around nice computer3 Novatel Wireless XU870 HSDPA modemI used to use a Novatel EV-DO card on Sprint for my mobile internetneeds but the MacBook Pro has an ExpressCard/34 slot and there was noEV-DO card available for it Luckily, since I was switching toCingular anyway I found this card It supports GPRS/EDGE/UMTS/HSDPAdata connections up to 36 Mbps and uses a standard SIM card fornetwork access It also has drivers for OS X available, so it's prettymuch a winner across the board I got a second 3G SIM card fromCingular, set it up as a modem in OS X and it was off to the races Ialso got international data roaming turned on for the SIM card so iteven worked overseas I've been surprised at how often it's been ableto find an HSDPA signal to connect to, it's worked really welleverywhere I've tried to use it including France and the UKSupposedly there's a firmware upgrade that will be coming in the nottoo distant future that will allow the modem to handle up to 72 Mbpsover the air Once it comes out I'll be sure to post my experienceswith itTechnorati Tags: gadgets, laptop, mobilehttp://www.secuobs.com/revue/news/52019.shtmlhttp://www.secuobs.com/revue/news/52019.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - Richard Bejtlich has been talking a lot about the difference betweenNetwork Security Monitoring NSM and "alert-centric" technologieslike Snort His basic premise is that "real" NSM requires more thanjust IDP alerts and packet logs, it requires event notifications, fullpacket logs of the entire network and flow data as well He alsoquotes me as saying "Richard, I wrote Snort so you don't have to lookat packets" This isn't quite right I think what I actually said was"Did you think about how much data you're going to record if you dothat on a high speed network We wrote IDS so that we wouldn't have torecord everything"I get it, I understand what the NSM guys are saying and I really don'tdisagree with them at all The problem I have is that if you try todeploy this concept in a large network environment with lots and lotsof sensors, you've got some big problems to overcome Let's look atthe problems1 Flow aggregation - As I see from Richard's latest post on Cico'sMARS product, he wants the raw flow data, not just statistical NetFlowrollups RNA does that already, as can Snort with the right optionsturned on This works fine as long as the network environment isrelatively small and you don't try to roll up all of the data for postprocessing and analysis If you do aggregate it to a centralcollector, then you've got the multiplication problem on theaggregation links, namely the more traffic the sensors see and themore sensors you have the more they pump up to the collector and haveto push into the database that you're using to be able to manage allthis information If you're in an environment where you're aggregatingmore than a few million flows per hour, that's a ton of data to manageif you figure ~40-bytes per flow record in binary format That's200MB of data just for flow data for an hour, almost 5GB per day150GB per month Those also have to get blasted into a database sothey can be worked with, so you're going to be translating all thatdata into SQL insert statements and then pumping it into the localdatabase on your aggregation machine or across the network to adatabase server or cluster That's a lot of processing, a lot ofnetwork bandwidth and a lot of disk, not to mention a lot of RAM tomaintain the indices in memory for the database It's not that thisisn't doable, but now we're talking about offloading the work acrossmultiple machines at a minimum and that's going to increase your costsdramatically Overall this isn't a huge problem the NetFlowanalysis/NBA guys do it for a living but it is a big one in any largeenterprise, it takes a lot of work to scale technology to work with iteffectively2 Traffic aggregation - If you thought the flow aggregation problemwas fun then start logging all the traffic on your network Let's takea fairly well utilized modern enterprise network backbone running at asustained 500Mbps, that's 625MBps of data to record on a singlesensor 225GB/hour of packet traffic, 54TB per day from a singlesensor All that data is going to need to be rolled up too, unlessyou're going to spool it into a local database and do distributedqueries across the network for packet traces At that kind of datadensity your NSM sensor is going to need a NAS device someplace nearbyso that the data can be stored, it's going to be really hard to dothat on a 1U appliance just due to physical drive space limitationsOnce you have all that data, you're going to need to be able to workwith it, so it's got to be in a database or it has to be indexed onthe filesystem in some logical fashion so that smaller chunks of datacan be rapidly located, decoded and presented to the user on demandThere are companies that build products to do this, I can't reallyspeak to their effectiveness I can hook a high-speed collectionprocess like daemonlogger up to a big disk and grab all this data, butonce again how much value are you really getting for recording allthat data vs the logistical overhead of trying to maintain all thatinformation in a usable fashion for extended periods of time What'sthe time horizon if this data Do I need to keep a week/month/year ofthis data live in a database for referential purposes If there'sgoing to be any expectation of success the amount of data that's kept"live" is going to have to have some pragmatic constraints3 Alert aggregation - This is what IDP vendors spend their timeworking on getting to their users We have pretty well establishedmetrics as to what is acceptable in this realm in terms of sustainableevent rates, data overload thresholds for analysts, data density andso on This is the de facto standard in IDP because this is the thingthat people are paying for, we've got to generate the events andeveryone wants to see them since that's what the technology issupposed to be doing This is a lot of data to deal with too, and thisis the raw information that analysts have to work with We do a lot atSourcefire to pare down the number of events analysts have to dealwith via our Impact Assessment technology that's enabled by RNA, so itis possible to do effectively in large environments even with lessthan optimal tuning of the sensor infrastructureWhen I started Sourcefire one of the things that I decided to do toget people to want to pay for something that was free ie Snort wasto try solving their data management problems If you look at most ofthe IDS vendors before Sourcefire was founded, they would sell you IDSsensors and management front-ends but they wouldn't solve the biggestproblem that most people would have once they deployed the technology,namely managing the information produced by the sensors As we allknow, IDS can generate immense amounts of data with just alerts and ifyou want to be able to work with that data it needs to go into adatabase that has been optimized for the data set Prior toSourcefire, you could buy $250k worth of sensors from vendor X andwhen you deployed the sensor grid you'd call vendor X and ask them howyou're going to manage all those alerts Their answer was typically"go call Oracle, they make a really nice database and we'll sell youprofessional services if you need help setting it up" This greatlyincreased the cost and complexity of deployment of the IDS solutionsWhen Sourcefire started I decided that this was an area where we couldadd real value, so we built what is now called Defense Center allowingcustomers to have a plug-n-play appliance that solved their datamanagement problems and provided a path to deploy largeinfrastructures of our gear quickly As you can see from our S-1filing, this was probably a Good IdeaA "real" NSM infrastructure is going to primarily be built around theidea of collecting, moving and storing data and then making it highlyavailable in a variety of presentation formats for users If you tryto do this on a network that's generating lots of traffic across lotsof sensors/segments, the likelihood of building a scalable solutionthat anyone is willing to pay for is vanishingly low You're going toneed hundreds of terabytes of disk, a dedicated out of band managementnetwork for moving data, huge database servers AND the management andsensing infrastructure to actually grab the dataNow we want to scale it I know from experience that there are largedistributed international enterprises out there that have remoteoffices sitting on the other side of 128kbps and below links Theyget really irritated when you saturate that link to pump out acontinuous stream of security data These organizations also have corenetworks with 10Gbps links that can sustain 2+Gbps of internal trafficfor hours That's a couple terabytes per hour of traffic you want tolog, give or take, just in the core Then you have the rest of theenterprise with 100+ sensors deployed that are seeing varying amountsof traffic but say none of them go below 10Mbps typically, so that'sanother TB of data every hour you've got to collect and forward to acentral aggregation point Then we throw in the flow data lots ofsmall records to insert and the event data more small records toinsert and you've got a data aggregation nightmare Concentratingthis data to a central collector or a load balanced set of collectorswill saturate a gigabit line so you're going to either have to figureout how to leave it local on the sensors and perform distributedqueries against it or you're going to have to deploy a bunch ofadditional network gear to absorb the loadThe cost of deploying a solution like this will make today's IDPdeployments look like rounding error and the amount of time requiredto sell this into an enterprise will make today's sales cycles looklike selling fast foodThen we've got training I know what the binary language of moisturevaperators, Rich knows the binary language of moisture vaperators,lots of Sguil users know it too The majority of people who deploythese technologies do not Giving them a complete session log of anFTP transfer is within their conceptual grasp, giving them a fullydecoded DCERPC session is probably not Who is going to make use ofthis data effectively My personal feeling is that more of theanalysis needs to be automated, but that's another topicOne of the comments made to one of Rich's posts saidIt seems that a lot of these SIM and IDS/IPS systems are reallynow being sold to small and medium enterprises without any regardto the amount of additional staff time and expertise that will berequired to maintain them Consequently I find that the ones I'veused aren't oriented towards making investigation of an incidenteasier but are there simply to send out more alerts under thepremise that more alerts is surely better because we're detectingand stopping more attacksThat's incorrect They're being sold to extremely large enterprisesFortune 100 and when they're sold in those environments there's anexpectation that they will scale There is more data that we can getto the users of these systems for sure, but everything is anunrealistic expectation given the realities of the large enterprisesthat these technologies are sold intoRecording everything doesn't scale today but maybe someday it willLike after the SingularityTechnorati Tags: NSM, data management, Snorthttp://www.secuobs.com/revue/news/52018.shtmlhttp://www.secuobs.com/revue/news/52018.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - I've been thinking about my previous post regarding NSM methods andthe "log everything" mentality that I believe is unworkable in mediumto large environments Given that I'm a guy who doesn't like to givepeople "it's impossible" for an answer and I don't like "unsolved"problems, I've been thinking about some of the other things that couldbe put into events that would make them more useful for NSM-styleincident analysis My thinking on this topic was further bolstered byBejtlich's recent post on his NSM processGiven that "alertocentrism" is a Bad Thing, what are some of the otherthings we could do with an engine like Snort that could add value tothe events that it generates I'm not going to recommend loggingeverything, although you certainly could do that pretty easily Inoticed from the post referenced above that flow analysis seems toconstitute a large portion of the time that is spent performing NSMGiven that Snort 2x and 3x already have the ability to log flowinformation albeit somewhat limited in stream4, what are the thingsthat we could do to improve alertsA Snort unified alert typically contains the following information:* An Event structure containing* generator ID* snort ID* snort ID Revision number* classification ID* priority* event ID* event reference* event reference time* Event packet information containing* packet timestamp* source IP* destination IP* source port/Icmp code* dest port/icmp type* protocol number* event flagsAdditionally, flow records from Snort stream4 look like this:* start time* end time* server responder IP* client initiator IP* server port* client port* server packet count* client packet count* server byte count* client byte countI've been thinking that one thing that could be done that would bepretty easy and add some value would be to add "point-in-time" flowsummary data to Snort events The idea behind doing this would be toadd the data for the flow that the event occurred upon to the eventdata Something like this:* Event structure as above* Event packet info as above* "Flow point" information including* flow start time* last packet time* initiator packet count* initiator bytes* responder packet count* responder bytes* initiator TCP flag aggregate if any* responder TCP flag aggregate* last packet originator initiator/receiver* alerts on flow count* flow flags bitmapI think that this kind of information could certainly be useful forputting an event into context within its flow, the analyst could seeif there has been bidirectional interaction prior to the event, get asense for the number of alerts on the flow prior to the current event,etcThere are some other things that could be done along with this Ithink that adding in flow point data along with doing things likepost-event packet logging would probably be more useful than what wehave today I know post-event logging is not what you want in afull-blown NSM context but it certainly helps to constrain the datamanagement issue associated with just logging every packet and it'sbetter than nothing I suppose we could also add things likepersistent logging to the system as an option thinking more in theSnort 30 timeframe to allow continuous logging of selected packettraffic, of course this is a DoS waiting to happen so it'd have to beturned off by default and have some pretty serious constraint logicassociated with it in terms of port/protocol/IP filteringI'm going to think about this more, anyone NSM-heads have any thoughtson the topicTechnorati Tags: alerts, NSM, programming, Snorthttp://www.secuobs.com/revue/news/52017.shtmlhttp://www.secuobs.com/revue/news/52017.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - I've been hearing quite a bit of punditry out there regarding Snort30's licensing language and lots of commentary on What It All Means,so I guess it's time for me to clear the airIf you want to know what Snort 30's licensing language is going tobe, try reading it It's available in the first Snort 30 pre-alpharelease I did last month and we're using the GPL Apparently it washard to locate because it was in a file called COPYING instead of onecalled LICENSE The origin of naming the license file COPYING comesfrom the FSF as I recall and is typical of most GPL projects Anyway,to avoid further confusion and so I can tell people to look at myblog if it comes up I'll post the preamble that we added to theCOPYING file before the GPL license language in Snort 30 right here:/*************** IMPORTANT SNORT LICENSE TERMS ****************************** * * The Snort Network Traffic Analysis Platform "Snort" software is the * copyrighted work of Sourcefire, Inc C 2007 Sourcefire, Inc All Rights * Reserved This program is free software; you may use, redistribute and/or * modify this software only under the terms and conditions of the GNU General* Public License as published by the Free Software Foundation; Version 2 with* the clarifications and exceptions described below If you wish to embed this * Snort technology into proprietary software, we sell alternative licenses * contact snort-license@sourcefirecom * * Note that the GPL requires that any work that contains or is derived from* any GPL licensed work also must be distributed under the GPL However,* there exists no definition of what is a "derived work" To avoid* misunderstandings, we consider an application to constitute a "derivative* work" for the purpose of this license if it does any of the following: * - Integrates source code from Snort* - Includes Snort copyrighted data files* - Integrates/includes/aggregates Snort into a proprietary executable* installer, such as those produced by InstallShield* - Links to a library or executes a program that does any of the above where* the linked output is not available under the GPL* * The term "Snort" should be taken to also include any portions or* derived works of Snort This list is not exclusive, but is just* meant to clarify our interpretation of derived works with some common* examples These restrictions only apply when you actually redistribute* Snort For example, nothing stops you from writing and selling a* proprietary front-end to Snort Just distribute it by itself, and* point people to http://wwwsnortorg/ to download Snort* * We don't consider these to be added restrictions on top of the GPL, but just* a clarification of how we interpret "derived works" as it applies to our* GPL-licensed Snort product This is similar to the way Linus Torvalds has* announced his interpretation of how "derived works" applies to Linux kernel* modules Our interpretation refers only to Snort - we don't speak* for any other GPL products* * If you have any questions about the GPL licensing restrictions on using* Snort in non-GPL works, we would be happy to help As mentioned* above, we also offer an alternative license to integrate Snort into* proprietary applications and appliances These contracts can generally* include a perpetual license as well as providing for priority support and* updates as well as helping to fund the continued development of Snort* technology Please email snort-license@sourcefirecom for further* information* * If you received these files with a written license agreement or contract* stating terms other than the terms above, then that alternative license* agreement takes precedence over these comments* * Source is provided to this software because we believe users have a right to* know exactly what a program is going to do before they run it This also* allows you to audit the software for security holes* * Source code also allows you to port Snort to new platforms, fix bugs,* and add new features You are highly encouraged to send your changes to* roesch@sourcefirecom for possible incorporation into the main distribution* By sending these changes to Sourcefire or one of the Sourcefire-moderated* mailing lists or forums, you are granting to Sourcefire, Inc the unlimited,* perpetual, non-exclusive right to reuse, modify, and/or relicense the code* Snort will always be available Open Source, but this is important * because the inability to relicense code has caused devastating problems for* other Free Software projects such as KDE and NASM We also occasionally* relicense the code to third parties as discussed above If you wish to* specify special license conditions of your contributions, just say so when* you send them * * This program is distributed in the hope that it will be useful, but WITHOUT* ANY WARRANTY; including without limitation any implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE See the GNU General* Public License for more details at http://wwwgnuorg/copyleft/gplhtml, * or in the COPYING file included with Snort * */There you goWhy did we add this preamble The GPL license is vague in a number ofways as to what constitutes a "derivative product" and there are lotsof confused vendors out there who, one way or another, "misinterpret"this language in ways that are very beneficial to themselves At thesame time those vendors infrequently, if ever, actually contributeanything to the projects that they use While it's not a stipulationof the GPL that you must contribute back to the projects that you useas core technologies in your products, it is a stipulation that youhave to hew to the license language That being the case, we took acue from Nmap and decided to add the preamble to the license languageto provide clarity for users as to what we believe constitutes aderivative product so that there's as little confusion as possible Ifyou're an end user and you're using Snort as your IDS/IPS technology,this has no effect on you If you're a commercial company that's usingSnort as part of a product offering in such a way that you've breachedthe terms of the GPL license, you have two choices You can distributethe source code for your product under the GPL or you can seek analternative license from SourcefireAs I said before, the template that we used for this language comesfrom Nmap, one of the most popular and wide-spread open sourcesecurity applications on the internet today As I have said before inmany places, Snort 30 is open source technology and is distributedunder the GPL Nothing has changed from the Snort 2x series exceptfor the clarifications to the license and the option to seek analternate license from Sourcefire As with Snort 2x, technologyintegrators that don't violate the terms of the license can continueas they have beforeRegarding forking the code base, that's always an option if you don'tlike the direction that the project is taking but if the goal islicense evasion then you're probably going to be disappointed Whenyou fork a GPL project you can't change the license on the forked codebase unless you replace every line of code from the original code basewith new code belonging to the group maintaining the fork But we allknow that the purpose of the parties who are discussing a fork doesn'thave anything to do with license evasion, rightIf you don't like Snort 30's license language you can keep usingSnort 2x, you can use one of the other free IDS/IPS enginetechnologies out there or you can write your own It's a prettystraightforward process to build one of these things, I did it in myspare timeTechnorati Tags:Snort, Open Source, GPL, licensing, Snort 30, intrusion detection,intrusion preventionhttp://www.secuobs.com/revue/news/52016.shtmlhttp://www.secuobs.com/revue/news/52016.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - There have been a lot of questions and speculation about the things weSourcefire have been changing in Snort's licensing recently and itneeds to be addressed so that we can clear the airThere are three things that people have been asking questions about orhaving issues with1 GPL v2 lock that we put in place on June 29th2 "Clarifications" in Snort's license language Snort 303 "Clarifications" with regard to assignments of ownership forcontributed code Snort 30Let me address these issues in order1 GPL v2 lockHere's what happened About 3 weeks ago I got a heads up that underGPL v2, a licensee can choose to use GPL v3 if we don’t specify whatversion of the GPL to use; conceivably we could have people forkingand changing license on us Seeing as GPL v3 didn't even "ship" untilJune 29th we didn't feel like we were going to be able to make anydecision on the language that was contained in the new version untilwe'd had some time to perform a formal legal review It also didn'thelp that they decided to release on the last day of the quarterAnother contributing factor to the decision for me was that Linusdecided to keep the Linux kernel at GPL v2, that in itself was enoughto get me to hit the pause button and take some serious time reviewingthis new license before making any decision Linus himself said "I'mnot arguing against the GPLv3 I'm arguing that the GPLv3 is wrong for_me_, and it's not the license I ever chose" It's not the license wechose either and we're not moving to it without a conscious decisionto do soIf we didn't want the code base moving to the new version then whatcould we do The simplest thing given the time constraints that wewere working within was just to change the language in the source fileheader preambles and not the license itself noting that we werespecifying Snort at GPL version 2 until we could make a solid andinformed decision about how we wanted to treat GPL v3For those of you with wholly contributed source files where the fileheaders were changed, many most/all of them referred to "theprogram" as being under an indistinct version number not just yoursource files and so rather than try to track everyone down in thetime frame we had to work with *I* made a unilateral decision to justmove forward with it and we'd clean up the mess afterwards I'm sorryfor the "bull in the china shop" routine but we felt like we needed tohave this language out there before GPL v3 shipped at noon EDT on June29th Clearly there were some mistakes made, obviously we shouldn'thave changed things like the BSD license on the strl* files and so on,we'll fix that too As Victor observed, this was done in something ofa hurry BTW, we didn't try to "slip it out on a Friday" per the noteon some blog, Friday was the deadline and we had to moveWhere do we go from here We're going to examine the language in thenew license and decide if we want to move forward with it This isgoing to take a while but we'll make an announcement when we make thefinal decision For those of you who have wholly authored source filesthat would like the language changed for your source files back to theoriginal, with the provision that the language reflect that you'rejust referring to your file and not the entirety of the program, justlet us me know and send us the verbage you want and we'll make thechange For those of you who object to this sort of thing all togetherthat would like to maintain your code as an external patch set forSnort instead of in the main source tree, give us the heads up andwe'll pull your code from the source trees Once again, this is withthe provision that we may reimplement the capabilities that your codeoffers as Sourcefire-authored code if it happens to be something thatwe consider important to the projectIf anyone has any other input I'd be happy to hear it Contrary towhat several groups with vested interests seem to be promoting,Sourcefire isn't interested in closing Snort's source code or makingthis a closed-source project The community continues to be importantto us and we have no plans on that ever changing2 Snort 30 "clarifications" and the GPLThere has been a fair amount of opinion being put forth by people inthe blogging world that Snort 30 will no longer be "open source" dueto the clarifications that we put in place This is just plain wrongSourcefire produces Snort as an open source project My interest asthe guy who started this whole thing and who has worked on andadvanced this project for closing in on 9 years now has always beenhow good we can make the technology and how well we can serve theneeds of the community Now that Snort has my company behind it, thepriorities really haven't changed but there's an interesting dynamicout there with companies that are using Snort as a part of theirproduct or service offering Many of them seem to expect us to work onthis technology and improve it continuously so that their offering iscutting edge but contribute nothing to the project and complainbitterly whenever we do something that might cost them some money tocontinue to use a best-of-breed technology like thisIt's Free as in "Free Speech", not Free as in "Free Money" peopleCompanies that use Snort as part of a service or product seem to behaving a tough time accepting this The goal of the new licensinglanguage is to define what we consider to constitute conditions underwhich something built on or around Snort is a derivative work subjectto the stipulations of the GPL ie putting the derivative code underthe GPL license Despite all the gnashing of teeth that has resultedfrom this clarification, what we've really done is take about the most"open" stance you can with a GPL project and put it out there, trueopen source champions should be applauding us for our positionThat didn't happen Instead we've gotten a litany of grousing from theblogerati, primarily because we've offered a commercial license forpeople who don't want to play by the rules of the GPL in their productand service offerings that will *gasp* cost money If you'relicensing technology from Sourcefire which all of you using the GPLversion of Snort are doing and you don't wish to live under the termsof that license, we're giving you another one to choose from If youdon't like having world-class security technology available for a feebecause it affects your cost structure, that's not my problem If youwant to use it for free then you have to live by the license butpeople always seem to interpret the GPL in ways that are optimallyadvantageous to them if they don't just take the code directly andbury it in their product The clarifications we put into Snort 3 arethere to get us all on the same page and to make sure that commercialusers of the technology understand that we're not a "venturetechnology" company, giving them technology for free to enable theirbusiness models which frequently compete against us in some regardThere's nothing wrong with using Snort as a part of your commercialoffering as long as you adhere to its license If you can't do thatthen we need to talkAt the same time we've taken many measures to ensure that the endusers of the technology are unaffected Want to integrate Snort orpart of Snort into your open source project No problem, it's freeWant to deploy 100 home-made Snort sensors in yournon-profit/enterprise/government organization Go for it Want tolearn how these systems work at the code level No problem, it's allthere Want transparency of your security technology and the contentthat drives it It's all there, as it should be Want to have accessto the internals to extend or correct or add your own value to theproject or just your operational environment All part of the opensource concept, make it happen Want to fork and make your own IPSproject built on the code-base You can do that, just make sure youunderstand what you're doing in maintaining proper licensing for theforked project and respect our IPI personally have *always* been the biggest advocate for the users ofSnort since the day this company was formed and I always will be3 Snort 30 and IP assignmentsThis is the most controversial provision of the clarifications that weput into the Snort 30 license Basically what it says is:* By sending these changes to Sourcefire or one of theSourcefire-moderated* mailing lists or forums, you are granting to Sourcefire, Inc theunlimited,* perpetual, non-exclusive right to reuse, modify, and/or relicensethe code* Snort will always be available Open Source, but this is important* because the inability to relicense code has caused devastatingproblems for* other Free Software projects such as KDE and NASM We alsooccasionally* relicense the code to third parties as discussed above If you wishto* specify special license conditions of your contributions, just sayso when* you send themSo what's that mean If you send a patch to the mailing lists or toSourcefire, if you contribute code to the Snort project we considerthat code and it's IP to be "assigned" to us The reason for doingthis should be pretty clear, we don't feel that contributing a 3-linepatch to a 200k+ LOC codebase means that the contributer has copyrightclaims over Snort at that point In the early years there were manypeople who contributed in any way to Snort but over the years sinceSourcefire was incorporated the total contribution by these externalcontributers has decreased substantially After that, Sourcefiredeveloped more and more of the code, especially the core functionalityof the detection engine and preprocessors, not to mention tons of therules as well I have felt for a long time that we need to have asense of proportionality about this and we should also have theability to be flexible with the code base in terms of licensingwithout needing to approach every contributer individually to getsign-off on any changes that we make That's why we've put thisprovision into Snort 30This "assumptive assignment" is exactly what projects like Nmap usePerhaps we should take the next step and use the FSF's model wherecontributers to projects like GCC need to sign a legal documentexplicitly to contribute to the project The FSF does this becausethey need to have flexibility but also because they need to get outfrom under any potential problems that may occur due to someoneinappropriately contributing IP from a 3rd party I don't like thatconcept because of the overhead associated with interacting with theproject, Snort's not a huge project like GCC so I've liked that peoplecan contribute as they see fit The FSF does take one additional step,they guarantee that the projects that people make assignments to willbe available as open source projects in perpetuity I think that maybewe need to make a statement like that but quite frankly it's alwaysbeen our position that Snort will always be available as Free Softwareand we have no intention to change our position everI think that the part of this provision that people have had the mosttrouble with is that we also retain the right to relicense thecontributed code under alternative licenses We have to be able to dothat if we're going to offer alternative licenses to Snort,maintaining a "patch free" code branch and a "patch tainted" branchdoesn't make any sense to me and probably not to you either Theassignment doesn't mean we're going to "steal" your code and"disappear" it CIA-style It means that we need to be able to retainthe right to offer it under our commercial license The code youcontribute will always be available to you and everyone else in theopen source code base, we're not going to steal it but we are going tomake it available to our commercial users If you've got a problemwith this, don't contribute the code to us, maintain it as an externalpatchThat's about it I'm sorry we haven't been as communicative with theOSS community as we probably should be, I personally have a lot ofdemands on my time and I'm the person at SF who's the most familiarwith the totality of the Snort project so I have a lot of input intothe process here and I'm also fairly parochial regarding communicatingconcepts like this to the user community In the future I'll try to bemore forthcoming with all of you and I hope you'll continue to bepatient with both me and Sourcefire; our hearts really are in theright place with the users of this technology but we also have to bepragmatic about how all of this is going to work given all of thecommercial use that Snort seesWe're trying to be pragmatic about these issues, I hope that peoplecan feel comfortable with the direction that we're taking things Ilook forward to reading people's responsesTechnorati Tags:Snort, Open Source, GPL, licensing, Snort 30, intrusion detection,intrusion preventionhttp://www.secuobs.com/revue/news/52015.shtmlhttp://www.secuobs.com/revue/news/52015.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - This is a repost of a message I sent out on snort-users last week Forthe sake of continuity I thought it'd be a good idea to post it heretooGeneralQ1 Do these licensing updates change Sourcefire's commitment to opensourceA No, Sourcefire remains committed to open source Snort will alwaysremain an open source product - periodSnort 2x licensing questionsQ2 What are Sourcefire's issues with GPL v3A Simply stated, similar to Linus Torvalds' stance - GPL v3 is notthe license we chose Without a complete legal review and opinion ofthe entire work we can't comment on the specifics We want to completedue diligence on the license and make an informed decision We willpublish our opinion when it's readyQ3 What is the practical impact to end users of the GPL v2 lockA None The lock provides us time to review GPL v3 and make aninformed decision End users are free to use, modify and redistributeSnort under GPL v2Q4 Is it within Sourcefire's right to change the language in thesource code preamble comments to lock the license at version 2 of theGPLA The new language that we incorporated for the 27x release changesa notification provision that applies to the GPL, IT DID NOT CHANGETHE GPL This is a permissible change because it's modifying thesuggested language for header preambles in Snort 27x, not thelicense itself If you read the GPL you'll see that this language issuggested in the section that comes AFTER the Terms and Conditions ofthe license The new language follows one of these suggestions andspecifies which version we want our licensees to followQ5 Is Sourcefire addressing the concerns raised by Victor and Willfrom the Snort-inline projectA Yes, we made some mistakes and have corrected them Today's releaseof 27 addresses the issues raised by Will and Victor If you haveconcerns regarding the headers or copyrights on code that you'vecontributed let us know and we'll take care of itQ6 Do the GPL v2 derivative works clarifications used in the Snort30-alpha code base apply to the 2x releases of SnortA No, these clarifications apply only to Snort 30Q7 Does the "assumptive assignment" clause from Snort 30 apply tothe 26/27 releases of SnortA No, the assignment provisions in the Snort 30 license do not applypast contributions Sourcefire is in no way attempting to takeownership of the copyrights of past contributersSnort 30 Licensing QuestionsQ8 Will Snort 30 be licensed under GPL currently v2 onlyA YesQ9 Is Sourcefire claiming ownership of all contributed codeA No The assignment clause in 30 will maintain your ownership ofcopyrights It is simply a licensing agreement granting us the rightto modify and relicense to 3rd partiesQ10 Does this apply to past contributionsA No Snort 30 is a completely new code base that is entirelydeveloped and copyrighted by Sourcefire If we incorporate pastcontributions to the 2x code base as work on the Snort 30 projectcontinues they will maintain their original copyright and licenseQ11 What if I refuse to accept the terms of the assignmentA As we said, simply tell us the terms under which you'recontributing code and we'll work with you to come to an agreement Ifwe can't, you're free to maintain it as an external patch under anylicense you wishQ12 What is the practical effect of the derivative worksclarificationsA For end users there are none You are free to use and modify Snortas you do today For anyone that modifies and redistributes Snort*and* adheres to the terms of the GPL, there are none You maycontinue to modify and redistribute Snort as you do today The onlyimpact is on organizations that redistribute Snort and fail to adhereto the terms of the GPLTechnorati Tags:Snort, Open Source, GPL, licensing, Snort 30, intrusion detection,intrusion preventionhttp://www.secuobs.com/revue/news/52014.shtmlhttp://www.secuobs.com/revue/news/52014.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - I decided at the last minute to head out to BlackHat See you who aregoing thereTechnorati Tags:BlackHat, Travel, Conferenceshttp://www.secuobs.com/revue/news/52013.shtmlhttp://www.secuobs.com/revue/news/52013.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - Daemonlogger 10 is available on my user page on snortorg It's got acouple new features but nothing major, if you're a Daemonlogger fanit's definitely worth a downloadhttp://www.secuobs.com/revue/news/52012.shtmlhttp://www.secuobs.com/revue/news/52012.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - Snort 30 is the next generation Snort engine that is currently underactive development at Sourcefire I have been acting as lead architectas well as a contributing developer on the project for many monthsnow As one of the people who's driving development of the system Ithought it would be worthwhile to start talking about what we'rebuilding because I know a lot of people are interested in learningmore about this next generation Snort engineSnort is 9 years old this month and has a lot of miles under its beltIt's one of the most widely deployed network security technologies inthe world and is therefore one of the most highly exposed in terms oflive network packets processed and well tested IDP code basesavailable today Snort 30 is a huge undertaking but I feel its aworthwhile effort to achieve some of the long term goals that we havefor the engine I believe that ultimately our users will benefittremendously from the design of the new engine and that it will be aplatform that will work well for the at least the next 9 yearsThere are several goals associated with Snort 30's development:0 Rewrite the core frameworks for Snort from the ground up to cleanout code base cruft and leverage external libraries where possible tocontain the scope of the rewrite This will also allow us to flushunused features and effectively reduce the size and complexity of thecode base making it easier to extend and ultimately lending thesecurity benefits of a smaller code base1 Build an "contextually aware engine", one that has the ability tounderstand what it's defending built around the concept of networkcontext Network context is essentially data about the environmentthat is being defended by Snort, the composition of the hosts in thenetwork as well as the local network composition This is important inSnort 3 in order to:* Reduce/simplify/eliminate tuning as much as possible leveragingnetwork context* Generate event priorities based on network context* Address network and transport layer evasion by leveraging networkcontext2 Abstract and compartmentalize Snort's subsystems to make components"separable"* Compartmentalize common functionality that any network trafficanalysis application would need to enable Snort to be a moreeffective platform for building arbitrary network trafficanalyzers3 Improve support for protocol encapsulation within the overallengine architecture to make handling things like enterprise/WANprotocols and IPv6 support more natural4 Add an interactive shell to the system so that it may be more fullyorchestrated at runtime5 Multithread the engine to take better advantage of multi-coreplatforms that are standard today* Make the engine parallalizable so that multiple analytics threadsmay run simultaneously on the same traffic6 Normalize Snort's language so that it's easier to read and writeSnort3BlockDiagrampngAs a result of these goals the engine architecture has a number ofmajor discrete software components1 Data Source The Data Source component encapsulates commonfunctionality required by any network traffic analyzer, functions thatwill have to be performed prior to running almost any analysis taskThe data source incorporates a number of components including:* Data Acquisition DAQ - The DAQ provides an interface between therest of the engine and the host OS packet facilities This iswhere we get packets from the underlying hardware and where wetalk to that hardware regarding the disposition of those packetsThe DAQ subsystem allows Snort 30 to incorporate arbitraryexternal packet interfaces including things like libpcap, IPQ anddivert sockets* Decoder - The Decoder performs basically the same tasks it did inSnort 2x Validate the packets, detect protocol anomalies andprovide a referential structure for the rest of the program tooperate upon* Flow Manager - The Flow Manager provides services for trackingconversations between endpoints on the network In Snort 30 italso contains features for "fastpathing" traffic, allowing it topass straight through the engine in the event that the analyticsmodules have decided that they're not interested in a particularflow any longer Snort 30 also includes a mechanism called "flowslots" that subsystems can use to store stateful "flow local"information This is the place that things like flowbits will getstored in Snort 30* IP Defragmenter - This module provides services for putting IPv4and IPv6 packets back together and will include mechanisms toallow for target-based fragment reassembly* TCP Stream Reassembler - As with the IP Defragmenter, providestarget-based services for reassembling TCP segments intonormalized streams and presenting them to the underlyinganalytics* Data Source API - An abstraction API between the facilitiesprovided by the data source and the rest of the Snort 30 softwareframework This API exists to that the rest of Snort 30 can workwithout caring whether the Data Source is implemented as hardwareor software2 Action System The Action System handles event queuing,notification and logging when the system fires events The supportedoutput types in Snort 30 will be text console, syslog and Unified2, a serialized binary stream format3 Attribute Management System AMS The AMS will store networkcontextual data about the operational environment being defended by aparticular Snort instance This subsystem will be addressablecontinuously at runtime and provide interactive interfaces to thecommand shell as well as analytics modules that can leverage its dataThe inclusion of the AMS will make all the goals in section 1 of thegoals section attainable4 Analytics System The Analytics System is where Snort detectionengine threads will be located The idea in Snort 30 is to put alldetection logic in analytics modules that run as separate threads, allthe other code exists to support the functions of the AnalyticsSystem Multiple threads may operate on the data coming from adispatcher instance simultaneously The Analytics System is structuredso that all interaction between the analytics modules and the rest ofthe Snort 30 framework is brokered by an API called the "SnortAbstraction Layer" SAL Note that arbitrary functions may beperformed by analytics modules in a given runtime instance Forexample the Sourcefire edition of this engine is going to includeRNA's functionality as an analytics module running side-by-side withSnort 305 Dispatcher The Dispatcher exists to coordinate information flowbetween the different components of Snort 30 and to manage trafficqueuing and disposition across analytics threads It also tiestogether all of the objects in a runtime instance of a Snort engine,uniting data source, analytics, action system and the attributemanager into a single manageable entity for the purposes of processand threat management from the command shell6 Snortd and the command shell Snortd is the daemon process thatprovides marshaling services for the objects that are instantiated ina particular framework instance The command shell also runs attachedwithin a thread to snortd The command shell provides interactiveobject management services for different software modules, runtimemanagement of the process and threads, health management and a fullscripting language for Snort 30 The command shell is running the Luascripting language, a lightweight embeddable scripting language thatis fast and portable as well as being very nice for implementingDomain Specific Languages If Snort's parser wasn't one of yourfavorite features in the past you should definitely like this changeFor those of you wondering if Snort 30 will handle Snort's existingrules language, of course it will We're not planning on throwing out9 years of accumulated detection functionalityThe analytics modules that are under development right now that I candiscuss include a Snort 2x detection engine implementation, RNA forSourcefire implementations and a Lua traffic analysis module forusers who are in environments where a scripting interface to trafficanalysis would be very usefulOver the coming days and weeks I am planning to post a subsystem bysubsystem design overview of the engine components so that users ofthe system may familiarize themselves with the system as we prepare torelease additional alpha code snapshots on our way to a Snort 30betaTechnorati Tags:intrusion detection, intrusion prevention, Open Source, programming,Snort 30http://www.secuobs.com/revue/news/52011.shtmlhttp://www.secuobs.com/revue/news/52011.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - I'm at the CanSecWest conference speaking this week If you're up herein not so sunny Vancouver I'll see you at the Conhttp://www.secuobs.com/revue/news/52010.shtmlhttp://www.secuobs.com/revue/news/52010.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - Daemonlogger 11 is available on my personal site for those of youinterested in packet logging and network tapping New featuresinclude:* Rollover size command line shortcuts eg "-s 1M" vs "-s1048576"* Disk utilization-based ringbuffer rollovers For example, you cannow tell Daemonlogger to write pcap files until the disk is 90%full and then "eat its tail" by deleting the oldest pcap file inthe logging directoryI also fixed a bug that was found by Wesley Shields to preventDaemonlogger from pruning files collected by previous runs of thesoftware that were in its logging directory This should also makeit safe to have multiple instances of Daemonlogger write to thesame logging directory without interfering with one anotherEnjoyTechnorati Tags:open source, sourcefire, tools, daemonloggerhttp://www.secuobs.com/revue/news/52009.shtmlhttp://www.secuobs.com/revue/news/52009.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - Things have changed a bit in the Snort 30 world since my last post so Ithought I'd provide an update as a foundation for moving forward withthis "series" I promise it'll be more than one articleIn Part 1 I discussed the architecture of the Snort 30 technology andsince then there have been some changes The largest change has beenorganizational in nature We've decided to name the core systemframework apart from the overall project since you can do more thanjust Snort-style intrusion detection with it So, as a result from nowon we'll be calling the software framework SnortSP the Snort SecurityPlatform and then the engines will be named separately The overallarchitectural umbrella that this all lives under is still going to becalled the "Snort 3 Architecture" and it will consist of differentsoftware components, chief among them will be SnortSP and the enginemodules that utilize itHere's a handy reference diagram:SnortSP engine block diagramjpgOk, now that that's out of the way, let's talk about the beta On June30th we released the initial beta of SnortSP et the Snort 282 Engineto open source beta It's located at http://wwwsnortorg/dl/snortspTo date we have done three releases of the code base with progressiveversions nailing down loose ends and fixing compilation issues and thelikeWe would love any feedback that people have on the betas, if you're aSnort fan you should definitely check it out and start getting yourfeet wet, this is the future of SnortFor my next post I'll be spending some time talking about the SnortSPcommand shell and some neat stuff you can do with itTechnorati Tags:IDS, IPS, open source, snort, sourcefirehttp://www.secuobs.com/revue/news/52008.shtmlhttp://www.secuobs.com/revue/news/52008.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - One of the biggest user-facing changes in the Snort 3 architecture isthe inclusion of a user shell interface to interact with the systemUp until now everything has been controlled strictly via the commandline interface at startup time and via signals sent to the processThere are many reasons for going with a command shell that I detailedin the first part of this series but basically since Snort is designedto be able to run continuously now it was essential to have a way tointeract with itUpon starting SnortSP you are sent directly to the command shellprompt+ Loaded pcap DAQ+ Loaded file DAQ+ Loaded afpacket DAQ* DAQ Modules Loaded* Loading decoder modules+ Loaded ethernet+ Loaded null+ Loaded arp+ Loaded ip+ Loaded tcp+ Loaded udp+ Loaded icmp+ Loaded icmp6+ Loaded gre+ Loaded mpls+ Loaded 8021q+ Loaded ipv6+ Loaded ppp+ Loaded pppoe+ Loaded gtp+ Loaded raw* Decoder initialized* Flow manager initialized* Data source subsystem loaded* Engine manager initializedControl thread running - 3082939280 18555* Loading command interface Loading SnortSP command metatable Loading data source command metatable Loading engine command metatable Loading output command metatable Loading analyzer command metatableExecuting etc/snortlua,,_ -* SnortSP If this is your first time running Snort you'll probably want to gethelp Every subsystem in the Snort 3 architecture has its own helpfunction available as do the engine modules, if you ever get lostworking with a module just invoke its object help function Forexample:snort ssphelp* SnortSP Commands:helpset_log_level debug|info|notice|warn|error|critical shutdownAvailable subsystems within SnortSP have their own help methods:dsrc - Data Sourceeng - Dispatcher/Engineanalyzer - Analytics Modulesoutput - Output ModulesFor example: dsrchelp will call the Data Source help functionAs you can see, the top level module for SnortSP is called "ssp" andyou can invoke its help function by calling it with a "ssphelp"function If you want to find out the data source subsystem'savailable functions simply call "dsrchelp" and so onOne cool thing: if your system has Readline support available the Luainterpreter will pick it up automatically and you'll have standardshell functionality available within SnortSP such as command historyand command line editingUnder the covers a few things are happening When a command is invokedin SnortSP there are a series of lookups performed by the code in thesrc/platform/lua_interfacec file Lua is really handy for wrapping Cfunction calls, it's one of the initial reasons I went with it forSnort Let's take a look at some of the simple functionality in thelua_interfacec file for the ssp* commandsstatic int set_log_level_wraplua_State *L {int level;if lua_isnumberL, 1{level = lua_tointegerL, 1;}else{const char *name = char * luaL_checkstringL, 1;if name ==NULL return 0;if strcasecmpname, "debug" == 0 level = S_LOG_DEBUG;else if strcasecmpname, "info" == 0 level = S_LOG_INFO;else if strcasecmpname, "notice" == 0 level = S_LOG_NOTICE;else if strcasecmpname, "warn" == 0 level = S_LOG_WARN;else if strcasecmpname, "error" == 0 level = S_LOG_ERROR;else if strcasecmpname, "critical" == 0 level = S_LOG_CRITICAL;else return 0;}log_set_current_log_levellevel;level = log_get_current_log_level;S_INFO"Log level set to %s", log_level_to_stringlevel;return 0;}static int shutdown_wraplua_State *L {stop_processing = 1;return 0;}static int platform_help_wraplua_State *L {printf"* "PLATFORM_NAME" Commands:"" help"" set_log_level debug|info|notice|warn|error|critical "" shutdown"" Available subsystems within "PLATFORM_NAME" have their own help methods:"" dsrc - Data Source"" eng - Dispatcher/Engine"" analyzer - Analytics Modules"" output - Output Modules"" For example: dsrchelp will call the Data Source help function";return 0;}static void platform_set_infolua_State *L {lua_pushliteral L, "_COPYRIGHT";lua_pushliteral L, "Copyright C 2008 Sourcefire Inc";lua_settable L, -3;lua_pushliteral L, "_DESCRIPTION";lua_pushliteral L, "Network Intrusion Prevention System command interface";lua_settable L, -3;lua_pushliteral L, "_VERSION";lua_pushliteral L, PLATFORM_NAME" 01";lua_settable L, -3;}static const struct luaL_reg platformlib = {{"help", platform_help_wrap},{"shutdown", shutdown_wrap},{"set_log_level", set_log_level_wrap},{NULL, NULL},};static int platform_dir_create_meta lua_State *L {luaL_newmetatable L, METATABLE;/* set its __gc field */lua_pushstring L, "__gc";lua_settable L, -2;return 1;}static int luaopen_platformlua_State *L {platform_dir_create_metaL;S_INFO" Loading "PLATFORM_NAME" command metatable";luaL_openlibL, PLATFORM_PROMPT, platformlib, 0;platform_set_infoL;return 1;}This code sets up a "metatable" in Lua which allows us to callfunctions from its root with dot notation The call to bind the rootname is a little hidden in this case, it's the luaL_openlib call inthe last function above We ended up using the PLATFORM_PROMPTsubstitution due to our habit of renaming the whole projectperiodically in its earlier daysThe array definition at the top of the code listing shows how thekeywords are being mapped to the function calls themselves When youput it all together, if you call "PLATFORM_PROMPThelp" it maps thatback to calling the platform_help_wrap function If you're familiarwith the internals of Snort 1x-2x you'll know that this mapping ofkeywords in the interpreter to function calls is very similar to howSnort's parser worked so it was a very comfortable transition for meAdd in the validation and scripting that you can get from lua and youcan do some really cool stuffLet's take a look at a sample lua function scripting the SnortSParchitecture to demonstrate the power of this arrangement One of thethings I wanted to do with SnortSP in its early days is test out thedecoders to make sure there were no huge flaws in any of them thatwould lead to crash, as you know Snort's decoders are "criticalinfrastructure" that must be fast and crash free, the rest of thesystem is counting on them The function I wrote would allow a user topoint Snort at a directory of pcap files and process each onesequentially until every file in the directory had been processedIn the old days we would have had to start a new instance of Snort2x for each file we wanted to process and take all the time to loadand shutdown In SnortSP you could in theory load a detectionconfiguration and then process each file, all without having torestart The way the function is setup right now it just processes thefile and prints the packet dump to the screen Here's a listingrequire"lfs" # load filesystem functionsfunction rundir path, mask # args are path to pcap files, filename maskonce = 0 # initialize the data source once onlyfor file in lfsdirpath do # grab the pathif stringmatchfile, mask ~= nil then # find files that match the maskprint"Processing File "path'/'fileifonce == 0 then # setup the dsrc params once onlyonce = 1dsrc1 = {name="src1",type="pcap", intf="file", filename=path'/'file,flags=1,snaplen=0,maxflows=16384,maxidle=10,flow_memcap=10000000,display="max"}dsrcnewdsrc1 # instantiate the dsrc once onlyengnew"e1" # instantiate the dispatcher once onlyenglink{engine="e1", source="src1"} # link dsrc to dispatcherprint"Starting engine"engstart"e1" # run itelseengrun_file"e1", path'/'file # just fun the file nowendendendifonce == 1 then # process the last file tooengrun_file"e1", ""endendThe first thing happening here is the inclusion of a Lua librarycalled "lfs" That's necessary for the filesystem interface, Luadoesn't ship with one natively I know After that it's the functiondefinition which takes two arguments, the path to the directorycontaining the pcap files and the filename mask to use to select theright files from the directory The first iteration through the configdata structures and engine objects are instantiated and configured andthe first file is processed, after that it just processes all the restof the files in the directoryThis is pretty cool Practically speaking you can script the startupand shutdown of Snort and script the operation of all the differentsystem objects as well For example, you can make external systemcalls to lookup runtime parameters like interface configuration oravailable memory and use that information to tune your runtimeconfiguration parameters of your engine You can also script testingnew software modules or even get into more crazy stuff likeautomatically turning engine modules on and off at certain times ofday or pretty much anything else you can imagineThis is just a brief tour of the functionality of the command line,there's more that can be done I'll leave it to interested readers toexplore the SnortSP command shell and I'd love to hear aboutinteresting things you do with itIn the next part of the series I'll start writing about the datasource subsystem and its componentshttp://www.secuobs.com/revue/news/52007.shtmlhttp://www.secuobs.com/revue/news/52007.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - We had some beautiful clear skies here over Labor Day weekend so I tookadvantage of it and managed to get my telescope out 3 nights in a rowto do some astrophotgraphy The results are my best yet but certainlyjust "decent" by advanced amateur standards these days I'm gettingbetter at it but I've still got a long way to goAnyway, here's the pictures For those of you interested in suchthings the gear used to collect the photons was a TEC-140 ApochromaticRefractor, SBIG ST-10XME CCD camera, and an Astro-Physics 900GTOmount I'm also using a Starlight Instruments Digital Feather Touchfocusing systemEnjoyhttp://www.secuobs.com/revue/news/52006.shtmlhttp://www.secuobs.com/revue/news/52006.shtml Secuobs.com : 2009-01-15 22:24:04 - Security Sauce - Daemonlogger 121 is available at its usual place This release is acleanup that allows compilation to work properly on systems whichdon't support the BSD TAILQ macros like some Linux'sI also neglected to announce the release of Deamonlogger 120 Thatrelease changed the default pruning mechanism for ringbuffer mode backto the original default of pruning the oldest file in the loggingdirectory and allows you to do per-run pruning by setting the -zswitch at the command line It also repaired a bug with size-basedrollovers not working properlyGet Daemonlogger herehttp://www.secuobs.com/revue/news/52005.shtmlhttp://www.secuobs.com/revue/news/52005.shtml