http://www.secuobs.com Observatoire de la securite Internet fr webmaster@secuobs.com 2009-03-09 17:29:50 - ADD XOR ROL : Hey all,I haven't given a reverse engineering trainings class in Amsterdam fora few years, but this year is different :- -- I will be at BHAmsterdam, and there are still seats open in the trainings class forApril 14th and 15thWhat will be done in the course Well, for one thing, we'll gobug-hunting in some interesting piece of code Furthermore, we'll talkquite a bit about C++ and it's effects in the binary We'll do a fairbit of differential debugging, some more bug-hunting, and a lot of IDAautomation Questions like* given a C++ executable, how do I recover an inheritance diagram ofthe classes * given a big and ugly executable, how do I find the interestingplaces to focus on * how do I make sure IDAPython and NaviPython make my life easier will be treated thoroughlySo, if you still have some trainings/travel budget left in spite ofthe crisis, you can find moredetails herehttp://www.secuobs.com/revue/news/68645.shtmlhttp://www.secuobs.com/revue/news/68645.shtml Secuobs.com : 2009-03-04 23:11:23 - ADD XOR ROL - I posted a while ago about the new DiffDeluxe comparison engine, and thatwe'd release it in Q1 2009 Well, we're almost there, the engine isnow in beta If you are a BinDiff user and wish to give the new enginea try, send mail to info@zynamicscom :-I mentioned in my last post on the topic that DiffDeluxe was designedto facilitate symbol porting, and to allow comparisons betweenexecutables that are "far away" from each otherIn the last post I wrote about Mozilla JS engine vs AcrobatEScriptdll Today I am going to try something slightly crazier: Inorder to evaluate how well these matching algorithms work, we will bediffing an executable that was compiled for ARM against a very similarexecutable compiled for x86My coworker Vincenzo is a big fan of all things OSX, and he brought upthe idea of comparing x86 and ARM versions of the OSX dynamic loader-- namely the disassembly of dyld on the iphone against thedisassembly of dyld on OSXNow, the first voices are going to yell: "You have names for allfunctions, BinDiffing is easy then" Well, true, but we will runDiffDeluxe without taking the names into account, and then just usingthe names to validate the resultsThe two executables have 704 x86 and 618 ARM functionsrespectively Without namematching, we match 345 functions Inspecting the symbols, we see thatwe have matched160 of these functions in full accordance with the symbols Let's havea look at some of the details:Cute, eh Let's look at some moreIt is almost surprising how far one can get without actually lookingat the instruction semanticsIf we take the names into account, matching functions becomes easy,but matching basic blocks properly ends up the difficulty With namematching enabled, DiffDeluxe matches 3809 basic blocks, out of 7904respective 5196So to summarize: The structural comparison is sufficiently strong toyield some useful results even accross two different CPUs While thereis still a good amount of room for improvement, I am quite happywith these results so far :-So, if you want to beta, and you already use BinDiff, drop us a line http://www.secuobs.com/revue/news/67402.shtmlhttp://www.secuobs.com/revue/news/67402.shtml Secuobs.com : 2009-02-05 19:36:59 - ADD XOR ROL - Hey all,I will be in Washington DC from the 16th to the 20th of FebruaryAmongst other things, I will be teaching a course at Blackhat DC Theeconomic crisis is clearly hitting -- eg there are still seatsavailable We will also get around to using some of the nice featuresof BinNavi v2 in class, which I am looking forwards toNow, aside from the course: If you are in the DC area and interestedin a product demo for BinDiff and the upcoming DiffDeluxe, BinNaviv2 including REIL, or the latest VxClass now available as serviceand virtual appliance, do not hesitate to drop a line toinfo@zynamicscom :-http://www.secuobs.com/revue/news/58924.shtmlhttp://www.secuobs.com/revue/news/58924.shtml Secuobs.com : 2009-01-05 11:03:40 - ADD XOR ROL - Correction of my last post: It appears that Clam has *some* unpackingsupport It is not as comprehensive as some of us would like, butprogress is being made :-http://www.secuobs.com/revue/news/48583.shtmlhttp://www.secuobs.com/revue/news/48583.shtml Secuobs.com : 2009-01-04 19:21:13 - ADD XOR ROL - Hey all,this might be a rather odd question, but given the unfortunate factthat ClamAV can't unpackeven the simplest packers, has nobody ever contemplated writingpacker-specific unpackersfor ClamAV Cheers,Halvarhttp://www.secuobs.com/revue/news/48459.shtmlhttp://www.secuobs.com/revue/news/48459.shtml Secuobs.com : 2008-12-27 01:24:34 - ADD XOR ROL - http://taossacom/indexphp/2008/10/13/bugs-vs-flaws/#more-83I didn't see this post beforehand, and I would like to comment on itmainly because commenting on his blog post might be the easiest wayof getting into a conversation with Mr McDonald these days ;, but Idon't have time right now Will fix this later this week hopefullyhttp://www.secuobs.com/revue/news/46597.shtmlhttp://www.secuobs.com/revue/news/46597.shtml Secuobs.com : 2008-12-26 22:27:36 - ADD XOR ROL - Hey all,apologies for the sensationalist title, but I found another amusingexample today where the same function was present in two differentexecutables -- in two differently obfuscated forms Amusingly,DiffDeluxe identified the "common components" between these twofunctions, effectively removing a lot of the obfuscationWhile this is clearly not a typical case, it nonetheless made mesmileMerry Christmas everyone http://www.secuobs.com/revue/news/46573.shtmlhttp://www.secuobs.com/revue/news/46573.shtml Secuobs.com : 2008-11-15 17:23:15 - ADD XOR ROL - is like a good joke This one, while requiring special circumstancesto succeed with high probability, was responsible for a lot oflaughter on my sidehttp://www.secuobs.com/revue/news/36195.shtmlhttp://www.secuobs.com/revue/news/36195.shtml Secuobs.com : 2008-11-11 22:28:15 - ADD XOR ROL - Hey all,we have re-activated the BinDiff / BinNavi User Forum underhttps://zynamicsfogbugzcom/defaultaspBinNavihttps://zynamicsfogbugzcom/defaultaspBinDiffThere is not a whole lot there at the moment, but that should changesoon :http://www.secuobs.com/revue/news/35320.shtmlhttp://www.secuobs.com/revue/news/35320.shtml Secuobs.com : 2008-11-11 16:56:39 - ADD XOR ROL - Hey all,for some research that I'm doing, I'm looking for a collection ofmalicious Office/PDF documents If anyone has such documents egbecause he was targeted in an attack, or because he found onesomewhere, I'd much appreciate submissions :http://www.secuobs.com/revue/news/35246.shtmlhttp://www.secuobs.com/revue/news/35246.shtml Secuobs.com : 2008-11-10 14:47:39 - ADD XOR ROL - Hey all,we have written about the SQL storage format for BinNavi quite a fewtimes on this blog, and how we'd like to encourage third parties touse it I am quite happy to say that Stefan Esser ofSektionEins GmbH has built code to export PHP byte code into thedatabase format The cute results can be seen underhttp://wwwsuspektorg/2008/11/05/php-bytecode-in-binnavi-20/http://www.secuobs.com/revue/news/34981.shtmlhttp://www.secuobs.com/revue/news/34981.shtml Secuobs.com : 2008-11-09 02:49:12 - ADD XOR ROL - One of my favourite things when travelling and interacting people fromother cultures is observing differences in conversational conventions-- and most importantly different forms and perceptions of"conversational humor" Aside from comedic protocol screw-ups egliterally translating an essentially untranslateable expression toanother language, earning -- at best -- puzzled looks and -- at worst-- thoroughly offending the conversation partner, it often providesinteresting insights into one's own culture and habitsThis weeks special: German forms of expressing optimismThere are many expressions in German that are horribly difficult totranslateOne of my favourites that could cause confusion is the German customof wishing people luck by wishing them "Hals- und Beinbruch"literally: 'broken neck and broken leg' or 'Kopf- und Bauchschuss'literally: 'shot in the head and stomach' or for sailors 'Mast-und Schotbruch' literally: 'broken mast and ripped sail' uponpartingA common reply for this would be "wird schon schiefgehen" literally:'I have no doubt it's going to go badly' Counterintuitively, thesemantics of this is optimistic -- eg whoever says that things aregoing to turn out badly indicates by this that he is not worried, andthat he actually expects that things will be fineIn essence, one expresses optimism by claiming that an improbablyhorrible outcome is a near-certaintyEven though I try hard to not have an all-too-obvious German accentany more, I do catch myself all the time in using the above pattern,even though it does not translate I deservedly earned puzzled lookstoday by clumsily attempting to use the following German saying toindicate my optimism about the future:"Lächle und sei froh, sagten sie mir, denn es könnte schlimmer kommenUnd ich lächelte und war froh, und es kam schlimmer"This has a certain elegance in German, which is totally lost in myclumsy translation:"Smile and be happy, they told me, because things could be a lotworse So I smiled and was happy, and things got a lot worse"Aside from the clumsiness of the expression when translated, thesemantics eg the intention to express optimism was thoroughly lost-- the effect was a thoroughly puzzled and slightly worried look by myconversation partner I think it is situations like these whereGermans earn their bad reputation for being thoroughly unfunnyOther things that are good for causing confusion between a nativeEnglish speaker who interacts with someone from the German-speakingworld are differences when it comes to acceptable replies to thequestion "How are you " The usual form of this in German is "Wiegehts ", essentially "How is it going " In the English speakingworld, acceptable replies seem to be restricted to "good", "goodgood", or "great"Proper replies to the question "How is it going" over here would be:"Muss" -- literal translation: 'it has to somehow'"Naja, ganz ok" -- 'well ok '"Könnte schlechter/besser gehen" -- 'could be worse/better'"Bergauf" or "Bergab" -- uphill / downhillIf the other party feels inclined to have a longer chat, they couldreply with"Yesterday, we stood on a cliff Today we have advanced by asignificant step"or "Katastrophe" This is usually followed with a short anecdote orcomplaint about something work-related From a social perspective,this does wonders as an ice-breakerWhenever I catch myself in such a situation, I realize that no matterhow much one travels, and no matter how much time one spends in adifferent cultural climate, certain components of the socialinteraction are nigh-impossible to changeAnyhow, time to go to sleephttp://www.secuobs.com/revue/news/34357.shtmlhttp://www.secuobs.com/revue/news/34357.shtml Secuobs.com : 2008-10-26 23:43:22 - ADD XOR ROL - So I do own a car contrary to what most people expect About a yearago, I bought a VW Caddy EcoFuel It runs on natural gas in normalmode and only uses the gasoline tank for starting and when thenatural gas has run outUp until 4 weeks or so ago I was pretty happy with it, but onemorning, the car refused to start unless I hit the gas heavily whilestarting I brought the car to the repair shop that belongs to thesame place where I bought the car After a few days of tinkering, theytold me that1 The particular car I own doesn't lock the tank when the rest ofthe car is locked and2 Somebody poured an unidentifiable liquid into my tank causing theproblems3 Because this is not a problem with the car itself, warrantydoesn't cover it4 Removing the tank and the fuel pump and cleaning everything isgoing to cost 1200 EUI am somewhat annoyed by some punk pouring an unidentifiable liquidinto my tank and agree to pay the money I also ask for the shop toretain a sample of the tank contents so I can at least find out whatwas poured into the tank, and perhaps get money back from myinsuranceThey agree When I come to pick up the car, the guys at the shop forsome bizarre reason cannot find the sample I sit and wait for ~1hour, and they finally produce an unlabelled can from somewhere Ok Iask them to sign a piece of paper certifying that this sample iscoming from my tank, and they tell me they will send it to me viaregular mail the next day So far so goodSo two weeks pass, and I call back 3 times for that piece of paper Atthe beginning of the third week, I have to take my guinea pigs to thevet in the morning yes, I don't only own a car, I also have guineapigs On my way back from the vet, the natural gas runs out, and thecar switches to gasoline mode -- while I am going about 130km/h with alarge truck behind me The only complication: My engine switches offAwesomeSo I manage to stop the car safely on the side of the autobahn and gettowed to the next Volkswagen shop About 2 hours after I leave my carthere, I get a call from the repair guy there, telling me that theycan see in the VW database which repairs were done on my car recently,but from what they can tell, these repairs never happened They callin an expert that is certified to appear in court to take pictures etwrite a report, and he also confirms: The tank was never removed, thegasoline pump never replaced, and the 1200 EU were apparently chargedwithout any of the stuff ever happeningClearly, I am somewhat surprised To my dismay, I am also told thatthe actual repairs will cost about 2000 EU, and that there is stillunidentified stuff in my tankSo all in all, I am currently stuck with1 1200 EU for repairs that never happened2 2000 EU for repairs that are happening now3 2 * 300 EU for chemical analysis of the two samples taken4 unspecified legal costs most likely covered by my insurance todeal with the situationAll in all, I am quite dissatisfied with VW on this front -- IMO theyshould've warned me that the tank doesn't lock, and they shouldn'thave "VW Certified Repair Shops" that appear to attempt to defraudcustomers I have trouble imagining that not actually performing therepairs was an "honest mistake" although I usually live by the mottothat "one should not attribute anything to malice that can beattributed to incompetence"Anyhow, let's see how this plays out As if I don't have other stuffto dohttp://www.secuobs.com/revue/news/31735.shtmlhttp://www.secuobs.com/revue/news/31735.shtml Secuobs.com : 2008-10-18 04:16:57 - ADD XOR ROL - I am at Virus Bulletin in Ottawa -- if anyone wants to meet to see ournew stuff, please drop mail to info@zynamicscom :It has been a while since I posted here -- partially because I had alot of work to finish, partially because, after having finished allthis work, I took my first long vacation in a very long whileSo I am back, and there are a number of things that I am happy to blogabout First of all, I now have in writing that I am officially an MScin Mathematics For those that care about obscure things likeextending the euclidian algorithm to the ring of boolean functions,you can check the thesis here:http://wwwzynamicscom/files/DiplomarbeitThomasDullienFinalpdfFor those that are less crazy about weird computational algebra: Ourteam here at zynamics has made good progress on improving the corealgorithms behind BinDiff further Our stated goal was to make BinDiffmore useful for symbol porting: If you have an executable and yoususpect that it might contain a statically linked library for whichyou have source access or which you have analyzed before, we wantBinDiff to be able to port the symbols into the executable you have,even if the compiler versions and build environments differsignificantly, and even if the versions of the library are not quitethe sameWhy is this important Let's say you're disassembling some piece ofnetwork hardware, and you find an OpenSSL-string somewhere in thedisassembled image Let's say you're disassembling an old PIX image634 perhabs and see the stringOpenSSL 095a 1 Apr 2000This implies that PIX contains OpenSSL, and that the guys at Ciscoprobably backported any fixes to OpenSSL to the 095a version Now,it would be fantastic if we could do the following: Compile OpenSSL095a with full symbols on our own machine, and then "pull-in" thesesymbols into our PIX disassemblyWhile this was sometimes possible with the BinDiff v20 engine andv21, which is still essentially the same engine, the results wereoften lacking in both speed and accuracy A few months back, Soerenand I went back to the drawing board and thought about the nextgeneration of our diffing engine -- with specific focus on the abilityto compare executables that are "far from each other", that differsignificantly in build environments etc and that only share smallparts of their code The resulting engine dubbed "DiffDeluxe" bySoeren is significantly stronger at this taskWhy did the original BinDiff v2 engine perform poorly There are anumber of reasons to this, but primarily because of the devastatingimpact that a "false match" can have on further matches in the diffingprocess, and due to the fact that in the described scenarios, most ofthe executable is completely different, and only small portions matchThe old engine had a tendency to match a few of the "unrelatedcomponents" of each executable, and these initial incorrect matchesled to further bad matching down the roadThis doesn't mean the BinDiff v2 engine isn't probably the bestall-round diffing engine you can find I think it is, even if someearly builds of the v20 suffered from silly performance issues --those of you that are still plagued by this please contact support@for a fix -- but for this particular problem some old architecturalassumptions had to be thrown overboardAnyhow, to cut a long story short: While the results generated byDiffDeluxe aren't perfect yet, they are very promising Let's followour PIX/OpenSSL scenario:DiffDeluxe operates with two "fuzzy" values for each function match:"Similarity" and "Confidence" Similarity indiciates how successfulthe matching algorithm was in matching basic blocks and instructionswithin the two functions, and confidence indicates how "certain"DiffDeluxe is that this match is a correct one This is useful to sortthe "good" and "bad" matches, and to inspect results before portingcomments/names Anyhow, let's look at some high-confidence matches:Well, one doesn't need to be a rocket scientist to see that thesefunctions match But in many situations, the similarity between twofunctions is not 100% evident: The following is a matched functionwith only 72% similarity but 92% confidence:So what is the overall result Out of the 3977 functions which we hadin libcryptoso, we were able to match 1780 in our Pix disassembly --but with a big caveat: A significant number of these have very lowsimilarity and confidence scores This isn't surprising: Thedifferences between the compiler used upon compile time of our Piximage sometime 6 years ago and the compiler we used gcc 41, -O3is drastic All in all, we end up with around 250 high-confidencematches -- which is not too bad considering that we don't know howmany functions from OpenSSL the Pix code actually containsIn order to have a more clear idea of how well these algorithmsperform, we need an example of which we know that essentially theentire library has been statically linked in For this, luckily, wehave Adobe Reader :-With all the Adobe patches coming up, let's imagine we'd like to havea look at the Javascript implementation in Acrobat Reader It can befound in Escriptapi Now, I always presume that everybody else is aslazy as me, so I can't imagine Adobe wrote their own Javascriptimplementation But when Adobe added Javascript to Acrobat Reader,there were few public implementations of Javascript around --essentially only the engine that is nowadays known as "SpiderMonkey",eg the Mozilla Javascript engine So I compiled SpiderMonkey into"libjsso" on my Linux machine and disassembled Escriptapi Then Iran DiffDeluxe The result:Escript contains about 9100 functions, libjsso contains about 1900After running the diff, we get 1542 matches Let's start verifying how"good" these matches are As discussed above, DiffDeluxe uses a"similarity" and "confidence" score to rate matches We get 203matches with similarity and confidence above 90% -- for thesefunctions, we can more or less blindly assume the matches are correctIf we have any doubts, we can inspect them:Well, there is little question that this match was accurateThe interesting question is really: How low can we go similarity- andconfidence-wise before the results start deteriorating too badly Let's go low -- for similarities below 40% For example thejs_ConcatStrings matchManual inspection of the screenshot on the right will show that thecode performs equivalent tasks, but that hardly any instructionsremain identicalProceeding further down the list of matches, it turns out that resultsstart deteriorating once both confidence and similarity drop below 03-- but we have around 950 matches with higher scores, eg we havesuccessfully identified 950 functions in Escriptapi While this issignifcantly less than the 1900 functions that we perhabs could haveidentified, it is still pretty impressive: After all, we do not knowwhich exact version of SpiderMonkey was used to compile Escriptapi,and significant changes could have been made to the codeClearly, we're a long way from matching 95% -- but we're very close tothe 50% barrier, and will work hard to improve the 50% to 75% andbeyond :-Anyhow, what does all this have to do with automatic classificationand correlation of malware I think the drastic differences induced by platform/compiler changesmake it pretty clear that statistical measures that do not focus onthe structure and semantics of the executable, but on some "simple"measure like instruction frequencies, fail All the time Behaviorialmethods might have a role to play, but they will not help you one bitif you acquire memory from a compromised machine, and are triviallyobfuscated by adding random noisy OS interactionI am happy to kill two birds with one stone: By improving thecomparison engine, I am making my life easier when I have todisassemble Pix -- and at the same time, I am improving the ourmalware classification engine Yay :-Anyhow, as mentioned above: I am at the Virus Bulletin conference --if anyone wishes to have a chat or have our products demo'ed, pleasedo not hesitate to send mail to info@zynamicscomhttp://www.secuobs.com/revue/news/30392.shtmlhttp://www.secuobs.com/revue/news/30392.shtml Secuobs.com : 2008-10-18 04:16:57 - ADD XOR ROL - Hey all,I forgot to mention a few things in the previous post:1 We're going to release BinDiff v21 on the 15th of October 2008This is still the "old" diffing engine, albeit with a number ofspeed et reliability improvements2 We're going to release BinNavi v20 on the 15th of October 2008The number of new features in this release is huge -- it's reallyquite significant You can read about it in detail on SP's blogI will post some more information myself in the next days Just afew mouth-watering keywords: Plugin API to extend Navi fromJava/JRuby/Jython/JavaScript, built-in intermediate language,hierarchical tagging / namespaces for structuring largedisassemblies, cross-module-graphing, managing multiple addressspaces in one project, many user interface improvements, fasterIDA-SQL export etc etc etc3 The DiffDeluxe engine will be part of the next BinDiff releasethereafter, probably no later than February 2008 If you are anexisting BinDiff customer and would like to try the DiffDeluxeengine in order to provide us with feedback, do not hesitate tocontact us -- it's available for testing now We're especiallyinterested in finding instances where DiffDeluxe performs worsethan BinDiff v21 Switching the core diffing engine is asignificant change, and I would not want to know of any instanceswhere the new engine is worse than the old onehttp://www.secuobs.com/revue/news/30391.shtmlhttp://www.secuobs.com/revue/news/30391.shtml Secuobs.com : 2008-10-18 04:16:57 - ADD XOR ROL - My brother wrote an article injecting some reality into the discussionabout the banking crisis on Spiegel Online The german version can beseen here I'll share a short summary of his arguments here and he'llcomplain about my distortions later ;Short version: The article describes why the situation is less direthan many pundits claim, and explains logical fallacies incommonly-heard argumentsIn the following, here's a summary of his arguments, in the form of"Myth -- Reality"1 The US government is taking on a total of 7000bn in liabilities-- about 5500bn by agreeing to step in for Fannie Mae / FreddieMac, and about 700bn in papers bought by doing the bailout Thisequates to roughly half of US GDP, and since the US is already indebt by about 65% of GDP, this would push the total indebtednessof the US to be clearly past 100% of GDP As a result, seriousdoubts would have to be cast on the US governments ability torepay debts and service interest on debtReality: Most of the 5500bn are backed by "proper" mortgages withdecent quality It is unclear whether the US gov will lose moneyon the Fannie Mae / Freddie Mac deal at all Even the 700bn in"toxic assets" the US is willing to buy have some underlyingvalue Realistic expectations at the total loss for the USgovernment in this deal runs in the area of 500bn, which would beless than 3% of GDP -- and therefore not a significant source ofproblems2 The liquidity that central banks are injecting into the marketsshould lead to hyperinflation Reality: The measures to helpliquidity in the markets do not increase the money supply in thelong run They are usually short-term credits given to strugglingbanks for a limited amount of time -- weeks or months After thistime, the creditors have to repay the loans, and the moneydisappears At the same time, the willingness by existing banks tolend decreases, thus decreasing the money supply in the economyThe statistics by central banks show that the actual money supplyM2 is growing a lot less slowly at the moment in spite of all theliquidity injections Since the money supply is only growing veryslowly at the moment, the inflationary pressures are low3 The banking crisis is responsible for the overall slowdown in theEU's economy, and the German government is thus not responsiblefor having to adjust their growth estimates downwards sharplyReality: Most indicators show that the slowdown started way beforethe crisis reached it's current urgence The indicators startedpointing down much earlier as a result of the heavy increase inenergy costs, the appreciation of the euro and the resulting lossin competitiveness, and Germany's botched reform of accountingrules for writing down investments in equipment The bankingcrisis is just the latest "kick" -- but the three previous oneswere all known early and could've been partially corrected4 This is the mother of all financial crises This banking crisisis the worst crisis in several generations, up to the 1930'scrash Reality: Dramatic banking crises are more common than wethink Since 1970, the IWF has counted 42 crashes in countrieslike Argentina, Indonesia, China, Japan, Finland or Norway Incomparison to these crises, the current crisis isn't even verydeep or expensive: The Paulson-bailout comes at a cost of 700bn,not even 5% of GDP, and only a fraction of this will be actuallylost According to the IWF, the average banking crisis in acountry came at the cost of 13% of GDP for that country's taxpayer The Indonesian crisis even came in at four times this Thebig difference to the other crises is that this one has caught onin the world's biggest economy, and as such reaches unknowndimensions in absolute termshttp://www.secuobs.com/revue/news/30390.shtmlhttp://www.secuobs.com/revue/news/30390.shtml