http://www.secuobs.com Observatoire de la securite Internet fr webmaster@secuobs.com 2010-04-13 08:49:07 - KernelTrap : Linux news It's been two weeks rather than the usual one, because we've been hunting a really annoying VM regression that not a lot of people seem to have seen, but I didn't want to release an -rc4 with it, began Linux creator Linus Torvalds, announcing the 2634-rc4 Linux kernel He explained, we had the choice of either reverting all the anon-vma scalability improvements, or finding out exactly what caused the regression and fixing it And we got pretty close to the point where I was going to just revert it all Linus continued Absolutely _huge_ kudos to Borislav Petkov who reported the problem and was able to not just reliably reproduce it, but also test new patches to try to narrow things down at a moments notice The thing took ten days of emails flying back and forth, and Borislav was there all the time, day and night, through several patches that tried to fix it several real bugs, but not the one he hit and lots of patches to just add instrumentation to get us nearer to the cause of the problem And finally, today, confirmation that we actually nailed the problem So if anybody has been seeing a oops or sometimes a GP fault in page_referenced , that should be gone now As for the rest of the changes, Linus noted, the bulk of the changes come from drivers - a new network driver cxgb4 , but also updates to the radeon and nouveau drivers And then there is the random updates everywhere Read on for the full changelog read more http://www.secuobs.com/revue/news/211464.shtmlhttp://www.secuobs.com/revue/news/211464.shtml Secuobs.com : 2010-04-09 23:03:06 - KernelTrap - Linux news If you're wondering why I'm taking a long time to respond to your patches, , began Theodore Ts'o on the linux-ext4 mailing list, in a thread that offered much insight into how and why to properly submit and test patches Patches that are accepted into mainline should do one and only one thing, Ted continued, so if someone suggests that you make changes to your submitted patch, ideally what you should do is to resubmit the patch with the fixes --- and not submit a patch which is a delta to the previous one He also noted that patch submitters often greatly outnumber maintainers dictating a higher standard of quality, consider that for some maintainers, there may be 10 or 20 or 30 or more patch submitters in their subsystem With that kind of submitter-to-maintainer ratio, the patch submitter simply has to do much more of the work, since otherwise the subsystem maintainer simply can't keep up Ted went on to acknowledge, I happen to believe that we need to encourage newcomers to the kernel developer community, and so I spend more time mentoring people who are new to the process He noted that his time was finite however, and that patches are accepted more quickly when they are easy to review and integrate Regarding the filesystem for which the patches had been submitted, he added, Ext4 is actually quite stable at this point Very large numbers of people are using it, and most users are quite happy For this reason, he pointed out that it is even more critical that the patches merged be of high quality That said, he continued, there is no such thing as code which is not buggy For any non-trivial program, it's almost certain there are bugs Ext4 is not exempt from these fundamental laws of software engineering 'Code is always buggy until the last user of the program dies' He tied this back to the importance of testing patches before submitting, keep in mind that the maxim that code which is not buggy also applies to your patches read more http://www.secuobs.com/revue/news/210638.shtmlhttp://www.secuobs.com/revue/news/210638.shtml Secuobs.com : 2010-04-03 04:03:38 - KernelTrap - Linux news Mel Gorman posted the seventh version of his Memory Compaction patches asking, are there any further obstacles to merging The patches, first posted in May of 2007, provide a mechanism for moving GFP_MOVABLE pages into a smaller number of pageblocks, reducing externally fragmented memory Mel explains that 'compaction' is another method of defragmenting memory, for example, lumpy reclaim is a form of defragmentation as was slub 'defragmentation' really a form of targeted reclaim Hence, this is called 'compaction' to distinguish it from other forms of defragmentation The core compaction patch explains that memory is compacted in a zone by relocating movable pages towards the end of the zone A single compaction run involves a migration scanner and a free scanner Both scanners operate on pageblock-sized areas in the zone The migration scanner starts at the bottom of the zone and searches for all movable pages within each area, isolating them onto a private list called migratelist The free scanner starts at the top of the zone and searches for suitable areas and consumes the free pages within making them available for the migration scanner The pages isolated for migration are then migrated to the newly isolated free pages read more http://www.secuobs.com/revue/news/208600.shtmlhttp://www.secuobs.com/revue/news/208600.shtml Secuobs.com : 2010-04-02 00:43:42 - KernelTrap - Linux news Netfilter maintainer Patrick McHardy recently announced a first alpha-release of nftables, slated to eventually replace iptables as the standard Linux packet filtering engine Nftables aims to simplify the kernel ABI, reduce code duplication, improve error reporting, and provide more efficient execution, storage and updates of filtering rules Patrick began with a high level overview of the three pieces that comprise the firewall, the kernel provides a netlink configuration interface, as well as runtime ruleset evaluation using a small classification language interpreter libnl contains the low-level functions for communicating with the kernel, the nftables frontend is what the user interacts with An insightful overview can be found on lwnnet Patrick explained that data is represented internally in a generic fashion, meaning it's possible to use any matching feature ranges, masks, set lookups etc with any kind of data He went on to add, the kernel doesn't have a distinction between matches and targets anymore, operations can be arbitrarily chained, fixing a common complaint that multiple rules are required to fi log and drop a packet Terminal operations will stop evaluation of a rule, even if further operations are specified Speaking about the the userspace frontend, he noted, the classification language is based on a real grammar that is parsed by a bison-generated parser currently, it might have to be replaced and converted to a syntax tree Patrick continued, the frontend supports both dealing with only a single rule at a time for incremental operations, as well as parsing entire files In the later case verification is performed on all rules and changes are only made after full validation Currently not implemented, but planned, is transactional semantic where changes are rolled back when the kernel reports an error read more http://www.secuobs.com/revue/news/208193.shtmlhttp://www.secuobs.com/revue/news/208193.shtml Secuobs.com : 2010-04-01 18:28:19 - KernelTrap - Linux news Arnd Bergmann noted that he's working on removing the BKL from the Linux kernel, I've spent some time continuing the work of the people on Cc and many others to remove the big kernel lock from Linux and I now have a bkl-removal branch in my git tree He went on to explain that his branch is working, and lets him run the Linux kernel, on a quad-core machine with the only users of the BKL being mostly obscure device driver modules Arnd noted that this effort has a long history, the oldest patch in this series is roughly eight years old and is Willy's patch to remove the BKL from fs locksc, and I took a series of patches from Jan that removes it from most of the VFS Arnd noted that his patch adds a global mutex to the TTY layer, which he called the 'Big TTY Mutex' and described as, the basic idea here is to make recursive locking and the release-on-sleep explicit, so every mutex_lock, wait_event, workqueue_flush and schedule in the TTY layer now explicitly releases the BTM before blocking Alan Cox suggested that this portion of the patch was best dropped for now, it would be nice to get the other bits in first removing BKL from most of the kernel and building kernels which are non BKL except for the tty layer That after Ingo's box from hell has run it a bit would reasonably test the assertion that the tty layer has no BKL requirements that are driven by code external to tty layer code Andrew Morton suggested that the patches be pushed upstream to their appropriate maintainers for an additional sanity check, Seems that there might be a few tricksy bits in here Please do push at least the non-obvious parts out to the relevant people read more http://www.secuobs.com/revue/news/208048.shtmlhttp://www.secuobs.com/revue/news/208048.shtml Secuobs.com : 2009-07-22 10:08:19 - KernelTrap - Be careful -- there are some serious dragons there in the presence of multiple threads http://www.secuobs.com/revue/news/123430.shtmlhttp://www.secuobs.com/revue/news/123430.shtml Secuobs.com : 2009-07-22 10:08:19 - KernelTrap - A weak coder becomes a strong coder by reading code and writing code - every day, for fun http://www.secuobs.com/revue/news/123429.shtmlhttp://www.secuobs.com/revue/news/123429.shtml Secuobs.com : 2009-07-22 10:08:19 - KernelTrap - Some secure protocols like SSH send encrypted keystrokes as they're typed By doing timing analysis you can figure out which keys the user probably typed keys that are physically close together on a keyboard can be typed faster A careful analysis can reveal the length of passwords and probably some of the password itself http://www.secuobs.com/revue/news/123428.shtmlhttp://www.secuobs.com/revue/news/123428.shtml Secuobs.com : 2009-07-22 10:08:19 - KernelTrap - The _real_ bug is clearly in the hardware design that allows you to brick those things without apparently even having a lock bit I'm hoping Intel doesn't treat this as just a software bug Some hw designer should be thinking hard about which orifice they put their head up in http://www.secuobs.com/revue/news/123427.shtmlhttp://www.secuobs.com/revue/news/123427.shtml Secuobs.com : 2009-07-22 10:08:19 - KernelTrap - KernelTrap has been updated to Drupal 6 We're running a fair bit of custom code, so please let me know if you find anything broken or if any of your favorite features are missing And yes, this upgrade is the first step toward getting regular posts on the website again read more http://www.secuobs.com/revue/news/123426.shtmlhttp://www.secuobs.com/revue/news/123426.shtml Secuobs.com : 2008-10-15 15:48:14 - KernelTrap - New functionality has been enabled that allows logged-in users tohighlight interesting mailing list discussions This new feature hasbeen provided out of necessity, as I'm finding myself withinsufficient time of late for keeping up with the many mailing lists Itrack to post articles on KernelTrap My goal is to inspire you toparticipate more in the process, occasionally clicking the newup-arrow on mailing list messages that you find interesting and worthyof attention In the upcoming weeks, improved interfaces will beprovided for navigating other people's votes, and for filtering ononly the mailing lists you're interested in Future KernelTrap storiesand quotes will be selected from those that are highlighted by thisvoting processread morehttp://www.secuobs.com/revue/news/28454.shtmlhttp://www.secuobs.com/revue/news/28454.shtml Secuobs.com : 2008-10-15 15:48:14 - KernelTrap - "It is about time to take a step back and describe what I have beenimplementing," began Daniel Phillips, referring to his new Tux3filesystem He provided a simple ASCII diagram that detailed thefilesystem's hierarchical structure, describing each of the elementsAbout one he noted, "the volume table is a new addition not central tothe goals of Tux3, but a nice feature to have given that it comesnearly for free One Tux3 volume can have an arbitrary number ofseparate filesystems tucked inside it, indexed by a simple integerparameter at mount time People say they like this idea and it imposesno significant complexity, so it goes in" Daniel continued:"Each volume has a metablock pointing at the forward log chain forthe volume, a version table that describes the hierarchicalrelationship between versions snapshots, an atime table to takecare of that horrid legacy Unix feature, and an inode tablecontaining files and attributes of files Versioning takesplace in three places, versioned pointers in the atime btree,versioned extents in a file data btree and versioned attributes inthe inode table Notice the absence of a journal, thefunctionality of which is provided by forward log elements that Idescribed in the Hammer thread and will eventually write aseparate post about"read morehttp://www.secuobs.com/revue/news/28453.shtmlhttp://www.secuobs.com/revue/news/28453.shtml Secuobs.com : 2008-10-15 15:48:14 - KernelTrap - A recent discussion on the Linux Kernel mailing list noted thatthreaded 64-bit applications suffer a drastic slowdown inpthread_create performance when stack utilization goes above 4GB IngoMolnar offered an explanation of the problem, "unfortunately MAP_32BITuse in 64-bit apps for stacks was apparently created without foresightabout what would happen in the MM when thread stacks exhaust 4GB Theproblem is that MAP_32BIT is used both as a performance hack for64-bit apps and as an ABI compat mechanism for 32-bit apps So wecannot just start disregarding MAP_32BIT in the kernel - we'd break32-bit compat apps and/or compat 32-bit libraries" The originalreport noted that once the shared stack goes above 4GB in size, threadcreation can take as long as 10 milliseconds, a slowdown described as"quite unacceptable"Ingo created a patch introducing a new MAP_STACK flag for glibc to beused instead of MAP_32BIT and avoid imposing the 32-bit performancelimitation on threaded 64-bit applications He noted, "glibc canswitch to this new flag straight away - it will be ignored by thekernel" The new flag was quickly merged upstream, and changes wereplanned for glibcread morehttp://www.secuobs.com/revue/news/28452.shtmlhttp://www.secuobs.com/revue/news/28452.shtml Secuobs.com : 2008-10-15 15:48:14 - KernelTrap - "The latest feature release GIT 160 is available at the usual places,"began Git maintainer, Junio Hamano, announcing the latest stablerelease of the distributed version control system originally writtenby Linus Torvalds Among the current changes, Junio noted, "with thedefault Makefile settings, most of the programs are now installedoutside your $PATH, except for 'git', 'gitk' and some server sideprograms that need to be accessible for technical reasons" Hecontinued, "by default, packfiles created with this version usesdelta-base-offsetencoding introduced in v144 Pack idx files are using version 2 thatallows larger packs and added robustness thanks to its CRC checking,introduced in v152 and v1445" Julio highlighted several otherchanges, including the addition of a 'sample' extension to thedefault trigger scripts to be sure they don't execute in a defaultinstall, and the removal of the 'stupid' merge strategy Other changesinclude:"Git-gui learned to stage changes per-line; Reduced excessiveinlining to shrink size of the 'git' binary; When an object iscorrupt in a pack, the object became unusable even when the sameobject is available in a loose form, we now try harder to fallback to these redundant objects when able; performance of'git-blame -C -C' operation is vastly improved; even moredocumentation pages are now accessible via 'man' and 'git help';longstanding latency issue with bash completion script has beenaddressed; pager configuration variable can be used toenable/disable the default paging behaviour per command;git-cvsserver learned to respond to 'cvs co -c'; 'git-diff -p'learned to grab a better hunk header lines in BibTex,Pascal/Delphi, and Ruby files and also pays attention to chapterand part boundary in TeX documents; error codes from gitweb aremade more descriptive where possible, rather than '403 forbidden'as we used to issue everywhere; git-merge has been reimplementedin C"read morehttp://www.secuobs.com/revue/news/28451.shtmlhttp://www.secuobs.com/revue/news/28451.shtml Secuobs.com : 2008-10-15 15:48:14 - KernelTrap - "I'd like to get a first round of review on my AXFS filesystem," beganJared Hulbert, describing his new Advanced XIP File System for LinuxXIP stands for eXecute-In-Place The new filesystem received quite abit of positive feedback Jared offered the following description:"This is a simple read only compressed filesystem like Squashfsand cramfs AXFS is special because it also allows forexecute-in-place of your applications It is a major improvementover the cramfs XIP patches that have been floating around forages The biggest improvement is in the way AXFS allows for eachpage to be XIP or not First, a user collects information aboutwhich pages are accessed on a compressed image for each mmapedregion from /proc/axfs/volume0 That 'profile' is used as an inputto the image builder The resulting image has only the relevantpages uncompressed and XIP The result is smaller memory sizes andfaster launches"read morehttp://www.secuobs.com/revue/news/28450.shtmlhttp://www.secuobs.com/revue/news/28450.shtml Secuobs.com : 2008-10-15 15:48:14 - KernelTrap - "Another week, another -rc," began Linus Torvalds, announcing the2627-rc4 Linux kernel, continuing, "this time the diffstat is almosttotally dominated by the addition of the musb driver that drives theMUSB and TUSB controllers integrated into omap2430 and davinci That,together with the removal of the auerswald USB driver replaced bylibusb version is more than half of the bulk of the patch, andobviously most users won't ever notice" Linus added:"Apart from those bulky USB updates, there's some arch updatesblackfin and ia64, network and input driver updates, and an XFSand UBIFS update The rest is mostly random stuff all over,probably best described by the appended shortlog A number ofregressions should be off the table, but more remain"read morehttp://www.secuobs.com/revue/news/28449.shtmlhttp://www.secuobs.com/revue/news/28449.shtml Secuobs.com : 2008-10-15 15:48:14 - KernelTrap - Linus Torvalds announced the 2627-rc5 Linux Kernel, noting that his"weekly releases" tend to happen every eight days, adding, "the bulkof it is all config updates, and with arm and powerpc leading thepack" Linus continued:"While the config updates amount to about three quarters of thediff, and if you don't use a rename-aware diff the blackfininclude file movement pretty much accounts for the rest, hiddenbehind all those trivial but bulky changes are a lot of smallchanges that hopefully fix a number of regressions"The most exciting well, for me personally - my life isapparently too boring for words was how we had some stackoverflows that totally corrupted some basic thread datastructures That's exciting because we haven't had those in a longtime The cause turned out to be a somewhat overly optimisticincrease in the maximum NR_CPUS value, but it also caused someintrospection about our stack usage in general Including thingslike a patch to gcc to fix insane stack usage for vararg functionson x86-64 But that one would only hit anybody who was a bit tooadventurous and selected the big 4096 CPU configuration The restof the regressions fixed are a bit more pedestrian"read morehttp://www.secuobs.com/revue/news/28448.shtmlhttp://www.secuobs.com/revue/news/28448.shtml Secuobs.com : 2008-10-15 15:48:14 - KernelTrap - Daniel Phillips noted that his new Tux3 versioning filesystem is nowoperating like a filesystem, "the last burst of checkins has broughtTux3 to the point where it undeniably acts like a filesystem: one canwrite files, go away, come back later and read those files by name Wecan see some of the hoped for attractiveness starting to emerge: Tux3clearly does scale from the very small to the very big at the sametime We have our Exabyte file with 4K blocksize and we can alsocreate 64 Petabyte files using 256 byte blocks" He went on to discusssome of the remaining features yet to be implemented, including atomiccommits, versioning, coalesce on delete, a version of the filesystemwritten in the kernel, extents, locking, and extended attributesReviewing the above list, Daniel decided he would work next on thecoalesce on delete functionality, noting, "without this we can stilldelete files but we cannot recover file index blocks, only empty them,not so good" He added that at this time he was only going to focus onfile truncation, "as soon as file truncation is added to the test mixwe will see much more interesting behavior from the bitmap allocator,and we will discover some great ways to generate horriblefragmentation issues Yummy" Daniel continued to point out that Tux3is an open source project, and as such is always looking for others toparticipate, "whoever wants to carve their initials on what isstarting to look like a for-real Linux filesystem, now is a great timeto take a flyer The code base is still tiny, builds fast, has lots ofinteractive feedback and is easy to work on And you get to put youremail address near the beginning of the list, which will naturallywrite its way into the history of open source Probably"read morehttp://www.secuobs.com/revue/news/28447.shtmlhttp://www.secuobs.com/revue/news/28447.shtml Secuobs.com : 2008-10-15 15:48:14 - KernelTrap - "The patches most people hopefully care about tend to be smalldetails," noted Linus Torvalds, announcing the 2627-rc6 kernel Hecontinued, "and so more regressions should hopefully be closed now,some by just reverting the commits that caused breakage I don't thinkanything special merits explicit comment, but you can get a flavor forthings by scanning the appended shortlog" Earlier in the announcementemail, Linus did note some specifics about which drivers caused thebulk of the patch:"Same old deal - except it's been almost two weeks since -rc5That said, the diff is actually about the same size, so I guessthat means things are calming down Most of the diff bulk-wiseis updates to the new gspca standard USB webcam driver, althoughsome of it is also removal of the dead miropcm20* driver"read morehttp://www.secuobs.com/revue/news/28446.shtmlhttp://www.secuobs.com/revue/news/28446.shtml Secuobs.com : 2008-10-15 15:48:14 - KernelTrap - "So yet another week, another -rc," began Linux creator, LinusTorvalds, announcing the 2627-rc8 Linux kernel He continued, "thisone should be the last one: we're certainly not running out ofregressions, but at the same time, at some point I just have to picksome point, and on the whole the regressions don't look _too_ scaryAnd -rc8 obviously does fix more of them" Linus went on to note thatmost of the changes since -rc7 are small, "and there aren't even awhole lot of them"Jiri Kosina cautioned that there is still an unknown bug affecting thee1000e driver currently in the 2627 kernel, "rendering the cardsunusable for most of the i-am-not-a-hacker users and remember, evenDave Airlie bricked his laptop completely to death, when trying torestore eeprom contents" When asked how to duplicate the bug, Jirinoted that the inability to reliably reproduce the bug added to thedifficulty in debugging the problem, "apparently it is some kind ofrace, as it usually takes multiple cycles to trigger"read morehttp://www.secuobs.com/revue/news/28445.shtmlhttp://www.secuobs.com/revue/news/28445.shtml