badass debugger + badass toy = geek pr0n

Today I finally got working a hacked-together minimal version of the iPhone debugger client for BinNavi. It's heavily based on Patrick Walton's (with HD's updates) weasel debugger. Once tied to BinNavi debug client framework the whole client-server interaction is trivial.

It feels just right, the best looking debugger together with the slickest device.. recipe for fun.. ;-)



The test application is telnet on the iPhone. On the iPhone's screen is the debug output from BinNavi's debug client. telnet is launched from an ssh session in OSX, where BinNavi is running.



For anybody trying to link Mach's debugging interface with a C++ iPhone application, remember the extern "C" when defining boolean_t exc_server(mach_msg_header_t *in, mach_msg_header_t *out); (which is not defined in the header files, as pointed in weasel's source code). Otherwise you'll get a nasty "Undefined symbols" message when linking.

extern "C" is also needed for catch_exception_raise(...) so exc_server can call it to handle exceptions. Documented here.
(I've used the standard iPhone toolchain on Debian, this is running on the firmware 1.1.3)

pydot 1.0.2 ... that took long

Finally! it was long due. Here it is pydot 1.0.2

Some weeks ago I started updating the code to support all the attributes and enhancements in GraphViz 2.16. In attempting to make it pass all the regression tests some severe shortcomings it had became apparent.
pydot users had also provided with insight into how to improve performance by redesigning the way the data for the objects is stored internally. All in all, the limitations I was facing led me to rewrite the whole core of pydot, which took much longer than I wanted but I feel it was well worth it as it's orders of magnitude better than the last release 0.9.

Performance-wise the new pydot stores graphs and their objects using a hierarchy of nested dictionaries and lists. Graph, Node, Edge objects are mere proxies to the data and are created on demand. So that now it's possible to have a graph with a 1 million edges and there will not be a single Edge instance (only if requested, then they will be created on demand, mapping the data and providing with all the methods to act on the data in the global dictionary).
Storing a graph with 1 million edges in pydot 1.0 has approximately the same memory requirements (~813MiB) as dealing with one with only 40.000 edges in pydot 0.9 (~851MiB), the 40.000 edges graph needs ~35MiB in pydot 1.0 . Handling graphs should be much faster, as no linear searches are performed in pydot 1.0.2

Exe_Dump_Utility, a web-enabled pefile

Gregory Piñero has put together Exe_Dump_Utility, a web-based version of pefile. Now it's possible to obtain the whole set of information processed by pefile online, without the need to install it. Neat!

Take Two: Packers, Time and Google Groups

I just had to do it... This morning I read about chronoscope in a post in the Google Code Blog and I could not help myself from wanting to tinker with it.

I wrote a Mathematica function to export a time-series of the format (timestamp, value) into the dataset format used by chronoscope.

Epoch[date_] :=   ToString[AbsoluteTime[DateList[ToString[date]]] -   AbsoluteTime[DateList["1970"]]]; ChronoscopeJsExport = Function[ {datasetName, id, label, axis, data},   jsData = datasetName <>   " = {\nId: \"" <> ToString[id] <> "\", \n" <> "domain: [" <>   StringJoin[ Riffle[ Map[ Epoch, data[[All, 1]] ], ", "] ] <>   "], \n" <> "range: [" <>   StringJoin[     Riffle[ Map[ ToString, data[[All, 2]] ], ", "] ] <> "], \n" <>   "label: \"" <> ToString[label] <> "\", \n" <>   "axis: \"" <> ToString[axis] <> "\"\n};";   jsData ];

And ran it through the packer time-series I harvested from Google Groups. Then I picked some widget demo code and put it all together in a mash-up. The results of the quick hack are here... much nicer to visualize than in the previous post. (and it's interactive!)

• Use the mouse-wheel to zoom
• Drag the plot left/right to browse around different date ranges
• You can pick any packer and the data will be plotted against the previously selected one

pefile 1.2.8

And yet another one. pefile 1.2.8 comes with the usual few bugfixes and a slew of enhancements. Some of them are:

• One can now "relocate" the image by invoking relocate_image(ImageBase) with a new ImageBase the PE file's relocations will be applied to produce the relocated image.


• Section entropy is computed faster (thanks to Gergely)


• MD5, SHA-1, SHA-256, SHA-512 hashes are calculated on a per-section basis (thanks Jim Clausing for the suggestion)


• Improved (rather fixed) handling of Unicode strings when parsing the resources information

For more details and downloads head to pefile's project page.

Hex-Rays unleashed

Hex-Rays, Ilfak Guilfanov's decompiler, has been unleashed. I have had the chance of playing a bit with the beta and it is really impressive, to say the least. This will save so many hours to reverse engineers...

pefile 1.2.7

Just pushed out an updated version of pefile with some minor enhancements and fixes:

• Added additional IMAGE_SUBSYSTEM_* flags
• Added processing of the Optional Header's DllCharacteristics
• Time/date fileds are now reported as UTC times
• Added warning message for suspicious entry point addresses
• Several minor parsing bugs fixed

Black Hat Slides

Although originally Halvar Flake and I were supposed to present together in a quick turbo-talk at Black Hat in Las Vegas, he unfortunately couldn't make it to the conference for reasons that have been already discussed.

I ended up sticking mostly to the original plan for the talk and presented some Python tools to automate reverse engineering and analysis processes.

I've just put the slides up here.

pefile 1.2.6

It's finally here! Took longer than I expected because of all the enhancements and because I decided to move pefile to Google Code.

Besides access to the source code through their subversion server, they also have a really cute wiki.

I've added documentation and examples and it should make it easier for people to contribute ideas and improvements.

I introduced some of the new features of pefile in my turbo-talk in the last Black Hat in Las Vegas.

Besides some bugfixes, pefile-1.2.6 can now parse PEiD's signatures, it will report on the entropy of each section and will display more warnings for suspicious values found when parsing PE files. Just check the example dumps of Tiny PE and 0x90.exe

Information on how to use the PEiD signature matching can be found here. I posted a while ago on what can be done with the signature parsing.

Windows XP and Bochs

This might be of interest for anyone out there attempting to get Windows XP to install inside Bochs with no luck.

I had not been able to get it to install in any recent version, for one reason or another it always failed during install with a problem regarding the "catalogs" (error message was along the lines of "Setup failed to install the product catalogs. This is a fatal error."). No matter what options I had compiled in Bochs.

I had got it to install in the past... a long long time ago, so I figured out that I might get lucky with other versions of Bochs, so I started trying. 1.4.1 nothing, 2.0.2 nothing, 2.1.1 nothing (trying all of them with different configuration options, that took a while)... but finally got to 2.2.6 and bingo! it made it through with no errors! Once installed the image runs just fine in the latest incarnation of Bochs. Here it is, Bochs running Windows XP on Fedora running inside Parallels on my OS X...

Virtualization(emulation) = madness^2 ?

I hope this saves somebody from hours of compiling, recompiling and reinstalling...

VxClass. Automated executable classification

Here at Sabre Security, we have been putting together a variety of technologies nurtured and developed over years of reverse engineering and malware analysis. All of it has taken form in VxClass, which is finally shaping up. We are quite proud and happy to see such a complex project (and one I've personally long dreamed about) actually working.



The first incarnation of VxClass is already able to automatically handle (unpack, analyze and classify) a wide range of Windows malware. The results are nearly addictive to look at. Accompanying this post are some screen-shots of the Web interface showing a listing of files and the automatically generated cluster of families.

VxClass will allow analysts or other tools to communicate with it and submit executable files. Those will be unpacked, analyzed and classified automatically according to their structural properties. The classification results as well as analysis databases can be retrieved through either an XMLRPC or a Web interface.



If you or your company would be interested on evaluating it or discussing if it might be an applicable technology for you, don't hesitate in dropping us a mail.

Also, I'm in the Bay Area for most of June and August (also in Black Hat in Las Vegas). So if you want to have me demo it or chat about it, drop me a line.

ida2sql, exporting IDA's dissasemblies to SQL

Because BinNavi nowadays reads all the disassembly information from a SQL database, we needed some means of exporting the information to it. ida2sql is the result, it is a monster set of Python scripts I wrote (all nicely wrapped in a couple of files for easy installation) that will export the information from an IDB (only Intel, ARM and PPC so far. The latter two in experimental mode) into a MySQL database.
It's available for download from my site together with installation and usage instructions. It needs the IDAPython plug-in to run.

Any feedback is welcome.

I posted a while ago about the database schema. One can do pretty neat things when having the dissasembly in such form...

pefile-1.2.5 released

Besides some small fixes the new release of pefile is able to report suspicious or malformed entries encountered while parsing. Any time that a non-critical (something that wouldn't prevent the file from running) problem is found it's added to a list of warnings that can be retrieved get_warnings() or shown show_warnings()

Example warning messages:

Suspicious NumberOfRvaAndSizes in the Optional Header. Normal values are never larger than 0x10, the value is: 0xdfffddde

Error parsing the import table. Invalid data at RVA: 0x400000 Error parsing the Import directory. Invalid Import data at RVA: 0x60 Error parsing export directory at RVA: 0x6c6c642e

IDAPython 0.9.0

Gergely Erdélyi has just put out the last release of IDAPython, 0.9.0

It can be found in his site together with some brief release notes. This release supports Python 2.5 among a good deal of other enhancements and additions.

pefile 1.2.2

I've just released an update to pefile. This new release includes:

• pefile-1.2.2 can now correctly parse the files from the Tiny PE challenge, which pushes the limits of valid parsing


• Added support for parsing the version information structures in the resources directory
, I'll blog about how to take advantage of this feature shortly

Additional information and download links can be found in pefile's page.

Intel binaries for Pythonika

I've just uploaded new Pythonika packages (tar.gz and zip) to my site. The only change is that there are now compiled versions for Intel of the MathLink module for OS X. For Python 2.3 and 2.5.

pefile 1.2

pefile has just gotten some nice updates.

- Added support for PE32+ files
- Merged the patches from the Offensive Computing people and other contributors
- Added support for writing changes back to the PE file. This should be used with care
- Miscellaneous other bugfixes and enhancements

As usual, more information and download links can be found in pefile's page.

Pythonika

I have finally managed to release Pythonika! I wrote Pythonika quite a while ago and was never getting around to push it out.

Pythonika is a MathLink module for Mathematica that makes it possible to write Python code within Mathematica's notebooks. It handles the conversion of Python and Mathematica objects transparently and allows to use all of Python's standard modules.

I'm a big fan of Python and I've been using Pythonika for a while. I hope more people will find it useful.

Pythonika is available at:

http://dkbza.org/pythonika.html

(an example notebook is available on the previous link as well as in the downloaded package)

The download includes source code and binaries for OSX/Windows/Linux.

pefile-1.1

I just released pefile-1.1. This release brings some new functionality besides some bugs fixed. A detailed list of changes is available here.

IDAPython 0.8.0 released

Dyce has released IDAPython 0.8.0. I know of plenty of people who have been waiting for this one, me for one. Also, the Windows version is linked against Python 2.4.