Exe_Dump_Utility, a web-enabled pefile

Gregory Piñero has put together Exe_Dump_Utility, a web-based version of pefile. Now it's possible to obtain the whole set of information processed by pefile online, without the need to install it. Neat!

pefile 1.2.8

And yet another one. pefile 1.2.8 comes with the usual few bugfixes and a slew of enhancements. Some of them are:

• One can now "relocate" the image by invoking relocate_image(ImageBase) with a new ImageBase the PE file's relocations will be applied to produce the relocated image.


• Section entropy is computed faster (thanks to Gergely)


• MD5, SHA-1, SHA-256, SHA-512 hashes are calculated on a per-section basis (thanks Jim Clausing for the suggestion)


• Improved (rather fixed) handling of Unicode strings when parsing the resources information

For more details and downloads head to pefile's project page.

pefile 1.2.7

Just pushed out an updated version of pefile with some minor enhancements and fixes:

• Added additional IMAGE_SUBSYSTEM_* flags
• Added processing of the Optional Header's DllCharacteristics
• Time/date fileds are now reported as UTC times
• Added warning message for suspicious entry point addresses
• Several minor parsing bugs fixed

Black Hat Slides

Although originally Halvar Flake and I were supposed to present together in a quick turbo-talk at Black Hat in Las Vegas, he unfortunately couldn't make it to the conference for reasons that have been already discussed.

I ended up sticking mostly to the original plan for the talk and presented some Python tools to automate reverse engineering and analysis processes.

I've just put the slides up here.

pefile 1.2.6

It's finally here! Took longer than I expected because of all the enhancements and because I decided to move pefile to Google Code.

Besides access to the source code through their subversion server, they also have a really cute wiki.

I've added documentation and examples and it should make it easier for people to contribute ideas and improvements.

I introduced some of the new features of pefile in my turbo-talk in the last Black Hat in Las Vegas.

Besides some bugfixes, pefile-1.2.6 can now parse PEiD's signatures, it will report on the entropy of each section and will display more warnings for suspicious values found when parsing PE files. Just check the example dumps of Tiny PE and 0x90.exe

Information on how to use the PEiD signature matching can be found here. I posted a while ago on what can be done with the signature parsing.

pefile and packer detection

I've always wanted some tool that I could run over large collections of executable files and would tell me what's packed and what's not and, ideally, also the packer. PEiD has wonderful signature libraries but my ideal tool would be easier to integrate with other components and not restricted to Windows.
The guys at OffensiveComputing had put together some code to, by making use of PEiD signatures and pefile, recognize packers.

I've decided that it's time for pefile to have such functionality by default and I've reimplemented the signature parsing and matching. The next version of pefile should include this new code.
I've also found some pretty extensive signature libraries and here are some of results of the test runs in some files I've laying around.

Of the 48.025 files (all malware) that I scanned, in ~42% no packer could be found using the current signature database. In the remaining ~58% the tests found 227 different packers and compiler signatures.



A more extense listing of the most frequently found packers looks like:



Given that I've run pefile in several tens of thousands of pieces of malware with all kind of exotic PE format contortions, I've managed to find and fix a couple of obscure bugs. The forthcoming release will be even stronger when facing files that push the limits of the PE format well-formedness.

pefile-1.2.5 released

Besides some small fixes the new release of pefile is able to report suspicious or malformed entries encountered while parsing. Any time that a non-critical (something that wouldn't prevent the file from running) problem is found it's added to a list of warnings that can be retrieved get_warnings() or shown show_warnings()

Example warning messages:

Suspicious NumberOfRvaAndSizes in the Optional Header. Normal values are never larger than 0x10, the value is: 0xdfffddde

Error parsing the import table. Invalid data at RVA: 0x400000 Error parsing the Import directory. Invalid Import data at RVA: 0x60 Error parsing export directory at RVA: 0x6c6c642e

Tiny (and crazy) PE

I did prepare a couple of new graphics for the last training I taught with Pedram in BlackHat DC.

One of them was to illustrate a bit the structure of the header mess that leads to the small footprint of the executables in Solar Eclipse's solution to the Tiny PE challenge.

I think it's a good example of how flexible and tolerant the Windows loader is and why loading PE files is something that tends to break most tools when a file pushes the limits.

I'll comment on some of the slides on this post without going into too much detail on how or why things work, that's well explained in Solar Eclipse's page. The general format is the same I use when doing the walk-through of the PE format's main headers in a sane file in order to illustrate how the headers are laid out on the file itself.

The red zeros in the following pictures mean data beyond the file size. That data is zeroed by Windows when the file is mapped in memory and Tiny PE relies on those zeros being there, as windows will try to access data in memory at that location, beyond the end of the file. If the memory wasn't zeroed first and contained random data it would be much harder to cook up the headers in the current "compressed" layout.

This first shot just shows the DOS header and the e_lfanew field. Which points to the the start of the NT headers.



The e_lfanew field contains 4, which is the offset within the file where the NT headers can be found. That's in the middle of what would otherwise be the DOS header. In the shot of NT headers we can see some of its fields.



The NT headers contain the File and Optional headers, the next picture shows some of the fields constituting the Optional header.



As the last entry in the Optional Header one can find the array of data directories.



Now, the loader would need to locate the section headers. These normally follow after the directories of the Optional header. But in this case, as it's illustrated in the following picture, they lay in what would be the middle of the Optional Header. The location of the section headers is calculated by adding the size of the Optional header (4) to its offset (0x1C). Amusingly enough, the Windows loader does not take into account the reported size of the optional header when it reads the header itself, but it does in order to find what follows.



And here are the fields of the section header...



I haven't taken the time to illustrate the import directory and the couple of additional details missing (It's just left as an exercise of mental contortion for the reader...). The original text by Solar Eclipse provides with the rest of the info for the interested souls.

pefile: parsing version information from the resources directory

A while ago I got some inquiries on how to go about reading the version information stored in PE files.

I had an idea of it being just a bunch of unicode strings without much of a structure but to follow along the rest of the PE file format, it does indeed have some structure. The only inconvenient was to find proper resources on how to parse it as Microsoft's docs mainly amount to, understandably, "just use the API". I eventually found a couple of references where a parser for the version information stored with a Portable Executable's resources directory was implemented.

After finally understanding how that information was stored, I added support in pefile so now a dictionary is conveniently returned whenever parseable version information exists in a PE file.

Some of the links in which I based my parsing implementation are:

• Full parser in C


• VS_VERSIONINFO structure definition


• VS_FIXEDFILEINFO structure definition


• VarFileInfo structure definition


• StringFileInfo structure definition

From those last two links one can follow into definitions for the other structures.

Now, before I forget how this all goes. The version info structure in composed of a list of substructures. Those substructures can be of StringFileInfo or VarFileInfo type. The former contains the usual textual information that can be seen on the Version tab on the Properties dialog for a PE image. The later specifies version information in a way that does not depend on the language and codepage.



StringFileInfo contains a list of StringTable structures and each of those contains a String structure. This last structure contains the Key, Value pairs that make for the textual version information.
VarFileInfo contains a list of Var structures (although normally is only one) and each of those contains a list of pairs of Word values with version information.

• VS_VERSIONINFO(VS_FIXEDFILEINFO)
  • StringFileInfo
    • StringTable (LangID)
      • String
  • VarFileInfo
    • Var
      • WORD, WORD

Example

If the file has version information, the following attributes will exist in the PE instance returned.

• VS_VERSIONINFO will contain the first three fields of the main structure: 'Length', 'ValueLength', and 'Type'


• VS_FIXEDFILEINFO will hold the rest of the fields, accessible as sub-attributes: 'Signature', 'StrucVersion', 'FileVersionMS', 'FileVersionLS',
  'ProductVersionMS', 'ProductVersionLS', 'FileFlagsMask', 'FileFlags', 'FileOS', 'FileType', 'FileSubtype', 'FileDateMS', 'FileDateLS'


• FileInfo is a list of all StringFileInfo and VarFileInfo structures.


• StringFileInfo structures will have a list as an attribute named 'StringTable' containing all the StringTable structures. Each of those structures contains a dictionary 'entries' with all the key/value version information string pairs.


• VarFileInfo structures will have a list as an attribute named 'Var' containing all Var structures. Each Var structure will have a dictionary as an attribute named 'entry' which will contain the name and value of the Var.

print hex(pe.VS_VERSIONINFO.Length) print hex(pe.VS_VERSIONINFO.Type) print hex(pe.VS_VERSIONINFO.ValueLength) print hex(pe.VS_FIXEDFILEINFO.Signature) print hex(pe.VS_FIXEDFILEINFO.FileFlags) print hex(pe.VS_FIXEDFILEINFO.FileOS) for fileinfo in pe.FileInfo: if fileinfo.Key == 'StringFileInfo': for st in fileinfo.StringTable: for entry in st.entries.items(): print '%s: %s' % (entry[0], entry[1]) if fileinfo.Key == 'VarFileInfo': for var in fileinfo.Var: print '%s: %s' % var.entry.items()[0]

0x35c 0x0 0x34 0xfeef04bdL 0x0 0x4 LegalCopyright: Mozilla Corporation InternalName: Firefox FileVersion: 1.8.1: 2006101023 CompanyName: Mozilla Corporation LegalTrademarks: Firefox is a Trademark of The Mozilla Foundation. Comments: ProductName: Firefox ProductVersion: 2.0 FileDescription: Firefox OriginalFilename: firefox.exe Translation: 0x0000 0x04b0

This should come quite handy, for instance, to people interested in creating databases of version information of collections of DLLs and EXEs...

pefile 1.2.2

I've just released an update to pefile. This new release includes:

• pefile-1.2.2 can now correctly parse the files from the Tiny PE challenge, which pushes the limits of valid parsing


• Added support for parsing the version information structures in the resources directory
, I'll blog about how to take advantage of this feature shortly

Additional information and download links can be found in pefile's page.

pefile 1.2

pefile has just gotten some nice updates.

- Added support for PE32+ files
- Merged the patches from the Offensive Computing people and other contributors
- Added support for writing changes back to the PE file. This should be used with care
- Miscellaneous other bugfixes and enhancements

As usual, more information and download links can be found in pefile's page.

pefile-1.1

I just released pefile-1.1. This release brings some new functionality besides some bugs fixed. A detailed list of changes is available here.

pefile is out!!

Finally, after many delays I've managed to find time to release a newer version of pefile (was pype). The name change comes from the fact that another project called PyPE already exists and was using the name earlier. So now it's pefile.

It can now parse Delayed Imports, thanks to Adam Morrison, and multiple fixes have been added. It should be able to take really corrupted and malformed PE files.