News for version 0.9.2

The new version of the Profiler is out with the following news:

removed virtual memory constraint: large files are now supported
added decompression bomb detection
added media preview for image files
added preview for several PE resources
added text preview for Office Word Documents
added format selection to open file dialog
display format choose dialog when more than one format has been detected
added XFA interactive forms detection inside PDFs
added from/to hex and base64 filters
automatically detect files in Zip archives missing a Central Directory
increased PySide integration
– fixed Office VBA extraction bug
– fixed bug in PDF V4 and V5 Revision encryption

Format detection & selection

To better help with the identification of files which can be interpreted as different formats, the individual file dialog features now some additions.

As you can see the identified formats for the currently selected file are listed (it’s a simple GIF file with a PDF appended at the end). The dialog gives the user also the ability to manually choose the format to use for loading the file. While all this could be achieved even before, it wasn’t as handy as it is now.

However, it wouldn’t make sense to display the file selection dialog when the user uses the shell integration or drops a file to open it. So, instead the Profiler displays a choice dialog for the format in case multiple formats are detected.

Conversion filters

Some new filters are available: from/to hex/base64.

While the actions in the Profiler already feautured a mechanism to do these conversions, having them as filters is extremely useful, because it allows to use them to load embedded files or to convert large portions of data.

Damaged Zip archives

While it has always been possible to manually extract through filters data or partial data from damaged Zip files (e.g. those missing a Central Directory), now the embedded data is automatically analyzed and ready for inspection. This means that even when a Zip archive is truncated and some compressed files are truncated as well, they will nonetheless be automatically detected and be available for inspection by the user.

As you can see many improvements have been introduced. The most important of them is of course the removal of the virtual memory constraints as it represents an important step in the roadmap of the Profiler. Stay tuned as the next version will be important as well!

XFA Interactive Form Inspection

The upcoming 0.9.2 version of the Profiler introduces detection of Acro/XFA interactive forms inside PDFs. This technology has been abused numerous times (some recent cases come to mind), so it is now being reported as a potential threat.

The video below shows the inspection of a XFA Interactive Form and how to load a base64-encoded GIF image embedded in it.

Stay tuned!

Previews

The upcoming version 0.9.2 of the Profiler adds previews for various things: images (all supported formats), several Portable Executable resources and Office Word Documents (text-only).

PE resources preview

Since media elements are rendered through third-party code, the Profiler displays a warning box before actually rendering a media element.

Preview warning

The ‘Allow all’ button allows media elements for the current session only. If the Profiler is running in a safe environment (like a VM), the user can decide to permanently disable the warning box and allow all media elements.

Preview settings

Last but not least, text-only preview of Office Word Documents has been introduced. This allows users to safely inspect the text content of a document without processing the file with an official viewer which could be the target of exploits.

Office document preview

While there are already enough new features to release, some smaller additions will be squeezed into 0.9.2 during the next days. Stay tuned!

Zip bomb revisited

The upcoming 0.9.2 release of the Profiler removes the virtual memory constraint, meaning that it is now able to open and process files of any size (the hex editor can edit large files as well). This feature has actually been in the TODO list from day 1 and I’ll write about the internals of it in some other post, in order to better demonstrate the capabilities gained by these changes to the core.

Also, because of the increased functionality, it made sense to add detection for Zip (decompression) bombs. Almost a year ago we’ve talked about Zip bombs, but it was limited to the safe exploration of such files. Let’s start with the new ‘Limits’ page in the setup.

Limits

  • What had been once the maximum size of a file is now the size of virtual memory the memory pool is allowed to use. Of course, the more virtual memory is granted, the faster it becomes to analyze large files. But it’s only a matter of speed, you can choose to give to the pool the bare minimum, it’ll work just as well.
  • The nesting option shouldn’t need any explanation since it hasn’t changed. But just for completeness: it specifies the maximum level of scanning into a root object. If more levels are available, it is signaled. For instance, level 0 specifies that children objects should not be scanned automatically (though they might still be opened manually by the user).
  • The maximum file size can be used to discard files larger than the specified size during batch scan operations. The default is 0, which stands for infinite.
  • The decompression bomb threshold is the limit we’re interested in for this post. It represents a cumulative size which can’t be exceeded. In other words if an archive contains 1 file of 100+ GBs (let’s use this number for the purpose of this example) it’s the same as whether one sub-archive contains 100 files of 1 GB each (plus a single byte in excess). If the threshold is exceeded, it will be reported as a threat. While 100 GBs is the default, you can speed up scanning by specifying a lower limit.
  • The maximum number of children files is cumulative as well. This constraint depends on virtual memory limitations (as all children are shown in a tree). 100.000 (the default) is a safe choice. If more children than the imposed limit are present, it is signaled.

So let’s again take the famous 42.zip as an example of Zip bomb and let’s scan it. We’ll get this in the summary:

Decompression bomb threat

Please note that the threat may not be reported in the summary of the root object itself, but in one of its children objects (once the bomb threshold has indeed been exceeded). But since we know there’s a threat (as reported by the risk factor) we can just jump to it by pressing F2 in the hierarchy view.

Widgets and Views

The last release of the Profiler featured some significant improvements. So while it also included initial PySide support, there wasn’t much time to make it really nice. One of the missing things was the ability to mix internal Profiler views (such as the hex editor) with PySide widgets. With the upcoming 0.9.2 release it will be possible to create a view and obtain a PySide widget with just one method:

widget = view.toWidget()

This way one can make use of advanced internal views of the Profiler and combine them with other custom controls. Let’s see a practical example.

Mixed widget

The widget in the screenshot combines a QTreeView with a directory model and a hex view. When a file is activated in the tree, it is opened by the hex editor. To try it out, just press Ctrl+Alt+R and enter the following code:

from Pro import *
from PySide import QtCore, QtGui

class MixedWidget(QtGui.QSplitter):
    def __init__(self, parent=None):
        super(MixedWidget, self).__init__(parent)

        self.setWindowTitle("Mixed widget")
        self.setOrientation(QtCore.Qt.Vertical)

        self.model = QtGui.QDirModel()
        tree = QtGui.QTreeView()
        tree.setModel(self.model)
        self.addWidget(tree)

        ctx = proContext()
        self.hex = ctx.createView(ProView.Type_Hex, "")
        self.addWidget(self.hex.toWidget())

        tree.activated.connect(self.updateFile)

    def updateFile(self, idx):
        if self.model.isDir(idx) == True:
            self.hex.clear()
        else:
            name = self.model.filePath(idx)
            self.hex.setFileName(name)


ctx = proContext()
w = MixedWidget()
v = ctx.createViewFromWidget(w)
ctx.addView(v)

Amazingly little code snippet, right? Please note that the ProHexView setFileName method is also a new addition to the SDK.

News for version 0.9.1

The new version of the Profiler is out with the following news:

added capability of opening multiple analysis views
added capability of switching root object in the workspace
added navigation in analysis views
added bookmarks
added PySide integration
– added user application data folder support
– added history for the Python command line and script dialog
– added save option to the keys input dialog
– improved notes: the toolbar now signals their presence
– updated Qt to 4.8.4

Also a new Demo version has been released, which as usual can be found on the product page.

UI Improvements & Bookmarks

The upcoming 0.9.1 version of the Profiler features some important UI improvements and the introduction of bookmarks. Among the UI improvements there’s:

  • the ability to switch root in the workspace
  • multiple analysis views displaying data of different roots
  • navigation

In this case a video is probably more worth than a thousand words.

These new features lay down the groundwork for some more interesting capabilities which will be added soon. Stay tuned! ­čÖé

PySide support

This is really a small addition which took just a couple of hours of work, but since it can come very handy, it’s worth dedicating a post to it. The upcoming 0.9.1 version of the Profiler adds explicit support for PySide. Thus, it will be possible to create Qt widgets and add them to the workspace.

Installing PySide

First of all, let’s install PySide. There are 3 ways to do this.

1) Install it from the qt-project page.

Make sure you select the package matching the current Python version used by the Profiler.

2) Install the package we compiled for you. It’s vanilla, directly from the original sources, but it has the advantage that it is guaranteed to work. In fact, at the time of writing the official package contains a bug (missing shiboken Python module) and so the first one is not really an option until it is not fixed.

Download
SHA1: 2024348E79890A167BB231098A6A16FC8BB02C9E

3) You can compile PySide yourself following the instructions at qt-project. At the end, use the installer created inside ‘c:\pyside-setup\dist’.

A code sample

Using it is even easier than the setup process. Basically ProContext has a new method called createViewFromWidget which takes as a parameter a widget created by PySide and returns a ProView which in turn can be added to the workspace.

Adding a widget to the workspace only takes the following line:

ctx.addView(ctx.createViewFromWidget(widget))

Therefore using an existing widget and adding it to the workspace is very easy. Let’s see a real-world widget like an official PySide sample: PySide/examples/effects/lighting.py. It’s sufficient to remove:

if __name__ == '__main__':

    import sys

    app = QtGui.QApplication(sys.argv)

    lighting = Lighting()
    lighting.setWindowTitle("Lighting and Shadows")
    lighting.resize(640, 480)
    lighting.show()

    sys.exit(app.exec_())

And add:

lighting = Lighting()
lighting.setWindowTitle("PySide widget")

ctx = proContext()
ctx.addView(ctx.createViewFromWidget(lighting))

Now we can add an action to execute the code or just insert it in the custom script box (Ctrl+Alt+R) and the view will be shown like this:

To set a custom icon for the view use setWindowIcon.

As usual stay tuned as the upcoming version is going to include some major additions and significant changes.

News for version 0.9.0

We’d like to wish everybody merry Christmas and a happy new year!

We’re just in time to place under the Christmas tree a new version of the Profiler with the following news:

added Java Class support including byte code disassembler and layout ranges
added .NET support including byte code disassembler and layout ranges
added DEX support including byte code disassembler and layout ranges
added dedicated view to display data like raw PDF objects
added PE MUI resources validation
– added Adler32 to filters
– updated jsbeautifier
separated malicious threats from intrinsic ones in the report view
– fixed update with unprivileged user account on Windows
– fixed several bugs

The main addition in this new release as seen previously is the managed trio Java, DEX and .NET.

Dedicated view to display raw data

Previously PDFs had 3 views to display objects: one for the dictionary, one for the decoded stream and one for the decoded stream shown as text. Now there’s also a ‘raw data’ view to show the object unmodified just as it is in the file.

Highlighted with different colors you can see the dictionary/value and the stream part. The same applies to child objects which are highlighted in the stream of their parent.

PE MUI resources validation

Following a short post about MUI resources, validation for them has been added. Also some bug fixes related to resource validation.

Separated malicious threats from intrinsic ones

In order to better separate intrinsic risk factors from malicious threats, they are now shown separately in the report view.

Have some nice holidays and stay tuned as we’ll try to add even more features the upcoming year.

Christmas hat image from freevector.com

DEX support

Support for Android’s DEX format is the last major feature of the upcoming 0.9.0 release of the Profiler. The support includes format, layout ranges and a Dalvik disassembler. Support for APK is implicit, since support for Zip archives has been added long ago.

All sections of the format are accessible. The central point for parsing a DEX file is the Classes view. Hence it’s also the most complex view of the format.

Also accurate layout ranges can help to analyze the format.

And finally a disassembler to quickly inspect the Dalvik bytecode.

Here’s a disassembled function from the Android SDK NotePad sample:

  private final void cancelNote()
  {
    const/4 v3, #int 0 // #0
    iget-object v1, v4, android.database.Cursor mCursor
    if-eqz v1, loc_74 // +34
    iget v1, v4, int mState
    if-nez v1, loc_90 // +38
    iget-object v1, v4, android.database.Cursor mCursor
    invoke-interface {v1}, void android.database.Cursor.close()
    iput-object v3, v4, android.database.Cursor mCursor
    new-instance v0, android.content.ContentValues
    invoke-direct {v0}, void android.content.ContentValues.()
    const-string v1, "note"
    iget-object v2, v4, java.lang.String mOriginalContent
    invoke-virtual {v0, v1, v2}, void android.content.ContentValues.put(java.lang.String, java.lang.String)
    invoke-virtual {v4}, android.content.ContentResolver getContentResolver()
    move-result-object v1
    iget-object v2, v4, android.net.Uri mUri
    invoke-virtual {v1, v2, v0, v3, v3}, int android.content.ContentResolver.update(android.net.Uri, android.content.ContentValues, java.lang.String, java.lang.String[])
loc_74:
    const/4 v1, #int 0 // #0
    invoke-virtual {v4, v1}, void setResult(int)
    invoke-virtual {v4}, void finish()
    return-void
loc_90:
    iget v1, v4, int mState
    const/4 v2, #int 1 // #1
    if-ne v1, v2, loc_74 // -11
    invoke-direct {v4}, void deleteNote()
    goto loc_74 // -16
  }

And the original source for comparison:

    private final void cancelNote() {
        if (mCursor != null) {
            if (mState == STATE_EDIT) {
                // Put the original note text back into the database
                mCursor.close();
                mCursor = null;
                ContentValues values = new ContentValues();
                values.put(NotePad.Notes.COLUMN_NAME_NOTE, mOriginalContent);
                getContentResolver().update(mUri, values, null, null);
            } else if (mState == STATE_INSERT) {
                // We inserted an empty note, make sure to delete it
                deleteNote();
            }
        }
        setResult(RESULT_CANCELED);
        finish();
    }

Next week will be dedicated to fixing reported bugs and adding some small improvements. After that and some testing the new version should be ready, so keep tuned as the new version should be deployed soon!