Mystery box reveals digital secrets

Arun Prasannan, member of CCL-Forensics’ R&D team. 

Every now and again, an unusual device arrives for analysis at CCL-Forensics, which proves interesting – but above all, significant to an investigation.

Earlier this month, a UK law enforcement agency submitted what can only be described as a ‘black box’.  It was plastic, no bigger than a packet of cigarettes, and from the outside, it had only a slot for a SIM card and a socket for power.

Working closely with the investigating agency, a member of CCL-Forensics’ R&D team carried out an in-depth analysis of what was inside the device, and what data it was capable of storing.

It was initially suspected that it was some kind of tracking device, and when disassembled, it was found to contain a battery, and two separate circuit boards, to one of which was attached a mercury switch which detected movement.  One board contained all the circuitry one would normally expect on a mobile phone, and had everything it needed to connect to a GSM network.  When examined VERY closely, it was labelled (in very small print) with an IMEI number.  From this, we could identify the board, and then research all the available documents about that piece of hardware.

Interestingly, it was a widely used GSM module found in many mobile devices such as GPS trackers, Fax machines and even some phones.

The SIM card was analysed separately, and it was strongly suspected that there was additional data on the board itself.

Our analysts procured a test module, and carried out a comprehensive technical analysis to validate what data it could store.  It was found to have the capacity to store call data (made, received, missed), SMS and contacts – as well as some call timers.  It was also determined that SMS messages could be extracted without changing their status. 

Following this comprehensive research, it was found that the suspect device DID contain a number of phone numbers and call times – which were presented back to the investigator in the case.  This was a level of potentially vital evidence which would have been missed without this very low-level investigation of the device and the data it contained.

It also highlights the talents of CCL-ForensicsR&D department, and the value investigators can derive by not simply opting for a ‘plug and play’ forensic examination.

For more information, please contact us at


Cracking Android PINs and passwords

In a previous blog post we described a method to retrieve an Android pattern lock from the raw flash of a device. However, since version 2.2 (known as “Froyo”) Android has provided the option of a more traditional numeric PIN or alphanumeric password (both are required to be 4 to 16 digits or characters in length) as an alternative security measure.

The very act of writing the last blog got us thinking whether it was possible to use a similar approach to recovering the PINs and passwords.

Our first port of call was to return to the Android source code to confirm how the data was being stored (see listing 1). Both the numeric PIN and alphanumeric passwords were found to be processed by the same methods in the same way, both arriving as a text string containing the PIN or password.

As with the pattern lock the code is sensibly not stored in the plain, instead being hashed before it is stored. The hashed data (both SHA-1 and MD5 hash this time) are stored as an ASCII string in a file named password.key which can be found in the same location on the file system as our old friend gesture.key, in the /data/system folder.

However, unlike the pattern lock, the data is salted before being stored. This makes a dictionary attack unfeasible – but if we can reliably recover the salt it would still be possible to attempt a brute force attack.

     * Generate a hash for the given password. To avoid brute force attacks, we use a salted hash.
     * Not the most secure, but it is at least a second level of protection. First level is that
     * the file is in a location only readable by the system process.
     * @param password the gesture pattern.
     * @return the hash of the pattern in a byte array.
     public byte[] passwordToHash(String password) {
        if (password == null) {
            return null;
        String algo = null;
        byte[] hashed = null;
        try {
            byte[] saltedPassword = (password + getSalt()).getBytes();
            byte[] sha1 = MessageDigest.getInstance(algo = "SHA-1").digest(saltedPassword);
            byte[] md5 = MessageDigest.getInstance(algo = "MD5").digest(saltedPassword);
            hashed = (toHex(sha1) + toHex(md5)).getBytes();
        } catch (NoSuchAlgorithmException e) {
            Log.w(TAG, "Failed to encode string because of missing algorithm: " + algo);
        return hashed;

Listing 1

Source: com/android/internal/widget/

The salt which is added to the data before hashing is a string of the hexadecimal representation of a random 64-bit integer. Necessarily, this number must then be stored, and the source code showed that the Android.Settings.Secure content provider was being used to store the value under the lockscreen.password_salt key.

On the Android file system the backing store for this content provider is found in an SQLite database settings.db in the /data/data/ directory (see fig. 1).

Fig 1: Salt as stored in the settings.db SQLite database

Once we knew how these two essential pieces of data were being stored we were able to consider how they might be recovered from a raw flash dump. In the case of the hashes, our approach was similar to the pattern lock. Knowing that:

  • The dump was broken into chunks of 2048 bytes (2032 for storing the data, the remaining 16 used for YAFFS2 file system tags)
  • The passcword.key file contains two hashes encoded as an ASCII string:  an SHA-1 hash (40 hexadecimal digits long) followed by a MD5 hash (32 hexadecimal digits long) which would make the file 72 bytes long in total starting at the top of the chunk
  • The hashes only contain the characters 0-9 and A-F
  • The remaining 1960 bytes in the data portion of the chunk will be zero bytes

Recovering the salt required a little extra thought. The salt is stored in an SQLite database which, because of the structure of SQLite data, meant that it would be all-but-impossible to predict where in the chunk the data might be stored. Worse still, we couldn’t even be sure of the length of the data as it was stored as a string. However, having a deeper understanding of the SQLite file format allowed us to derive a lightweight and reliable way to recover the salt.

Fig 2: Raw data making up the Salt field in the settings.db database

Figure 2 shows the raw data for the salt record in the settings.db. An SQLite record is made up of two parts, a record header and the record data. The record header is made up of a sequence of values; the first value gives the length of the record header in bytes (including this value), the following values (serial type codes in SQLite parlance) define the data types which follow in the record data.

In our case our serial type codes represent two data types: a null and two strings. The null is unimaginatively represented by the zero-byte highlighted by the red box (if you take a look at Figure 1 you may notice that the first field is displayed as a numeric value 34; this column is defined in the schema as the type INTEGER PRIMARY KEY which means that the value is not actually stored in the record itself, hence being replaced by null. The reasons for this are out of the scope of this blog post, but if you’re particularly interested Alex is more than happy to explain, at length, another time!).

The other two values (highlighted by yellow and green boxes respectively) define strings; in serial type codes a string is represented by a value larger than thirteen, the length of the string can be found by applying the following formula where x is the value of the serial type code:

(x – 13)/2

We can test this by considering the second field: we know that this field will always contain the string: “lockscreen.password_salt” which is 24 characters (and bytes) long. The serial type code associated with this data is the value highlighted with the yellow box: 0x3D which is 61 in decimal. Applying the formula: 61 – 13 gives us 48, divided by 2 gives us 24 which is the length of our string.

The field containing the salt is also a string, but its length can vary depending on the value held. We know from the source code that it is a 64 bit integer being stored which gives us a range of -9223372036854775808 to 9223372036854775808 which, allowing for the negative sign means that the value, stored as a string, takes between 1 and 20 characters. Reversing the formula, the second field’s serial type code must be odd and fall between 15 and 53 (0x0F and 0x35).

Using this information we can create a set of rules which should allow us to search for this record in the raw dump so that we can extract the salt:

  • Starting with the record header:
    • First field is always null (and therefore a zero byte)
    • The next field is always a string of length 24 (serial type code 0x3D)
    • The third field is always a string with length 1-20 (odd serial type codes 0x0F-0x35)
  • Moving on to the record data:
    • The first null field doesn’t feature – it’s implicit in the record header
    • The first field to actually appear will always be the string: ”lockscreen.password_salt”
    • The next field will be a string of a positive or negative integer.

This allows us to define a regular expression that should reliably capture this record:


Understanding the record structure also means that once we have captured the record we can ensure that we extract the whole salt value; we can simply read the appropriate serial type code and apply the formula to get the length of the salt’s string.

Satisfied that we could reliably extract the data we needed to recover the PINs or passcodes we crafted a couple of Python scripts – one to find and extract the data in the flash dump, and the other to brute force the hashes recovered (using the salt). On our test handset (an Xperia X10i running Android 2.3) we set a number of PINs and passcodes and found that even on a fairly modest workstation (Python’s hashing modules are gratifyingly efficient) PINs of up to 10 digits could be recovered within a few hours.

The passwords obviously have a much larger key-space so took longer, but the attack still seems feasible for shorter passwords and the script can easily be modified to only use Latin characters and digits rather than any other special characters or work from a password dictionary which could expedite the process.

Fig 3: Running the BruteForceAndroidPin script standalone

Once again we are happy to be releasing the scripts so that other practitioners can make use of this method. The scripts can be downloaded here:

Alex Caithness and Arun Prasannan of the R&D team

Unlocking Android Pattern Locks

What do you do if you have to examine an Android device which has a pattern lock enabled, but USB debugging is not initialised?

If a physical level acquisition can be performed, our technique can be used to retrieve the lock pattern from the device.

The lock pattern is entered by the user joining the points on a 3×3 grid in their chosen order. The pattern must involve a minimum of 4 points and each point can only be used once (used points can be crossed subsequently in the pattern but they will not be registered).

Each point is indexed internally by Android from 0 to 8. So the pattern in Figure 1 would be understood as “0 3 6 7 8 5 2”.

Figure 1

Storing the pattern “in the plain” would constitute a significant security flaw. Android instead stores an SHA-1 hash of the pattern interpreted as a string of bytes. In order to represent the pattern in our example, the byte string: 0x00030607080502 would be hashed to produce: 618b589aa98dfee743e7120913a0665a4a5e8317.

This behaviour can be confirmed by taking a look at the Android source code shown below:

 * Generate an SHA-1 hash for the pattern. Not the most secure, but it is
 * at least a second level of protection. First level is that the file
 * is in a location only readable by the system process.
 * @param pattern the gesture pattern.
 * @return the hash of the pattern in a byte array.
private static byte[] patternToHash(List pattern) {
    if (pattern == null) {
        return null;

    final int patternSize = pattern.size();
    byte[] res = new byte[patternSize];
    for (int i = 0; i < patternSize; i++) {
        LockPatternView.Cell cell = pattern.get(i);
        res[i] = (byte) (cell.getRow() * 3 + cell.getColumn());
    try {
        MessageDigest md = MessageDigest.getInstance("SHA-1");
        byte[] hash = md.digest(res);
        return hash;
    } catch (NoSuchAlgorithmException nsa) {
        return res;

Source: com/android/internal/widget/

The pattern lock hash is stored as a byte string in a file named gesture.key which can be found in the /data/system folder on the device’s file system.

Although storing the pattern as a hash is a sensible approach as it is a “one-way” operation from the original data to the hash, Android does not apply a salt to the original data; this, combined with the fact that there is a finite number of possible patterns, makes it fairly trivial to create a dictionary of all valid hashes. If you can gain access to the gesture.key file you can easily recover the original pattern using a dictionary attack – it would take seconds.

However; sensibly, the gesture.key file is stored in an area of the device’s file system which is not accessible in normal operation, requiring root access to the device in order to extract the file; of course if you have root access you also have access to every other file on the device as well, so recovering the pattern becomes far less useful. (

There are situations, though, where root access cannot be gained (for example, where USB debugging cannot be activated on the device) which spurred us on to investigate whether the pattern lock could be easily recovered from a raw read of the device’s storage, the likes of which could be made by using JTAG or chip-off techniques.

One possible solution would be to rebuild the entire file system from a raw dump, and from this identify the relevant file; however, time constraints ruled this out. Instead we decided to examine whether it was possible to identify the contents of the gesture.key file among the rest of the data in the dump. We set a known pattern lock on a test handset (an HTC Wildfire in this case) and acquired the contents of its flash memory by using JTAG.

Figure 3: JTAG test pads on an HTC Wildfire

Loading the dump into a hex viewer, we noted that it was organised into chunks of 2048 bytes: the first 2032 bytes are allocated for the data in files while it appears that the remaining 16 bytes are used to store metadata (tags) relating to the YAFFS2 file system.  We then searched for the SHA-1 hash which corresponds to our known lock pattern. This data was found to be stored as expected, occupying the first 20 bytes of the block; interestingly we also found that the remaining 2012 bytes were zero-bytes – an observation which was repeated in our other tests.

These observations left us with a set of rules which we believed would allow us to identify a gesture.key file in the raw dump without having to rebuild the entire file system:

  • The dump is organised into blocks of 2048 bytes each
  • The gesture.key file should contain a 20-byte long SHA-1 hash at the start of a block
  • The 20-byte sequence at the start of the block must appear in our dictionary of hashes
  • The following 2012 bytes should all be zero-bytes

Bearing these rules in mind, we wrote a simple Python script which reads a dump 2048 bytes at a time, disregarding blocks which do not fit our required pattern of 20 arbitrary bytes followed by 2012 zero-bytes. If a block does match these criteria, the script then attempts to look up the prospective hash (the first 20 bytes) in our pre-compiled dictionary which was enough to remove any false positives and recover the correct pattern.

One potential shortcoming of this technique is that in its current form it will also identify any previously set patterns which are still present in flash memory. Due to the nature of flash media and the YAFFS file system, which is still found to be used by the majority of Android devices, the gesture.key file does not get over-written in place. Rather the new version is written to a new block with a higher sequence number.

Despite this potential issue, we have used this technique a number of times since with great success and are happy to be making public the scripts for generating the dictionary and for searching the data for research purposes. creates an SQLite database file (AndriodLockScreenRainbow.sqlite) which contains a list of hashes for all possible patterns. takes one argument: the flash dump file, and expects AndriodLockScreenRainbow.sqlite to be present in the same folder.

C:\wildfire> WILDFIRE_JTAG.bin

[0, 3, 6, 7, 8, 5, 2] 618b589aa98dfee743e7120913a0665a4a5e8317

The scripts can be downloaded from our website: / We hope that researchers and analysts will find the technique useful, and we welcome any comments or suggestions.

Alex Caithness and Arun Prasannan of the R&D team

Forensic software tools – get ‘em while they’re hot, they’re lovely!

The R&D team at CCL-Forensics are a busy bunch. Over the past couple of years, they’ve developed a number of forensic software tools to examine the evidence that standard tools can’t reach.

Here’s a quick overview of what’s on offer. Follow the links to find out more, or give us a shout by phone (01789 261200) or email ( – we’re always happy to talk geek with like-minded practitioners.

epilog allows investigators to recover deleted data from SQLite databases, a widely-used format in many devices including mobile phones, computers and SatNavs). Many off-the-shelf tools will only allow you to view live records.

PIP is our XML and plist parsing tool. It allows investigators to present often-complex data from XML files quickly, efficiently, and in a user-friendly format. Apple’s property list files – both XML and binary formats – present no obstacle to PIP at all.

dunk! is a splendidly-named tool for digging around in cookies. Unlike standard tools, it analyses known cookie types to uncover potential new evidence and help give context to other browser artefacts. This includes showing the path the user took to arrive at a particular webpage by parsing Google Analytics cookies, revealing a wealth of information previously unavailable to practitioners.

rubus  is FREE! We like to give a little love back to the community, so with this in mind, we made our BlackBerry backup deconstruction tool available. Not having found a tool that would do the job, we made our own – enabling analysts to reverse engineer BlackBerry backup data stored in .ipd files.

The tools all went through beta-testing first, and were pronounced ready to unleash upon the world. Since then, they’ve been subject to an introductory pricing period, and have been bought and used successfully around the world.

Now that we’re confident in the tools we’ve developed, we’re also confident in their value to our customers. So with that in mind, if you haven’t bought the tools already, you may want to think about doing so! The introductory pricing period finishes at the end of March – and although they’ll still be extremely good value for money, they will be a little more expensive.

We’ve had useful feedback from our customers in the past, which has helped us to further develop our tools, and we always welcome comments and suggestions on our software. Feel free to comment below, or get in touch with us in more traditional ways!