Friday, 21-Oct-2011 12:10:55 UTC
There exists no standard for what constitutes a supported device...
I read with interest the following on Wikipedia:
Mobile Forensic Controversies
"In general there exists no standard for what constitutes a supported device in a specific product. This has led to the situation where different vendors define a supported device differently. A situation such as this makes it much harder to compare products based on vendor provided lists of supported devices. For instance a device where logical extraction using one product only produces a list of calls made by the device may be listed as supported by that vendor while another vendor can produce much more information. Furthermore different products extract different amounts of information from different devices. This leads to a very complex landscape when trying to overview the products. In general this leads to a situation where testing a product extensively before purchase is strongly recommended. It is quite common to use at least two products which complement each other."
Given this difficulty in undertaking a true comparison of tools we thought it may be useful to give some input.
It seems like every day another vendor in the mobile phone forensics industry claims to have a higher level of cell phone forensic support. We understand how confusing this must be, so we wanted to highlight some of the less transparent methodologies being practiced.
Examples of which are:
-
Claiming a Logical file system dump is some form of Physical support for a mobile device. It isn't - it's an extension of the logical support available and shouldn’t be classed differently.
-
Stating a mass of confusing numbers. E.g. “Database of over 4000 GSM, GPS and CDMA devices including 2350 supported and 880 fully validated devices.” Meaning it is very difficult to assess which devices have actually ever been tested and validated thoroughly.
-
Making assumption based support claims. E.g. “Supports more than 3500 phones, because product X enables viewing and acquisition of file systems on 80% of all CDMA handsets.” Again which ones have actually been tested by the manufacturer for true forensic verification or is it all just theoretical?
-
Counting support for the same phone more than once; e.g.
- Using slightly different methods to recover data off the same phone through a logical extraction process Nokia E71 (No Client) and Nokia E71 (Using Client) is somehow 2 phones supported?
- Counting support for the same device twice because it is both GSM & CDMA enabled e.g. HTC Ozone XV 6175 GSM/CDMA magically gets counted twice.
- Counting the same device 3 times because the same model has a different name when supplied by different network operators Samsung SCH-i500 Galaxy S (Android) / Samsung SCH-i500 Fascinate (Android) / Samsung SCH-i500 Mesmerize (Android)
-
Publishing a comparison table of rival products, ensuring that your own software details are regularly updated to reflect your product in the best possible light. Then neglecting to update any competitor information, so their performance looks significantly worse. This is a significant issue in mobile forensics because the market is moving so fast, all vendors need to update their software regularly.
Some Guidance on Mobile Forensic Tool Comparisons
Given these less than transparent approaches to marketing taken by others, we thought the following tips may help you:
1) The actual number of phones claimed as supported is no longer any indicator of the quality of the solution you are buying.
It has started to become a bit meaningless as a way of explaining support in mobile forensics. One phone is supported whether you can only pull off the contacts list or if you can extract the entire contents, security codes and perform a full physical dump and decode. It is still just one phone.
We faced the same problem in trying to demonstrate the difference and so we switched terminology away from using the term “Phones Supported” to use the term “Device Profiles” to try and help explain more clearly where the product improvements are occurring for customers. Why did we do this? Well consider this example:
If in January we offer Logical extraction support for 3,000 different mobile phones and then on our next release in March we do a lot of work enabling physical dumping of these same handsets and include 500 of the same phones with support for physical dumping extraction, then on the next release we still only support 3,000 mobile phones supported?
If for the next release we then focus on physical decoding of those same 500 phones that we just released dumping support for, then after the second release in June, we still only support 3,000 phones?
Six months research and development work and 3 releases later and the phone support count remains the same? That’s not helpful for us, or for our customers, so we decided to change the terminology to more accurately reflect the work involved and the improvements we are delivering. So that’s why we use the term “Device Profile” and not “Phones Supported” in our documentation.
2) Be very careful about what is meant by the term ‘support’ – it does not always mean tested.
Check how a solution provider is counting. They should have actual possession of the device claimed and be able to run it through an examination process obtaining repeatable results, before claiming it as supported if this is a true forensic tool. That way if there is ever a technical challenge at a later date, the company can go back to the handset in their possession and run further tests to establish what if any problems there may be.
If however the support claim is based on theory - a projection of what should be possible based on the technical specifications of a device. A device that was never actually tested or in the possession of the manufacturer - well that’s not forensic science that is theory.
It may work a lot of the time, but eventually there will be a problem and no doubt it will happen on a critical case. How then does the vendor validate disputed support claims without access to the original handset?
The truth is that doing mobile forensics properly is both very difficult and very expensive, you have to acquire an original of every device you claim support for and then thoroughly test it before claiming it as supported. Even then things can go wrong and we have plenty of experience of making mistakes and learning along the way. It is not unusual for us to even find bugs in the manufacturers own proprietary code.
We are not claiming to be perfect, I am sure MSAB could improve on some things (no doubt our competitors will be pleased to highlight them). However we do want to operate with integrity in this market for the sake of our customers.
