Since my iPhone 3GS died, I have been using my dad’s Samsung Jet as a temporary replacement phone. I really can’t stand the resistive touch screen – tapping backspace will at times hit the T9 button when I’m composing an SMS. Also, I miss the display of SMSes as a conversation with both sent and received messages in a single place.
I obsess over keeping chat history, so naturally I want to find a way to preserve these messages on the phone. I keep hitting the maximum limit of about 200 odd SMSes and I have to delete old messages. The first step would be to download the SMSes onto my PC, using the Samsung New PC Studio (NPS) software.
The Nokia PC Suite used to save the SMSes in some CSV-like format, so I wonder if the Samsung NPS does the same thing. Peering into the AppData directory, I found a couple of files and (empty) folders. I inspected the
.dat files, and I found that they were of the type “Standard Jet DB”. To open them, I renamed them to
.mdb and opened them in MS Access. One file had password protection, and the other did not. Using this other database, it seems that the sync-ed SMSes were stored in the
MESSAGE table. Great!
So I began looking for a way to access this database. Since I’m using 64-bit Windows, I was having difficulty using the Windows Scripting Host (WSH) and the ActiveX objects, but JetSQLConsole worked fine. JetSQLConsole was written in C/C++ and it relied on QT, so recompiling it for my purposes is going to be a little bit troublesome. In its current state, it provides an SQL console to the database, but I would prefer programmatic access to the database, so eventually I settled for Python + pywin32 extensions.
iPhone SMS Database
The iPhone (or iOS, rather) uses an SQLite database to store the SMSes. In this database, there are a couple of tables, indices and triggers. The tables we are interested in is the
message table, the
msg_group table for grouping the SMSes into different conversations, and a mapping table called
Accessing this database will not be easy (or should I say convenient) if you do not jailbreak your iPhone. There are two methods for accessing this database:
- jailbreak the iPhone, then use something like iPhone Explorer to extract the SMS database
- sync to backup the iPhone, then locate this database in the backup to copy it
Since we are importing SMSes into the iPhone, the database will need to be copied back into the phone. If you choose to use method #2, you will need to perform a restore to get the updated database back into the phone.
Putting it Together
Armed with the knowledge of both databases and their schema, all that was left is to create a converter between the two. The typical use case would be:
- connect the Samsung Jet to download SMSes from the phone into NPS
- copy the iPhone SMS database (
/var/mobile/Library/SMS/sms.db) to the script directory
- run the script to import the Samsung SMSes into the iPhone SMS database
- copy the updated iPhone SMS database back into the phone
Since I will be running it frequently, I did not want to have to worry about importing duplicate SMSes. I made the script check for duplicate SMSes using the SMS text, the timestamp and the number and direction (sent or received) it came from. With most older phones, sending a blank SMS was allowed; the script will ignore blank messages.
As I did not have iTunes on the Windows machine, I had to use my Mac, copy the database to the Mac, then to the PC, run the script, then copy it back to the Mac, and then onto the iPhone. To streamline this process, I decided to install iPhone Explorer and only the necessary dependencies, instead of everything that comes with iTunes. You will need only the AppleMobileDeviceSupport(64) and the AppleApplicationSupport packages.
The script uses the Python port of libphonenumber, which parses phone numbers. This is needed because my telco prefixes the country code even if the SMS is local, and obviously my address book doesn’t have those, so I needed a way to match these. The databases is filled with both versions of these numbers, depending on whether the SMS was outgoing or incoming, and sometimes the other party’s telco.
I’ve made the script available on Bitbucket. You can use the “get source” link to grab a copy of it, or alternatively, by using Mercurial.
Update: I gotten my iPhone 4S since Nov 2011 and successfully managed to restore the SMS database from my iPhone 3GS backup.
The iPhone 3GS was running iOS 4.3.2. I ran my SMS import script every few days to update the iPhone’s SMS database, and I still connected my iPhone 3GS to iTunes to perform regular backups. When I got my new 4S, which was running iOS 5.0, I simply “restored” the latest backup to the new phone. All my SMS history (except those special cases I’ve mentioned) was on the new phone, as if I had been using an iPhone all the while.
As such, I will not be updating my script to work with iOS versions greater than 4.3.x. In iOS 5 the SQLite database schema would most probably have changed to accommodate iMessage. However, I would be happy to accept patches from someone who has implemented the functionality.