Posted: Feb 09, 2010 by Steve KradelTagged software, ldap
Today I'm very excited to announce Zetetic Connect, a fast, super-flexible way to get information out of (and into) LDAP directories and relational databases databases for the web, iPhone, BlackBerry, and Android.
Connect grew out of a client's need to publish Active Directory data to the web; so many businesses and schools rely on directory systems, but there just aren't a lot of mature applications that help people take full advantage of those directories or to spot and correct missing data. Connect does that and, going two steps farther, hosts an extremely robust, high-performance engine that can unify multiple directories and data stores behind the scenes, and allows users from any of those directories to sign in.
For example, if part of your company uses Microsoft Exchange, another division uses Lotus Domino, and you've just acquired a third group that uses Zimbra for messaging, Zetetic Connect can quickly create an online phonebook merging all three. Just launch Connect, search for "Smith", and you'll get the contact and organizational details for all the Smiths across all systems, live. The same powerful simplicity applies to contacts, groups and distribution lists, sites and locations, and virtually any other data in your directories. (Of course, you can choose what to include or exclude from each.) Users can easily maintain their own data, such as phone and mobile numbers, according to rules you define.
Interested? »
We're currently running a small number of commercial evaluations for interested customers, including system deployment by Zetetic's LDAP-meisters. Contact us now if you'd like a demonstration and evaluation of the system, since availability is limited.
Your mobile users will be impressed by Connect's user interface, which is tailored especially for the iPhone, BlackBerry, and Android, and has the look and feel of a native application. (See: screenshots.) Connect is Unicode and internationalization-friendly too, so your users get Connect in their native languages, top-to-bottom.
Behind the scenes, we've leveraged many years of directory experience to build an extremely robust framework that actively monitors your directory servers' performance and availability, so users won't notice if individual servers go out of service. Connect also does some pretty serious and smart caching and threading, and features a wealth of special optimizations for Active Directory. That said, it works great with any LDAP server, SSL on or off, and any schema.
You can even use the Zetetic Connect core as a pure-LDAP, federation-capable membership and role provider for any ASP.NET application, a topic upon which I'll touch in a future blog post.
I’m currently putting together a quick set of extensions to the Javascript Date object, inspired by this article over here by Lawrence O’Sullivan. Mostly, I just want to be able to conveniently pull up relational dates (like the beginning of date’s week), similarly to the Rails date helper methods:
now = new Date();
alert(now.beginning_of_week());
alert(now.end_of_month());
Back before I realized how powerful Rails date helper methods were, I had put together some of my own calculations for Tempo, and then subsequently ditched them. But I wanted to pull them back up for this since I needed to roll it myself, and I’m quite happy those old SVN commits are still sitting in my Gmail:
Publicly exposing exploits on Twitter does not make you an 31337 security consultant. It just makes you a weener.
While our TOS clearly prohibits this sort of activity, I generally don’t mind at all when people poke around. I used to read 2600, I don’t believe in giving people a hard time for figuring out how things work. But I like to think that somewhere along the way we grow up or graduate and we don’t just go throwing exploits out into the wild. You let the affected service know about it so they can patch it before you go publishing your leetness to the world.
Turns out we missed one bit of user input that should have been sanitized, and have since patched it, and then we did a full review of all our Rails views to make sure there aren’t any others we missed. We’ve been pretty religious about it. It doesn’t look like any of our users were adversely affected, although there was about a 24-hour span between the time when our hero published the vulnerability and when I saw it on Twitter.
Babyshaker can’t really be within the realm of those super-strict Human Interface Guidelines that can get your app rejected from the App Store, can it?
John Gruber is often complaining about Michael Arrington because established pundits (and DF author Gruber is established in the blogosphere) don’t like it when competitors pee in their sandbox. Thus the need to slag the guy at every opportunity. I really love to disagree with Gruber, so I haven’t quit reading the guy yet, but more and more I find myself thinking, “wow, you’re such a jerk!” Not that I have an affinity for Arrington, but the constant Arrington-hating (and the constant general hating from Gruber) is getting really droll.
One wonders if these guys ever get worked up about real life issues, if they ever bother to pick up the phone and call their Congressional representative?
Mobile devices are notoriously difficult to secure. With a small footprint it is dangerously easy to leave your phone in the back of a taxi, or forget to pick it up from a table after dinner at a restaurant. iPhone and iPod Touch devices are no exception – most applications store their data in an easily readable format. Anyone with access to your device can browse through your personal information. Even worse, since the devices back up to your computer, any knowledgeable person with access to your workstation can read your data as well.
This is balanced by immense convenience. An iPhone makes a perfect digital wallet: a single device with all your important information literally at your fingertips. When we set out to write Strip, our iPhone Password Manager and Data Vault, we wanted to build a system that we could trust to store Passwords, Credit Card Numbers, Passport Identifiers, SSNs and bank account information. Most of the other applications in the App store used undocumented encryption or weaker field level security. We wanted to encrypt EVERY bit of Strip’s data on the device. Naturally the ideal approach would be to have the encryption managed transparently, so Strip didn’t need to know about the details.
Enter SQLCipher, a specialized build of the excellent SQLite database that performs transparent and on-the-fly encryption. Using SQLCipher, an application uses the standard SQLite API to manipulate tables using SQL. Behind the scenes the library silently manages the security aspects, making sure that data pages are encrypted and decrypted as they are written to and read from storage. SQLCipher uses the widely trusted and peer-reviewed OpenSSL library for all cryptographic functions.
SQLite is already the predominant API for persistent data storage on the iPhone so the upside for development is obvious. As a programmer you work with a stable and well documented API. All security concerns are cleanly decoupled from application code and managed by the underlying framework. The framework code of the SQLCipher and OpenSSL projects are both open source, so users can be confident that an application isn’t using insecure or proprietary security code.
Using SQLCipher in an iPhone application is straightforward once you setup your project properly. The easiest way to integrate SQLCipher into an application is to use XCode project references as described here.
Get OpenSSL
SQLCipher relies on OpenSSL for several encryption requirements including the AES-256 algorithm, pseudo random number generation, and PBKDF2 key derivation. OpenSSL isn’t a framework that is usable directly on the iPhone so we will setup our project to build and link against it as a static library.
Start by going over to the OpenSSL download page to a recent stable version (0.9.8k as of this writing). Download the source file and extract it to a folder on your system. Make a note of the source directory path for later.
OpenSSL can be a tricky system to compile properly from source. It’s even more complex when you’re dealing with multiple target architectures, targeting i386 for the simulator but armv6 for a device. Luckily we’ve built a handy XCode project template to make it easy called openssl-xcode. The project actually relies on the OpenSSL configure and make system to build the libraries. However, it automatically detects the appropriate build settings for architecture (i386, ppc, arv6), build tools, and SDK. This makes it ideal for inclusion in an iPhone project. Just git clone or download openssl-xcode from GitHub and move the openssl.xcodeproj file into the OpenSSL source directory.
Get SQLCipher
SQLCipher is a standalone package that includes the entire SQLite source distribution. You can git clone or download the latest version from the SQLCipher repository on GitHub. Again, make note of the source directory path you use for later in the setup process.
XCode Setup
SQLCipher uses project references to manage build dependencies and static linking. In order to allow linking you must setup your XCode environment to use a central build location. We will also need to configure Source Trees OpenSSL and SQLCipher respectively.
Open XCode, Choose the XCode Menu, and then Preferences.
On the Building tab, Change the “Place Build Products In” to “Customized location” and pick a location for the build directory.
Switch to the Source Tree tab.
Add a SQLCIPHER_SRC that references the path to the SQLCipher sourcecode
Add a OPENSSL_SRC to reference the path to the OpenSSL sourcecode
Project Setup
Now that we have XCode global preferences setup we will move on to the project configuration.
Open your XCode application project, and click on the top level Project item. Hit option-command-a to add a resource. Navigate to the OpenSSL directory and choose the openssl.xcodeproj to add the reference. When the add resource window comes up make sure that “Copy items” is not checked, and add it to the appropriate target. Repeat the same, This time choose the sqlcipher.xcodeproj file in SQLCipher source directory.
Let’s change the Path Types as a convenience to multi-developer teams. Using paths relative to the Source Trees we defined will allow each developer to place the SQLCipher and OpenSSL in any locations and the build will still work.
Hit option-i on the sqlcipher.xcodeproj reference and change the Path Type to “Relative to SQLCIPHER_SRC”.
Hit option-i on the openssl.xcodeproj reference and change the Path Type to “Relative to OPENSSL_SRC”
Hit option-command-a to add another resource
Now we will add build dependencies to ensure that the SQLCipher is compiled before the application code. Open the Info panel for your Application Target and Choose the General Tab. We will add two dependencies, one for OpenSSL crypto, and one for SQLCipher.
Then switch to the Build tab to add our source paths to the include directories to make relevant header files available. Make sure that the Configuration is set to “All Configurations”. Look for the “Header Search Paths” setting and add references to $(SQLCIPHER_SRC) and $(OPENSSL_SRC). Check “recursive” on both.
Finally, we will tell XCode to link against the built libraries. Expand the sqlcipher and openssl .xcodeproj references and select libsqlcipher.a and libcrypto.a. Drag and drop them on “Link Binary With Libraries” under your application target.
Building SQLCipher
At this point you should be able to build your XCode project without errors. Note that the first time you build your application for a particular architecture (Simulator for instance), it will take much longer than usual. This is because SQLCipher and OpenSSL are compiled from source for the specific architecture. You can keep track of the status under Build Results. Subsequent builds for the same platform will be much quicker since the libraries don’t need to be recompiled.
In Code
Now that you’ve incorporated the SQLCipher library into your project you can start using the system immediately. Telling SQLCipher to encrypt a database is as easy as opening a database and using “PRAGMA key” or the sqlite3_key function.
#import <sqlite3.h>
...
sqlite3 *db;
if (sqlite3_open(@"/path/to/database", &db) == SQLITE_OK) {
sqlite3_exec(db, "PRAGMA key = 'BIGsecret', NULL, NULL, NULL);
if (sqlite3_exec(db, (const char*) "SELECT count(*) FROM sqlite_master;", NULL, NULL, NULL) == SQLITE_OK) {
// password is correct, or, database has been initialized
} else {
// incorrect password!
}
}
The call to sqlite3_key or PRAGMA key should occur as the first operation after opening the database. In most cases SQLCipher uses PBKDF2, a salted and iterated key derivation function, to obtain the encryption key. Alternately, can tell SQLCipher to use a specific binary key in blob notation (note that you must provide exactly 256 bits of key material):
Once the key is set SQLCipher will automatically encrypt all data in the database! If you don’t set a key then SQLCipher will operate identically to a standard SQLite database.
Verify
After your application is wired up to use SQLCipher you should take a quick peek at the resulting data files to make sure everything is in order. An ordinary SQLite database will look something like the following under hexdump. Note that the file type, schema, and data are clearly readable.
Fire up your SQLCipher application in simulator you can look for your database files under /Users/<username>/Library/Application Support/iPhone Simulator/User/Applications/<HEX app id>/Documents. Try running hexdump on your application database. With SQLCipher you should get output that looks completely random, with no discerning characteristics at all:
new-host-2:Documents sjlombardo$ hexdump -C encrypted.db
00000000 84 d1 36 18 eb b5 82 90 c4 70 0d ee 43 cb 61 87 |.?6.?..?p.?C?a.|
00000010 91 42 3c cd 55 24 ab c6 c4 1d c6 67 b4 e3 96 bb |.B<?U$???.?g??.?|
00000020 8e df fa bc c3 9c 92 8a 4e 40 59 6f b5 95 f3 80 |.????...N@Yo?.?.|
...
00000bd0 91 16 9e 89 d9 4e ac f7 1c c9 d1 d7 aa bb a7 51 |....?N??.??ת??Q|
00000be0 dc 77 5c 6c de c6 d3 be 43 49 48 3e f3 02 94 a9 |?w\l??ӾCIH>?..?|
00000bf0 8e 99 ee 28 23 43 ab a4 97 cd 63 42 8a 8e 7c c6 |..?(#C??.?cB..|?|
00000c00
Conclusion
SQLCipher is an easy way to incorporate full database encryption into an iPhone application. For more information on SQLCipher and the underlying security features please check out the SQLCipher project site. To see it in action please try Strip and Strip Lite in the iTunes App Store!
Apple enforces a number of restrictions on applications in the App Store. Among the most painful is the lack of feature-limited trials. Applications are either completely free, or the customer must pay up front, sight unseen. The proliferation of “Lite” applications is a direct result of this shortcoming. Publishers will often create two application versions: the first is fully functional but costs money, the second is “Lite” and comes with limited features but zero price tag. The goal is for prospective customers to try the free version first and then decide to buy the paid full version separately if they like it.
Because free applications have greater exposure this approach is widely advocated as a means to increase visibility and grow sales. When building a game or other stateless application the approach makes complete sense. However, utility applications often maintain information entered by the device owner. Application authors are faced with a dilemma because the iPhone’s security sandbox prevents one application from reading another application’s files. Thus, when customers upgrade from the Lite application they are penalized by having to re-enter all data!
When we developed our data vault product, Strip, we felt this situation was unacceptable. We wanted to offer Strip Lite, free for people to try, limited to 10 entries. Upgrading customers should be able to import their data from the Lite version. No pain.
The Approach
Strip stores user data in an encrypted SQLite Database. Since we really wanted a way to copy the data file from Strip Lite to Strip we developed a fairly clever solution: use a registered URL handler as a means of communication between application versions. This is how it works:
A customer downloads Strip Lite for free and enters some unique private data. All information is written to a database in the application’s document directory.
When the 10 record limit is hit Strip Lite asks them to upgrade. An upgrade button launches to explain the process and allow the user to download the paid version of the product.
Once Strip is purchased and installed, our customer will launch Strip Lite and click an “Export to Strip” button. Strip Lite reads the application database into an NSData object, Base64 encodes it and then Launches a “strip://” URL with the encoded file contents as a URL parameter.
The full version of Strip is registered as a handler for “strip://” URLs. The OS launches Strip and passed the URL to the handleOpenURL method of the application delegate. It parses and decodes the data and writes the database to the desired location in the new applications Document folder.
Our customer now has access to all the data the originally entered into Strip Lite!
Project Setup
This tutorial assumes that your project is already configured with two targets, one for your full version, and one for the Lite.
Start by creating a new InfoLite.plist file for your Lite version. We use 2 separate .plist files to ensure that the Full version is the only registered URL handler (we don’t want the Lite version responding to upgrade events itself!). It is easiest to just make a copy of the existing Info.plist file named InfoLite.plist.
Next, Open up the Build preferences for the Lite target. Change the Info.plist File property to read InfoLite.plist. This will ensure that the Lite target includes the proper file.
Now open the Info.plist (used by the Full Version) and register a custom URL handler. This is a straightforward process of adding elements as shown in the following screenshot.
Please note that the URL identifier and URL scheme must be unique for your application. Mobile Orchard has a great stepwise overview of this process for reference.
Finally, the application must include code to perform Base64 encoding itself. This functionality is, unfortunately, not directly accessible in the framework SDK. There are a few options in Cocoa, including using OpenSSL’s libcrypto, but the easiest method is to use the encoding libraries from Google Toolbox for Mac which is conveniently distributed under the Open Source Apache License. Download GTMDefines.h, GTMBase64.h and GTMBase64.m, then add them to your project. The GTM code compiles perfectly on the iPhone and even includes web safe variants that are immediately suitable for use in URLs without percent encoding.
The Code
With the project setup complete the Paid version will respond to the custom URL launch! This will facilitate an export from the Lite version to be received by the Full version.
The Lite Version
The Lite application should be extended to add a button to trigger the export on an about, settings, or upgrade page.
The controller action code will read the data file, encode it and then format a URL with the protocol prefix / URL scheme configured in the previous step.
The final step is to make your application handle the Open URL event and reverse the encoding process.
Add a handleOpenURL method to the AppDelegate implementation as follows. This will receive the URL, sanity check it, and then parse the query string directly. The resulting binary data is written to the file in the full application’s data directory.
#import "GTMBase64.h"
...
- (BOOL)application:(UIApplication *)application handleOpenURL:(NSURL *)url {
if([@"/importDatabase" isEqual:[url path]]) {
NSString *query = [url query];
NSString *importUrlData = [GTMBase64 webSafeDecodeString:query];
// NOTE: In practice you will want to prompt the user to confirm before you overwrite their files!
[importUrlData writeToFile:@"/path/to/FullApplication/Documents/file" atomically:YES];
return YES;
}
return NO;
}
End Game
We have transferred files of up to 100k this way, and the size of a URL string is theoretically only limited by available memory. That said, this approach is most suitable for applications with small to medium size data transfer requirements. Best of all though, it can be easily used for SQLite databases, XML files, text files, or even extended to handled compressed archives.
Strip has seen steady growth since it’s release as people upgrade from Strip Lite to the full version. Based on download reports and direct comments from customers we believe that much of this is directly attributable to the simple upgrade path. Until (and if at all) Apple begins to allow in-app purchases from within free applications this is a simple and painless way to migrate data between Lite and Paid application versions.
The experience is absolutely not as good as the iPhone, but I guess at the end of the day, it’s still much more usable than the Android, and for me, it’s worth supporting an alternative because well, despite Schiller’s outreach, the the App Store really is abominable, not just in its practice/actuality, but also, after having given it some thought, and reflecting on its implications, as a general model.
Mobile devices are the next generation general computing/network access platform and having a device manufacturer as a post-facto gatekeeper is just not right. Getting rid of end-to-end not only reverses the freedoms that spawned the innovation on the Internet, but also creates a bottleneck on software development/distribution that I’ve never seen in modern general computing…
While I share his sentiments, I’m not quite ready to jump into development for the Palm Pre.
Having an existent base of robust client work is awesome. So is having a revenue-generating web service! And so is building new apps for the iPhone! But all this takes time, and we’ve decided it was time to add another person to the team.
Bret Morgan, CEO of DBL Systems and creator of BandsOnABudget has come on board to be an extra pair of hands, helping us to maintain the current version of Tempo and build out the next revision, which includes a complete UI re-design from "nGenWorks"http://ngenworks.com!
Products highlight
Tempo
Simple Time Tracking, Mobile Access, Powerful Reporting
Connect grew out of a client's need to publish Active Directory data to the web; so many businesses and schools rely on directory systems, but there just aren't a lot of mature applications that help people take full advantage of those directories or to spot and correct missing data. Connect does that and, going two steps farther, hosts an extremely robust, high-performance engine that can unify multiple directories and data stores behind the scenes, and allows users from any of those directories to sign in.
