I recently wrote some major updates to OpenLAPI. The update allowed our OpenLAPI to be shipped with a J2ME application, wrapping the Location API (JSR-179) if it is available on the device, otherwise falling back to GPS over Bluetooth mode, or a variety of backends (e.g. CellID lookup).
I was going to blog about the technical challenges that I encountered in doing this, but the changes were rolled back and will never see the light of day. Instead I’m going to tell you all about how stupid the security model is on J2ME and how the Unified Testing Initiative is killing J2ME. I’ll also explain why it means libraries like OpenLAPI will always have to be device specific.
How MIDlet signing used to be
In the beginning of J2ME, it was decided that applications should be signed by a trusted authority or the user would be prompted to grant access to sensitive APIs. Practically, this meant that J2ME developers had to buy a signing certificate from somebody like Verisign or Thawte, costing about $500 for an unlimited number of signatures per year. Or you could upload your own self-made certificates to your handset for development purposes. David Hayes wrote an excellent tutorial on the subject.
The benefit was… nothing… actually. The user is still prompted to allow the application to access the network and instead of the user getting a confusing technical note during installation about “this application is not signed”, they get a confusing technical note saying “this application has been signed by XYZ”. So nobody signed their MIDlets.
Then along came the JSRs
MIDP2/CLDC1.1 was an upgrade to the initial J2ME offering, adding a bunch of APIs that everybody was craving (mostly for games). They did this really well… the APIs were developed with input from the developers and I am especially impressed by how nice javax.microedition.lcdui.game.* is.
Having laid down the foundations of J2ME, standards for optional libraries were created to allow support for device hardware such as sms and cbs (JSR-120), bluetooth (JSR-82), file system Access (JSR-75), cameraphone and speakers (JSR-135), webservices (JSR-172) and location awareness (JSR-179).
This was a great idea! However most of these new APIs require a trusted signature on the application. So, forking out for a Verisign or Thawte signature made it impossible to write hobby projects. This was the death of open source J2ME beyond gaming.
Then along came the clowns (network operators)
When people started having to sign their applications, they realised that Versign and Thawte and Geotrust and their friends were not supported on the devices in circulation! The reason was simple… J2ME never specified a list of default certificates that should be supported.
Handset manufacturers would ship with seemingly random collections of certificates meaning that developers had to get their key signed by every authority they could find. This pushed the entrance price up to a few thousand dollars a year.
But it got worse… the network operators were “customising” the certificate stores on handset requiring everything to be signed by them as well as the handset manufacturer! And the handset manufacturers stopped allowing developers to upload their own certificates in order to run their own code on their own device.
The Unified Testing Initiative
Seeing a problem, the J2ME working groups decided that they needed a single place where applications could be verified as non-malicious and signed with a certificate that would be guaranteed to exist at runtime. Wonderful news! Until somebody seen that they could make a fortune by bleeding the developers dry.
The central repository is the Java Verified Program. It lets you upload your jar/jad file and then it does some automated tests (which are b0rk3n… more on this later). It then lets you decide which devices you wish to test on before letting you select who to send your application to.
Yes, it needs to be manually checked! And only by commercial entities that struck an insanely lucrative deal. To give some examples… it costs about $300 per test, per device! That means if you want to write a J2ME application that uses a “restricted API” on all supporting devices, you could potentially end up spending $30,000 per bug fix! It’s not unrealistic to spend half a million dollars per application. It gets worse… some operators may also require you to dual sign your applications with them, so this may well only be half the cost of the signing process.
I noted that the automated process is broken. This is the reason why a single OpenLAPI jar cannot be part of a UTI-signed application. The whole point of OpenLAPI is to detect JSR-179 and use it if it is available, otherwise using alternatives. It does this using a combination of reflection and wrappers which know when a cast is valid. The UTI process detects that javax.microedition.location.* is used, and plain refuses to allow the tests on anything other than high-end Nokia phones. Defeating the entire purpose! This is why the next release of OpenLAPI will not be as amazing as it could have been.
Conclusion
I really do not see any reason why the signing process has to be bundled with a testing process. And given that J2ME is entirely sandboxed anyway, I don’t see why a human needs to check the application for malicious behaviour.
The decision to restrict JSRs to signed MIDlets was possibly a wise one (identity management through SSL certificates is A Good Thing™). However, it killed open source and hobby J2ME development.
The decision to standardise the certificates on a device was a logical and common-sense answer to a problem that had plagued everybody.
The decision to unify the certification and testing, relying on third party developers in a lucrative position to charge whatever they wanted, whenever they wanted or your application doesn’t hit the shelf is possibly the stupidist thing I have ever heard!
What should have happened is that the J2ME standards committee should have given out a list of standard certificates that must be on a device. They should also have suggested that handset manufacturers allow the uploading of self-signed certificates so that open source and hobby programmers could write programs for their own handsets.
What has now happened is that nobody can afford the prices of testing and signing anymore, so developers stick to creating games. Innovation has been completely killed and if anybody really wants access to the hardware, they either write it entirely in Symbian or use a Symbian component to speak to Java. My understanding is that the Symbian signing process still allows Verisign and self-signed certificates.
So that is why MIDlet Signing is killing J2ME. Long live the iPhone! Let’s hope Apple give all these operators and manufactures a good kick up the butt, just so we can feel some form of vengeance for their incompetences.
Saves me the trouble of writing a similar post, I gave up on code signing when I first tried it 3 years ago as it was immediately clear that it was a nice idea that had been destroyed by the greed of operators and some manufacturers (Moto are bad, others are better).
Anders Borg wrote:
August 10th, 2007 at 9:10 am