Android has gone through quite a few changes during its short 6 years of life. The Android that drives most of the world's smartphones of today would be almost unrecognizable to what was launched in late 2008. We've seen massive visual changes, expansion to almost every conceivable form factor, and a completely fleshed-out content ecosystem for multimedia and apps. As the operating system matured, some elements have successfully grown with it, and others have become dead weight. Naturally, progress calls for the replacement of those pieces that haven't scaled well. We've seen an excellent example of this when ART came to replace Dalvik as the standard Android runtime. With the release of Lollipop, a similar project emerged that promises to replace a part of the existing app development toolchain with a pair of new compilers called Jack and Jill.

Presumably named after the nursery rhyme characters, Jack and Jill will ultimately replace the javac and dex compilers used by most app developers today. Google hasn't published its plans for the new project, but it appears to be a big step towards cutting down on compile times, but it may also be angled towards escaping some of the politics surrounding Java and massively improving the entire development experience.

How It Works

In an article on Saikoa's blog (this is the developer of ProGuard and DexGuard), Eric Lafortune explains the roles Jack and Jill will play in the toolchain and the inputs and outputs of each compiler. The current method is effectively a 2-step process that begins with javac converting source code into standard Java bytecode, then the dex compiler is responsible for turning that into Dalvik bytecode so it can be run on Android. If there are additional pre-compiled libraries, they go straight into the dex compiler. In between these steps, tools like ProGuard may also be used to optimize and obfuscate the output.

Image Credit: Saikoa

The new process is similar, but there are some important distinctions. To begin with, the Jack compiler converts raw code directly to .dex format, removing Java bytecode from the process entirely. In the event that a Java bytecode library is included, the Jill compiler is called to convert it to an intermediate format called .jayce .jack that will be passed to the Jack compiler along with the source code for the app. For the most part, Jill will only have to be called once when a library is added or updated, since the output can be cached and reused for future compilations. Additionally, ProGuard support has been baked directly into the Jack compiler, meaning it can operate with the same configuration files and produce the same output without requiring a separate step. There is still room for external tools to step in, but they will have to target the jayce jack or dex formats instead of Java bytecode.

Image Credit: Saikoa

Where Things Stand

Google hasn't made an official statement or published any documentation about the new project, so we're left with a few mysteries about what it's meant to achieve and how far along it is. The only place we have to look is AOSP where repositories for Jack and Jill have been open sourced. The first code submission for Jill was made on Feb 6, 2014, and Jack followed about 6 weeks later on Mar 19, 2014, but development obviously began much earlier. So far, the project could probably be considered experimental, particularly since the repositories are still organized so that the code resides exclusively in development branches, rather than the master branches where it will eventually end up.

If you're interested in trying out the Jack and Jill compilers, they are already available in version 21.1 of the Android SDK, and the Gradle plugin (as of v0.14.0) supports a new flag called useJack. It's as simple as adding a single line to the build script for your app (not the project's build script). Just add useJack true to the defaultConfig section and recompile. I'm still getting about the same performance (or worse) with Jack than I was with the javac/dex recipe, but that will probably change in the future.

Since the compiler is distributed with the SDK rather than versions of Android, the updates will roll out on a fairly steady schedule. It's impossible to guess when we'll see Jack and Jill replace javac and dex as the default compilers; but they already seem mostly stable, so we see the switch before too long.

What Is There To Gain

Google's reasons for creating two new compilers are a bit mysterious, but there are a few plausible explanations. The first and most obvious is to combat the slow compile times developers often complain about. Efforts have been made to optimize the current process, but there are only so many things that can be done. Ultimately, almost every time an app is compiled most of the code will go through two fairly complicated procedures before the final output is ready. By removing the conversion to Java bytecode, at least one largely unnecessary step can be eliminated.

Bigger advantages may also come from total ownership over the toolchain. The use of javac means Android developers are essentially reliant on implementations controlled by other groups like Oracle, Eclipse, or the OpenJDK project. Google is free to fork any of these, but at a certain point the design, architecture, and even politics will always stand in the way of rapid iteration. With a brand new compiler focused directly at Android, it's possible to add features, improve output, and directly manage the outcome without worrying about how each change affects the larger ecosystem built around Java. There are certainly increased costs associated with taking over development of the compiler, but it may be worth it in the long run.

Another potentially significant piece of this puzzle may relate to the ongoing feud with Oracle. Despite a few strong legal wins for Google, the decision to use Java as a primary development language has led the two giants down a road of lawsuits that almost rivals the inane bickering between Apple and Samsung. Complaints have been brought up regarding patents, misuse, and even if method names can be copyrighted. (Eyeroll.) A pair of new compilers certainly won't be enough to end the territorial squabbling, but removing Java bytecode almost entirely from the equation would be a good start.

While this may be wishful thinking, Google may have more ambitious goals lined up for this project. One of the most distinct additions to Jack is ANTLR v3.4, a very powerful tool for generating language parsers. This may prove to be the first step towards language-level customizations, support for additional languages, or very advanced language inlining. For example, it may soon be possible write raw SQLite queries in the middle of code, or write an app in C# or Python without the use of special libraries (assuming the runtime supports all of the necessary features).

Conclusion

Much like ART, Google is undertaking a project that could have lasting effects on the future of Android. The potential may be much greater than is immediately obvious. Jack and Jill may turn out to be just a way to reduce compile time for developers and put some distance between Google and Oracle, but as the project matures, it might just lead to an evolution in the way developers build the apps we all rely on.

Update [Dec 11]: Google just released a post about Jack and Jill. We now know the intermediate format is called .jack instead of .jayce. and we also see that the acronyms for the two compilers are Java Android Compiler Kit (JACK) and Jack Intermediate Library Linker (JILL).

Thanks, Eric Lafortune

Source: Saikoa blog