"Because the history of computing has taught us is that data will not be contained. Data breaks free. It expands to new media, crashes through barriers; painfully, maybe even dangerously. But, uh, there it is… Data finds a way." - Jeff Goldblum as Jeff Goldblum (Jurassic Gift Shop)
When we last left our favorite removable storage device, OEMs had begun adopting Google’s policy for restricting write access to SD cards. Given the nature of the Android community, it was probably safe to assume the story wouldn’t simply end with some people rooting to re-enable classic file system access and the rest passively accepting that the SD slot was just for decoration.
File managers and media players have been trickling out with changelogs claiming that they’ve added support for SD cards on unrooted KitKat devices. While a few of these have turned out to be a bit inaccurate, it became clear that some developers had actually cracked the code. After some digging, we came across a post from late 2012 on the Total Commander forums that sheds quite a bit of light on the situation. To explain how this works, we have to cover a little bit of background.
The Media Scanner
Long-time Android fans may remember the Media Scanner for being a fairly obnoxious, battery-gobbling service that could run amok at the drop of a hat. It’s still around, but these days it’s behaving much better. The purpose of the Media Scanner is to look through primary and secondary storage for audio, video, and image files which are then recorded in a database so other apps don’t have to repeatedly do their own scans.
While the Media Scanner may be oriented towards conventional multimedia, it still keeps track of every other file, as well. This is done to support a few other features, particularly MTP mode (used for file transfers over a USB cable) which pulls the file structure from the database instead of directly from the filesystem.
The Media Scanner normally does a full scan just after a reboot and anytime an SD card is mounted. Applications can also initiate a scan of an individual file by using the MediaScannerConnection class. This would generally be done by an app after it has downloaded something from the Internet, taken a picture, or recorded a video.
When apps want to get at this information, they will call out to the Media Store.
The Media Store
The secret to modifying files on the SD card is in the MediaStore class. It's a Content Provider with access to the database built by the Media Scanner. This is mostly useful to apps like the gallery and music players, but anything that wants to see locally stored images, audio, or video will probably use it.
We’re really interested in a child class called MediaStore.Files. Its job is to give links (URIs) to apps so they can query the database for files on a specific storage device. Through this mechanism, developers can get a specially crafted link that can be used to open a file with write access.
It seems that this method works because the links actually call on the Media Store to open and edit files. Since the Media Store runs with system privileges, it is free to make changes on external storage devices. In essence, apps are able to masquerade as a system service for the purpose of writing to the SD card.
The Current State Of Affairs
Unfortunately, making use of this technique isn’t very intuitive, it's prone to bugs, and there seems to be some unpredictable inconsistencies across devices from different OEMs. In other words, this is obviously not the way the Media Store was intended to be used. Among the devices we've been able to test, the Samsung Galaxy Note 3 has shown the least friction while the Google Play Edition LG G Pad 8.3 has been much more hit-and-miss. The HTC One M8 mounts the SD slot slightly differently than most devices, so the developers we've talked to are still looking into it.
We’re aware of a few apps that have already adopted this method in some way or another, but none of them are truly complete and flawless implementations.
The full set of functions would include:
- creating new files/folders
- copying files/folders
- deleting files/folders
- moving files/folders
- renaming files/folders
Total Commander appears to be the first to have discovered and implemented this workaround. There are still a few known bugs and some devices, like the HTC One M8 with its /sdcard2 mount point, aren’t supported yet, but it’s almost completely working on the Note 3. While the code has been in Total Commander for quite some time, a recently posted beta cleaned up several bugs and makes it the best implementation we've seen.
ES File Explorer partially works with the Note 3 and One M8, but there are still limitations across all of the devices we've been able to experiment on. On the Note 3, ES File Explorer cannot rename, and on the One M8, it can't rename or write files but can delete them and create folders, for example. As I said, support is hit-and-miss across various devices.
Left: Total Commander can do everything on a Note 3. Right: ES File Explorer can copy, delete, and move files/folders, but cannot rename.
Not long ago, MX Player added support for deleting files, and File Expert recently reached out to let us know a beta release is coming soon with most of the standard functionality. FolderSync also supports copying files from primary storage to the SD card on the Note 3, but that's the only thing working, so far. There may be more apps to add to this list, but they haven't hit our radar.
Does This Mean We Get Our SD Cards Back?
I hate to say it, but I wouldn’t count on this workaround being available in future versions of Android. Google left no room for interpretation, the documentation is very clear.
"Apps must not be allowed to write to secondary external storage devices, except in their package-specific directories as allowed by synthesized permissions." -- Android Documentation
Using the Media Store to modify files on the SD card (outside of an app’s private folder) is clearly circumventing the rules. It’s safe to assume this loophole was unintended and there’s even a chance that it will be marked as a security risk. On the other hand, disabling this workaround will probably break a few apps that use the Media Store for the intended purposes. Google will probably have to add checks to ensure file permissions are respected by the Media Store when 3rd-party apps are making changes. We’ll have to wait to find out what happens here.
In the meantime, this does give app developers a way to continue treating the SD card as they always have. It may not last forever, but it’s a decent crutch until something better comes along. The code posted a year and a half ago by Total Commander’s developer Christian Ghisler surely isn’t up-to-date, but it’s a great starting point for anybody willing to take the time to work with it.
In reality, most users aren’t completely in the cold. Most OEM devices still ship with a stock file manager included. If it is a system app -and it should be- then it can still be used for basic tasks. Of course, this doesn’t solve the problem for numerous 3rd-party apps that should have write access to the SD card, nor does it help existing or future Google Play Edition devices that don’t ship with file managers.
Here’s to hoping for a better solution with the next version of Android.
Update: With the release of Android 5.0 Lollipop, new APIs have been added that allow an app to request permission from users to access folders on external storage. This approach gives users the same level of functionality that has always been available, with greatly improved security. See Android 5.0 Makes SD Cards Great Again, Extends API To Allow Full Directory Access, Automatic MediaStore, And Improves Security