Mark L. Murphy's Blog, page 32

Android Studio 3.0: What "Injecting Monitoring Logic" Means

Michael Comella published a nice bit of analysis
last week, showing the ramifications of this innocuous sentence in the
documentation for the Android Studio 3.0 profiler:

To show you advanced profiling data, Android Studio must inject monitoring logic into your compiled app.

When you run your app from Android Studio, in addition to deploying the
APK, Android Studio also pushes three files to the root of your app���s
portion of internal storage:

libperfa_x86.so
perfa.jar
perfd

These will be pushed over if:

You run the app from the Run option in the IDE (versus installing the app
by other means)

You run the app on an Android 8.0+ device (and possibly on older ones as well,
if you opt into that)

Have ever had the Android Profiler tool window open in Android Studio since
you started up this copy of the IDE

It would appear that these three files contain code that gets injected into your
app���s process as part of profiling it, in line with the quoted sentence.

Tactically, this can screw up some tests. Mr. Comella found out about the
problem when testing Firefox Focus, Mozilla���s privacy-centric browser for
Android. One of the tests involves ensuring that no extra stuff gets left
behind after a browsing session, and the test tripped up over these unrecognized
files.

(as an aside, testing to ensure that the files in internal storage are what
you expect is a really cool test)

From a security standpoint, on the one hand, there does not appear to be an immediate problem.
In my testing, these files are not packaged into the APK itself, but instead
get pushed over separately. Hence, these files should not make it into your
production environment. Still, I really hope that whatever mechanism Android
Studio uses to inject this stuff is adequately secured.

My only complaint is that we cannot turn this off, short of closing up
Android Studio, re-opening it, and making sure not to touch the Android Profiler
tool. If the Profiler needs this injected stuff to work, that���s fine, but there
still should be better developer-level control over the code injection.
With luck, this will get addressed sometime.

So, my hat���s off to Mr. Comella for documenting this, and to Nathan Freitas
for raising awareness further.

Android's Architecture Components Version 0.6 Released

Subscribers now have
access to an update to Android���s Architecture Components,
known as Version 0.6, in PDF, EPUB, and MOBI/Kindle formats. Just log into
your Warescription page and download
away, or set up an account and subscribe!

This is yet another small update, with two new chapters:

An overview of the repository pattern

Configuring SQLite beyond what Room directly supports

It also updates the samples to use 1.0.0 of the main Architecture Components,
and to use 3.0.1 of the Android Gradle Plugin. Plus, the samples were updated
for Java 8 support, including lambda expressions.

And, as usual, it fixes a variety of errata, and makes other
small improvements.

AccessibilityServices, Play Store Bans, and API Design

TL;DR: When designing an API, keep features with privacy and security ramifications
separate from other features, such that both you and developers using the API have a very
clear sense of what is and is not sensitive.

Many of you may have heard about Google���s threats to remove apps from the
Play Store that have an AccessibilityService that is not primarily for
accessibility. Many outlets reported on this, including
XDA Developers
and Ars Technica.
According to the purported emails, Google is stating that if your
app has requested the BIND_ACCESSIBILITY_SERVICE permission:

you must explain to users how your app is using the android.permission.BIND_ACCESSIBILITY_SERVICE to help users with disabilities use Android devices and apps

Later, the email states that you should remove the app if
���the permission is being used for something other than helping users with disabilities use Android devices and apps���.

On the one hand, I completely understand the rationale for instituting this sort
of ban. Accessibility APIs have been a privacy and security problem for some time,
even if many people are only recently noticing it. It is easier for Google to
whitelist various apps that have legitimate need to use those APIs than it is
to root out all malicious uses.

However, whenever you see bath water being thrown out,
look for an associated baby.

In Android 8.0, Google added
fingerprint gesture support.
While relatively few
devices have hardware support for this, it effectively turns the fingerprint
sensor into a tiny trackpad. At present, you can detect swipes in the four
cardinal directions (up, down, left, right), and you can take some action
based upon this via a registered callback.

However, they put fingerprint gesture support in
AccessibilityService. Hence, only apps that request the
BIND_ACCESSIBILITY_SERVICE permission can find out about fingerprint gestures.

It is unclear why Google thinks that fingerprint gestures are primarily for
users who need assistive technologies. After all, trackpads are fairly commonplace
elsewhere, used by people with a wide range of abilities. Google even supports
trackpads with Android apps on Chromebooks, without requiring the app to have
an AccessibilityService.

Moreover, by putting fingerprint gestures into the accessibility APIs,
not only would app developers need to fuss with an AccessibilityService,
and not only would enabling that service trigger all sorts of privacy and security
warnings to the user, but with the new Play Store policy, any app seeking
fingerprint gestures needs to be for users with disabilities, period.

While the ban is aiming to defend users against unnecessary privacy and security
challenges, there is nothing in today���s fingerprint gesture API that is
sensitive, and it���s unclear what they could add in the future that would
be sensitive. Even if the gestures somehow get integrated with actual
fingerprint scans, there���s a separate API for fingerprint scans, not tied into
accessibility.

So we get a new general-purpose feature, and courtesy of an engineering choice
(where the API resides) and a policy choice (ban on unnecessary use of
accessibility APIs), most developers cannot use that feature.

This brings me back to the TL;DR from the outset. Keep APIs with privacy and security
ramifications separate from other APIs. Make it very obvious to all parties what
portions of your API may put users at risk. Conversely, don���t attach normal APIs
onto an API set that is sensitive just because it���s easier or, in the absence
of those privacy and security ramifications, it might make sense. Bright lines
help everybody understand what is and is not safe. And, in the future, should
bans like this one be required, it is simpler to only affect those misusing the
sensitive APIs if the sensitive APIs are clearly separated out from the remaining
APIs.

In this case, given that accessibility APIs are going to have privacy and security
ramifications indefinitely, IMHO the better approach would have been to have fingerprint
gestures be somewhere else. For example, they could be just simple MotionEvent
sources, not significantly different than how trackpads work elsewhere. Even
if there are technical reasons for keeping fingerprint gestures out of the regular
input framework, there could be a FingerprintGestureManager system service
or some such that provided this ability, independent of the accessibility APIs.
It���s not out of the question that this could be addressed in the timeframe
of Android P (Peppermint? Pistachio? Pomegranate?), should Google
choose to do so. Only time will tell if we can put this baby back into some
bath water.

After all, you have to be very careful with some babies.

The Storage Situation: Removable Storage

Back in 2014, I wrote a series of blog posts to try to clear up confusion
around where you can read and write files in Android. This post is an updated
edition of the original post on removable storage.

There is a lot of confusion regarding Android���s storage model. That confusion
got a lot worse with Android 4.4���s changes to that storage model, and it has
not really improved much since. There
are countless Stack Overflow questions and the like where they clearly do not
quite grok the various pieces of Android���s storage model.

This is the third post in a three-part series covering this storage model, to help
clarify what is going on. Monday, we looked at
internal storage.
Yesterday, we looked at
external storage.
Today, we will look at removable storage, the source
of an unfortunate amount of angst.

What Your Users Think ���Removable Storage��� Means

Many of your users will have a device that has some sort of removable media.
Often times this is a micro SD card. Some tablets or docks have a full SD card slot.
Plus, USB mass storage is possible via USB On-The-Go and USB Type C connectors (not to mention
devices or docks with a full USB host port).

Your users will think that they can work with removable storage much like they
can with a desktop or notebook.

Unfortunately, your users are largely mistaken, and are even more mistaken
with Android 4.4+. That���s because Google���s approach towards removable storage is���
unconventional.

What Google Thinks ���Removable Storage��� Means

In the beginning, as was noted yesterday, external storage was often in the form
of a removable micro SD card. At that time, many developers got in the habit of
thinking that external storage == removable storage.

However, as Android 3.0 and higher started rolling out en masse, developers soon
started to realize two things:

External storage != removable storage on most of those devices

There���s nothing in the Android SDK for removable storage

Wait, Wut?

That���s right: until Android 4.4, there was no official support for removable media
in Android. Quoting Dianne Hackborn:

���keep in mind: until Android 4.4, the official Android platform has not supported SD cards at all except for two special cases: the old school storage layout where external storage is an SD card (which is still supported by the platform today), and a small feature added to Android 3.0 where it would scan additional SD cards and add them to the media provider and give apps read-only access to their files (which is also still supported in the platform today).

Android 4.4 is the first release of the platform that has actually allowed applications to use SD cards for storage. Any access to them prior to that was through private, unsupported APIs. We now have a quite rich API in the platform that allows applications to make use of SD cards in a supported way, in better ways than they have been able to before: they can make free use of their app-specific storage area without requiring any permissions in the app, and can access any other files on the SD card as long as they go through the file picker, again without needing any special permissions.

But��� But��� But��� What About All These Apps That Use Removable Media?

They fall into three buckets:

Some are just relying on MediaStore indexing. So, for example, a video player can
find out about videos on all available media by querying the MediaStore, and if
the device manufacturer hasn���t broken the MediaStore indexing of removable media,
the player will be able to play back videos on removable media.

Some are apps that ship with the hardware. The hardware manufacturer knows the
device and what the rules of the game are for that device. The hardware manufacturer
is also far less concerned about cross-device compatibility, as their apps aren���t
(usually) shipping on the Play Store. Hence, a hardware manufacturer has carte blanche
to work with removable media.

Some are apps written by developers who decided to go past the boundaries of
the Android SDK. There are various recipes online for examining various Linux
system files (and file-like substances) to determine what ���mount points��� exist,
and from there apply some heuristics to determine what represents removable media.
While reliability across devices could easily be an issue, beyond that, these
techniques worked��� until Android 4.4, when everything changed.

What Happened in Android 4.4: The Good News

Two things happened in the timeframe of Android 4.4 that affect removable media.

On the plus side, we gained some measure of official Android SDK support for removable media.
Specifically getExternalFilesDirs() and getExternalCacheDirs() (note the plural form)
will not only return directories that we can use on ���real��� external storage,
but also will return directories that we can use on any available and supported
removable media. Our apps do not need any specific permissions to use any of those
directories.

Also, the Storage Access Framework
gives device manufacturers some options for exposing removable media to our apps
in a more controlled fashion. Quoting Jeff Sharkey:

However, apps can create/write files outside of their package-specific directories on secondary external storage devices by using the new CREATE_DOCUMENT intent, which involves the user in the file creation process.

What Happened in Android 4.4: The Angst-Riddled News

Since Android 4.2, there has been a request from Google for device manufacturers to
lock down removable media. Generally, this was ignored.

For Android 4.4+, Google amended the Compatibility Test Suite (CTS) that device manufacturers
must comply with in order to ship a device containing Google���s proprietary apps
(e.g., Play Store, Maps, Gmail; otherwise known as ���GMS���). Quoting
Dave Smith:

However, new tests were added in CTS for 4.4 that validate whether or not secondary storage has the proper read-only permissions in non app-specific directories, presumably because of the new APIs to finally expose those paths to application developers. As soon as CTS includes these rules, OEMs have to support them to keep shipping devices with GMS (Google Play, etc.) on-board.

As a result, apps can read files on removable media using the various undocumented
and unsupported tricks for finding removable media. However, apps cannot write to or
otherwise modify such removable storage. Note that device manufacturers themselves
may have ways of dealing with this, but ordinary app developers do not.

The android-platform Google Group has been the home of
a rather epic discussion thread
regarding Google���s decision here, including Dianne Hackborn���s entry linked to earlier
in this post.

Despite the hand-wringing, I do not expect Google to backpedal on this issue.
For every user that complains about how Android 4.4+ makes their micro SD card
useless, there���s a pundit complaining about how Android lets any app run amok
and read stuff on storage, raising privacy and security issues. In fact,
I would not be the
least bit surprised if read access to removable media is blocked in the future,
beyond an app���s own directory on that media.

Where Can I Learn More?

Two of the better resources for this topic are ones I linked to earlier in this
post:

Dianne Hackborn���s
explanation of what���s going on

Dave Smith���s
explanation of how this was implemented

OK, So What Are We Supposed to Do?

Ideally, use the Storage Access Framework:

ACTION_OPEN_DOCUMENT (akin to a legacy system file-open dialog)
ACTION_OPEN_DOCUMENT_TREE (akin to a legacy system directory-open dialog)
ACTION_CREATE_DOCUMENT (akin to a legacy system save-as dialog)

On Android 7.0+, you can also use StorageVolume to request access to specific
directory trees on removable storage.

All of these give you a Uri back, representing a document or a document tree.
You would use those with things like DocumentFile and ContentResolver to
work with the content (or tree of content).

The big advantage of this approach is that your code is now storage-agnostic.
The same code will work with:

External storage
Removable storage
Google Drive
Samba/CIFS network file servers (if the user has Google���s app for that installed)
Anything else exposed by a third-party documents provider

While your code will be somewhat different than when using java.io.File, it���s
not that different:

You still use InputStream and OutputStream to read and write content,
but you get them from ContentResolver (e.g., openInputStream()), rather
than from FileInputStream and FileOutputStream

You still can traverse document trees, but you do so using DocumentFile,
not File

The biggest difference is that you should no longer assume that there is
a meaningful filename, using DocumentFile to get a ���display name��� to show
the user.

Certain types of apps, such as media players, might be better served using
the MediaStore and ignoring the filesystem entirely. Not only can you
query the MediaStore to find out the available media of relevance, but you
can create a MediaStore Uri that has the MediaStore stream the content
to you, even for content that you might not be able to access.

But, That Doesn���t Fit My UI

From what I���ve seen, the #1 source of complaints about the ���lost��� removable
storage access comes from developers of file managers. Their argument is that
they need to be able to list all sources of files and traverse them.

That���s just not how Android works. With some tweaks to the UX, you can get
something that is far more Android-friendly.

For example, rather than considering your app to be the ���source of truth��� for
document trees, let the user do that, in the form of a ���bookmark��� system. Your
app would come with a pre-configured bookmark for external storage. If the user
taps an action bar item (or whatever) to add a new bookmark, you use
ACTION_OPEN_DOCUMENT_TREE and let them choose a tree from whatever they want.
You take persistable permissions on the Uri that you get back, then save
the Uri as part of your app data (e.g., in a database). You then use DocumentFile
for traversing the document tree for presentation in your UI.

With this approach, you fit better with Android, by using the Storage
Access Framework. Along the way, you offer more functionality to the users, as now your ���file
manager��� can manage content from a variety of places, including things that
are not local files. Similar approaches can be used for other types of apps,
switching from a ���file��� metaphor to a ���content��� metaphor and leveraging
the Storage Access Framework.

The Storage Situation: External Storage

Back in 2014, I wrote a series of blog posts to try to clear up confusion
around where you can read and write files in Android. This post is an updated
edition of the original post on external storage.

There is a lot of confusion regarding Android���s storage model. That confusion
got a lot worse with Android 4.4���s changes to that storage model, and it has
not really improved much since. There
are countless StackOverflow questions and the like where they clearly do not
quite grok the various pieces of Android���s storage model.

This is the second post in a three-part series covering this storage model, to help
clarify what is going on. Yesterday, we looked at
internal storage.
Today, we will look at external storage.

What Your Users Think ���External Storage��� Means

Many Android device users will have no idea what ���external storage��� means.
There is nothing in the device UI that will necessarily use that term.
At best, your users will think that an SD card is external storage.
That���s not quite right. Except on Android 4.4+, in which case it is only
partially right.

What Google Thinks ���External Storage��� Means

The Android SDK documentation
has this to say in terms of a definition of ���external storage���:

Every Android-compatible device supports a shared ���external storage��� that you can use to save files. This can be a removable storage media (such as an SD card) or an internal (non-removable) storage. Files saved to the external storage are world-readable and can be modified by the user when they enable USB mass storage to transfer files on a computer.

Much of what has been written in the documentation and elsewhere
about external storage was for the pre-Android 4.4 world. In those halcyon days of yesteryear,
there was a single volume known as ���external storage���, and it was effectively defined as
���the stuff that shows up when the user plugs their device into a computer using a
USB cable���. Even that wasn���t completely accurate, as some manufacturers would also
allow access to their devices��� removable media via the USB cable as well. And
Android 4.4 added yet more wrinkles in terms of removable media��� which is why
removable media gets its own blog post tomorrow.

For the purposes of this blog post ��� and to line up with what most other written
material will refer to ��� ���external storage��� is defined as the directory tree returned
by Environment.getExternalStorageDirectory().

The Many Places Where External Storage Is Stored

External storage, like a tumbleweed, goes where the wind blows it.

In Android 1.x and most 2.x devices, external storage was generally some form
of removable media, typically a micro SD card. More importantly, for all Android 1.x
and 2.x devices, external storage was a separate partition with a separate filesystem.
While a few Android 2.3 devices elected to use on-board flash for external storage,
that was still a separate partition from the partition that held internal storage.
As a result, we wound up with devices that might advertise having several GB worth
of storage, but that storage tended to be mostly for external storage, as the partitions
could not be resized by ordinary users.

Android 3.0 changed this around, allowing internal and external storage to each be
on the same partition, just in separate directory trees. This provided a lot more
flexibility for users, as now there was no artificial hard distinction between space
for internal storage and space for external storage. Device manufacturers still could
elect to have external storage be a separate partition, or even be on removable media,
but typically they did not.

The Many Paths Under Which External Storage is Stored

For us as developers, the actual path under which this magical ���external storage��� was
accessed varied over the years, from /sdcard, to locations under /storage, to
eventually the current /mnt/shell/emulated/0. And just as secondary users of an
Android 4.2+ tablet get their own internal storage, they get their own external storage,
with its own root directory.

Hence, as I mentioned yesterday:

NEVER HARDCODE PATHS

Use various methods to get the base directory to work from��� though this too
has gotten more complicated over the years.

In the beginning, everyone used Environment.getExternalStorageDirectory(),
which pointed to the root of external storage. This led to external storage
being just a big basket of random content.

Later, Google offered more organization:

getExternalFilesDir() and getExternalCacheDir() on Context, pointing
to an application-specific directory on external storage, one that would be
deleted when the app is uninstalled

Environment.getExternalStoragePublicDirectory(), for centralized places
to store well-known file types, like photos and movies

Note that the Context methods have plural forms on Android 4.4+
(getExternalFilesDirs() and getExternalCacheDirs()), which ties into removable
media, which we will get into more tomorrow.

External Storage and Permissions

Just as the location ��� physically and logically ��� of external storage keeps
changing, so does our ability to work with it.

Originally, apps could do whatever they wanted.

Android 1.5 added the WRITE_EXTERNAL_STORAGE permission, which apps had to hold
to be able to write files to external storage. That way, the user would be informed
at install time that the app intended to modify external storage contents. Any app
could still read from external storage, though.

Android 4.4 started enforcing a READ_EXTERNAL_STORAGE permission, so you cannot
even read from external storage if you do not hold some permission. Note that
WRITE_EXTERNAL_STORAGE implies READ_EXTERNAL_STORAGE, so you only need one of those,
not both. And, for getExternalFilesDir() and getExternalCacheDir(), you do not
need either of those permissions ��� you can read and write in those directories
with impunity. Android now has an android:maxSdkVersion attribute on
<uses-permission>, specifically so that you can drop WRITE_EXTERNAL_STORAGE if
you no longer need it, because you are only working with getExternalFilesDir()
and getExternalCacheDir().

Poking Around External Storage

As a developer, assuming that you can find where external storage really resides
for your version of Android, you have unfettered access to it from the Device
File Explorer in Android Studio, for both emulators and production devices.

You can also use a USB cable, much like your users will use. However, bear in mind
that what is presented to the USB interface is not what is on external storage���
but, instead, is what has been indexed on external storage in the MediaStore.
Hence, unless you take steps
to ensure that new files that you create get indexed, they may not be immediately visible.

Under the covers, Android is using the Media Transfer Protocol for its USB
communications. This enables a lot more flexibility than does Mass Storage Mode
(a.k.a., what thumb drives use) that Android used originally. However, some MTP
clients may cache directory listings, so even after you get your file
indexed by MediaStore, an already-attached client may need to be refreshed.

Frequently-Asked Questions

Here are some FAQs about what the SDK refers to as external storage:

How Do I Secure My Files on External Storage?

In general, you don���t. The user has complete access to external storage, and other
apps can get to external storage if they hold the right permissions.

One thing that you can do is use something like Facebook���s Conceal.
This encrypts your files on external storage, but uses a generated encryption
key kept on internal storage. From a security standpoint, the net effect
is to make external storage closer to internal storage in terms of read access.
Note, though, that Conceal cannot prevent other apps, or the user, from deleting
your files on external storage, or perhaps trying to write to them and corrupting
the files as a result.

The Storage Situation: Internal Storage

Back in 2014, I wrote a series of blog posts to try to clear up confusion
around where you can read and write files in Android. This post is an updated
edition of the original post on internal storage.

There is a lot of confusion regarding Android���s storage model. That confusion
got a lot worse with Android 4.4���s changes to that storage model, and it has
not really improved much since. There
are countless StackOverflow questions and the like where they clearly do not
quite grok the various pieces of Android���s storage model.

This is the first post in a three-part series covering this storage model, to help
clarify what is going on. Today, we will look at ���internal storage������ including
its various definitions.

What Your Users Think ���Internal Storage��� Means

Your users will eventually wander into Settings > Storage on their device ���
or the equivalent location, depending on their device model ��� and
may see a screen that describes ���Internal Storage���:

And that will be that. The user thinks that all of the on-board flash
is ���internal storage���.
Fortunately, Google has started to change that with Android 8.0, going with a
general ���storage��� term instead of ���internal storage���.

However, users may still see ���internal storage��� in places like an Explorer
window in Windows, when their device is connected via USB.

What Google Thinks ���Internal Storage��� Means

Alas, that is not what the Android SDK thinks ���internal storage��� means, which is
where some of the confusion lies.

If you read the Android documentation on internal storage,
the description is��� hand-wavy:

You can save files directly on the device���s internal storage. By default, files saved to the internal storage are private to your application and other applications cannot access them (nor can the user). When the user uninstalls your application, these files are removed.

In truth, the Android SDK���s definition of ���internal storage��� is a specific
directory, unique to your app, where your app can place files. As suggested in the
docs, those files by default are read-write for your app and deny-all for any
other app.

(exception: users running file managers with superuser privileges on rooted devices
can access anything)

There are a handful of base methods on Context that give you access to
internal storage, including:

getCacheDir()
getDir()
getDatabasePath()
getFilesDir()
openFileInput()
openFileOutput()

Other methods will rely upon these, such as openOrCreateDatabase(). Other
classes also will rely upon these, such as SQLiteOpenHelper and
SharedPreferences.

Where Internal Storage Is Stored��� Sometimes

If you look around various blog posts, StackOverflow answers, and books that
came out in 2012 or earlier, you will be told that your app���s ���internal storage���
resides at:

/data/data/your.application.package.name

(where your.application.package.name is replaced by your application ID, as is
declared in the package attribute in the manifest or modified via Gradle)

Inside of there will be some directories automatically created by Android as
you use some of those Context methods. For example, getFilesDir() returns a
File object that points to a files/ directory inside of your app���s
internal storage.

Where Internal Storage Is Stored��� The Rest of the Time

However, this is not always where your app���s internal storage resides. If there
is one rule for developers that you should take away from this blog post series,
it is:

NEVER HARDCODE PATHS

Every now and then, I will see developers do something like this:

File f=new File("/data/data/their.app.package.name/files/foo.txt&...

This is really really dumb.

First, it is more typing than using getFilesDir():

File f=new File(getFilesDir(), "foo.txt");

More importantly, internal storage is not always at the same place. Notably,
we have the notion of separate user profiles, starting in
Android 4.2 for
tablets and Android 5.0 for phones.
Each user gets his or her own ���internal storage��� area. While the aforementioned
directory is still used for the primary user, that is not guaranteed, and it will
not be used for secondary accounts.

Poking Around Internal Storage

The Device File Explorer tool in Android Studio 3.0+ can browse all of internal
storage on an emulator, plus internal storage of debuggable apps on production
devices. This should handle most of your scenarios:

Fromn the command line, you can use the
run-as option with adb at the command line. For example, to download a
database from the primary user���s internal storage to your development machine, you
might use:

adb shell 'run-as your.application.package.name cp /data/data/your.application.package.name/databas... /sdcard'

Note that:

You will need to change the destination to wherever on your device external
storage is mapped to (shown here as /sdcard/, which will not work on all devices)

You may need to use cat instead of cp on older devices

Once the file is on external storage, you can use adb pull to download
it to your development machine, or access it by other conventional means
(e.g., via mounting the device as a drive on your development machine).

Limitations of Internal Storage

On ancient Android 1.x and 2.x devices, internal storage was usually on a dedicated
filesystem partition, and that partition was usually rather tiny. The HTC Dream
(a.k.a., T-Mobile G1), the original Android device, had a whopping 70MB of
internal storage for use by all apps.

(And, no, that���s not a typo. We measured storage in megabytes back then.)

By the time the 2.3 devices were rolling out, internal storage might be as
big as 1GB.

Android 3.0 changed the storage model around, in such a fashion as internal
storage got more space. Devices that advertise as having 4GB, 8GB, 16GB, etc.
of storage space usually had all of that (less existing contents) available
for internal storage. We will get into what changed in Android 3.0 and its impacts
on the storage model as part of tomorrow���s blog post on external storage.

For Android 1.x and 2.x, internal storage was only really for small files, and you
needed to use external storage for everything else. Android 3.0+ means that
for most devices and most users, internal storage is fine for files that are not
meant to be used routinely by other apps or accessible by the user independently
from your app. Some power users, though, will find that even on-board flash is
insufficient for what they want to store, so they turn to removable storage���
which is a whole ���nuther can of worms that we will get into later in the week.

Frequently-Asked Questions

Here are some FAQs about what the SDK refers to as internal storage:

Should I Make Files on Internal Storage World-Readable or World-Writeable?

Oh, $DEITY, no. Use FileProvider and serve that content via that
ContentProvider implementation. Then, you at least have the option of using
Android���s permission system to manage access to those files, versus having
any app on the system be able to monkey with those files.

Well, How About android:sharedUserId?

I also counsel against this.

android:sharedUserId is an attribute you can place in your manifest that indicates
a logical user ID to be used for your app. Any other app that is installed that
is signed by the same signing key and requests the same android:sharedUserId
will use the same Linux user from a security standpoint. The net effect is that
those two apps will be able to work with each other���s files with impunity, as those
files are all owned by the same Linux user.

This attribute is really designed for pre-installed apps, such as some software
suite pre-loaded by the device manufacturer, carrier, or ROM mod maintainer. In
particular, once you ship your app, you cannot reliably change your android:sharedUserId
value without locking your user out of any existing files��� as Android does not
change the ownership of existing files when it changes the Linux user account that
your app runs as.

There are various risks in having multiple processes work simultaneously with
files. Some subsystems, like SQLite, have built-in logic to deal with this.
But if you are doing your own file access yourself (e.g., via File and Java I/O), then
you have to somehow deal with simultaenous access, which can get tricky.

You also have to deal with what happens when one app is uninstalled, taking away
files that another app was using. In a hub-and-spoke model, like an app and a suite
of plugins, perhaps this is not that risky. In other models, where apps are closer
to being peers, you cannot afford to have your app���s data vanish because the user
elected to uninstall some separate app.

Finally, you do not know what the future might bring. Right now, you might view
your set of apps as a tightly-coupled suite. Somebody who acquires those apps,
or acquires your firm, might wish to go a different route. Using data sharing
options that are more loosely coupled, like a ContentProvider, gives you greater
flexibility than does assuming that your apps should co-mingle their files.
In an ideal world, your app should treat other apps as a fairly-reliable but not
always avaialble resource, just like you treat your own Web service.

How Do I Prevent Users of Rooted Devices From Accessing My Files on Internal Storage?

Simple: don���t put files on internal storage. Users of rooted devices can access
whatever they want on the device, so the only way to prevent them from accessing your
data is to not have it on the device.

Some developers will attempt to encrypt their files, using a hard-coded password,
to prevent rooted device users from using those files. This does erect a ���speed bump���,
but it is a small one. All it takes is one person with interest to reverse-engineer
your app, determine how to decrypt those files, then write up a blog post or
discussion board entry on how to do it.

On the whole, there are relatively few people with rooted devices ��� I estimate it
at well under 1%. IMHO, you will have better success by focusing your
engineering work on writing a better app, rather than spending that time trying
to block rooted device users.

The Busy Coder's Guide to Android Development Version 8.8 Released

Subscribers now have
access to the latest release of The Busy Coder���s Guide to Android Development,
known as Version 8.8, in all formats. Just log into
your Warescription page and download
away, or set up an account and subscribe!

Mostly, this update is about Android Studio 3.0 and the corresponding version
of the Android Gradle Plugin and Gradle itself. This includes:

Updating the tutorials

Making major changes to all of the Gradle coverage, both in the core
chapters and the trails

Adding a new chapter on icons, specifically to explain the new Image
Asset Wizard and the adaptive icon stuff

Replacing the chapter on dependencies with a brand-new edition

Updating all of the places that used Jack-based lambda expressions to
use the new native ones, and updating all of the Rx samples to use lambda
expressions

Updating all the samples to use implementation instead of compile,
using flavor dimensions wherever product flavors were used, etc.

Making significant changes to some performance tuning chapters to reflect the
changes in Android Studio���s profiling tools

A few chapters were removed as a result of this. Some were redudant chapters
in the Gradle coverage. Also, the Android Things chapter was removed,
to not violate the Android Things Console terms of service.
And, the chapter on writing reusable components was removed, as it was just
scary old.

This update also brings us up to speed with the final release of Android 8.0 and
has your usual smattering of other miscellaneous fixes and improvements.

The combination of all of these changes, plus standardizing on ���Logcat��� instead
of ���LogCat��� and related nomenclature changes,
means that there are a lot of sections with updates that will
show up in bold-italic in the table of contents.

For those with the APK edition of the book,
there is a new appinar, on GraphQL and Android, tied to the
book GraphQL and Android.

There should be another update to this book before the end of 2017.

Android Things, Terms of Service, and a Pulled Chapter

My chapter on Android Things will be pulled in the forthcoming
update to The Busy Coder���s Guide to Android Development.

I had fallen behind on dealing with Things since it went to 5, then 5.1, then
5.1.1, over the past couple of months. I went to start integrating some of those updates,
only to find that it appears that you can no longer obtain Android Things images except by means
of the Android Things Console. That, in turn, requires agreeing to the
Android Things Console Terms of Service.

Of note, it is against the Terms of Service to discuss the Android Things Console
(Section 2.2, ���You will not��� release or disclose the results of your use,
development, or evaluation of the Services to any third party���, where ���the
Services��� is defined as the Android Things Console).

My choices are:

Agree to the terms, and then be not allowed to explain to developers how to use
the Android Things Console, or

Don���t agree to the terms, and then be unable to explain to developers how to use
the Android Things Console (since I don���t know what it looks like, cannot produce
screenshots, etc.)

There���s little value in
my book having an out-of-date chapter with a massive hole in it. So, it���ll get
pulled.

Perhaps someday Google will update these Terms of Service and fix this. In the
meantime, I���ll work on other, um, things.

Android Studio 3.0 and FLAG_TEST_ONLY

Your AndroidManifest.xml file can have an android:testOnly attribute.
This value then shows up in PackageManager and ApplicationInfo
as FLAG_TEST_ONLY.
This has been around since API Level 4, which is practically forever in
Android terms. According to what little docs we have,
it serve as:

Option to indicate this application is only for testing purposes. For example, it may expose functionality or data outside of itself that would cause a security hole, but is useful for testing. This kind of application can not be installed without the INSTALL_ALLOW_TEST flag, which means only through adb install.

On the whole, this may seem unremarkable, considering that few people have
remarked on it since it was introduced in 2009. However, there are two things
to note about it:

You cannot install an app with android:testOnly="true" by conventional means,
such as from an Android file manager or from a download off of a Web site

Android Studio 3.0 sets android:testOnly="true" on APKs that are run
from the IDE

For many developers, this is of little consequence. However, if you have ever
taken an APK lying around your build/ directory and sent it to somebody,
that will only work if you did something other than run the app in
Android Studio in order to create that APK, such as:

Built the app using the ���Build APK(s)��� menu option

Built the app using the assembleDebug or assembleRelease Gradle tasks

Built the app using something else that might use those tasks, such as a CI
server

If, instead, you ship the APK that Android Studio built just by you running
the app, the APK cannot be installed normally.
adb install, for example, will report:

Failed to install app-debug.apk: Failure [INSTALL_FAILED_TEST_ONLY: installPackageLI]

adb install -t will install such a test-only app, however.

I don���t really have a problem with Android Studio 3.0 adding this flag. After
all, the APK that you run from the IDE may bear little resemblance to the
APK that your users will use (hello, Instant Run!). Using this flag to steer
developers towards shipping APKs built by other means is fairly reasonable.

Unfortunately, this change is not covered in
the Android Studio 3.0 release notes,
or
the Android Studio 3.0 migration document.
It is mentioned in
the ���Build and Run Your App���
page, though the docs there do not mention the need for the -t switch.

So, just make sure that when you send an APK to somebody, you built it by
some means other than just running it from Android Studio.

The Curious Case of the Missing google()

The idea of having Gradle be the single ���source of truth��� was that so builds
could be the same whether you were building in Android Studio, from a CI server,
from ./gradlew, from gradle, or anything else. That has never been completely
the case. For example, environment variables are unavailable to Android Studio
builds but are available to other types of builds. However, having the same
builds across the board was the stated objective, even if the implementation
didn���t completely realize that objective.

Android Studio 3.0 adds a new difference in behavior: where the Android Plugin
for Gradle comes from.

Try this: create a new Android Studio 3.0 project, and in the project-level
build.gradle, delete google() from the repositories in buildscript, so
you are left with:

buildscript {
repositories {
jcenter()
}
dependencies {
classpath 'com.android.tools.build:gradle:3.0.0'
}
}

Your project will still successfully build in Android Studio��� but it will fail
to build from anything else. The Android Plugin for Gradle is no longer in
jcenter(), and we removed google(), so command-line builds will fail to find
the plugin:

* What went wrong:
A problem occurred configuring root project 'MyApplication'.
> Could not resolve all files for configuration ':classpath'.
> Could not find com.android.tools.build:gradle:3.0.0.
Searched in the following locations:
https://jcenter.bintray.com/com/andro...
https://jcenter.bintray.com/com/andro...
Required by:
project :

Unfortunately, this is officially
working as intended.

In this particular case, it���s not a huge deal. It���s easy enough to add
google(). But it does beg some questions:

According to the response from Google, ���Android Studio comes with its own
version of the gradle plugin in a private repository that is not accessible
when building from the command line���. What if we do not want to use that version?
For example, we might want to pull all plugins and other dependencies from
a private artifact repository for security reasons. How do we tell Android
Studio to not use its own private copy of the plugin and instead use the
one that we tell it to use?

In what other ways will builds from Android Studio diverge from command-line
builds?