Mark L. Murphy's Blog, page 25

The Death of External Storage: How Can I Stay Away From Files?

Under the fairly innocuous title of
���scoped storage���,
Google has announced that external storage, as Android developers have used it,
effectively is dead, for Android Q and onwards.

I am going to spend some time on this over the course of the week, as this change
needs some attention:

Monday, I described what has changed and what you are supposed to use.
Yesterday, I covered the limited options for actually getting a file on external or removable storage.
Today, I will review how you might still wind up with references to files��� that you cannot access

One of the salvos in the War on Files came in Android 7.0, with the
introduction of FileUriExposedException. This would be raised if you put
a file:// Uri into an Intent and tried using that Intent to start an
activity, etc. It would crash the app and steer you towards using FileProvider
to make that content available to other apps.

The hope was that this would limit the number of file:// Uri values flitting
through the OS, and I am sure that it helped. However:

It does not come into play for apps with a targetSdkVersion below 24, so
legacy apps that are not getting updates will still emit file:// Uri values

I am sure that there are creative ways of passing a Uri around that bypass
those checks

FileUriExposedException is thrown by StrictMode, and so lots of people
suggest that you simply disable that check to get past the ���problem���

Most likely, in a world with READ_EXTERNAL_STORAGE and WRITE_EXTERNAL_STORAGE,
those file:// Uri values would work, assuming that your app held the appropriate
permissions. On Android Q, you cannot hold those permissions. It is very unlikely
that a file:// Uri that another app could use (to create it) will be one that
your app can use (to consume it). In other words, if you get a file:// Uri, you will
probably crash when you go to open that file for reading or writing.

Sometimes, an <intent-filter> controls what Uri values we receive. For example,
if you have an activity that supports ACTION_VIEW, you will have an <intent-filter>
in your manifest that, among other things, declares what sorts of content you can
accept, through <data> elements.

For example, you might have an <activity> like this in your manifest:

<activity android:name=".MainActivity">
<intent-filter>
<action android:name="android.intent.action.MAIN" />

<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<intent-filter>
<action android:name="android.intent.action.VIEW" />

<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE" />

<data android:scheme="file" />
<data android:scheme="https" />
<data android:scheme="content" />
<data android:mimeType="text/plain" />
</intent-filter>
</activity>

Here we have a <data android:scheme="file" />
element advertising support for the file scheme.

You do not want those <data android:scheme="file" /> elements on Android Q,
as most likely you will be unable to use the Uri that you receive.

What you can do is set up an <activity-alias> for those:

Create an <activity-alias> pointing to the affected activity

Clone the relevant <intent-filter> elements from the <activity> onto the <activity-alias>

Remove the <data android:scheme="file" /> elements from the <activity>

Remove the other schemes from the <activity-alias>, leaving only the <data android:scheme="file" /> for declaring schemes

Define a boolean resource, one that is true by default but false in a res/values-v29/ edition

Use that boolean resource in android:enabled on the <activity-alias>

You will wind up with a paired <activity> and <activity-alias> like this:

<activity android:name=".MainActivity">
<intent-filter>
<action android:name="android.intent.action.MAIN" />

<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<intent-filter>
<action android:name="android.intent.action.VIEW" />

<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE" />

<data android:scheme="https" />
<data android:scheme="content" />
<data android:mimeType="text/plain" />
</intent-filter>
</activity>

<activity-alias
android:name=".FileAlias"
android:enabled="@bool/supportFileScheme"
android:targetActivity=".MainActivity">
<intent-filter>
<action android:name="android.intent.action.VIEW" />

<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE" />

<data android:scheme="file" />
<data android:mimeType="text/plain" />
</intent-filter>

</activity-alias>

The net result is that the <activity-alias> will be used on older devices,
where you can still consume the file:// Uri values. But that <activity-alias>
will be disabled on Android Q and above. This helps reduce the number of unusable
Uri values that your app will get.

You might wonder if Android Q would do this sort of thing automatically, basically
ignoring <data android:scheme="file" /> elements in manifests. At least as of
Q Beta 1, it does not, though it is possible it might get added in later releases.

Unfortunately, there are other ways that you can wind up with a Uri from another
app that are not controllable via <intent-filter> by scheme.

The big one is ACTION_SEND. For reasons that were never clear to me, ACTION_SEND
does not work like ACTION_VIEW and other Uri-centric Intent actions.
While you can use <data> to filter by MIME type, AFAIK scheme and related restrictions
are ignored. As a result, you can wind up with EXTRA_STREAM values that contain
file:// Uri values.

There other similar sources of Uri, like ACTION_SEND_MULTIPLE and the clipboard,
that suffer from a similar lack of scheme-based filtering.

IMHO, the best solution is, if the scheme is file and you are on Android Q or
higher, to detect that combination of conditions and display a dedicated error
UI. You can explain that the other app is out of date or has a bug and is incompatible
with your app. Try to give the user an understanding of what is causing the problem
(sending you stuff from this older app) and how to get past it (switch to a newer
app).

At worst, make sure that you are cleanly handling any IOException that might come
from trying to use that Uri (e.g., from ContentResolver and openInputStream()).
Do not just say ���oh, we���ll let ACRA/Crashlytics/Firebase Crash Reporting/whatever
handle it���, as inevitably you wind up showing some generic ���oops!��� error message.
Leave that for the ���unknown unknowns���, where you cannot predict the problem in
advance. For something like this, try to offer an error message that is more specific
and more actionable.

Tomorrow, I���ll review some reasons why all of this is happening.

The Death of External Storage: I Can Haz File?

Under the fairly innocuous title of
���scoped storage���,
Google has announced that external storage, as Android developers have used it,
effectively is dead, for Android Q and onwards.

I am going to spend some time on this over the course of the week, as this change
needs some attention. Yesterday,
I described what has changed and what you are supposed to use. Today, I will
cover why ���what you are supposed to use��� will not always work and what to do about it.

In an ideal world, any stream that we get from an arbitrary Uri would meet all
our needs. Unfortunately, that is not the case.

Take, for example, OkHttp. It is one of the most popular libraries for Android.
Among other things, it allows you to do HTTP POST and PUT operations, including
supplying bulk data for upload.
However, its primary class for that ��� RequestBody
��� only supports byte[],
File, String, and the Okio ByteString classes as sources of what should be uploaded.
It does not support Uri, as OkHttp is not an Android library. And it
does not support InputStream, as the content may need to be re-read
multiple times, depending on server responses and OkHttp configuration.

Which means that, on Android Q, uploading a file using OkHttp may be a problem,
if you do not have a file in the first place.

For small content, you could try to read it all into memory and then use the resulting
byte[]/String/ByteString with OkHttp. That will not work with larger content,
though.

Libraries with file-only APIs are going to be troublesome. That is why I suggested
nearly six years ago
that libraries try to focus on streams, not files. But there are other scenarios
where you may need a file:

AFAIK, the NDK knows nothing about Uri, and even using an InputStream
may be difficult

Some framework classes, such as SQLiteDatabase, need files, because they need
random access

Some media APIs may work with some streams (those backed directly by files on
the filesystem) but not others (those needing the provider to do work on the
content, such as decrypting it)

So, what can you do?

One possibility is to ask the user to put the file in one of the locations on
external or removable storage where you do have filesystem access, such as
getExternalFilesDir(). This makes it easy on you, but it is not very user-friendly.
These locations are awkward: Android/data/one.package.name.among.hundreds/files
is a pain to navigate to. Plus, since other user-installed apps have no access to your app���s
corner of external storage, the user will be limited to whatever system apps
have access or breaking out the USB cable and doing it from their desktop or notebook.

Another possibility is to make a copy of the data to some file that you control,
perhaps on internal storage. You use the Uri that you have to get an InputStream,
then copy the bytes from there to some FileOutputStream on a file that you control.
Then, you use the file with whatever needs it.
This eliminates the user headache, but now you are
working off of a copy, with all the problems that entails:

Changes made to your copy are not reflected in the original

Making the copy may take a while and take up lots of space

You need to get rid of the copy at some point, and knowing when to do that
may be difficult to determine

And so on

In some cases, you might be able to work around the problem by using something
other than a stream. ContentResolver also offers APIs to let you get a ParcelFileDescriptor
or AssetFileDescriptor, and those in turn let you get to java.io.FileDescriptor objects.
Perhaps the API that you are using will accept one of those instead of a File.
Just bear in mind that not every Uri that you get will be able to supply
a file descriptor, as not every Uri is backed by a file on the filesystem.
You will need to ���gracefully degrade��� when you cannot get a file descriptor,
at the very least explaining to the user that your app is incompatible with
wherever that Uri came from.

The best long-term answer is to get more APIs rewritten to avoid needing files.
In some cases, that will mean pushing down this sort of work-off-a-cached-copy
into the library, though at least it may know better when to get rid of the copy.
Or, the library could offer some way for you to provide fresh streams when needed.

Going back to the OkHttp example, it is at least conceivable that there
could be a RequestBody.create() variant that took a StreamSupplier parameter
instead of a File. StreamSupplier would be an interface with a single abstract method
(e.g., openInputStream()) that OkHttp could call as many times as needed, whenever
it would need to start reading the bytes from the top. You, as the user of OkHttp,
would simply create a StreamSupplier that turned around and called openInputStream()
on a ContentResolver. In principle, this should address OkHttp���s need for starting
over. I have not looked at the guts of OkHttp to see whether
something like this could work, and it would not surprise me if there are strong
technical reasons why this would be impractical for them. It���s also a hack. But,
tough times call for tough measures, and we may need libraries to take this sort
of approach to deal with the Uri-centric world we are in.

One way or another, we���re going to have to do something.

Tomorrow, I will have a bit of info on the other side of the problem: how to
avoid or detect when a legacy app tries handing you a file, one that you may not
be able to access.

The Death of External Storage: What? And What Now?

Under the fairly innocuous title of
���scoped storage���,
Google has announced that external storage, as Android developers have used it,
effectively is dead, for Android Q and onwards.

I have been hoping that I���m wrong about this, but so far my tests bear out what
the docs say. Also, the developers involved in this issue
seem to have similar concerns.

I am going to spend some time on this over the course of the week, as this change
needs some attention.
Today, I am going to focus on what has changed and what you are now expected
to do.

READ_EXTERNAL_STORAGE and WRITE_EXTERNAL_STORAGE
are not merely deprecated, but are ineffective:

Apps targeting Q cannot request them ��� those permissions will be treated as if the
user previously rejected them with ���don���t ask again���, and the app will be unable
to access arbitrary locations on external or removable storage

Apps targeting previous versions can request them, but they do not actually
grant any rights, and the app will be unable
to access arbitrary locations on external or removable storage

Probably.

I say ���probably��� in part because Google, in its infinite wisdom, decided to hide this
behavior in Q Beta 1 devices and emulators. Instead, out of the box, things work as they have
for the past few releases. You have to run an adb command to get Android to
behave in the documented fashion:

adb shell sm set-isolated-storage on

Perhaps this is because Google is still debating the wisdom of all of
this and might elect to punt on this feature until Android R, or Q MR1, or
something.

Perhaps this is because Google is considering implementing the restriction, but
limiting it to apps that target Q. This does not excuse hiding the behavior as
they did, but it would blunt the impact, as many developers would have a bit
over a year to adapt their apps.
Certainly, that would seem to be
the hope of the folk starring this issue.

My fear is that perhaps this
is some sort of epic-level trolling by Google. Every week that passes where
developers do not realize that they need to adapt their apps is one less week
they have to do that work, before Q ships as Android 10.0.

I also say ���probably��� because I found a loophole big enough to drive a truck through.
I assume that it is a bug, and so I filed a security issue
for it. We will see what happens. If it turns out that this loophole actually
is intended behavior, then the situation improves a fair bit. Once I get clarity
on this ��� assuming somebody actually looks at the issue ��� I will
write about it.

Note that there is one class of apps that is temporarily immune to these effects:
apps that are already installed on the device at the point when the device
gets upgraded to Android Q. Those apps get the same basic behavior as
before. If the app is uninstalled and reinstalled, it is treated as a fresh
install. And, I suspect that if the app is updated to one that targets Q,
the app will then be subject to normal Q behavior.

On the surface, this may sound great. After all, it means that even if you do
nothing, on Day 1 of the Android 10.0 rollout, all your users will be unaffected.

But if you are getting a steady stream (or flood) of new users, some of them
will have Q devices, and those fresh installs do not get ���grandfathered��� in
the way existing installs do. So, you still will have unhappy Q users, just perhaps
fewer of them.

There are three major ways that you can work with external and removable
storage given these limitations.

First, you can still use all the external-storage methods on Context, such
as getExternalFilesDir() and getExternalCacheDir(). These work as they have
since their introduction. They do not require any permissions, and you have
ordinary filesystem access. However, you only have access to a few specific
directories.

Second, you can use the Storage Access Framework. Primarily, that���s ACTION_OPEN_DOCUMENT,
ACTION_CREATE_DOCUMENT, and ACTION_OPEN_DOCUMENT_TREE. These give you the
Android equivalent of ���open-file���, ���new-file���, and ���choose-directory��� dialogs
that you may be used to from other programming environments. These have the
advantage of not only supporting external and removable storage, but also supporting
third-party document providers, such as Google Drive. The big disadvantage is
that you are not working with files anymore ��� working with Uri and ContentResolver
limits your flexibility. I have written a lot about the Storage Access Framework
over the years ��� here is a FAQ-style blog post
on the subject.

Third, you may be able to use the MediaStore. What works there depends on
a number of factors:

If your targetSdkVersion is 28 or below, you can query the MediaStore
without issue and can find whatever content it has indexed.

If your targetSdkVersion is Q, by default, your app can only see things
in the MediaStore that your app put there in the first place, such as
by manually insert()-ing entries into the MediaStore and writing content
to the returned Uri. However, your app
cannot see content that the user put on the device by other means, whether
that is another app, a desktop file manager, or anything else.

If your targetSdkVersion is Q, and you hold ROLE_MUSIC or ROLE_GALLERY
in the new roles system,
your app can work with the MediaStore as if its targetSdkVersion is lower,
but only for the type of content associated with the role. For other types
of content, the role does not help, and you are limited to only having access
to your own content. However, only one app can hold a role at a time. Unless your
app is extremely important to the user, or legitimately is a full local
music player or gallery-style app, the role solution is unlikely to help.

And, AFAIK, that���s it.

Tomorrow, I���ll explore a bit more about what your options are if you absolutely,
positively, have to have access to a file on external storage.

Random Musings on the Q Developer Preview 1

Each time Google releases a new developer preview, I rummage through
the API differences report
the high-level overviews,
and even the release blog post,
to see if there are things that warrant more attention from
developers. I try to emphasize mainstream features that any developer
might reasonably use, along with things that may not
get quite as much attention, because they are buried in the JavaDocs.

As with the previous two releases, Q seems to be lacking a prominent
user-facing feature. Q is certainly better, but I have no idea what I will
tell my mother that she���ll gain if her phone gets the Q update at some point.

However, there are a number of changes for developers.

What Will Piss Off Developers

Google has done quite a bit over the past several releases to improve privacy
and security in Android. That tends to be a double-edged sword, though, where
the changes break existing app functionality. So, for example, developers of
file managers are going to be rather disgruntled with scoped storage,
which says that you can no longer access all of external storage via the
filesystem.

The roles system is an
interesting concept. At one level, it might mostly be there for the benefit
of the Play Store and other app distribution channels, to better identify whether
apps are justified in holding certain permissions. However, the relationship
between roles and permissions is almost completely undocumented. For example,
does ROLE_SMS ���Can send and receive SMS messages; can read contacts��� mean
that no other app can read the contacts, despite requesting the READ_CONTACTS
permission? The fact that we need to run experiments and attempt to reverse-engineer
the documentation is never a good sign.

Other things will just pour salt into existing wounds. For example, the ���non-SDK interfaces���
push continues in Q, so developers that dodged bullets in P might now run into
problems in Q, if their preferred hack is no longer available.

Plus, there is your usual barrage of changes that won���t affect everyone but
might dramatically impact some developers. Starting an activity from the background
has long been considered to be poor form; developers who stuck with it
now have problems.
SYSTEM_ALERT_WINDOW has been dodgy for years; developers who stuck with it
now have problems.
Background access to the clipboard is gone.
Using locations in the background is overhauled. And so on.

What Will Piss Off Me

Google says that they have changed onResume() onPause(), and android:resizeableActivity,
but they did not seem to document what changed. ��\_(���)_/��
At least on the onResume()/onPause() stuff, prior reports indicate
that multi-window environments ��� including foldables ��� will now
leave apps in the resumed state even if they no longer have the input focus.
There appears to be a new onTopResumedActivityChanged()
activity callback to find out if this activity is now the ���top resumed activity���,
which I am guessing means ���the activity with the input focus���.

What Has Me Worried

There is a whole new set of APIs around ���content capture���. Unfortunately,
the central class has ���TODO��� for its documentation.
However, it is tied to the detestable autofill APIs and
seems to report when views come and go.
In the hands of responsible individuals, this has interesting characteristics
for app testing. In the hands of irresponsible individuals, this could be yet
another privacy leak.

Similarly, there is an InspectionCompanion ���used to inspect views���. This, at
least, seems like it may be only some kind of in-process capability, even though
it is largely undocumented.

There is a new GET_AND_REQUEST_SCREEN_LOCK_COMPLEXITY permission, which
���allows an application to get the screen lock complexity and prompt users to update the screen lock to a certain complexity level���.
On the surface, this seems fine, but this is a normal permission, so users
are not prompted about it. Historically, this sort of functionality was limited
to device administration apps.

There is a new SMS_FINANCIAL_TRANSACTIONS permission, which
���allows financial apps to read filtered sms messages���. However, I don���t know
how one defines either ���financial apps��� or ���filtered sms messages���. The
protection level of this permission is undocumented.

There is a new setAllowSystemGeneratedContextualActions() method on Notification.Builder
that ���determines whether the platform can generate contextual actions for a notification. By default this is true.���
Frankly, the notion that Android will automatically put actions into our notifications
creeps me out.

What Has Me Wondering How Far Down the Rabbit Hole Goes

Google has been pushing us away from background services for a couple of releases,
forcing a lot of developers to switch to foreground services. Now, with Q,
there are ���types��� of foreground services, described via an android:foregroundServiceType
attribute (in the manifest?). Available types include phoneCall,
mediaPlayback, location, dataSync, and connectedDevice. The
startForeground() call can now also pass in the type of foreground service
it will be (or a FOREGROUND_SERVICE_TYPE_MANIFEST flag to say ���read the values
from the manifest���). So, now we have services, then foreground services, then
types of foreground services. What���s next? Subtypes? Plus, it remains to be seen
what the system does with this type information.

What Has Me Intrigued

RecoverableSecurityException
makes its return, having been in one of the past rounds of developer previews
but removed from the SDK prior to release. If you catch one of these, you
have a dedicated message and RemoteAction for being able to help fix whatever
the problem is.

There is a new
android:useAppZygote
attribute for <service> in the manifest.
It allows an isolated service (android:isolatedProcess) to have its process
forked from ���an Application Zygote, instead of the regular Zygote���. From the docs,
my guess is that this is here for Chrome and similar apps that may want to fork
lots of isolated processes for executing possibly-unsafe code. However, isolation
from the regular app zygote may also mean that attacks that affect the app
zygote (and therefore all normal app processes) might not affect these isolated services.

There is a new isRunningInUserTestHarness() method on ActivityManager.
This apparently is not to detect normal instrumentation tests, but rather some
other style of tests for ���a device farm/testing harness (such as Firebase Test Lab)���.
The idea is that you would use this flag to skip over stuff that might screw
up such tests. It remains to be seen exactly where this is true or false.

In Android 8.0, we gained the ability to launch an activity into a particular
display. This is a useful alternative to the Presentation API. Apparently,
though, there are some external displays that will show up in the system as
displays, but for which you cannot show an activity on them. There is a new
isActivityStartAllowedOnDisplay() method on ActivityManager to use to determine
if a particular display is good for activities. My best guess is that this
is for foldables, but that���s just a guess.

There are a bunch of new methods on NotificationManager. A lot center around
the notion that you can allow some other app to post notifications on your
app���s behalf (a ���delegate���).

Font, FontFamily, and related classes are now part of the framework APIs.

An entire MediaPlayer2 and related classes were added, apparently as a replacement
for MediaPlayer. I have not worked with ExoPlayer, so I do not know how
MediaPlayer2 might relate to ExoPlayer in terms of API or implementation.

There is now a DnsResolver class ���for asynchronous dns querying���. In related
news, AWLTCLB (Abbreviations Would Like Their Capital Letters Back).

Uri has toSafeString(), which attempts to redact some information to make
the resulting text safe for logging purposes. My guess is that the resulting
string will then be useless for debugging, but perhaps I am being pessimistic.

They added a FileUtils class, mostly containing various copy() methods.

Environment now offers dedicated public directories for audiobooks (DIRECTORY_AUDIOBOOKS)
and screenshots (DIRECTORY_SCREENSHOTS).

PowerManager now offers methods to get the ���thermal status��� (i.e., is the device
being throttled to reduce its temperature) and register a callback for status changes.
This may be useful for games or other high-intensity apps, so they can provide
a UI indicator that explains why things are now running more slowly (so the device
can cool down).

The new settings panels seem nice, though I���m uncertain why bloggers keep insisting
this has something to do with slices. We show a settings panel via startActivity(),
so most likely they are just activities themed as a bottom sheet dialog. Then
again, it would be nice if slices were used for something. Just sayin���.

What Is Now an Ex-API

The entire preference system, outside of SharedPreferences, is deprecated.
This includes PreferenceManager and the entire preference UI. Developers
are steered towards the AndroidX replacements.

ACTION_INSTALL_PACKAGE and ACTION_UNINSTALL_PACKAGE are deprecated.
Instead, we appear to need to use the more-cumbersome PackageInstaller
API.

ACTION_NEW_OUTGOING_CALL is deprecated. There are new CallRedirectionService
and CallScreeningService APIs for apps that formerly did that sort of work
by listening for ACTION_NEW_OUTGOING_CALL broadcasts.

The ���preferred activities��� APIs in PackageManager are now deprecated.

NetworkInfo is deprecated, along with the methods on ConnectivityManager
that return a NetworkInfo.

StorageVolume.createAccessIntent() is deprecated. More importantly, it will
fail fast. This was used to request access to one of the Environment
public directories, as an alternative to needing READ_EXTERNAL_STORAGE
and therefore getting access to the entire external storage area. It would appear
that the scoped storage feature not only replaces this but is incompatible
with this. So, if you are using createAccessIntent(), you will need to add
code to take a new approach on Android Q and higher devices.

Where Things Go From Here

There are six beta releases
on the docket. Beta 4, due in early June, is supposed to have final APIs, so
I���ll have a few more ���random musings��� posts to write in the upcoming weeks.

Plus, I���ll be giving some of this new stuff a workout, and I���ll have new
material for subscribers covering Android Q soon.

Ethical Analytics, Please

A couple of weeks ago, The Wall Street Journal ran an article entitled:
���You Give Apps Sensitive Personal Information. Then They Tell Facebook.���.

More accurately, there are some developers who:

Use analytics services offered by firms (e.g., Google, Facebook) that some people believe behave unethically
Send personal data ��� such as whether the user is trying to get pregnant ��� to those analytics services

I hold out some hope that the developers fought these decisions and were overruled
by pointy-haired bosses.
Alas, I fear that too many people focus on what is being sent to their own
Web services and do not think through what is being handed over to analytics
services. And, since many places outsource analytics collection and analysis,
privacy and security concerns can get magnified.

Fortunately, or perhaps unfortunately, I have not had to deal with analytics
much personally. If I were implementing analytics, I would fight tooth and nail
to do so ethically. Here are some of the things that I would be advising.

Own the Analytics Server

The biggest problem that people will have with what the Journal reported
isn���t that apps collect private information. The problem is that they are perceived
to share that private information with the likes of Facebook and Google. Facebook
in particular has been slipshod, at best, over the years in terms of data privacy.

While you as a developer might think that Google and Facebook don���t look at your
app���s analytics data, and while it���s possible that this is true, the perception
is that you���re sharing the analytics data with Google and Facebook. After all,
you���re sending it to their servers.

Ideally, analytics wind up being managed by some server that you control directly.
The most likely option for that is for the analytics to be sent to some server
that you provision by one means or another (Docker, VPS, etc.). This implies that
you license the server and host it yourself, which is more complicated than simply
outsourcing it. However, that complexity should be manageable and would greatly
reduce the privacy concerns that other parties have with your analytics.

A theoretical option ��� one that I suspect may not exist ��� would be
end-to-end encryption (E2E) of the analytics data. The machines that allow you
to examine the analytics, generate reports, and so on do not have to be the
same machines that are collecting the analytics from your apps. In principle,
the ���middleman��� servers that collect the analytics could be collecting encrypted
payloads of analytics records. A separate ���analytics analyzer��� that you operate
yourself would collect those records, decrypt them, and let you see the results.
The ���middleman��� servers could then be an outsourced service, with the encryption
preventing that provider from getting at the actual analytics data itself.

Encrypt Data In Motion

Speaking of encryption, ensure that the app���s communications to the analytics
server is using an adequate level of TLS or other on-the-wire encryption, so that
nefarious people do not sniff on your network packets to steal information in
transit.

Only Log Constants

I suspect that most users would be reasonably comfortable with you recording
information about what ���screens��� (activities, fragments, etc.) the user visits
and including that as part of your analytics data. I suspect that most users
would be far less comfortable with you recording their location (e.g., GPS fix).
Unfortunately, there is no easy way to automatically detect that the app is
logging sensitive data.

However, you could try to enforce that you only log constants. For whatever
client-side API the analytics service offers, create a Lint rule that will complain
if you try logging something that is not a string literal or string constant.

(and hopefully there is a way for Lint rules to detect Kotlin string interpolation���)

Obviously, this would block far more benign things than logging user locations.
However:

It is safe to say that hard-coded constants are not going to contain user-sensitive data

Automatic detection is much better than relying on manual audits that may never happen

Opt-In (Or At Least Opt-Out)

My guess is that current (e.g., GDPR) or future legislation will require apps
to allow users to control whether analytics get collected or not. Ideally, you
���get ahead of the curve��� and offer this now. Ideally, it would be an opt-in
choice, so the default is that analytics are not collected. At worst, make it
an opt-out option in your PreferenceFragment or other settings screen.

Decline Unnecessary Metadata

The analytics client library might provide APIs to automatically collect lots
of data about the environment: device model, OS version, screen resolution, and
so on. We see this a lot with crash logging, but analytics may offer to collect
similar stuff.

Try to minimize this. In particular, try to stop the collection of metadata
that you are not going to need.

While fixed values like device model are not user-sensitive, too much metadata
does start to make it possible to identify users across devices. The same sort
of stuff that Panopticlick uses for Web
browsers could be collected by analytics libraries in a native Android app.

Use an Open Source Client Library (and Vet It)

Try to use services that open source their SDKs. This allows you or
some consultant to examine the library and
see if it is doing something that you or your users might regret.

With luck,
somebody else has already performed that analysis and has published a report
that you can use. Just bear in mind that any such report will be for a specific
version of the SDK, and so periodically you will need to find a newer report
or vet the updated SDK yourself.

I know that there are some open source analytics options, and so most of what
I recommend here should be possible. And, I will admit that this is more work
than a lot of organizations will want to deal with. With luck, an ethical
analytics service will emerge that emphasizes these sorts of features, and perhaps
more, to help you avoid charges of invasions of privacy.

But, in general, treat your analytics data the same way that you treat your
���real��� data. And, treat your users the same, from a privacy and security
standpoint, for both types of data. Do not consider analytics privacy and security to be something
that you can ignore��� unless you elect to skip analytics outright.

"Elements of Android Jetpack" Version 0.3 Released

Subscribers now have
access to Version 0.3 of Elements of Android Jetpack,
in PDF, EPUB, and MOBI/Kindle formats. Just log into
your Warescription page to download it or read it online,
or set up an account and subscribe!

This update adds three new chapters:

���Adopting Fragments���

���Navigating Your App���, covering the Navigation component from Jetpack

���Thinking About Threads and LiveData���

Plus, a number of bugs were fixed.

The next update to this book should be in 4-6 weeks.

The March of the API Versions Continues

Two days ago, Google made the 2019 installment
of what is likely to be an annual affair: the announcement of API requirements
for the year. In particular, a targetSdkVersion of API Level 28 is required
for distribution on the Play Store:

For new apps starting in August 2019
For updates to existing apps starting in November 2019

This is in line with what Google started in late 2017.
So, in that respect, Google���s announcement on Thursday is not surprising.
And, at this point in time, users of legacy devices are going to be stuck with legacy apps,
by and large, given the version ratchet that occurred last year.

However, you still have work to do and things to think about.

AndroidX All the Things!

It appears that 28.0.0 is the end of the line for the Android Support Library. In principle,
you can set your targetSdkVersion to 28 and use 28.0.0 for your main
dependencies, such as appcompat-v7.

However, Google is focusing their energies on AndroidX and the androidx edition
of all of this support code. I see little evidence that any bugs in 28.0.0 will
get fixed by patch releases.

And, by late 2020, you will need to have a targetSdkVersion of 29 or higher.
That will be the exclusive province of AndroidX, barring a significant change
in Google���s approach.

As result, you have about 18 months to get migrated to AndroidX, if you are not
there already. And, if 28.0.0 support library bugs aren���t getting fixed by
patch releases, you should consider migrating to AndroidX sooner, just so
that you can get updated libraries as they become available.

API Levels Beyond the Play Store

Roughly speaking, you can divide the overall Android app ecosystem into three
categories:

Cases where the user installs an app from the Play Store

Cases where the user has a Google Play ecosystem device, but they install
some app from another distribution channel (e.g., F-Droid)

Cases where the user does not have a Google Play ecosystem device

For the first category, the 2019 requirements are tied to API Level 28.

In theory, the other two categories have no targetSdkVersion requirements.
However, in Google���s announcement, they point out that:

Some other distribution channels are going to start enforcing a minimum targetSdkVersion
(they specifically cite some major app stores in China and a targetSdkVersion of 26
for 2019)

Google Play Protect will start complaining about apps with a too-low targetSdkVersion
later this year (which affects the second of the three categories listed above)

The Play Protect rules are slightly more complicated. They will raise a warning if:

The targetSdkVersion is below the API level of the device, and

The targetSdkVersion is below 26 (for 2019, climbing in subsequent years)

If you are set for the Play Store, you will be set for these other requirements
as well. If you have a more complex distribution strategy, and that strategy
involves different targetSdkVersion values��� you will need to determine if
your strategy still works.

The announcement also claims:

Existing apps that have been released (via any distribution channel) and are not receiving updates will be unaffected ��� users will not be warned when installing them.

This wouldn���t seem to be possible. Google has no reliable way to know whether
���existing apps��� are not receiving updates���, except for those distributed through
the Play Store, which already have more stringent requirements. So, it remains to be
seen what this will really mean, or if it is merely semi-random PR fluff.

Android developers are used to the annual rite of passage of thinking about the ramifications
of the next version of Android��� and we���re due for a Q fairly soon. Thursday���s
announcement reinforces what we found out 15 months ago: we now have a second
annual chore, where we ensure that we can raise our apps��� targetSdkVersion to the
required level in time.

Plan accordingly!

The Lessons from Android Things

Android Things was:

Announced in December 2016
Released as a 1.0 in May 2018
Killed off (to a large extent) in February 2019

In principle, ���OEM partners��� can still use Android Things��� for whoever Google
decides it wants to have as a partner.

Given Google���s propensity for killing off stuff, this development is not a major shock.
However, it is undoubtedly a death knell for many projects in the pipeline that relied upon
Android Things, particularly projects from smaller organizations.

So, what can we learn from this? Here are two key lessons:

Lesson #1: The Future Is Murky

You never know when the environment around you is going to change. What the world
is like today (���Android Things is awesome! Everyone should use it!���) may not
be what the world is like tomorrow (���Android Things is OK, if you���re big, you���re working
on particular products, and Google likes you!���).

For example, while Fuchsia might be a thing someday,
even after it is released, there is the potential that Google might pull the plug on
it. Conversely, if Fuchsia is released, it���s entirely possible that Google quickly
pulls the plug on Android and starts scaling back its support and services. We
just have no way to know what may transpire.

Lesson #2: Build In Business Redundancy

The ���thing��� that worried me most about the Android Things plan is how dependent
firms would become on Google. If Google is the one that is distributing firmware
updates to Things, what happens when Google decides that it does not want to do
that anymore?

I guess some people are going to find out.

The more you rely on particular suppliers or service providers, the greater your risk.
Should somebody that you depend upon become undependable, you and your customers
may suffer.

In the case of IoT, if you use a managed firmware update service like Google offered,
what is your plan if that service goes away? How will you distribute firmware updates
without them? Is it even possible for you to distribute firmware updates, or do
they hold signing keys or other credentials that block you or anyone else from updating
the firmware?

For Android app developers:

Have more than one app distribution channel. Do not assume that your users will
always be able to get your app through one particular channel, in case
that channel stops liking you.
And while you may think that your app is immune to such problems��� so did the
developers of many apps that have subsequently been banned.

A side effect of the above: always sign the app yourself. You are what you
sign. If you rely on Google to sign the app (as is the case with App Bundles),
will you be able to sign it yourself if Google elects to stop distribution
of your app? If not, even if you have another distribution channel, will existing
users be able to get updates through that channel?

Have more than one means of accepting payments. This is one of the big risks
with in-app purchases: if your channel stops distributing your app, you also lose
your revenue. Ideally, organize your app and business such that there are legitimate
reasons to purchase outside of the app (e.g., to be able to use your Web app),
so you can support both channel-based payments and payments via other payment processors.

Note that while I am using Google as an example here, they are far from the
only source of these type of disasters. Apple, PayPal, Facebook, and countless other
companies view you as a source of revenue at most. Don���t depend upon them. Use them,
but make sure that you have alternatives, either live (e.g., supporting
multiple payment processors) or at least tested (e.g., self-distribution of the app).

In the 1960���s, it was ���don���t trust anybody over 30���.
In the 21 Century, perhaps it is ���don���t trust any firm with over $1 billion valuation���.

"Exploring Android" Version 0.4 Released... Again

Any subscribers who already downloaded Version 0.4 of Exploring Android
should download a fresh copy from your Warescription page.
I had some flawed instructions in Tutorial 2 about where and how to get
the starter project. That is now fixed in the corrected books.

I apologize for the screwup!

"Exploring Android" Version 0.4 Released

Subscribers now have
access to an update to Exploring Android,
known as Version 0.4, in PDF, EPUB, and MOBI/Kindle formats. Just log into
your Warescription page and download
away, or set up an account and subscribe!

I am rebooting this book as a second-generation title, so the tutorials will
use Kotlin and the Jetpack libraries. Version 0.4 therefore is a bit of step
backwards, as 11 tutorials��� worth of changes have been applied. The rest of
the tutorials ��� or their equivalents ��� will be added in the coming
months.

Due to the extensive number of changes, changebars were turned off for this
update.

Also, there is a bug in my full-text search support, where this book is not
showing up. I will fix that in the coming days. However, the online reader
does work, so subscribers can read the book that way in addition to downloading
the book.