Mark L. Murphy's Blog, page 22

Android Summit 2019!

I���ll be back at the Android Summit this year,
being held August 14-15 in the Washington DC area!

I am leading a short workshop: ���From RxJava to Coroutines���. As the name suggests,
we will take an existing small RxJava-based app and convert it to using Kotlin���s
coroutines:

Replacing Completable and Single with simple suspend functions

Replacing Observable with channels and flows

Replacing tests with��� other, more coroutine-y tests

I am honored and moderately frightened that the organizers put my workshop immediately
after Chet Haase���s opening keynote. I promise to do my level best, so attendees avoid
any post-Chet letdown.

The Summit has expanded to two days this year and has quite the lineup of speakers.
If you can get to the DC area, we���d love to see you there!

Need a Magic Constant? Choose Something Obscure

We all have various constants in our code, whether they are integers for use
with startActivityForResult(), strings for Logcat tags, and so forth. Most
of these are internal to our code and do not affect others.

Sometimes, though, we have an API that we expose that others might call, where
certain parameters are up to the caller��� except for some magic constants. For
example, you might have a key/value data store, where you want to reserve some
keys for your internal use, but others can stuff whatever they want into that data
store using their own choices of keys. Those reserved keys are magic constants.

(IMHO, try to avoid such APIs, keeping your internal data separate so there
is no chance of a key collision��� but sometimes this API structure is unavoidable)

So, the caller of your API can choose any value for their keys, except certain
magic constants. In that case:

Practice defensive programming and validate that a caller-supplied key is
not one of your magic constants,

Document what the magic constants are, and

Choose obscure values

When it comes to NotificationChannel, we got two out of the three.

NotificationChannel has a DEFAULT_CHANNEL_ID constant. This is a magic constant,
as you cannot use it as the ID of your own channel. If you try,
you get a somewhat cryptic stack trace.
But, that stack trace is because of the defensive programming, so that���s good.
And the magic constant is documented, at least to an extent.

However, the value of DEFAULT_CHANNEL_ID is "miscellaneous".

That���s a reasonably common English word, one that somebody might well choose
without noticing that it is not an eligible value.

Particularly with strings, you have a near-infinite space to choose magic constants
from. So, it is easy to choose a magic constant that is unlikely to collide with
any value that others might choose:

Use a UUID

Use a passphrase generated by diceware

Use a series of digits from a random number generator

And so on

Even if memory use is a consideration, it would be better to choose a random
six-digit number than to use a common word. There are lots of obscure values
that we could use that are shorter than "miscellaneous".

After all, neither the compiler nor the OS really cares about the actual value.
Your own code does not care about the individual characters inside of the magic
constant. It just needs to know that it can look up things using that magic constant
and get what it expects.

Magic constants are part of your API and they need to be designed along with the
rest of the API. It���s just that whereas normally you want things to be human-readable,
this is one case where you do not. Or, at the very least, choose a less-common
human-readable word, such as "axolotl".

(NOTE: no actual axolotls were harmed in the creation of this blog post)

Leaky APIs are Leaky

For years, I have been advising developers against doing the sort of thing
seen in this Stack Overflow question,
where the developer is hacking in arbitrary SQL to a query() on a
ContentResolver. This approach is deeply risky, for two reasons:

It assumes that the ContentProvider is using a SQL-compliant database, such
as SQLite, and can process arbitrary SQL queries

It assumes a particular way that the ContentProvider is assembling the
SQL statement to execute, in terms of how the various query() parameters
get concatenated or otherwise combined

In this case, apparently, the rules changed with Android Q, and the previous
hacked SQL no longer is valid.

Tactically, the problem lies with the developers who try hacking in SQL.
Strategically, the problem lies with the developers of MediaStore and
other ContentProvider implementations. If you allow this sort of hack,
this sort of hack is going to get used.

In other words, your API is the realm of the possible, which may or may not
line up with what you think your API is. Any capabilities that your API leaks
may get ruthlessly exploited by developers. If consumers of your component are
doing something, and it works, that ���something��� is part of your API, whether
you like it or not, because you leaked that capability.

We see this sort of thing all the time in the Android SDK, such as all the
reflection games that developers played to access private classes, methods,
and fields. Today, though, I���d like to focus on how the publisher of the API
should be dealing with this sort of thing. There are several steps that
you should consider taking:

Document Your Expectations: The documentation for your API should explain,
in detail, what you are expecting and supporting for the inputs to that API.
In the case of a ContentProvider, you should be documenting what the valid
values are for things like the group-by clause and the order-by clause.

Sanitize Your Inputs: Where practical, add runtime support to validate
that the inputs match what you documented and reject things that do not (e.g.,
throw an IllegalArgumentException). Make sure that your form of rejection explains
what the developer is doing wrong, and ideally explains how to fix it.

Monitor For Transgressions: Keep tabs on how your API is used, by watching
for references to it in developer support sites, whether those sites are your
own (e.g., support board, GitHub issues) or not (e.g., Stack Overflow). If you
see developers misusing your API, point it out, and make notes as to how they
are misusing it so you can take further steps.

Document (or Support) the Anti-Patterns: Where you find those transgressions, document
them, explaining why they are not supported. Or, find some way of supporting
what the developers are trying to do, either via their approach or something else,
and document that. One way or another, you need an official response to the misuse
of your API, lest that misuse spread to other developers, fueled by these support sites.

Write Lint Rules: In some cases, you cannot sanitize the inputs at runtime
(e.g., would add too much overhead). In those cases, if your API is in the form
of an Android library, consider adding a Lint rule to it. Timber users are familiar
with this, as Timber ships with Lint rules,
such as the legendary ���Using ���Log��� instead of ���Timber������ message. This can help
steer developers away from known anti-patterns proactively.

Break Things, Carefully: At some point, you may decide to change your API
in a way that breaks some hacks, such as what changed in Android Q to MediaStore.
Where possible, try to have custom error messages that explain the change and
steer the developer towards supported alternatives, rather than just failing with
some generic error message (e.g., near "GROUP": syntax error) that might cause
developers to think that your API itself is broken.

The worst thing to do is to ignore all of this. That���s largely what has happened
with the system-supplied providers. As a result, lots of developers are relying
on leaked capabilities (e.g., hacking in arbitrary SQL) and those developers are
screwed when the API changes such that the capabilities are no longer leaked.

Playing ���API cop��� is not the most fun role of a developer advocate, but for
prominent APIs with lots of developers using them, it is a necessary role.

Android Q, uiMode, and Configuration Changes

If you are a regular follower of my blog posts, I have complete confidence that
you are a fine Android app developer and that all of your apps support configuration changes.

However, there seem to be a lot of apps out there that don���t.

In many cases, that is because the apps do not support orientation changes.
The UIs are locked to portrait or landscape. Orientation changes are the most
common source of configuration changes on phones. The other configuration
change triggers are less common, with the second-most common one probably
being changing locale. So, some developers think they can skip dealing with
configuration changes.

Android Q puts far greater emphasis on the uiMode configuration change, though.
That gets invoked whenever the device switches between normal and dark mode.
The user might do that manually, and it might kick in when the device battery
gets low. It would not shock me if some devices offer other options for automatically
toggling into dark mode, just as stock Android 9.0 offers time-based
���night light��� mode (a blue light filter).

And, if your app does not handle configuration changes properly, your UI will
lose context when the device switches between normal and dark mode, as your
visible activities and their fragments get destroyed and recreated. Symptoms
include:

Losing information in your widgets, because that information is not part
of automatic saved instance state logic (e.g., enabled status)

Forgetting about a dialog, because you are showing it directly instead of using a
DialogFragment, so the dialog vanishes after the configuration change

Reloading data from disk or the network, because you are not using
a ViewModel, retained fragment, or other way to keep that data around across
the configuration change

So, even if your UI is locked to a single orientation, please add configuration
change support. Android Q gives us another easy way to test configuration
changes: add the ���Dark Theme��� tile to the notification shade, then toggle between
normal and dark modes using the tile.

Also, if you are manually managing configuration changes via android:configChanges,
there is a good chance you will want to manage uiMode manually as well. For
example, if your app involves continuous playback (e.g., a video player), you
might not want to interrupt playback just because the device switched between
normal and dark mode.

"Elements of Android Q" Version 0.4 Released

Subscribers now have
access to Version 0.4 of Elements of Android Q,
in PDF, EPUB, and MOBI/Kindle formats. Just log into
your Warescription page to download it,
or set up an account and subscribe!

The chapter on scoped storage was overhauled
again, owing to the substantial
changes that showed up in Q Beta 4.

Two short new chapters were added on dark mode and
gesture navigation, replacing previous sections in the
���Other Changes��� chapter.

That ���Other Changes��� chapter, and bits of other chapters, were
updated for Q Beta 4. Material was added on the audio capture APIs as well.

At this point, I am expecting two more updates to this book:

One when Android Q ships as Android 10, to make any tweaks based on the final
shipping versions plus perhaps add a bit more of new material

One when the next generation of Pixel devices ships, to cover any Android Q
things that are at least initially unique to those devices

It is possible that there will be another update sooner than those, if, say,
Q Beta 5 makes unexpected changes to Android Q. That is unlikely.

DevFest DC Update!

On Friday, I will be speaking at
DevFest DC on
the changes to storage in Android Q.

That���s not new.

What is new is that my presentation has been moved to 10am from its original
~4pm slot.

So, if you were aiming to attend my talk, make sure you get there for the morning
sessions!

"Elements of Android Jetpack" Version 0.4 Released

Subscribers now have
access to Version 0.4 of Elements of Android Jetpack,
in PDF, EPUB, and MOBI/Kindle formats. Just log into
your Warescription page to download it or read it online,
or set up an account and subscribe!

I apologize for the long time between updates of this title ��� I was focusing
on Exploring Android, Elements of Android Q, and helping out
customers.

Besides many bug fixes, this update adds several new chapters:

���Adding Some Architecture���

���Working with Content���, covering the Storage Access Framework

���Collecting Preferences���

���Requesting Permissions���

���Handling Files���

���Accessing the Internet���

���Storing Data in a Room���

It also updates everything for Android Studio 3.4.1.

The next update for this book most likely will be in August. Between now and
then, I will be working on Elements of Kotlin and other good stuff
for you!

The Death of External Storage: The End of the Saga(?)

If Q Beta 4 really does have the final APIs, then we may now have the final
implementation of scoped storage. While external storage as we know it is still
going away, it will not be for a while, and the user experience should be reasonable.

So, let���s review where we are now, with another set of fictionally-asked questions
(FAQs):

What Are the Options?

Apps can either have normal or legacy storage.

With legacy storage, everything behaves as it did in Android 4.4 through 9.0:

You can use getExternalFilesDir() and similar directories without permissions

You can work with the rest of external storage if you hold READ_EXTERNAL_STORAGE
or WRITE_EXTERNAL_STORAGE

Without legacy storage, apps still can use getExternalFilesDir() and similar directories without permissions.
However, the rest of external storage appears to be inaccessible via filesystem APIs. You can neither read
nor write. This includes both files created by other apps and files put on the device
by the user (e.g., via a USB cable).

It is conceivable that there are some types
of content that are still visible via the filesystem, as the documentation has:

An app that has a filtered view always has read/write access to the files that it creates, both inside and outside its app-specific directory

So, I cannot rule out scenarios where the app can work outside of getExternalFilesDir()
and similar directories via the filesystem APIs. I just have not found one yet.

What Changed From Q Beta 3?

Q Beta 3 also had two modes: legacy and sandboxed. Apps with sandboxed external
storage could read and write everywhere on external storage��� because they were
not working with the real external storage. Instead, they would read and write
from a sandbox. While this allowed existing code to keep working, it was costly
from a user experience standpoint, as many users would not know to wander
into the Android/sandboxes/ directory to find an app���s sandboxed edition of
external storage.

Now, instead of apps having a ���sandboxed��� separate bit of external storage, they
have a ���filtered��� view of the real external storage.

What Changed From Q Beta 1 and 2?

Too much changed to list here. Can���t we focus on more pleasant topics?

What is the User Experience?

Whether apps have normal or legacy external storage does not matter to the user,
to a large degree. Files show up wherever they would have shown up originally.
This is particularly important for apps with a lower targetSdkVersion that may
never get updated ��� users can use those apps the same way they have for
years.

What Am I Supposed to Use, Then?

You are welcome to continue using getExternalFilesDir(), getExternalCacheDir(),
getExternalMediaDirs(), getExternalCacheDirs(), and getExternalFilesDirs(),
if you were using those before.

However, the Storage Access Framework (e.g., ACTION_OPEN_DOCUMENT) is the primary way
that apps should work with user-supplied content.

Apps that have a focus on media ��� audio, video, and images ��� can use
the MediaStore. Note, though, that you need READ_EXTERNAL_STORAGE to be able
to see other apps��� content in the MediaStore.

One thing that you are not supposed to use is an <intent-filter> supporting
the file scheme. You will not be able to read files written by other apps, so
if you get a Uri like that, probably it is useless to you. The technique
that I wrote about previously,
to use <activity-alias> to support file only on older devices, should still
work.

I Don���t Like Change ��� How Do I Stick With What Worked Before?

For Android Q, you can add android:requestLegacyExternalStorage="true" to your
<application> element in the manifest. This opts you into the legacy storage
model, and your existing external storage code will work.

Technically, you only need this once you update your targetSdkVersion to 29.
Apps with lower targetSdkVersion values default to opting into legacy storage
and would need android:requestLegacyExternalStorage="false" to opt out.

What Happens Next Year?

The documentation still has:

Scoped storage will be required in next year���s major platform release for all apps, independent of target SDK level.

IMHO, this is unwise. Saying that it is required for targetSdkVersion 29 and
higher is reasonable. Saying that it is required for all targetSdkVersion
values means that lots of legacy apps will crash, as while they hold WRITE_EXTERNAL_STORAGE,
they would be ineligible to write to previously-valid locations.

My hope is that Android R will ���only��� deprecate and ignore
developer-supplied android:requestLegacyExternalStorage values,
while setting the defaults to be:

true for targetSdkVersion 28 and older

false for 29 and newer

That ensures the maximum compatibility with legacy apps while still enforcing the
new rules for actively-maintained apps.

It is unlikely that Android Q itself will change, though I cannot rule out an
Android 10.1 (Q MR1) that messes with this stuff.

So, by August 2020 or thereabouts ��� whenever Android R ships ��� you
will need to adapt to the new normal.

The idea is that you start adapting now. For some apps, switching to the
Storage Access Framework will be easy. For some apps, it will be painful.
Do not wait until 2020. Start migrating your apps now to using the alternative
approaches. The Storage Access Framework (mostly) works back to Android 4.4, for
example, and so many apps will have access to that set of Intent actions.

You can create a dedicated build type or product flavor where you set
android:requestLegacyExternalStorage="false" and opt out of the legacy storage
support. There, you can see what breaks and start creating plans to fix it.

Hey, Why Am I Seeing Deprecation Warnings?

If your code refers to Environment.getExternalStorageDirectory()
or Environment.getExternalStoragePublicDirectory(), you will see that they are
deprecated. They still work, but the deprecation warning is yet another nudge
to remind you that you need to stop using those.

Once you no longer have the legacy storage model, those directories are unusable,
which (presumably) is why they are deprecated.

How Can My Library Know What To Do?

In a library, you do not control whether the app is in normal or legacy mode.
If you need different code for those two cases, you can call
Environment.isExternalStorageLegacy(), which will return true if the app
is in legacy mode, false otherwise.

What Happens When the App is Uninstalled?

Any files that you wrote to external storage in getExternalFilesDir() get removed,
as normal.

If you were thinking of switching your Environment.getExternalStorageDirectory() and
Environment.getExternalStoragePublicDirectory() code to use getExternalFilesDir(),
that will work, but the cost is that your files go away when the app is uninstalled.

For files that are owned by the user and should remain after your app is removed,
use the Storage Access Framework or MediaStore.

So, There Is Nothing I Need to Worry About Today?

Well, there may be. Some script-kiddie workarounds to avoid existing limits may
cause your app to break on Android Q. Here are two examples:

Inhibiting FileUriExposedException

Back in Android 7.0,
Google added logic to StrictMode to see if you have a Uri in your Intent
that has the file scheme. If it does, and you use that Intent for something
like startActivity(), your app would crash with a FileUriExposedException.
The right solution is to use FileProvider or otherwise get a content
Uri. However, some developers elected to reconfigure StrictMode to block
that check and prevent the exception.

Technically, that hack still works, in that you should not crash with the exception.
However, apps that opt out of the legacy filesystem support cannot access your
file. So they crash when they try to use your Uri,
and your users lose whatever functionality you were trying
to offer by starting that third-party activity.

Reading DATA

Some developers, particularly for ACTION_PICK from the MediaStore, would
query the MediaStore and read the DATA column to try to get a filesystem path
corresponding to the picked content. That has not been reliable in quite some
time, but I am sure that some developers are still using it.

Well, in Android Q, the DATA column is blocked, and you will not be able to get
the values.

You will need to use the Uri that you get as intended, with a mix of ContentResolver
(e.g., openInputStream()) and DocumentFile.fromSingleUri() (e.g., getName()).

Is There Anything Else? This Is Getting Rather Long.

Keep an eye out for future Q beta releases, as while the API is supposed to be
stable, there still might be functionality changes.

If I stumble upon any new problems, I���ll write about them. If you encounter scoped storage bugs
new to Q Beta 4, ping me on Twitter or
reach out via email.

Random Musings on Q Beta 4

Each time Google releases a new developer preview beta, I putter around
the API differences report
the high-level overviews,
and even the release blog post,
to see if there are things that developers should pay more attention to.
I try to emphasize mainstream features that any developer
might reasonably use, along with things that may not
get quite as much attention, because they are buried in the JavaDocs.

We are now up to Beta 4, so this extends upon my notes for
Beta 1,
Beta 2,
and Beta 3.

Q Beta 4 is supposed to have the final APIs. However, I
was surprised as to how much is different in Q Beta 4.

Hey, Can We Call It API Level 29 Now?

Yes, we can. We no longer have to dance around the API level. Build.VERSION_CODES.Q
has the proper value instead of the 10000 placeholder. And you are able to
publish apps with targetSdkVersion set to 29 if you so choose��� though you
may or may not be ready for that just yet.

So, What���s Up with Scoped Storage?

In a nutshell, it changed again, based on the description in the docs.

The sandboxes are gone. Instead, there is one unified external storage
location that all apps and the user sees. However, apps only see their
own files in external storage, in general. So, instead of scoped storage
being sandboxed, it is filtered instead.

From the user���s standpoint, this should be simpler. Now files that apps write
to external storage will be where they had been previously. This is really
important for legacy apps that are not being regularly updated and which might
never adapt to the Storage Access Framework. Yet, at the same time, apps should
still unable to manipulate other apps��� files through the filesystem, meaning that
users still get enhanced security.

Also, apps are still able to opt out of the filtered view, at least until next
year sometime, when targetSdkVersion 29 becomes required for the Play Store
and select other app distribution channels.

My biggest fear right now is that there have been so many changes over the past
three months that we are at risk of more bugs than we might otherwise have had.

Note that getExternalStorageDirectory() and getExternalStoragePublicDirectory()
on Environment
are now deprecated, to further steer developers towards using the Storage Access
Framework or MediaStore.

I will be ���kicking the tires��� on this stuff today and will write up more about
the current state of scoped storage tomorrow.

What Else Got Deprecated or Removed?

The biggest surprise ��� and a pleasant one ��� is that android:sharedUserId
is deprecated. Moreover, it is planned to be dropped entirely in some future
release. For device manufacturers, android:sharedUserId was a handy shortcut
for data sharing, but even there it really was not a particularly good practice.
For ordinary app developers, android:sharedUserId was a
footgun, one that I have been advising
against people using since the very beginning. Having an app suite is awesome,
but please use IPC for data sharing, not some sort of shared file that may or may
not be managed properly for simultaneous multi-process access.

DownloadManager deprecated some things, as a side effect of scoped storage:

addCompletedDownload()
allowScanningByMediaScanner() on DownloadManager.Request
setVisibleInDownloadsUi() on DownloadManager.Request

Items listed in the new MediaStore.Downloads collection will be what appears
in the Downloads UI now. So, if you have content that was not downloaded to the
Downloads/ directory by DownloadManager, and you want it to appear in the Downloads
UI, you need to write it to a Uri supplied by MediaStore.Downloads, I guess.

A bunch of fields from MediaStore.MediaColumns were removed, including ORIENTATION,
DURATION, and DATE_TAKEN. If your app has been querying on those columns, they
may no longer be available to you.

Also, MediaPlayer2 and related classes vanished without a trace.

They Didn���t Add Anything New, Did They?

Well, yes, they did, a lot more than I would expect at this late stage. Here are
a few things of note:

Activity now has onGetDirectActions() and onPerformDirectAction(). A
DirectAction is an opaque identifier of something that somebody can do with
or in the activity. The idea is that an activity can supply the available actions
in onGetDirectActions(), and a VoiceInteractor can let the user act upon
one. The chosen action then gets delivered to onPerformDirectAction(), for
the activity to go do something. Unfortunately, there isn���t a lot to go on
in terms of how all of this is supposed to work. Hopefully, more documentation
is forthcoming.

Intent now has getIdentifier() and setIdentifier(). The identifier
���is an arbitrary identity of the Intent to distinguish it from other Intents���.
My assumption is that this does not affect Intent routing and is basically a specific
���extra��� bit of data that gets passed along. It is unclear what the value of this
is over, well, extras.

Intent also adds CATEGORY_APP_FILES to identify file managers.

There are four new DisplayMetrics screen densities: 140, 180, 200, 220.
Those are the first new low-end densities we have had in years, and it is unclear
what hardware would have such screens. It���s possible this is tied to desktop mode,
if those are meant to be used for certain external displays. Regardless, most
developers will not need to worry about these, as Android will scale mdpi or
hdpi drawables for you. But, if you have other code that cares about these
DENSITY_ constants on DisplayMetrics, you have four more to deal with.

android:hasFragileUserData is a new manifest setting (I���m guessing on <application>).
���If true the user is prompted to keep the app���s data on uninstall���. This
seems ripe for abuse, but I can see where it might be useful for some apps.

What Was Renamed?

One common thing late in the beta sequence is to have classes and methods be renamed,
as somebody lost a fight in a code review or something. Of particular note:

isExternalStorageSandboxed() on Environment is now isExternalStorageLegacy()

ContentResolver.TypeInfo is now ContentResolver.MimeTypeInfo

They Must Be Done With Changes Now, Right?

Perhaps not.

There is still a reference to
RecoverableSecurityException being enabled in a future beta.
Certain types of I/O might throw that exception,
which contains a RemoteAction that you can use to bring up some UI to help
recover from that exception. In particular, if you do not have rights to modify
some media, you might get a RecoverableSecurityException, where the RemoteAction
would bring up UI to allow the user to grant you write access. It is unclear if
this might show up in a future Q beta, whether it already is in Beta 4 (without
the docs being changed), or what.

OK, So Where Do We Go From Here?

In theory, there should be few changes from here on out in terms of the API. There
may yet be bug fixes, which is good, as a bunch of the bugs that I filed are still
reproducible on Q Beta 4. But if you have been holding off testing your app with
Android Q, waiting for things to stabilize, you should not wait any longer. I
expect Android Q to ship in final form to end users in 2-4 months, and you will
want to make sure that your app will survive that upgrade.

As for me, I will update Elements of Android Q shortly to reflect some
of these changes and explore more dusty corners of this release, such as desktop
mode.

The Storage Access Framework: Counterpoints

Last Friday, XDA Developers ran an article entitled
���The Storage Access Framework is the only way for apps to work with all your files in Android Q. And it���s terrible.���.
As you might expect from that title, the author rails against the Storage
Access Framework. The commenters do as well, by and large.

I agree that the Storage Access Framework API has issues, and I agree with
some of the arguments posed in that article. Other arguments, though, seem to
reflect the fact that the author develops a file manager, and therefore takes a
particular view of Android app development that does not reflect app development
in general.

So, let me offer some counterpoints.

With Q, Google is introducing (and requiring) ���Scoped Storage,��� which makes Android work more like an iPhone, where storage is isolated to each app. An app can only access its own files, and if it���s uninstalled, all its files are deleted.

Note the comparison to the iPhone. We���ll come back to that.

SAF has been available since Android 5.0 Lollipop, but developers tend to not use it unless required, as it has a difficult and poorly documented API, a poor user experience, poor performance, and poor reliability (largely in the form of device vendor-specific implementation issues).

The API for getting a Uri from the Storage Access Framework is fairly trivial.
The API for using the Uri is disjointed, with a mish-mash of ContentResolver
and DocumentFile being needed if you want something reasonable. And the documentation
is limited, but that is fairly common. But once you find the right methods
on those two classes, the API is not especially difficult and resembles that
of traditional file I/O.

(hey, Yi��it, if you���re reading this: how about a sexy new StorageX library in the
Architecture Components? ����)

The SAF API has gaps, to be certain, such as:

There is no ACTION_CREATE_DOCUMENT_TREE analog for ACTION_CREATE_DOCUMENT
and ACTION_OPEN_DOCUMENT_TREE, leading to some developers to
try hacks that prove to be unreliable

There is no way to specify that we need a read-write document in ACTION_OPEN_DOCUMENT,
leading to users getting screwed by Google���s ���Audio��� documents provider

Performance will be worse than with the filesystem API, but whether it matters for
the app will vary ��� more on this later.

And, ideally, Google would have been doing more from a testing standpoint, such as:

Writing a conformance test suite, so DocumentsProvider implementations can be
easily tested to determine if they comply with expectations; and

Applying that conformance test suite to any pre-installed DocumentsProvider
implementations as part of the Compatibility Test Suite

The most obvious user-facing change with SAF is the experience of granting an app access to storage. For an app to get access, it makes a request to the OS, which then displays a directory chooser screen. On this screen the user selects the root of a folder hierarchy in which that app will be able to read and write files.

Few apps need access to the complete contents of an arbitrary directory on external
or removable storage. Many apps do not need access to external or removable storage
at all. Many of those that do can get by with just getExternalFilesDir(). And
even those that need access to arbitrary content often need just a single file.
For this,
the resulting UI is not significantly different than the ���file open��� and ���file save-as���
dialogs that people have been using in desktop operating systems for decades.
Even the dialog that the author decries here is the same as the ���choose directory���
dialog that desktop users encounter from time to time.

File I/O performance takes somewhat of a hit under SAF, but the most outstanding problem lies in file directory operations, where it���s ~25 to 50 times slower than the conventional file access possible in Pie.

I have not run the benchmarks, but that result does not shock me. There are two
rounds of IPC for every SAF API call (your app to the SAF, then the SAF to the
DocumentsProvider). That is going to add overhead, compared to doing the same
thing with the filesystem.

Few apps will need to do this, though. File managers do, as they are not only working
with arbitrary directories on external/removable storage, but arbitrary
directories with arbitrary contents. Few apps are file managers or have the
storage-access characteristics of a file manager. Plus, this is the sort of thing
that the SAF could improve upon over time, such as via more aggressive caching.

An even greater performance issue is that some apps will have to copy files to their local ���scoped storage��� area before they are able to work with them.

I agree. I have been begging library authors to support streams for years.

Many Android apps take advantage of the amazing number of open-source Java libraries in the developer community, and these libraries commonly require direct filesystem access to work.

The real problem comes from libraries that need random access
to the contents of files. Those are basically SAF-resistant and will require data
copying. However, some libraries
take a File as a parameter, then turn right around, open a FileInputStream, and
work with that. Such libraries should be able to accept an InputStream as an
alternative to a File, and if they don���t, that could get fixed. For open source
libraries, code contributions may be welcome.

Of course, more of these libraries would already have been adapted had more developers
asked them to adapt rather than using script-kiddie workarounds to try to avoid
using a Uri correctly. Just sayin���.

Google touts the security and privacy benefits of this change, but technically speaking, there is no improvement.

Recall that the author compared scoped storage to the iPhone. Well, in the eyes
of the privacy and security experts that I follow, the iPhone is the gold standard, and Android is
a trash fire. People hold fundraisers to get iPhones in the hands of at-risk
people who otherwise would only be able to afford an Android device.
I hope that privacy and security experts will appreciate Android���s move towards
being more iPhone-like in this area.

Being able to control storage access on a more granular basis is a fairly massive
win. Just because App X needs access to one particular file on external storage
does not mean that App X should be granted access to all of external storage.
Yet that is what the classic READ_EXTERNAL_STORAGE and WRITE_EXTERNAL_STORAGE
permissions offer.

When you grant an app access to the root directory of your storage via SAF, it can read, write, and send any file it wants to its nefarious developer in the exact same fashion it could when you granted an app access to storage in Pie.

Few apps need access to the complete contents of an arbitrary directory on external
or removable storage. Hopefully, users will think through whether they want to
grant that right when an app asks for directory access.

There is more work that could be done to improve privacy and security here.
For example, Android needs a Settings screen where users can review what durable storage access
grants are outstanding and be able to revoke them. Users ought to be able to control
whether access is durable or not at the point of choosing the directory (or even how long the
permission grant should be, with values between ���session only��� and ���forever���).
A warning dialog
might be warranted if the user grants durable access to a root directory, to help ensure
the user understands the ramifications of that choice.

Ideally, in the long term, most users will get the ���gimme access to everything forever���
dialog once or twice a year. Hopefully everything else will get handled with things that are
more tightly scoped in terms of breadth (e.g., individual files) or time (e.g., access
for less time than ���forever���).

Some power users are going to get irritated, though. I am not going to deny that.

The only ���security improvement��� comes about because it���s now a more arduous process for a user to do this.

Users have used file dialogs in desktop programs for decades. It is possible that
those dialogs are a cause of global warming or tooth decay or something, but I doubt it.

The official stated reason in the Android Q beta documentation is to ���give users more control over their files and to limit file clutter.���

That part of the documentation needs to get fixed. The revisions in Q Beta 3
basically eliminated that rationale.

If Google is truly concerned about giving users more control over files and clutter, they should architect a solution that directly addresses that, rather than falsely branding the current Android Q design as such an improvement.

It is an improvement. It may not be the best possible improvement; there will
be collateral damage stemming from this particular approach. And, as I have written about
extensively, I think the rollout had loads of issues. But, it is an improvement.
And while it may sound hyperbolic, I can honestly say that lives might be saved
in the long run by this change.