The Joy of a Fast Photo Proximity Search in Lightroom

Nikon D4 + Nikkor 24-70mm f/2.8 @ 31mm — 1/60 sec, f/3.5, ISO 6400 —
map & image data — nearby photos

Boring Sunset Over Kyoto

the Shogunzuka overlook (将軍塚)

— Today —

I popped up to Kyoto's Shogunzuka Overlook (将軍塚) today for the first time
since March, hoping
for a nice sunset.

As you can see above, I didn't get it.

However, due to my blog's
proximity search feature, we can follow the “nearby photos”
link under the photo to see other shots from Shogunzuka that I've posted over the years, including:

Nikon D700 + Nikkor 70-200mm f/2.8 @ 70mm — 1/320 sec, f/6.3, ISO 5000 —
map & image data — nearby photos

June 2012




Nikon D700 + Voigtländer 125mm f/2.5 — 1/320 sec, f/11, ISO 320 —
map & image data — nearby photos

November 2011




Nikon D200 + Nikkor 70-200mm f/2.8 @ 70mm — 1/1500 sec, f/3.2, ISO 640 —
map & image data — nearby photos

July 2007






Nikon D700 + Voigtländer 125mm f/2.5 — 1/8000 sec, f/2.5, ISO 200 —
map & image data — nearby photos

November 2011






Nikon D700 + Nikkor 24-70 mm f/2.8 @ 70mm — 1/200 sec, f/2.8, ISO 3200 —
map & image data — nearby photos

November 2008

I added the proximity-search feature to my blog several years ago, and I
use it often in various ways because proximity search can be extremely
useful, but my blog proximity search is obviously limited to photos I've
actually published on my blog. My full catalog of photos in Adobe Lightroom
is much, much bigger.

Over the years I've written a bazillion (~45) plugins for Lightroom
that supports a photographer in all kinds of helpful ways,
but one of the first (already five
years ago!) was my Lightroom catalog Proximity
Search plugin.

To take advantage of proximity search, your photos have to be geoencoded to begin
with (that is, each photo must be associated with latitude/longitude
coordinates of where it was shot). At the time I first released the plugin,
there was no way to geoencode photos within Lightroom, so one had
to somehow take care of it before importing photos into your catalog. It was a pain.

But, while working on that plugin I figured
out a tricky way to build another plugin that
allows you to geoencode within Lightroom, and my Geoencoding Support
plugin was released a couple of weeks later;
it remained the only way to geoencode within Lightroom until Adobe
introduced the Map Module last year. (My Geoencoding Support plugin is all
the more useful now that Lightroom supports location editing, because it
extends the usefulness of location information far beyond what Lightroom
provides.)

Anyway, I was happy that I could do a
proximity search on my photos, but the unfortunate reality was that
Lightroom's catalog interface for plugins was simply too slow, creating
a high barrier to use: I'd use it sparingly, only
when it was worth the several-minute wait for a
result.

Still, even today it's better than what Lightroom itself now supplies,
at least on my machine. If I set Lightroom's Map
Module to an area I'm interested in, then switch to Library and select
“All Photographs”, then go back to Map to see which ones show up,
Lightroom completely locks up for eight minutes. Locks up. Eight minutes. Painful.

(I hear that Lightroom is faster in this respect on Windows; someone who
tested my catalog said the lockup was only two minutes there.)

I don't understand why plugin access to the Lightroom catalog is so
slow, but to gain some insight I tried accessing the Lightroom-catalog
SQLite database directly. The results? The same search that locked up
Lightroom for eight minutes took 0.6 seconds.

Even though the very first Lightroom-related article on my blog was a
post in 2006 about accessing the Lightroom
database directly, I've purposefully stayed away from doing so within
my plugins as a matter of principle, keeping instead to the official plugin infrastructure.

But come on, an 800-fold speedup is just too much to pass up, especially
for a feature that blooms in usefulness when you can use it on the spur of
the moment with little friction. So for the first time in my
plugin-development life, I did an end around Lightroom's interface, adding
a “Fast Full-Catalog Proximity Search” plugin-extra feature to my Proximity
Search and Geoencoding Support plugins last week. This is the fast search
that I mentioned on my
previous post.

The search is nominally invoked from the “File > Plugin Extras >
Geoencoding Support” menu, but on my system I mapped it to a keyboard
shortcut, so while looking at an image that's geoencoded, a quick tap
brings up this dialog:

Upon activation, the plugin goes outside of Lightroom to grab the data,
then import just those results back in, creating a collection with them. It took just a few seconds to isolate the 548 photos from that general area over the years.

One bummer about the workaround that achieves this: it doesn't work on
Windows, nor on Lr4 or earlier, so in those situations the “Fast
Full-Catalog Proximity Search” plugin-extra item reverts to the slower,
official, much-less-compelling method. )-:

(If you can figure out a way to give sqlite3.exe readonly access a locked database, please let me know.)

Still, if you're using Lr5 on a Mac and find this useful, please let
Adobe know; perhaps they'll add this kind of thing directly into Lightroom...
where it will presumably work on Windows as well.