Jeremy Keith's Blog, page 21

Websites in the dock

I updated my Mac to the newest operating system, Sonoma. I did this to try out the new ���add to Dock��� feature in Safari. It���s like the ���add to Homescreen��� action on iOS.

Before I get into what���s good, I have to start by ranting about a big problem on both desktop and mobile: discovery.

First of all, you have to know that a square with an arrow sticking out of it means ���share���.

Okay, I can buy it. It���s no better or worse than the three horizontal lines of a hamburger icon, or the three horizontal dots of a kebab icon. And whereas the hamburger and kebab icons are used as a catch-all cupboard to sweep all your rubbish into, at least this icon has a specific meaning. It means share, right?

Well, it used to. But now it���s also home to ���add to Homescreen��� and ���add to Dock���. Neither of those actions are sharing.

Stretching semantics, I suppose you could say you���re sharing something with yourself.

Anyway, this is the biggest issue with progressive web apps on both iOS and Mac. Regardless of how well they work, there���s not much point if most people don���t know the option exists.

(Update: Jen rightly points out that you can also get to ���add to Dock��� from the file menu. Doh! How did I miss that���)

Discovery aside, I was interested to see how Safari handles web apps on desktop compared to how Chrome does it.

I���ve had one or two web apps in my dock for a while, installed through Chrome. Google Meet is one of them. I use it quite a bit, and honestly it feels no different to using a native desktop app.

One annoyance is that the Chrome browser also automatically launches whenever I launch the Google Meet icon in my dock. This wouldn���t matter if Chrome were my default browser, but I use Firefox. So whenever I���m using the Google Meet web app, there���s a Google Chrome icon hanging around in my dock, saying ���gizza job, I can do that.���

I opened Google Meet in Safari and then selected ���add to Dock��� from the square with an arrow sticking out of it. Then I quit Safari. I was delighted to see that when I launched the Google Meet web app from the dock, it didn���t automatically launch Safari! Excellent!

Even better, links within a Safari-installed web app respect your default browser choice. What I mean is, previously when I had Google Meet installed via Chrome, if I clicked an external link in Google Meet it opened in Chrome. But now with the Google Meet installed via Safari, external links open in Firefox, my browser of choice. Very good!

But the Safari-installed version of Google Meet is, alas, over-zealous with permissions. I have to grant access to my microphone and camera every single time I launch it. Previously���with the version installed via Chrome���it remembered my choice.

Now I don���t know if the behaviour in the Safari-installed version is a deliberate choice made for security reasons, or if it���s a bug. I suspect it���s a bug. After all, on iOS you get access to more permissions if a site is added to the homescreen���it���s the only way to ask for permissions for notifications, for example.

I added a few more sites to my dock: mastodon.social and thesession.org. They both work really well as standalone apps.

Interestingly, if I click a link to thesession.org from within the mastodon.social standalone web app (or the other way around), the link opens in my default browser. So the web apps don���t ���own��� the domains. That���s fine, although I wonder if it violates the principle of least surprise���perhaps the expectation is that the installed web app is the preffered owner of that link.

I also tried adding Google Calendar to my dock. Ironically, I can only do this with Safari. For some reason, Google have chosen not to make their calendar a progressive web app ���which means there���s no option to install it from Google Chrome.

They���re missing a trick there. It���s the perfect candidate for being a standalone app.

Actually, I take that back. Gmail is the perfect candidate for being a standalone app ���and yet it���s not a progressive web app. Very odd!

With Safari, you can add any website to the dock. It doesn���t need to be a progressive web app. But the installation experience works best if there���s a manifest file pointing to some nice icons.

As it turned out, Google Calendar runs like an absolute dog in Safari (and therefore as a Safari-installed web app). Before you cry conspiracy, note that it runs absolutely fine in Firefox. I know because I use it in Firefox every day. But I can���t add it to my dock from Firefox because Mozilla turned their back on progressive web apps a while back. A bad decision.

Google Calendar isn���t the only thing that runs slowly in Safari���s engine. This page on The Session has a very large DOM and a badly-coded in-page search (I know, I know, I need to improve it). It works fine in other browsers but Safari struggles mightily.

Still, aside from a few annoying little things around permissions and performance���and the situation with discovery���it feels great to have websites that act just like other apps. I like that some of the icons in my dock are native, some are web apps, and I mostly don���t notice the difference.

I wonder if there���s much point using wrappers like Electron any more? I feel like they were mostly aiming to get that parity with native apps in having a standalone application launched from the dock.

Now all you need is a website.

Crawlers

A few months back, I wrote about how Google is breaking its social contract with the web, harvesting our content not in order to send search traffic to relevant results, but to feed a large language model that will spew auto-completed sentences instead.

I still think Chris put it best:

I just think it���s fuckin��� rude.

When it comes to the crawlers that are ingesting our words to feed large language models, Neil Clarke describes the situtation:

It should be strictly opt-in. No one should be required to provide their work for free to any person or organization. The online community is under no responsibility to help them create their products. Some will declare that I am ���Anti-AI��� for saying such things, but that would be a misrepresentation. I am not declaring that these systems should be torn down, simply that their developers aren���t entitled to our work. They can still build those systems with purchased or donated data.

Alas, the current situation is opt-out. The onus is on us to update our robots.txt file.

Neil handily provides the current list to add to your file. Pass it on:

User-agent: CCBotDisallow: /User-agent: ChatGPT-UserDisallow: /User-agent: GPTBotDisallow: /User-agent: Google-ExtendedDisallow: /User-agent: OmgilibotDisallow: /User-agent: FacebookBotDisallow: /

In theory you should be able to group those user agents together, but citation needed on whether that���s honoured everywhere:

User-agent: CCBotUser-agent: ChatGPT-UserUser-agent: GPTBotUser-agent: Google-ExtendedUser-agent: OmgilibotUser-agent: FacebookBotDisallow: /

There���s a bigger issue with robots.txt though. It too is a social contract. And as we���ve seen, when it comes to large language models, social contracts are being ripped up by the companies looking to feed their beasts.

As Jim says:

I realized why I hadn���t yet added any rules to my robots.txt: I have zero faith in it.

That realisation was prompted in part by Manuel Moreale���s experiment with blocking crawlers:

So, what’s the takeaway here? I guess that the vast majority of crawlers don’t give a shit about your robots.txt.

Time to up the ante. Neil���s post offers an option if you���re running Apache. Either in .htaccess or in a .conf file, you can block user agents using mod_rewrite:

RewriteEngine OnRewriteCond %{HTTP_USER_AGENT} (CCBot|ChatGPT|GPTBot|Omgilibot| FacebookBot) [NC]RewriteRule ^ ��� [F]

You���ll see that Google-Extended isn���t that list. It isn���t a crawler. Rather it���s the permissions model that Google have implemented for using your site���s content to train large language models: unless you opt out via robots.txt, it���s assumed that you���re totally fine with your content being used to feed their stochastic parrots.

Trabaja remoto

August was a month of travels. You can press play on that month���s map to follow the journey.

But check out the map for September too because the travels continue. This time my adventures are confined to Europe.

I���m in Spain. Jessica and I flew into Madrid on Saturday. The next day we took a train ride across the Extremaduran landscape to C��ceres, our home for the week.

This is like the sequel to our Sicilian trip. We���re both working remotely. We just happen to do be doing it from a beautiful old town with amazing cuisine.

We���re in a nice apartment that���crucially���has good WiFi. It���s right on the main square, but it���s remarkably quiet.

There���s a time difference of one hour with Brighton. Fortunately everything in Spain happens at least an hour later than it does at home. Waking up. Lunch. Dinner. Everything is time-shifted so that I���m on the same schedule as my colleagues.

I swear I���m more productive working this way. Maybe it���s the knowledge that tapas of Iberican ham await me after work, but I���m getting a lot done this week.

And when the working week is done, the fun begins. C��ceres is hosting its annual Irish fleadh this weekend.

I���ve always wanted to go to it, but it���s quite a hassle to get here just for a weekend. Combining it with a week of remote work makes it more doable.

I���m already having a really nice time here and the tunes haven���t even started yet.

Simon’s rule

I got a nice email from someone regarding my recent posts about performance on The Session. They said:

I hope this message finds you well. First and foremost, I want to express how impressed I am with the overall performance of https://thesession.org/. It’s a fantastic resource for music enthusiasts like me.

How nice! I responded, thanking them for the kind words.

They sent a follow-up clarification:

Awesome, anyway there was an issue in my message.

The line ‘It’s a fantastic resource for music enthusiasts like me.’ added by chatGPT and I didn’t notice.

I imagine this is what it feels like when you���re on a phone call with someone and towards the end of the call you hear a distinct flushing sound.

I wrote back and told them about Simon���s rule:

I will not publish anything that takes someone else longer to read than it took me to write.

That just feels so rude!

I think that���s a good rule.

Secure tunes

The caching strategy for The Session that I wrote about is working a treat.

There are currently about 45,000 different tune settings on the site. One week after introducing the server-side caching, over 40,000 of those settings are already cached.

But even though it���s currently working well, I���m going to change the caching mechanism.

The eagle-eyed amongst you might have raised an eagle eyebrow when I described how the caching happens:

The first time anyone hits a tune page, the ABCs getting converted to SVGs as usual. But now there���s one additional step. I grab the generated markup and send it as an Ajax payload to an endpoint on my server. That endpoint stores the sheetmusic as a file in a cache.

I knew when I came up with this plan that there was a flaw. The endpoint that receives the markup via Ajax is accepting data from the client. That data could be faked by a malicious actor.

Sure, I���m doing a whole bunch of checks and sanitisation on the server, but there���s always going to be a way of working around that. You can never trust data sent from the client. I was kind of relying on security through obscurity ���except it wasn���t even that obscure because I blogged about it.

So I���m switching over to using a headless browser to extract the sheetmusic. You may recall that I wrote:

I could spin up a headless browser, run the JavaScript and take a snapshot. But that���s a bit beyond my backend programming skills.

That���s still true. So I���m outsourcing the work to Browserless.

There���s a reason I didn���t go with that solution to begin with. Like I said, over 40,000 tune settings have already been cached. If I had used the Browserless API to do that work, it would���ve been quite pricey. But now that the flood is over and there���s a just a trickle of caching happening, Browserless is a reasonable option.

Anyway, that security hole has now been closed. Thank you to everyone who wrote in to let me know about it. Like I said, I was aware of it, but it was good to have it confirmed.

Funnily enough, the security lesson here is the same as my conclusion when talking about performance:

If that means shifting the work from the browser to the server, do it!

Speedy tunes

Performance is a high priority for me with The Session. It needs to work for people all over the world using all kinds of devices.

My main strategy for ensuring good performance is to diligently apply progressive enhancement. The core content is available to any device that can render HTML.

To keep things performant, I���ve avoided as many assets (or, more accurately, liabilities) as possible. No uneccessary images. No superfluous JavaScript libraries. Not even any web fonts (gasp!). And definitely no third-party resources.

The pay-off is a speedy site. If you want to see lab data, run a page from The Session through lighthouse. To see field data, take a look at data from Chrome UX Report (Crux).

But the devil is in the details. Even though most pages on The Session are speedy, the outliers have bothered me for a while.

Take a typical tune page on the site. The data is delivered from the server as HTML, which loads nice and quick. That data includes the notes for the tune settings, written in ABC notation, a nice lightweight text format.

Then the enhancement happens. Using Paul Rosen���s brilliant abcjs JavaScript library, those ABCs are converted into SVG sheetmusic.

So on tune pages there���s an additional download for that JavaScript library. That���s not so bad though���I���m using a service worker to cache that file so there���ll only ever be one initial network request.

If a tune has just a few different versions, the page remains nice and zippy. But if a tune has lots of settings, the computation starts to add up. Converting all those settings from ABC to SVG starts to take a cumulative toll on the main thread.

I pondered ways to avoid that conversion step. Was there some way of pre-generating the SVGs on the server rather than doing it all on the client?

In theory, yes. I could spin up a headless browser, run the JavaScript and take a snapshot. But that���s a bit beyond my backend programming skills, so I���ve taken a slightly different approach.

The first time anyone hits a tune page, the ABCs getting converted to SVGs as usual. But now there���s one additional step. I grab the generated markup and send it as an Ajax payload to an endpoint on my server. That endpoint stores the sheetmusic as a file in a cache.

Next time someone hits that page, there���s a server-side check to see if the sheetmusic has been cached. If it has, send that down the wire embedded directly in the HTML.

The idea is that over time, most of the sheetmusic on the site will transition from being generated in the browser to being stored on the server.

So far it���s working out well.

Take a really popular tune like The Lark In The Morning. There are twenty settings, and each one has four parts. Previously that would have resulted in a few seconds of parsing and rendering time on the main thread. Now everything is delivered up-front.

I���m not out of the woods. A page like that with lots of sheetmusic and plenty of comments is going to have a hefty page weight and a large DOM size. I���ve still got a fair bit of main-thread work happening, but now the bulk of it is style and layout, whereas previously I had the JavaScript overhead on top of that.

I���ll keep working on it. But overall, the speed improvement is great. A typical tune page is now very speedy indeed.

It���s like a microcosm of web performance in general: respect your users��� time, network connection and battery life. If that means shifting the work from the browser to the server, do it!

Multi-page web apps

I received this email recently:

Subject: multi-page web apps

Hi Jeremy,

lately I’ve been following you through videos and texts and I’m curious as to why you advocate the use of multi-page web apps and not single-page ones.

Perhaps you can refer me to some sources where your position and reasoning is evident?

Here���s the response I sent���

Hi,

You can find a lot of my reasoning laid out in this (short and free) online book I wrote called Resilient Web Design:

https://resilientwebdesign.com/

The short answer to your question is this: user experience.

The slightly longer answer���

For most use cases, a website (or multi-page app if you prefer) is going to provide the most robust experience for the most number of users. That���s because a user���s web browser takes care of most of the heavy lifting.

Navigating from one page to another? That���s taken care of with links.

Gathering information from a user to process on a server? That���s taken care of with forms.

This frees me up to concentrate on the content and the design without having to reinvent the wheels of links and form fields.

These (let���s call them) multi-page apps are stateless, and for most use cases that���s absolutely fine.

There are some cases where you���d want a state to persist across pages. Let���s say you���re playing a song, or a podcast episode. Ideally you���d want that player to continue seamlessly playing even as the user navigates around the site. In that situation, a single-page app would be a suitable architecture.

But that architecture comes at a cost. Now you���ve got stop the browser doing what it would normally do with links and forms. It���s up to you to recreate that functionality. And you can���t do it with HTML, a robust fault-tolerant declarative language. You need to reimplement all that functionality in JavaScript, a less tolerant, more brittle language.

Then you���ve got to ship all that code to the user before they can use your site. It might be JavaScript code you���ve written yourself or it might be a third-party library designed for building single-page apps. Either way, the user pays a download tax (and a parsing tax, and an execution tax). Whereas with links and forms, all of that functionality is pre-bundled into the user���s web browser.

So that���s my reasoning. At least nine times out of ten, a multi-page approach is leaner, more robust, and simpler.

Like I said, there are times when a single-page approach makes sense���it all comes down to whether state needs to be constantly preserved. But these use cases are the exceptions, not the rule.

That���s why I find the framing of your question a little concerning. It should be inverted. The default approach should be to assume a multi-page approach (which is the way the web works by default). Deciding to take a JavaScript-driven single-page approach should be the exception.

It���s kind of like when people ask, ���Why don���t you have children?��� Surely the decision to have a child should require deliberation and commitment, rather than the other way around.

When it comes to front-end development, I���m worried that we���ve reached a state where the more complex over-engineered approach is viewed as the default.

I may be committing a fundamental attribution error here, but I think that we���ve reached this point not because of any consideration for users, but rather because of how it makes us developers feel. Perhaps building an old-fashioned website that uses HTML for navigations feels too easy, like it���s beneath us. But building an ���app��� that requires JavaScript just to render text on a screen feels like real programming.

I hope I���m wrong. I hope that other developers will start to consider user experience first and foremost when making architectural decisions.

Anyway. That���s my answer. User experience.

Cheers,

Jeremy

Performative performance

Web Summer Camp in Croatia finished with an interesting discussion. It was labelled a town-hall meeting, but it was more like an Oxford debating club.

Two speakers had two minutes each to speak for or against a particular statement. Their stances were assigned to them so they didn���t necessarily believe what they said.

One of the propositions was something like:

In the future, sustainable design will be as important as UX or performance.

That���s a tough one to argue against! But that���s what Sophia had to do.

She actually made a fairly compelling argument. She said that real impact isn���t going to come from individual websites changing their colour schemes. Real impact is going to come from making server farms run on renewable energy. She advocated for political action to change the system rather than having the responsibility heaped on the shoulders of the individuals making websites.

It���s a fair point. Much like the concept of a personal carbon footprint started life at BP to distract from corporate responsibility, perhaps we���re going to end up navel-gazing into our individual websites when we should be collectively lobbying for real change.

It���s akin to clicktivism���thinking you���re taking action by sharing something on social media, when real action requires hassling your political representative.

I���ve definitely seen some examples of performative sustainability on websites.

For example, at the start of this particular debate at Web Summer Camp we were shown a screenshot of a municipal website that has a toggle. The toggle supposedly enables a low-carbon mode. High resolution images are removed and for some reason the colour scheme goes grayscale. But even if those measures genuinely reduced energy consumption, it���s a bit late to enact them only after the toggle has been activated. Those hi-res images have already been downloaded by then.

Defaults matter. To be truly effective, the toggle needs to work the other way. Start in low-carbon mode, and only download the hi-res images when someone specifically requests them. (Hopefully browsers will implement prefers-reduced-data soon so that we can have our sustainable cake and eat it.)

Likewise I���ve seen statistics bandied about around the energy-savings that could be made if we used dark colour schemes. I���m sure the statistics are correct, but I���d like to see them presented side-by-side with, say, the energy impact of Google Tag Manager or React or any other wasteful dependencies that impact performance invisibly.

That���s the crux. Most of the important work around energy usage on websites is invisible. It���s the work done to not add more images, more JavaScript or more web fonts.

And it���s not just performance. I feel like the more important the work, the more likely it is to be invisible: privacy, security, accessibility ���those matter enormously but you can���t see when a website is secure, or accessible, or not tracking you.

I suspect this is why those areas are all frustratingly under-resourced. Why pour time and effort into something you can���t point at?

Now that I think about it, this could explain the rise of web accessibility overlays. If you do the real work of actually making a website accessible, your work will be invisible. But if you slap an overlay on your website, it looks like you���re making a statement about how much you care about accessibility (even though the overlay is total shit and does more harm than good).

I suspect there might be a similar mindset at work when it comes to interface toggles for low-carbon mode. It might make you feel good. It might make you look good. But it���s a poor substitute for making your website carbon-neutral by default.

Lovers in a dangerous time

Being in Croatia last week got me thinking about the country���s history.

I remember the break-up of Yugoslavia, but I was quite out of touch with the news for a while back in 1991. That���s because I was hitch-hiking and busking around Europe with my musical partner Polly from Cornwall. I had my mandolin, she had her fiddle.

We went from Ireland to England to France to Germany to Czechoslovakia (still a single country back then), to Austria to Italy, back to France, and back to England. A loop around Europe.

We set off on August 21st, 1991. The only reason I know the date is because I remember we had been to a gig in Cork the night before.

Sonic Youth were playing in Sir Henry���s (a great venue that no longer exists). The support band was a group from Seattle called Nirvana. I remember that some of my friends decided to skip the support band to stay in the pub next door until Sonic Youth came on because the pints were cheaper there.

By the time Polly and I got back from our travels, Nirvana were the biggest band on the planet. It all happened very quickly.

The same could be said for the situation in Yugoslavia.

I remember when we were stuck for a day at a petrol station in the alps trying to get from Austria to Italy. There was a bureau de change listing currency exchange rates. This was before the euro came in so there were lots of different currencies; pounds, francs, lira, deutsche marks. Then there was the listing for the Yugoslav dinar. It read:

We buy: 00.00 We sell: 00.00

That really struck me, seeing the situation summarised so clinically.

But what really got to me was an encounter in Vienna.

Polly and I did well in that city. On our first evening of busking, not only did we make some good money, but we also met a local folk singer. This young man very generously took us in and put us up in his flat.

At some point during our stay, we were on one of the city���s trams. That���s when we met another young couple who were on the road. Somehow there was always a connection between fellow travellers. I can���t remember who spoke to who first, but we bonded straight away.

It soon became clear that our situations were only superfically similar. This was a young couple deeply in love. One of them was Serbian. The other was Croatian. It wasn���t safe for either of them back where they used to call home.

I could return home at any point. I always knew that when I was sleeping rough, or struggling to make enough money to eat.

They couldn���t return. All they wanted was to be together somewhere safe. They started asking us about Ireland and England. ���Do you think they���d give us asylum?��� they asked with so much hope. It broke my heart to see their desperation, the pleading look in their eyes.

I felt so useless. I wished there was something I could���ve done for them.

I think about them a lot.

Travels

He drew a deep breath. ���Well, I���m back,��� he said.

I know how you feel, Samwise Gamgee.

I have returned from my travels���a week aboard the Queen Mary 2 crossing the Atlantic, followed by a weekend in New York, finishing with a week in Saint Augustine, Florida.

The Atlantic crossing was just as much fun as last time. In fact it was better because this time Jessica and I got to share the experience with our dear friends Dan and Sue.

There was dressing up! There was precarious ballet! There were waves! There were even some dolphins!

The truth is that this kind of Atlantic crossing is a bit like cosplaying a former age of travel. You get out of it what you put it into it. If you���re into LARPing as an Edwardian-era traveller, you���re going to have a good time.

We got very into it. Dressing up for dinner. Putting on a tux for the gala night. Donning masks for the masquerade evening.

It���s actually quite a practical way of travelling if you don���t mind being cut off from all digital communication for a week (this is a feature, not a bug). You adjust your clock by one hour most nights so that by the time you show up in New York, you���re on the right timezone with zero jetlag.

That was just as well because we had a packed weekend of activities in New York. By pure coincidence, two separate groups of friends were also in town from far away. We all met up and had a grand old time. Brunch in Tribeca; a John Cale concert in Prospect Park; the farmer���s market in Union Square; walking the high line ���good times with good friends.

New York was hot, but not as hot as what followed in Florida. A week lazing about on Saint Augustine beach. I ate shrimp every single day. I regret nothing.

We timed our exit just right. We flew out of Florida before the tropical storm hit. Then we landed in Gatwick right before the air-traffic control chaos erupted.

I had one day of rest before going back to work.

Well, I say ���work���, but the first item in my calendar was speaking at Web Summer Camp in Croatia. Back to the airport.

The talk went well, and I got to attend a performance workshop by Harry. But best of all was the location. Opatija is an idyllic paradise. Imagine crossing a web conference with White Lotus, but in a good way. It felt like a continuation of Florida, but with more placid clear waters.

But now I���m really back. And fortunately the English weather is playing along by being unseasonably warm . It���s as if the warm temperatures are following me around. I like it.