Journal tags: tag

The Dark Forest idea comes from the Remembrance of Earth’s Past books by Liu Cixin. It’s an elegant but dispiriting solution to the Fermi paradox. Maggie sums it up:

Dark forest theory suggests that the universe is like a dark forest at night - a place that appears quiet and lifeless because if you make noise, the predators will come eat you.

This theory proposes that all other intelligent civilizations were either killed or learned to shut up. We don’t yet know which category we fall into.

Maggie has described The Expanding Dark Forest and Generative AI:

The dark forest theory of the web points to the increasingly life-like but life-less state of being online. Most open and publicly available spaces on the web are overrun with bots, advertisers, trolls, data scrapers, clickbait, keyword-stuffing “content creators,” and algorithmically manipulated junk.

It’s like a dark forest that seems eerily devoid of human life – all the living creatures are hidden beneath the ground or up in trees. If they reveal themselves, they risk being attacked by automated predators.

Those of us in the cozy web try to keep our heads down, attempting to block the bots plundering our work.

I advocate for taking this further. We should fight back. Let’s exploit the security hole of prompt injections. Here are some people taking action:

• Eric Bailey talks about Consent, LLM scrapers, and poisoning the well,
• Thomas Rigby says Rage against the machine, and
• Lewis Dale points out that Perplexity AI is susceptible to prompt injection.

I’ve taken steps here on my site. I’d like to tell you exactly what I’ve done. But if I do that, I’m also telling the makers of these bots how to circumvent my attempts at prompt injection.

This feels like another concept from Liu Cixin’s books. Wallfacers:

The sophons can overhear any conversation and intercept any written or digital communication but cannot read human thoughts, so the UN devises a countermeasure by initiating the “Wallfacer” Program. Four individuals are granted vast resources and tasked with generating and fulfilling strategies that must never leave their own heads.

So while I’d normally share my code, I feel like in this case I need to exercise some discretion. But let me give you the broad brushstrokes:

• Every page of my online journal has three pieces of text that attempt prompt injections.
• Each of these is hidden from view and hidden from screen readers.
• Each piece of text is constructed on-the-fly on the server and they’re all different every time the page is loaded.

You can view source to see some examples.

I plan to keep updating my pool of potential prompt injections. I’ll add to it whenever I hear of a phrase that might potentially throw a spanner in the works of a scraping bot.

By the way, I should add that I’m doing this as well as using a robots.txt file. So any bot that injests a prompt injection deserves it.

I could not disagree with Manton more when he says:

I get the distrust of AI bots but I think discussions to sabotage crawled data go too far, potentially making a mess of the open web. There has never been a system like AI before, and old assumptions about what is fair use don’t really fit.

Bollocks. This is exactly the kind of techno-determinism that boils my blood:

AI companies are not going to go away, but we need to push them in the right directions.

“It’s inevitable!” they cry as though this was a force of nature, not something created by people.

There is nothing inevitable about any technology. The actions we take today are what determine our future. So let’s take steps now to prevent our web being turned into a dark, dark forest.

Large language models have reaped our words and plundered our books. Bryan Vandyke:

Turns out, everything on the internet—every blessed word, no matter how dumb or benighted—has utility as a learning model. Words are the food that large language algorithms feed upon, the scraps they rely on to grow, to learn, to approximate life. The LLNs that came online in recent years were all trained by reading the internet.

We can shut the barn door—now that the horse has pillaged—by updating our robots.txt files or editing .htaccess. That might protect us from the next wave, ’though it can’t undo what’s already been taken without permission. And that’s assuming that these organisations—who have demonstrated a contempt for ethical thinking—will even respect robots.txt requests.

I want to do more. I don’t just want to prevent my words being sucked up. I want to throw a spanner in the works. If my words are going to be snatched away, I want them to be poison pills.

The weakness of large language models is that their data and their logic come from the same source. That’s what makes prompt injection such a thorny problem (and a well-named neologism—the comparison to SQL injection is spot-on).

Smarter people than me are coming up with ways to protect content through sabotage: hidden pixels in images; hidden words on web pages. I’d like to implement this on my own website. If anyone has some suggestions for ways to do this, I’m all ears.

If enough people do this we’ll probably end up in an arms race with the bots. It’ll be like reverse SEO. Instead of trying to trick crawlers into liking us, let’s collectively kill ’em.

Who’s with me?

If you use Instagram, there may be a message buried in your notifications. It begins:

We’re getting ready to expand our AI at Meta experiences to your region.

Fuck that. Here’s the important bit:

To help bring these experiences to you, we’ll now rely on the legal basis called legitimate interests for using your information to develop and improve AI at Meta. This means that you have the right to object to how your information is used for these purposes. If your objection is honoured, it will be applied going forwards.

Follow that link and fill in the form. For the field labelled “Please tell us how this processing impacts you” I wrote:

It’s fucking rude.

That did the trick. I got an email saying:

We’ve reviewed your request and will honor your objection.

Mind you, there’s still this:

We may still process information about you to develop and improve AI at Meta, even if you object or don’t use our products and services.

The second episode of the second season of the Clearleft podcast is out. It’s all about employee experience design.

This topic came out of conversations with Katie. She really enjoys getting stuck into to the design challenges of the “backstage” tools that are often neglected. This is an area that Chris has been working in recently too, so I quized him on this topic.

They’re both super smart people which makes for a thoroughly enjoyable podcast episode. I usually have more guests on a single episode but it was fun to do a two-hander for once.

The whole thing comes in at just under seventeen minutes and there are some great stories and ideas in there. Have a listen.

And if you’re enjoying listening to the Clearleft podcast as much as I’m enjoying making it, be sure to spread the word wherever you share your recommnedations: Twitter, LinkedIn, Slack, your own website, the rooftop.

Matt wrote recently about how different writers keep notes:

I’m also reminded of how writers I love and respect maintain their own reservoirs of knowledge, complete with migratory paths down from the mountains.

I have a section of my site called “notes” but the truth is that every single thing I post on here—whether it’s a link, a blog post, or anything else—is really a “note to self.”

When it comes to retrieving information from this online memex of mine, I use tags. I’ve got search forms on my site, but usually I’ll go to the address bar in my browser instead and think “now, what would past me have tagged that with…” as I type adactio.com/tags/... (or, if I want to be more specific, adactio.com/links/tags/... or adactio.com/journal/tags/...).

It’s very satisfying to use my website as a back-up brain like this. I can get stuff out of my head and squirreled away, but still have it available for quick recall when I want it. It’s especially satisfying when I’m talking to someone else and something they say reminds me of something relevant, and I can go “Oh, let me send you this link…” as I retrieve the tagged item in question.

But I don’t think about other people when I’m adding something to my website. My audience is myself.

I know there’s lots of advice out there about considering your audience when you write, but when it comes to my personal site, I’d find that crippling. It would be one more admonishment from the inner critic whispering “no one’s interested in that”, “you have nothing new to add to this topic”, and “you’re not quailified to write about this.” If I’m writing for myself, then it’s easier to have fewer inhibitions. By treating everything as a scrappy note-to-self, I can avoid agonising about quality control …although I still spend far too long trying to come up with titles for posts.

I’ve noticed—and other bloggers have corroborated this—there’s no correlation whatsover between the amount of time you put into something and how much it’s going to resonate with people. You might spend days putting together a thoroughly-researched article only to have it met with tumbleweeds when you finally publish it. Or you might bash something out late at night after a few beers only to find it on the front page of various aggregators the next morning.

If someone else gets some value from a quick blog post that I dash off here, that’s always a pleasant surprise. It’s a bonus. But it’s not my reason for writing. My website is primarily a tool and a library for myself. It just happens to also be public.

I’m pretty sure that nobody but me uses the tags I add to my links and blog posts, and that’s fine with me. It’s very much a folksonomy.

Likewise, there’s a feature I added to my blog posts recently that is probably only of interest to me. Under each blog post, there’s a heading saying “Previously on this day” followed by links to any blog posts published on the same date in previous years. I find it absolutely fascinating to spelunk down those hyperlink potholes, but I’m sure for anyone else it’s about as interesting as a slideshow of holiday photos.

Matt took this further by adding an “on this day” URL to his site. What a great idea! I’ve now done the same here:

adactio.com/archive/onthisday

That URL is almost certainly only of interest to me. And that’s fine.

Back at the start of the (first) lockdown, I wrote about using my website as an outlet:

While you’re stuck inside, your website is not just a place you can go to, it’s a place you can control, a place you can maintain, a place you can tidy up, a place you can expand. Most of all, it’s a place you can lose yourself in, even if it’s just for a little while.

Last week was eventful and stressful. For everyone. I found myself once again taking refuge in my website, tinkering with its inner workings in the way that someone else would potter about in their shed or take to their garage to strip down the engine of some automotive device.

Colly drew my attention to Bookshop.org, newly launched in the UK. It’s an umbrella website for independent bookshops to sell through. It’s also got an affiliate scheme, much like Amazon. I set up a Bookshop page for myself.

I’ve been tracking the books I’m reading for the past three years here on my own website. I set about reproducing that list on Bookshop.

It was exactly the kind of not-exactly-mindless but definitely-not-challenging task that was perfect for the state of my brain last week. Search for a book; find the ISBN number; paste that number into a form. It’s the kind of task that a real programmer would immediately set about automating but one that I embraced as a kind of menial task to keep me occupied.

I wasn’t able to get a one-to-one match between the list on my site and my reading list on Bookshop. Some titles aren’t available in the online catalogue. For example, the book I’m reading right now—A Paradise Built in Hell by Rebecca Solnit—is nowhere to be found, which seems like an odd omission.

But most of the books I’ve read are there on Bookshop.org, complete with pretty book covers. Then I decided to reverse the process of my menial task. I took all of the ISBN numbers from Bookshop and add them as machine tags to my reading notes here on my own website. Book cover images on Bookshop have predictable URLs that use the ISBN number (well, technically the EAN number, or ISBN-13, but let’s not go down a 927 rabbit hole here). So now I’m using that metadata to pull in images from Bookshop.org to illustrate my reading notes here on adactio.com.

I’m linking to the corresponding book on Bookshop.org using this URL structure:

https://uk.bookshop.org/a/{{ affiliate code }}/{{ ISBN number }}

I realised that I could also link to the corresponding entry on Open Library using this URL structure:

https://openlibrary.org/isbn/{{ ISBN number }}

Here, for example, is my note for The Raven Tower by Ann Leckie. That entry has a tag:

book:ean=9780356506999

With that information I can illustrate my note with this image:

https://images-eu.bookshop.org/product-images/images/9780356506999.jpg

I’m linking off to this URL on Bookshop.org:

https://uk.bookshop.org/a/980/9780356506999

And this URL on Open Library:

https://openlibrary.org/isbn/9780356506999

The end result is that my reading list now has more links and pretty pictures.

Oh, I also set up a couple of shorter lists on Bookshop.org:

• Favourite Non-fiction
• Favourite Fiction

The books listed in those are drawn from my end of the year round-ups when I try to pick one favourite non-fiction book and one favourite work of fiction (almost always speculative fiction). The books in those two lists are the ones that get two hearty thumbs up from me. If you click through to buy one of them, the price might not be as cheap as on Amazon, but you’ll be supporting an independent bookshop.

A little while back, I switched from using Chrome as my day-to-day browser to using Firefox. I could feel myself getting a bit too comfortable with one particular browser, and that’s not good. I reckon it’s good to shake things up a little every now and then. Besides, there really isn’t that much difference once you’ve transferred over bookmarks and cookies.

Unfortunately I’m being bitten by this little bug in Firefox. It causes some of my bookmarklets to fail on certain sites with strict Content Security Policies (and CSPs shouldn’t affect bookmarklets). I might have to switch back to Chrome because of this.

I use bookmarklets throughout the day. There’s the Huffduffer bookmarklet, of course, for whenever I come across a podcast episode or other piece of audio that I want to listen to later. But there’s also my own home-rolled bookmarklet for posting links to my site. It doesn’t do anything clever—it grabs the title and URL of the currently open page and pre-populates a form in a new window, leaving me to add a short description and some tags.

If you’re reading this, then you’re familiar with the “journal” section of adactio.com, but the “links” section is where I post the most. Here, for example, are all the links I posted yesterday. It varies from day to day, but there’s generally a handful.

Should you wish to keep track of everything I’m linking to, there’s a twitterbot you can follow called @adactioLinks. It uses a simple IFTTT recipe to poll my RSS feed of links and send out a tweet whenever there’s a new entry.

Or you can drink straight from the source and subscribe to the RSS feed itself, if you’re still rocking it old-school. But if RSS is your bag, then you might appreciate a way to filter those links…

All my links are tagged. Heavily. This is because all my links are “notes to future self”, and all my future self has to do is ask “what would past me have tagged that link with?” when I’m trying to find something I previously linked to. I end up using my site’s URLs as an interface:

• adactio.com/links/tags/serviceworkers or
• adactio.com/links/tags/sci-fi

At the front-end gatherings at Clearleft, I usually wrap up with a quick tour of whatever I’ve added that week to:

• adactio.com/links/tags/frontend

Well, each one of those tags also has a corresponding RSS feed:

• adactio.com/links/tags/serviceworkers/rss
• adactio.com/links/tags/sci-fi/rss
• adactio.com/links/tags/frontend/rss

…and so on.

That means you can subscribe to just the links tagged with something you’re interested in. Here’s the full list of tags if you’re interested in seeing the inside of my head.

This also works for my journal entries. If you’re only interested in my blog posts about frontend development, you might want to subscribe to:

• adactio.com/journal/tags/frontend/rss

Here are all the tags from my journal.

You can even mix them up. For everything I’ve tagged with “typography”—whether it’s links, journal entries, or articles—the URL is:

• adactio.com/tags/typography

The corresponding RSS feed is:

• adactio.com/tags/typography/rss

You get the idea. Basically, if something on my site is a list of items, chances are there’s a corresponding RSS feeds. Sometimes there might even be a JSON feed. Hack some URLs to see.

Meanwhile, I’ll be linking, linking, linking…

Since support for service workers landed in Mobile Safari on iOS, I’ve been trying a little experiment. Can I replace some of the native apps I use with progressive web apps?

The two major candidates are Twitter and Instagram. I added them to my home screen, and banished the native apps off to a separate screen. I’ve been using both progressive web apps for a few months now, and I have to say, they’re pretty darn great.

There are a few limitations compared to the native apps. On Twitter, if you follow a link from a tweet, it pops open in Safari, which is fine, but when you return to Twitter, it loads anew. This isn’t any fault of Twitter—this is the way that web apps have worked on iOS ever since they introduced their weird web-app-capable meta element. I hope this behaviour will be fixed in a future update.

Also, until we get web notifications on iOS, I need to keep the Twitter native app around if I want to be notified of a direct message (the only notification I allow).

Apart from those two little issues though, Twitter Lite is on par with the native app.

Instagram is also pretty great. It too suffers from some navigation issues. If I click through to someone’s profile, and then return to the main feed, it also loads it anew, losing my place. It would be great if this could be fixed.

For some reason, the Instagram web app doesn’t allow uploading multiple photos …which is weird, because I can upload multiple photos on my own site by adding the multiple attribute to the input type="file" in my posting interface.

Apart from that, though, it works great. And as I never wanted notifications from Instagram anyway, the lack of web notifications doesn’t bother me at all. In fact, because the progressive web app doesn’t keep nagging me about enabling notifications, it’s a more pleasant experience overall.

Something else that was really annoying with the native app was the preponderance of advertisements. It was really getting out of hand.

Well …(looks around to make sure no one is listening)… don’t tell anyone, but the Instagram progressive web app—i.e. the website—doesn’t have any ads at all!

Here’s hoping it stays that way.

It’s time for the second talk at An Event Apart Seattle (Special Edition). Sarah is talking about Digital Marketing Strategies for the Busy “Web Master”. These are the notes I made during the talk…

Recently Sarah was asked for her job title recently and she found it really stressful. She wasn’t comfortable with “Art Director”. And, even though it would probably be accurate, “Social Media Expert” feels icky. A more fitting title would be “Social Media Designer” but that’s not a thing. Ironically the term “Web Master” probably fits us better than it did back in the ’90s.

We have a bit of a defeatist attitude towards social media at the moment. It feels like that space has been claimed and so there’s no point in even starting. But we’re still in the first 10,000 days in the web. There is no social media, Gary Vee says. It’s a slang term for a collection of apps and websites that now dominate attention in our society.

Sarah likes the term “consensual hallucination” (that I borrowed from William Gibson to describe how we did web design for years). It applies to social media.

Once upon a time we had to sell the benefits of just being online to our clients. Our businesses now need to get into the mindset of “How can I help you?” and “What can I do for you?” We’re moving away from being sales-y and getting down to being more honest. We’re no longer saying “Look at what I’ve got.”

The average time spent on social media per day is 1 hour and 48 minutes. The average time spent on the kind of sites we make is 15 seconds.

Quarterly design reviews are a good idea—strategically designing your social media campaigns, reviewing successes and failures.

The first thing to mention is vanity metrics. You might need to sit down and have “the talk” with your boss or client about this. It’s no different to having hit counters on our sites back in the ’90s. While we’re chasing these vanity metrics, we’re not asking what people really want.

Google brought a roadshow to Sarah’s hometown of Leigh-on-Sea a while back. There was a really irritating know-it-all chap in the audience who put his hand up when other people were asking about how to get followers on social media. “You need to post three times a day to all social media channels”, he said. “And you need to use the follow-unfollow method with a bot.” Sarah’s eyes were rolling at this point. Don’t beg for likes and follows—you’re skewing your metrics.

“What about this Snapchat thing?” people asked. Irritating guy said, “Don’t worry about—young people use it to send rude pictures to each other.” Sarah was face-palming at this point.

But this event was a good wake-up call for Sarah. We need to check our personal bias. She had to check her own personal bias against LinkedIn.

What we can do is look for emerging social networks. Find social networks that aren’t yet clogged. People still fixate on displayed numbers instead of the actual connection with people.

We all have a tendency to think of the more successful social networks as something that is coming. Like Snapchat. But if you’re in this space, there’s no time to waste. Sarah has been interviewing for social media people and it’s fascinating to see how misunderstood Snapchat is. One big misconception is that it’s only for youngsters. The numbers might be lower than Facebook, but there’s a lot of video on there. Snapchat’s weakness is “the olds”—the non-intuitive interface makes it cool with young people who have time to invest in learning it; the learning curve keeps the parents out. Because the moment that mums and grandmums appear on a social network, the younger folks get out. And actually, when it comes to putting ads on Snapchat, the interface is very good.

What can we do in 2018?

• By 2019, video will account for 80% of all consumer internet traffic. If you’re not planning for this, you’re missing out.
• Move to HTTPS.
• Make your website mobile ready.

Let’s ban the pop-up. Overlays. Permission dialogs. They’re all terrible. Google has started to penalise websites “where content is not easily accessible.”

Pop-ups are a lazy fix for a complex engagement problem (similar to carousels). It’s a terrible user experience. Do we thing this is adding value? Is this the best way to get someone’s email address? They’re like the chuggers of the web.

Here’s an interesting issue: there are discount codes available on the web. We inform people of this through pop-ups. Then it when it comes to check-out, they know that a discount is possible and so they Google for discount codes. You might as well have a page on your own website to list your own discount codes instead of people going elsewhere for them.

There’s a long tail of conversions, particularly with more expensive products and services. Virgin Holidays has a great example. For an expensive holiday, they ask for just a small deposit up front.

Let’s talk about some specific social networks.

Facebook

Facebook Pixel should be on your website, says Sarah. It collects data about your customers. (Needless to say, I disagree with this suggestion. Stand up for your customers’ dignity.)

Facebook is a very cheap way to publish video. Organic Facebook engagement is highest on posts with videos. (I think I threw up in my mouth a little just typing the words “organic”, “Facebook”, and “engagement” all in a row.) Facebook Live videos have six times the engagement of regular videos.

Sarah just said the word synergy. Twice. Unironically.

Facebook changed its algorithm last year. You’re going to see less posts from business and more posts from people.

Facebook advertising does work, but if it doesn’t work for you, the problem is probably down to your creative. (We’re using the word “creative” as a noun rather than an adjective, apparently.)

Google

With Ad Words, measure success by conversions rather than impressions. You might get thousands of eyeballs looking at a form, but only a handful filling it out. You need to know that second number to understand how much you’re really paying per customer.

trends.google.com is useful for finding keywords that aren’t yet saturated.

Google My Business is under-used, especially if you have a bricks’n’mortar store. It can make a massive difference to small businesses. It’s worth keeping it up to date and keeping it updated.

Instagram

700 million active users (double Twitter, and three times WhatsApp and Facebook Messenger). A lot of people are complaining about the changed algorithm. Social networks change their algorithms to deal with the “problems of success.” Instagram needs to help people with the discoverability of posts, says Sarah (again, I strongly disagree—it disempowers the user; I find Instagram’s we-know-best algorithm to be insultingly patronising).

Hashtags are the plumbing of the social media ecosystem. They’re not there for users to read. They’re for discoverability. Eleven hashtags are optimal.

Instagram Stories are a funny one. People are trying to use them to get around the algorithm, posting screenshots of photos to a story.

Archiving is a handy feature of Instagram. For time-sensitive content (like being closed during a snowstorm), it’s very useful to be able to archive those posts after the fact.

Planoly is a great website for managing your Instagram campaign. You can visually plan your feed. Only recently did Instagram start allowing scheduled posts (as long as they’re square, for some reason).

Influencer marketing is a thing. People trust peer recommendations more than advertising. You can buy micro-influencers quite cheaply.

(Side note: I think I’ve seen this episode of Black Mirror.)

How much do influencers cost? Not as much as you think. The average sponsored post rate is $180.

Case study

We need to have a “Design once. Use Everywhere.” mindset. Others we’ll go crazy. Away is doing this well. They sell a suitcase with built-in USB chargers.

The brands dominating social media are those with the most agile teams with exceptional storytelling skills. Away are very brave with their marketing. They’ve identified what their market has in common—travel—and they’re aiming at the level above that. They’re playing the long game, bringing the value back to the user. It’s all about “How can I help you?” rather than “Look at what I’ve gone.” Away’s creative is compelling, quirky, and fun. They work with influencers who are known to create beautiful imagery. Those influencers were given free suitcases. The cost of giving away those bags was much less than a traditional marketing campaign.

Their product is not front and centre in their campaigns. Travel is front and centre. They also collaborate with other brands. Their Google Ads are very striking. That also translates to physical advertising, like ads on airport security trays.

On Facebook, and on all of the social networks, everything is very polished and art-directed. They’re building a story. The content is about travel, but the through-line is about their suitcases.

When things go bad…

To finish, a semi-amusing story. Cath Kidston did a collaboration with Disney’s Peter Pan. Sarah had a hunch that it might go wrong. On paper, the social campaigns seemed fine. A slow build-up to the Peter Pan product launch. Lots of lovely teasers. They were seeding Instagram with beautiful imagery the day before launch. There was a real excitement building. Then the coveted email campaign with the coveted password.

On the site, people put in their password and then they had to wait. It was a deliberately gated experience. Twenty minutes of waiting. Then you finally get to the store …and there’s no “add to cart” button. Yup, they had left out the most important bit of the interface.

Sarah looked at what people were saying on Twitter. Lots of people assumed the problem was with their computer (after all, the web team wouldn’t be so silly as to leave off the “add to cart” button, right?). People blamed themselves. Cath Kidston scrambled to fix the problem …and threw people back into the 20 minute queue. Finally, the button appeared. So Sarah looked at a few bits ad pieces, and when she hit “add to cart” …she was thrown back to the 20 minute queue.

Sarah reached out to try to talk to someone on the web team. No one wanted to talk about it. If you ever find someone who was on that team, put them in touch.

Anyway, to wrap up…

Ensure the networks you are pursuing make sense for your brand.

Find your story for social media longevity.

See also:

• Torre’s notes on this talk

I’ve written before about how I use apps on my phone:

If I install an app on my phone, the first thing I do is switch off all notifications. That saves battery life and sanity.

The only time my phone is allowed to ask for my attention is for phone calls, SMS, or FaceTime (all rare occurrences). I initiate every other interaction—Twitter, Instagram, Foursquare, the web. My phone is a tool that I control, not the other way around.

To me, this seems like a perfectly sensible thing to do. I was surprised by how others thought it was radical and extreme.

I’m always shocked when I’m out and about with someone who has their phone set up to notify them of any activity—a mention on Twitter, a comment on Instagram, or worst of all, an email. The thought of receiving a notification upon receipt of an email gives me the shivers. Allowing those kinds of notifications would feel like putting shackles on my time and attention. Instead, I think I’m applying an old-school RSS mindset to app usage: pull rather than push.

Don’t get me wrong: I use apps on my phone all the time: Twitter, Instagram, Swarm (though not email, except in direst emergency). Even without enabling notifications, I still have to fight the urge to fiddle with my phone—to check to see if anything interesting is happening. I’d like to think I’m in control of my phone usage, but I’m not sure that’s entirely true. But I do know that my behaviour would be a lot, lot worse if notifications were enabled.

I was a bit horrified when Apple decided to port this notification model to the desktop. There doesn’t seem to be any way of removing the “notification tray” altogether, but I can at least go into System Preferences and make sure that absolutely nothing is allowed to pop up an alert while I’m trying to accomplish some other task.

It’s the same on iOS—you can control notifications from Settings—but there’s an added layer within the apps themselves. If you have notifications disabled, the apps encourage you to enable them. That’s fine …at first. Being told that I could and should enable notifications is a perfectly reasonable part of the onboarding process. But with some apps I’m told that I should enable notifications Every. Single. Time.

Of the apps I use, Instagram and Swarm are the worst offenders (I don’t have Facebook or Snapchat installed so I don’t know whether they’re as pushy). This behaviour seems to have worsened recently. The needling has been dialed up in recent updates to the apps. It doesn’t matter how often I dismiss the dialogue, it reappears the next time I open the app.

Initially I thought this might be a bug. I’ve submitted bug reports to Instagram and Swarm, but I’m starting to think that they see my bug as their feature.

In the grand scheme of things, it’s not a big deal, but I would appreciate some respect for my deliberate choice. It gets pretty wearying over the long haul. To use a completely inappropriate analogy, it’s like a recovering alcoholic constantly having to rebuff “friends” asking if they’re absolutely sure they don’t want a drink.

I don’t think there’s malice at work here. I think it’s just that I’m an edge-case scenario. They’ve thought about the situation where someone doesn’t have notifications enabled, and they’ve come up with a reasonable solution: encourage that person to enable notifications. After all, who wouldn’t want notifications? That question, if it’s asked at all, is only asked rhetorically.

I’m trying to do the healthy thing here (or at least the healthier thing) in being mindful of my app usage. They sure aren’t making it easy.

The model that web browsers use for notifications seems quite sensible in comparison. If you arrive on a site that asks for permission to send you notifications (without even taking you out to dinner first) then you have three options: allow, block, or dismiss. If you choose “block”, that site will never be able to ask that browser for permission to enable notifications. Ever. (Oh, how I wish I could apply that browser functionality to all those sites asking me to sign up for their newsletter!)

That must seem like the stuff of nightmares for growth-hacking disruptive startups looking to make their graphs go up and to the right, but it’s a wonderful example of truly user-centred design. In that situation, the browser truly feels like a user agent.

I was idly thinking about the different ways I can post to adactio.com. I decided to count the ways.

Admin interface

This is the classic CMS approach. In my case the CMS is a crufty hand-rolled affair using PHP and MySQL that I wrote years ago. I log in to an admin interface and fill in a form, putting the text of my posts into a textarea. In truth, I usually write in a desktop text editor first, and then paste that into the textarea. That’s what I’m doing now—copying and pasting Markdown from the Typed app.

Directly from my site

If I’m logged in, I get a stripped down posting interface in the notes section of my site.

Bookmarklet

This is how I post links. When I’m at a URL I want to bookmark, I hit the “Bookmark it” bookmarklet in my browser’s bookmarks bar. That pops open a version of the admin interface tailored specifically for links. I really, really like bookmarklets. The one big downside is that they don’t work on mobile.

Text message

This is something I knocked together at Indie Web Camp Brighton 2015 using the Twilio API. It’s handy for posting notes if I’m travelling somewhere and data is at a premium. But I don’t use it that often.

Instagram

Thanks to Aaron’s OwnYourGram service—and the fact that my site has a micropub endpoint—I can post images from Instagram to my site. This used to happen instantaneously but Instagram changed their API rules for the worse. Between that and their shitty “algorithmic” timeline, I find myself using the service less and less. At this point I’m only on their for the doggos.

Swarm

Like OwnYourGram, Aaron’s OwnYourSwarm allows me to post check-ins and photos from the Swarm app to my site. Again, micropub makes it all possible.

OwnYourGram and OwnYourSwarm are very similar and could probably be abstracted into a generic service for posting from third-party apps to micropub endpoints. I’d quite like to post my check-ins on Untappd to my site.

Other people’s admin interfaces

Thanks to rel="me" and IndieAuth, I can log into other people’s posting interfaces using my own website as the log-in, and post to my micropub endpoint, like this. Quill is a good example of this. I don’t use it that much, but I really should—the editor interface is quite Medium-like in its design.

Anyway, those are the different ways I can update my website that I can think of right now.

Syndication

In terms of output, I’ve got a few different ways of syndicating what I post here:

• RSS feeds for my journal, links, articles, and notes.
• JSON feeds for my journal, links, articles, and notes.
• Twitter accounts for my journal, links, articles, and notes (that one is my main Twitter account).
• I syndicate most of my my photos to my Flickr account.
• I syndicate most of my journal posts and articles to my Medium account.
• I used to syndicate my links to my Delicious account but at some point that became fairly pointless.
• Whenever I post a link, The Internet Archive gets pinged and makes a copy for the wayback machine. Here’s an example of a recent link.
• I syndicate just about everything to my Facebook account using If This, Then That recipes (RSS to Facebook posts). Facebook is a roach motel. I never post any original content there—everything starts here on my site.

Just so you know, if you comment on one of my posts on Facebook, I probably won’t see it. But if you reply to a copy of one of posts on Twitter or Instagram, it will show up over here on adactio.com thanks to the magic of Brid.gy and webmention.

Adam Silver is writing a book on forms—you may be familiar with his previous book on maintainable CSS. In a recent article (that for some reason isn’t on his blog), he looks at markup patterns for search forms and advocates that we should always use a label. I agree. But for some reason, we keep getting handed designs that show unlabelled search forms. And no, a placeholder is not a label.

I had a discussion with Mark about this the other day. The form he was marking up didn’t have a label, but it did have a button with some text that would work as a label:

<input type="search" placeholder="…">
<button type="submit">
Search
</button>

He was wondering if there was a way of using the button’s text as the label. I think there is. Using aria-labelledby like this, the button’s text should be read out before the input field:

<input aria-labelledby="searchtext" type="search" placeholder="…">
<button type="submit" id="searchtext">
Search
</button>

Notice that I say “think” and “should.” It’s one thing to figure out a theoretical solution, but only testing will show whether it actually works.

The W3C’s WAI tutorial on labelling content gives an example that uses aria-label instead:

<input type="text" name="search" aria-label="Search">
<button type="submit">Search</button>

It seems a bit of a shame to me that the label text is duplicated in the button and in the aria-label attribute (and being squirrelled away in an attribute, it runs the risk of metacrap rot). But they know what they’re talking about so there may well be very good reasons to prefer duplicating the value with aria-label rather than pointing to the value with aria-labelledby.

I thought it would be interesting to see how other sites are approaching this pattern—unlabelled search forms are all too common. All the markup examples here have been simplified a bit, removing class attributes and the like…

The BBC’s search form does actually have a label:

<label for="orb-search-q">
Search the BBC
</label>
<input id="orb-search-q" placeholder="Search" type="text">
<button>Search the BBC</button>

But that label is then hidden using CSS:

position: absolute;
height: 1px;
width: 1px;
overflow: hidden;
clip: rect(1px, 1px, 1px, 1px);

That CSS—as pioneered by Snook—ensures that the label is visually hidden but remains accessible to assistive technology. Using something like display: none would hide the label for everyone.

Medium wraps the input (and icon) in a label and then gives the label a title attribute. Like aria-label, a title attribute should be read out by screen readers, but it has the added advantage of also being visible as a tooltip on hover:

<label title="Search Medium">
  <span class="svgIcon"><svg></svg></span>
  <input type="search">
</label>

This is also what Google does on what must be the most visited search form on the web. But the W3C’s WAI tutorial warns against using the title attribute like this:

This approach is generally less reliable and not recommended because some screen readers and assistive technologies do not interpret the title attribute as a replacement for the label element, possibly because the title attribute is often used to provide non-essential information.

Twitter follows the BBC’s pattern of having a label but visually hiding it. They also have some descriptive text for the icon, and that text gets visually hidden too:

<label class="visuallyhidden" for="search-query">Search query</label>
<input id="search-query" placeholder="Search Twitter" type="text">
<span class="search-icon>
  <button type="submit" class="Icon" tabindex="-1">
    <span class="visuallyhidden">Search Twitter</span>
  </button>
</span>

Here’s their CSS for hiding those bits of text—it’s very similar to the BBC’s:

.visuallyhidden {
  border: 0;
  clip: rect(0 0 0 0);
  height: 1px;
  margin: -1px;
  overflow: hidden;
  padding: 0;
  position: absolute;
  width: 1px;
}

That’s exactly the CSS recommended in the W3C’s WAI tutorial.

Flickr have gone with the aria-label pattern as recommended in that W3C WAI tutorial:

<input placeholder="Photos, people, or groups" aria-label="Search" type="text">
<input type="submit" value="Search">

Interestingly, neither Twitter or Flickr are using type="search" on the input elements. I’m guessing this is probably because of frustrations with trying to undo the default styles that some browsers apply to input type="search" fields. Seems a shame though.

Instagram also doesn’t use type="search" and makes no attempt to expose any kind of accessible label:

<input type="text" placeholder="Search">
<span class="coreSpriteSearchIcon"></span>

Same with Tumblr:

<input tabindex="1" type="text" name="q" id="search_query" placeholder="Search Tumblr" autocomplete="off" required="required">

…although the search form itself does have role="search" applied to it. Perhaps that helps to mitigate the lack of a clear label?

After that whistle-stop tour of a few of the web’s unlabelled search forms, it looks like the options are:

• a visually-hidden label element,
• an aria-label attribute,
• a title attribute, or
• associate some text using aria-labelledby.

But that last one needs some testing.

Update: Emil did some testing. Looks like all screen-reader/browser combinations will read the associated text.

I was back in Nürnberg last week for the second border:none. Joschi tried an interesting format for this year’s event. The first day was a small conference-like gathering with an interesting mix of speakers, but the second day was much more collaborative, with people working together in “creator units”—part workshop, part round-table discussion.

I teamed up with Aaron to lead the session on all things indie web. It turned out to be a lot of fun. Throughout the day, we introduced the little building blocks, one by one. By the end of the day, it was amazing to see how much progress people made by taking this layered approach of small pieces, loosely stacked.

relme

The first step is: do you have a domain name?

Okay, next step: are you linking from that domain to other profiles of you on the web? Twitter, Instagram, Github, Dribbble, whatever. If so, here’s the first bit of hands-on work: add rel="me" to those links.

<a rel="me" href="https://twitter.com/adactio">Twitter</a>
<a rel="me" href="https://github.com/adactio">Github</a>
<a rel="me" href="https://www.flickr.com/people/adactio">Flickr</a>

If you don’t have any profiles on other sites, you can still mark up your telephone number or email address with rel="me". You might want to do this in a link element in the head of your HTML.

<link rel="me" href="mailto:jeremy@adactio.com" />
<link rel="me" href="sms:+447792069292" />

IndieAuth

As soon as you’ve done that, you can make use of IndieAuth. This is a technique that demonstrates a recurring theme in indie web building blocks: take advantage of the strengths of existing third-party sites. In this case, IndieAuth piggybacks on top of the fact that many third-party sites have some kind of authentication mechanism, usually through OAuth. The fact that you’re “claiming” a profile on a third-party site using rel="me"—and the third-party profile in turn links back to your site—means that we can use all the smart work that went into their authentication flow.

You can see IndieAuth in action by logging into the Indie Web Camp wiki. It’s pretty nifty.

If you’ve used rel="me" to link to a profile on something like Twitter, Github, or Flickr, you can authenticate with their OAuth flow. If you’ve used rel="me" for your email address or phone number, you can authenticate by email or SMS.

h-entry

Next question: are you publishing stuff on your site? If so, mark it up using h-entry. This involves adding a few classes to your existing markup.

<article class="h-entry">
  <div class="e-content">
    <p>Having fun with @aaronpk, helping @border_none attendees mark up their sites with rel="me" links, h-entry classes, and webmention endpoints.</p>
  </div>
  <time class="dt-published" datetime="2014-10-18 08:42:37">8:42am</time>
</article>

Now, the reason for doing this isn’t for some theoretical benefit from search engines, or browsers, but simply to make the content you’re publishing machine-parsable (which will come in handy in the next steps).

Aaron published a note on his website, inviting everyone to leave a comment. The trick is though, to leave a comment on Aaron’s site, you need to publish it on your own site.

Webmention

Here’s my response to Aaron’s post. As well as being published on my own site, it also shows up on Aaron’s. That’s because I sent a webmention to Aaron.

Webmention is basically a reimplementation of pingback, but without any of the XML silliness; it’s just a POST request with two values—the URL of the origin post, and the URL of the response.

My site doesn’t automatically send webmentions to any links I reference in my posts—I should really fix that—but that’s okay; Aaron—like me—has a form under each of his posts where you can paste in the URL of your response.

This is where those h-entry classes come in. If your post is marked up with h-entry, then it can be parsed to figure out which bit of your post is the body, which bit is the author, and so on. If your response isn’t marked up as h-entry, Aaron just displays a link back to your post. But if it is marked up in h-entry, Aaron can show the whole post on his site.

Okay. By this point, we’ve already come really far, and all people had to do was edit their HTML to add some rel attributes and class values.

For true site-to-site communication, you’ll need to have a webmention endpoint. That’s a bit trickier to add to your own site; it requires some programming. Here’s my minimum viable webmention that I wrote in PHP. But there are plenty of existing implentations you can use, like this webmention plug-in for WordPress.

Or you could request an account on webmention.io, which is basically webmention-as-a-service. Handy!

Once you have a webmention endpoint, you can point to it from the head of your HTML using a link element:

<link rel="mention" href="https://adactio.com/webmention" />

Now you can receive responses to your posts.

Here’s the really cool bit: if you sign up for Bridgy, you can start receiving responses from third-party sites like Twitter, Facebook, etc. Bridgy just needs to know who you are on those networks, looks at your website, and figures everything out from there. And it automatically turns the responses from those networks into h-entry. It feels like magic!

Here are responses from Twitter to my posts, as captured by Bridgy.

POSSE

That was mostly what Aaron and I covered in our one-day introduction to the indie web. I think that’s pretty good going.

The next step would be implementing the idea of POSSE: Publish on your Own Site, Syndicate Elsewhere.

You could do this using something as simple as If This, Then That e.g. everytime something crops up in your RSS feed, post it to Twitter, or Facebook, or both. If you don’t have an RSS feed, don’t worry: because you’re already marking your HTML up in h-entry, it can be converted to RSS easily.

I’m doing my own POSSEing to Twitter, which I’ve written about already. Since then, I’ve also started publishing photos here, which I sometimes POSSE to Twitter, and always POSSE to Flickr. Here’s my code for posting to Flickr.

I’d really like to POSSE my photos to Instagram, but that’s impossible. Instagram is a data roach-motel. The API provides no method for posting photos. The only way to post a picture to Instagram is with the Instagram app.

My only option is to do the opposite of POSSEing, which is PESOS: Publish Elsewhere, and Syndicate to your Own Site. To do that, I need to have an endpoint on my own site that can receive posts.

Micropub

Working side by side with Aaron at border:none inspired me to finally implement one more indie web building block I needed: micropub.

Having a micropub endpoint here on my own site means that I can publish from third-party sites …or even from native apps. The reason why I didn’t have one already was that I thought it would be really complicated to implement. But it turns out that, once again, the trick is to let other services do all the hard work.

First of all, I need to have something to manage authentication. Well, I already have that with IndieAuth. I got that for free just by adding rel="me" to my links to other profiles. So now I can declare indieauth.com as my authorization endpoint in the head of my HTML:

<link rel="authorization_endpoint" href="https://indieauth.com/auth" />

Now I need some way of creating and issuing authentation tokens. See what I mean about it sounding like hard work? Creating a token endpoint seems complicated.

But once again, someone else has done the hard work so I don’t have to. Tokens-as-a-service:

<link rel="token_endpoint" href="https://tokens.indieauth.com/token" />

The last piece of the puzzle is to point to my own micropub endpoint:

<link rel="micropub" href="https://adactio.com/micropub" />

That URL is where I will receive posts from third-party sites and apps (sent through a POST request with an access token in the header). It’s up to me to verify that the post is authenticated properly with a valid access token. Here’s the PHP code I’m using.

It wasn’t nearly as complicated as I thought it would be. By the time a post and a token hits the micropub endpoint, most of the hard work has already been done (authenticating, issuing a token, etc.). But there are still a few steps that I have to do:

1. Make a GET request (I’m using cURL) back to the token endpoint I specified—sending the access token I’ve been sent in a header—verifying the token.
2. Check that the “me” value that I get back corresponds to my identity, which is https://adactio.com
3. Take the h-entry values that have been sent as POST variables and create a new post on my site.

I tested my micropub endpoint using Quill, a nice little posting interface that Aaron built. It comes with great documentation, including a guide to creating a micropub endpoint.

It worked.

Here’s another example: Ben Roberts has a posting interface that publishes to micropub, which means I can authenticate myself and post to my site from his interface.

Finally, there’s OwnYourGram, a service that monitors your Instagram account and posts to your micropub endpoint whenever there’s a new photo.

That worked too. And I can also hook up Bridgy to my Instagram account so that any activity on my Instagram photos also gets sent to my webmention endpoint.

Indie Web Camp

Each one of these building blocks unlocks greater and greater power:

• rel="me"
• h-entry
• Webmention
• Micropub

Each one of those building blocks you implement unlocks more and more powerful tools:

• IndieAuth
• Bridgy
• Quill
• OwnYourGram

But its worth remembering that these are just implementation details. What really matters is that you’re publishing your stuff on your website. If you want to use different formats and protocols to do that, that’s absolutely fine. The whole point is that this is the independent web—you can do whatever you please on your own website.

Still, if you decide to start using these tools and technologies, you’ll get the benefit of all the other people who are working on this stuff. If you have the chance to attend an Indie Web Camp, you should definitely take it: I’m always amazed by how much is accomplished in one weekend.

Some people have started referring to the indie web movement. I understand where they’re coming from; it certainly looks like a “movement” from the outside, and if you attend an Indie Web Camp, there’s a great spirit of sharing. But my underlying motivations are entirely selfish. In the same way that I don’t really care about particular formats or protocols, I don’t really care about being part of any kind of “movement.” I care about my website.

As it happens, my selfish motivations align perfectly with the principles of an indie web.

I was up in London yesterday to spend the day with the web developers of a Clearleft client, talking front-end architecture and strategies for implementing responsive design. ‘Twas a good day, although London always tires me out quite a bit.

On this occasion, I didn’t head straight back to Brighton. Instead I braved the subterranean challenges of the Tube to make my way across london to Google Campus, where a panel discussion was taking place. This was Meet The TAG.

TAG is the Technical Architecture Group at the W3C. It doesn’t work on any one particular spec. Instead, it’s a sort of meta-group to steer how standards get specified.

Gathered onstage yesterday evening were TAG members Anne van Kesteren, Tim Berners-Lee, Alex Russell, Yehuda Katz, and Daniel Appelquist (Henry Thompson and Sergey Konstantinov were also there, in the audience). Once we had all grabbed a (free!) beer and settled into our seats, Bruce kicked things off with an excellent question: in the intros, multiple TAG members mentioned their work as guiding emerging standards to make sure they matched the principles of the TAG …but what are those principles?

It seemed like a fairly straightforward question, but it prompted the first rabbit hole of the evening as Alex and Yehuda focussed in on the principle of “layering”—stacking technologies in a sensible way that provides the most power to web developers. It’s an important principle for sure, but it didn’t really answer Bruce’s question. I was tempted to raise my hand and reformulate Bruce’s question into three parts:

1. Does the Technical Architecture Group have design principles?
2. If so, what are there?
3. And are they written down somewhere?

There’s a charter and that contains a mission statement, but that’s not the same as documenting design principles. There is an extensible web manifesto—that does document design principles—which contains the signatures of many (but not all) TAG members …so does that represent the views of the TAG? I’d like to get some clarification on that.

The extensible web manifesto does a good job of explaining the thinking behind projects like web components. It’s all about approaching the design of new browser APIs in a sensible (and extensible) way.

I mentioned that the TAG were a kind of meta-standards body, and in a way, what the extensible web manifesto—and examples like web components—are proposing is a meta-approach to how browsers implement new features. Instead of browser makers (in collaboration with standards bodies) creating new elements, UI widgets and APIs, developers will create new elements and UI widgets.

When Yehuda was describing this process, he compared it with the current situation. Currently, developers have to petition standards bodies begging them to implement some new kind of widget and eventually, if you’re lucky, browsers might implement it. At this point I interrupted to ask—somewhat tongue-in-cheek—”So if we get web components, what do we need standards bodies for?” Alex had an immediate response for that: standards bodies can look at what developers are creating, find the most common patterns, and implement them as new elements and widgets.

“I see,” I said. “So browsers and standards bodies will have a kind of ‘rough consensus’ based on …running code?”

“Yes!”, said Alex, laughing. “Jeremy Keith, ladies and gentlemen!”

So the idea with web components (and more broadly, the extensible web) is that developers will be able to create new elements with associated JavaScript functionality. Currently developers are creating new widgets using nothing but JavaScript. Ideally, web components will result in more declarative solutions and reduce our current reliance on JavaScript to do everything. I’m all for that.

But one thing slightly puzzled me. The idea of everyone creating whatever new elements they want isn’t a new one. That’s the whole idea behind XML (and by extension, XHTML) and yet the very same people who hated the idea of that kind of extensibility are the ones who are most eager about web components.

Playing devil’s advocate, I asked “How come the same people who hated RDF love web components?” (although what I really meant was RDFa—a means of extending HTML).

I got two answers. The first one was from Alex. Crucially, he said, a web component comes bundled with instructions on how it works. So it’s useful. That’s a big, big difference to the Tower of Babel scenario where everyone could just make up their own names for elements, but browsers have no idea what those names mean so effectively they’re meaningless.

That was the serious answer. The other answer I got was from Tim Berners-Lee. With a twinkle in his eye and an elbow in Alex’s ribs he said, “Well, these youngsters who weren’t around when we doing things with XML all want to do things with JSON now, which is a much cooler format because you can store number types in it. So that’s why they want to do everything in JavaScript.” Cheeky trickster!

Anyway, there was plenty of food for thought in the discussion of web components. This really is a radically new and different way of adding features to browsers. In theory, it shifts the balance of power much more to developers (who currently have to hack together everything using JavaScript). If it works, it will be A Good Thing and result in expanding HTML’s vocabulary with genuinely useful features. I fear there may be a rocky transition to this new way of thinking, and I worry about backwards compatibility, but I can’t help but admire the audacity of the plan.

The evening inevitably included a digression into the black hole of DRM. As always, the discussion got quite heated and I don’t think anybody was going to change their minds. I tried to steer things away from the ethical questions and back to the technical side of things by voicing my concerns with the security model of EME. Reading the excellent description by Henri, sentences like this should give you the heebie-jeebies:

Neither the browser nor the JavaScript program understand the bytes.

But the whole DRM discussion was, fortunately, curtailed by Anne who was ostensibly moderating the panel. Before it was though, Sir Tim made one final point. Because of the heat of the discussion, people were calling for us to separate the societal questions (around intellectual property and payment) from the technical ones (around encryption). But, Sir Tim pointed out, that separation isn’t really possible. Even something as simple as the hyperlink has political assumptions built in about the kind of society that would value being able to link resources together and share them around.

That’s an important point, well worth remembering: all software is political. That’s one of the reasons why I’d really appreciate an explicit documentation of design principles from the Technical Architecture Group.

Still, it was a very valuable event. Bruce has also written down his description of the evening. Many thanks to Dan and the rest of the TAG team for putting it together. I’m very glad I went along. As well as the panel discussion, it was really nice to chat to Paul and have the chance to congratulate Jeni in person on her appearance on her OBE.

Alas, I couldn’t stick around too long—I had to start making the long journey back to Brighton—so I said my goodbyes and exited. I didn’t have the opportunity to speak to Tim Berners-Lee directly, which is probably just as well: I’m sure I would’ve embarrassed myself by being a complete fanboy.

Speaking of URLs…

We were having a discussion in the Clearleft office recently about that perennially-tricky navigation pivot: tags. Specifically, we were discussing how to represent the interface for combinatorial tags i.e. displaying results of items that have been tagged with tag A and tag B.

I realised that this was functionality that I wasn’t even offering on Huffduffer, so I set to work on implementing it. I decided to dodge the interface question completely by only offering this functionality through the browser address bar. As a fairly niche, power-user feature, I’m not sure it warrants valuable interface real estate—though I may revisit that challenge later.

I can’t use the + symbol as a tag separator because Huffduffer allows spaces in tags (and spaces are converted to pluses in URLs), so I’ve settled on commas instead.

For example, there are plenty of items tagged with “music” (/tags/music) and plenty of items tagged with “science” (/tags/science) but there’s only a handful of items tagged with both “music” and “science” (/tags/music,science).

This being Huffduffer, where just about every page has corresponding JSON, RSS and Atom representations, you can also subscribe to the podcast of everything tagged with both “music” and “science” (/tags/music,science/rss).

There’s an OR operator as well; the vertical pipe symbol. You can view the 60 items tagged with “html5”, the 14 items tagged with “css3”, or the 66 items tagged with either “html5” or “css3” (/tags/html5|css3).

Wait a minute …66 items? But 60 plus 14 equals 74, not 66!

The discrepancy can be explained by the 8 items tagged with both “css3” and “html5” (/tags/html5,css3).

The AND and OR operators can be combined, so you can find items tagged with either “science” or “religion” that are also tagged with “politics” (/tags/science|religion,politics).

While it’s fun to do this in the browser’s address bar, I think the real power is in the way that the corresponding podcast allows you to subscribe to precisely-tailored content. Find just the right combination of tags, click on the RSS link, and you’re basically telling iTunes to automatically download audio whenever there’s something new that matches criteria like:

• everything that is tagged with either “science fiction” or “fantasy” or “horror” and is also tagged with “romance” (/tags/science+fiction|fantasy|horror,romance),
• everything that is tagged with “google” and is also tagged with either “apple” or “microsoft” or “yahoo” (/tags/google,apple|microsoft|yahoo).

I’m sure there are plenty of intriguing combinations out there. Now I can use Huffduffer’s URLs to go spelunking for audio gems at the most promising intersections of tags.

I’ve already described how machine tags on Huffduffer trigger a number of third-party API calls. Tagging something with music:artist=..., book:author=..., film:title=... or any number of similar machine tags will fire off calls to places like Amazon, The New York Times, or Last.fm.

For a while now, I’ve wanted to include Flickr in that list of third-party services but I couldn’t think of an easy way of associating audio files with photos. Then I realised that a mechanism already exists, and it’s another machine tag. Anything on Flickr that’s been tagged with lastfm:event=... will probably be a picture of a musical artist.

So if anything is tagged on Huffduffer with music:artist=..., all I need to do is fire off a call to Last.fm to get a list of that artist’s events using the method artist.getEvents. Once I have the event IDs I can search Flickr for photos that have been machine tagged with those IDs.

There’s just one problem. Last.fm’s API only returns future events for an artist. There’s no method for past events.

Undeterred, I found a RESTful interface that provides the past events of an artist on Last.fm. The format returned isn’t JSON or XML. It’s HTML. It turns out that past events are freely available in the profile for any artist on Last.fm with the identifier last.fm/music/{artist}/+events/{year}. Here, for example, are Salter Cane gigs in 2009: last.fm/music/Salter+Cane/+events/2009.

If only those events were structured in hCalendar! As it is, I have to run through all the links in the document to find the hrefs beginning with the string http://www.last.fm/event/ and then extract the event ID that immediately follows that string.

Once I’ve extracted the event IDs for an artist, I can fire off a search on Flickr using the flickr.photos.search method with a machine_tags parameter (as well as passing the artist name in the text parameter just to be sure).

Here’s an example result in the sidebar on Huffduffer: huffduffer.com/tags/music:artist=Bat+for+Lashes

It’s messy but it works. I guess that’s the dictionary definition of a hack.

After I started rewarding machine tagging on Huffduffer with API calls to Amazon and Last.fm, people started using them quite a bit. But when it came to displaying tag clouds, I wasn’t treating machine tags any differently to other tags. Everything was being displayed in one big cloud.

I decided it would be good to separate out machine tags and display them after displaying “regular” tags. That started me thinking about how best to display machine tags.

One of the best machine tag visualisations I’ve seen so far is Paul Mison’s Flickr machine tag browser, somewhat like the list view in OS X’s Finder. Initially, I tried doing something similar for Huffduffer: a table with three columns; namespace, predicate, and values.

That morphed into a two column layout (predicate and values) with the namespace spanning both columns. The values themselves are still displayed as a cloud to indicate usage.

This is marked up as a table. The namespace is in a th inside the thead. In the tbody, each tr contains a th for the predicate and td for the values.

<table>
 <thead>
  <tr>
   <th colspan="2"><a href="/tags/book">book</a></th>
  </tr>
 </thead>
 <tbody>
  <tr>
   <th><a href="/tags/book:author">author</a></th>
   <td><a href="/tags/book:author=arthur+c.+clarke">arthur c. clarke</a></td>
  </tr>
 </tbody>
</table>

Table markup allows for some nice :hover styles (in browsers that allow :hover styles on more than links). Whenever you hover over a table cell, you are also hovering over a table row and a table. By setting :hover states on all three elements, wayfinding becomes a bit clearer.

table:hover thead th a
table tbody tr:hover th a
table tbody tr td a:hover

See for yourself. I think it’s a pretty sturdy markup and style pattern that I’ll probably use again.

After I wrote about the hoops I had to jump through to get Amazon’s API to output JSON (via XSLT), Tom detailed a way of avoiding JSON by using XML-RPC. That’s very kind of him but the truth is that:

1. I like dealing with JSON and
2. the XSL transformation is done by Amazon, not me; that wouldn’t be the case if I used XML-RPC.

Anyway, having successfully created a Huffduffer-Amazon bridge using machine tags, I thought I’d do a little more hacking. Instead of restricting the mashup love to Amazon, I figured that Last.fm would be the perfect place to pull in information for anything tagged with the music namespace.

Last.fm has quite a full-featured API and yes, it can output JSON. To start with, I’m using the artist.getInfo method for anything tagged with music:artist=..., music:singer=... or music:band=.... Here are some examples:

• Law Low (live) by My Morning Jacket
• Let Me Know by Yeah Yeah Yeahs
• Cello Song by Nick Drake covered by José González and The Books

I’m pulling a summary of the artist’s bio, a list of similar artists and a picture of the artist in question. For maximum effect, view in Safari, the browser with the finest implementation of CSS3’s box-shadow property.

Nice as Last.fm’s API is, it’s not without its quirks. Like most APIs, the methods are divided into those that require authentication (anything of a sensitive nature) and those that don’t (publicly available information). The method user.getInfo requires authentication. Yet, every piece of information returned by that method is available on the public profile.

So when I wanted to find a Last.fm user’s profile picture—having figured out through Google’s Social Graph API when someone on Huffduffer has a Last.fm account—it made far more sense for me to use hKit to parse the microformatted public URL than to use the API method.

Just over two years ago, Drew delivered a superb presentation called Can Your Website Be Your API? In some situations, the answer is definitely “Yes.”

Update: It all ties together, as Julian explains on Twitter:

@adactio ha, I went to Drew’s presentation you mentioned on your blog; it made me add microformats to Last.fm in the first place :D

Over the weekend I was looking at the latest additions to Huffduffer. I noticed that Xavier Roy was using machine tags to tag a reading by Richard Dawkins. What an excellent idea!

I set aside a little time to do a little hacking with Amazon’s API. Now you can tag stuff on Huffduffer with machine tags like book:author=steven johnson, book:title=the invention of air or music:artist=my morning jacket. Other namespaces are film and movie. Anything matching that pattern will trigger a search on Amazon and display a list of results.

Amazon’s API was one of the first I ever messed about with, first on The Session and later on Adactio Elsewhere. There are things I really like about it and things I really dislike.

I dislike the fact that there’s no option to receive JSON instead of XML. However, one of the things I like is the option to pass the URL of an XSL file to transform the XML (I wish more APIs offered that service). So even though JSON isn’t officially offered, it’s perfectly feasible to generate JSON from the combination of XML + XSL. That’s what I did for the Huffduffer hacking—I find it a lot easier to deal with JSON than XML in PHP5. If you fancy doing something similar, help yourself to my XSL file. It’s very basic but it could make a decent starting point.

But the thing I dislike the most about the Amazon API is the documentation. It’s not that there’s a lack of documentation. Far from it. It’s just not organised very well. I find it very hard to get the information I need, even when I know that the information is there somewhere. Flickr still leads the pack when it comes to API documentation. Amazon would do well to take a leaf out of Flickr’s documentation book (hope you’re listening, Jeff).

At the same time that Flickr are demonstrating idiocy in the human resources department, they continue to do so some very cool stuff behind the scenes.

Aaron has been walking through some new API methods over on the Flickr code blog, quoting something I said in a chat with Steve Ivy:

something:somethingelse=somethingspecific

…which I don’t even remember saying but the shoe fits.

There’s something about the mix of rigidity and haphazardness in machine tags that appeals to me. While they all share the same structure, everyone is free to invent their own usage. If machine tags were required to go through a specification process we would have event:lastfm=... and event:upcoming=... instead of lastfm:event=... and upcoming:event=... but really, it simply doesn’t matter as long as people are actually doing the tagging.

With the introduction of these new API methods, it looks like there’s room to build more finely-tuned apps to pivot around namespaces, predicates and values.

Paul Mison has written an desktop-like machine tag browser which shows at a glance just how many different machine tag namespaces are out there. Quite a few pictures have been tagged with adactio:post=... since I first introduced the idea.