Tags: php

I’m quite sure this is of no interest to anyone but me, but I finally managed to fix a longstanding weird issue with my website.

I realise that me telling you about a bug specific to my website is like me telling you about a dream I had last night—fascinating for me; incredibly dull for you.

For some reason, my site was being brought to its knees anytime I syndicated a note to Mastodon. I rolled up my sleeves to try to figure out what the problem could be. I was fairly certain the problem was with my code—I’m not much of a back-end programmer.

My tech stack is classic LAMP: Linux, Apache, MySQL and PHP. When I post a note, it gets saved to my database. Then I make a curl request to the Mastodon API to syndicate the post over there. That’s when my CPU starts climbing and my server gets all “bad gateway!” on me.

After spending far too long pulling apart my PHP and curl code, I had to come to the conclusion that I was doing nothing wrong there.

I started watching which processes were making the server fall over. It was MySQL. That seemed odd, because I’m not doing anything too crazy with my database reads.

Then I realised that the problem wasn’t any particular query. The problem was volume. But it only happened when I posted a note to Mastodon.

That’s when I had a lightbulb moment about how the fediverse works.

When I post a note to Mastodon, it includes a link back to the original note to my site. At this point Mastodon does its federation magic and starts spreading the post to all the instances subscribed to my account. And every single one of them follows the link back to the note on my site …all at the same time.

This isn’t a problem when I syndicate my blog posts, because I’ve got a caching mechanism in place for those. I didn’t think I’d need any caching for little ol’ notes. I was wrong.

A simple solution would be not to include the link back to the original note. But I like the reminder that what you see on Mastodon is just a copy. So now I’ve got the same caching mechanism for my notes as I do for my journal (and I did my links while I was at it). Everything is hunky-dory. I can syndicate to Mastodon with impunity.

See? I told you it would only be of interest to me. Although I guess there’s a lesson here. Something something caching.

The Demise of the Mildly Dynamic Website

It me:

Broadly, these are websites which are still web pages, not web applications; they’re pages of essentially static information, personal websites, blogs, and so on, but they are slightly dynamic. They might have a style selector at the top of each page, causing a cookie to be set, and the server to serve a different stylesheet on every subsequent page load.

This rings sadly true to me:

Suppose a company makes a webpage for looking up products by their model number. If this page were made in 2005, it would probably be a single PHP page. It doesn’t need a framework — it’s one SELECT query, that’s it. If this page were made in 2022, a conundrum will be faced: the company probably chose to use a statically generated website. The total number of products isn’t too large, so instead their developers stuff a gigantic JSON file of model numbers for every product made by the company on the website and add some client-side JavaScript to download and query it. This increases download sizes and makes things slower, but at least you didn’t have to spin up and maintain a new application server. This example is fictitious but I believe it to be representative.

Also, I never thought about “serverless” like this:

Recently we’ve seen the rise in popularity of AWS Lambda, a “functions as a service” provider. From my perspective this is literally a reinvention of CGI, except a) much more complicated for essentially the same functionality, b) with vendor lock-in, c) with a much more complex and bespoke deployment process which requires the use of special tools.

Pinboard is Eleven (Pinboard Blog)

I probably need to upgrade the Huffduffer server but Maciej nails why that’s an intimidating prospect:

Doing this on a live system is like performing kidney transplants on a playing mariachi band. The best case is that no one notices a change in the music; you chloroform the players one at a time and try to keep a steady hand while the band plays on. The worst case scenario is that the music stops and there is no way to unfix what you broke, just an angry mob. It is very scary.

I wrote yesterday about how messing about on your own website can be a welcome distraction. I did some tinkering with adactio.com on the weekend that you might be interested in.

Let me set the scene…

I’ve started recording and publishing a tune a day. I grab my mandolin, open up Quicktime and make a movie of me playing a jig, a reel, or some other type of Irish tune. I include a link to that tune on The Session and a screenshot of the sheet music for anyone who wants to play along. And I embed the short movie clip that I’ve uploaded to YouTube.

Now it’s not the first time I’ve embedded YouTube videos into my site. But with the increased frequency of posting a tune a day, the front page of adactio.com ended up with multiple embeds. That is not good for performance—my Lighthouse score took quite a hit. Worst of all, if a visitor doesn’t end up playing an embedded video, all of the markup, CSS, and JavaScript in the embedded iframe has been delivered for nothing.

Meanwhile over on The Session, I’ve got a strategy for embedding YouTube videos that’s better for performance. Whenever somebody posts a link to a video on YouTube, the thumbnail of the video is embedded. Only when you click the thumbnail does that image get swapped out for the iframe with the video.

That’s what I needed to do here on adactio.com.

First off, I should explain how I’m embedding things generally ‘round here. Whenever I post a link or a note that has a URL in it, I run that URL through a little PHP script called getEmbedCode.php.

That code checks to see if the URL is from a service that provides an oEmbed endpoint. A what-Embed? oEmbed!

oEmbed is like a minimum viable read-only API. It was specced out by Leah and friends years back. You ping a URL like this:

http://example.com/oembed?url=https://example.com/thing

In this case http://example.com/oembed is the endpoint and url is the value of a URL from that provider. Here’s a real life example from YouTube:

https://www.youtube.com/oembed?url=https://www.youtube.com/watch?v=-eiqhVmSPcs

So https://www.youtube.com/oembed is the endpoint and url is the address of any video on YouTube.

You get back some JSON with a pre-defined list of values like title and html. That html payload is the markup for your embed code.

By default, YouTube sends back markup like this:

<iframe
width="480"
height="270"
src="https://www.youtube.com/embed/-eiqhVmSPcs?feature=oembed"
frameborder="0
allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"
allowfullscreen>
</iframe>

But now I want to use an img instead of an iframe. One of the other values returned is thumbnail_url. That’s the URL of a thumbnail image that looks something like this:

https://i.ytimg.com/vi/-eiqhVmSPcs/hqdefault.jpg

In fact, once you know the ID of a YouTube video (the ?v= bit in a YouTube URL), you can figure out the path to multiple images of different sizes:

• 120 × 90: https://i.ytimg.com/vi/-eiqhVmSPcs/default.jpg
• 320 × 180: https://i.ytimg.com/vi/-eiqhVmSPcs/mqdefault.jpg
• 480 × 360: https://i.ytimg.com/vi/-eiqhVmSPcs/hqdefault.jpg
• 1280 × 720: https://i.ytimg.com/vi/-eiqhVmSPcs/maxresdefault.jpg

(Although that last one—maxresdefault.jpg—might not work for older videos.)

Okay, so I need to extract the ID from the YouTube URL. Here’s the PHP I use to do that:

parse_str(parse_url($url, PHP_URL_QUERY), $arguments);
$id = $arguments['v'];

Then I can put together some HTML like this:

<div>
<a class="videoimglink" href="'.$url.'">
<img width="100%" loading="lazy"
src="https://i.ytimg.com/vi/'.$id.'/default.jpg"
alt="'.$response['title'].'"
srcset="
https://i.ytimg.com/vi/'.$id.'/mqdefault.jpg 320w,
https://i.ytimg.com/vi/'.$id.'/hqdefault.jpg 480w,
https://i.ytimg.com/vi/'.$id.'/maxresdefault.jpg 1280w
">
</a>
</div>

Now I’ve got a clickable responsive image that links through to the video on YouTube. Time to enhance. I’m going to add a smidgen of JavaScript to listen for a click on that link.

Over on The Session, I’m using addEventListener but here on adactio.com I’m going to be dirty and listen for the event directly in the markup using the onclick attribute.

When the link is clicked, I nuke the link and the image using innerHTML. This injects an iframe where the link used to be (by updating the innerHTML value of the link’s parentNode).

onclick="event.preventDefault();
this.parentNode.innerHTML='<iframe src=https://www.youtube-nocookie.com/embed/'.$id.'?autoplay=1></iframe>'"

But notice that I’m not using the default YouTube URL for the iframe. That would be:

https://www.youtube.com/embed/-eiqhVmSPcs

Instead I’m swapping out the domain youtube.com for youtube-nocookie.com:

https://www.youtube-nocookie.com/embed/-eiqhVmSPcs

I can’t remember where I first came across this undocumented parallel version of YouTube that has, yes, you guessed it, no cookies. It turns out that, not only is the default YouTube embed code bad for performance, it is—unsurprisingly—bad for privacy too. So the youtube-nocookie.com domain can protect your site’s visitors from intrusive tracking. Pass it on.

Anyway, I’ve got the markup I want now:

<div>
<a class="videoimglink" href="https://www.youtube.com/watch?v=-eiqhVmSPcs"
onclick="event.preventDefault();
this.parentNode.innerHTML='<iframe src=https://www.youtube-nocookie.com/embed/-eiqhVmSPcs?autoplay=1></iframe>'">
<img width="100%" loading="lazy"
src="https://i.ytimg.com/vi/-eiqhVmSPcs/default.jpg"
alt="The Banks Of Lough Gowna (jig) on mandolin"
srcset="
https://i.ytimg.com/vi/-eiqhVmSPcs/mqdefault.jpg 320w,
https://i.ytimg.com/vi/-eiqhVmSPcs/hqdefault.jpg 480w,
https://i.ytimg.com/vi/-eiqhVmSPcs/maxresdefault.jpg 1280w
">
</a>
</div>

The functionality is all there. But I want to style the embedded images to look more like playable videos. Time to break out some CSS (this is why I added the videoimglink class to the YouTube link).

.videoimglink {
    display: block;
    position: relative;
}

I’m going to use generated content to create a play button icon. Because I can’t use generated content on an img element, I’m applying these styles to the containing .videoimglink a element.

.videoimglink::before {
    content: '▶';
}

I was going to make an SVG but then I realised I could just be lazy and use the unicode character instead.

Right. Time to draw the rest of the fucking owl:

.videoimglink::before {
    content: '▶';
    display: inline-block;
    position: absolute;
    background-color: var(--background-color);
    color: var(--link-color);
    border-radius: 50%;
    width: 10vmax;
    height: 10vmax;
    top: calc(50% - 5vmax);
    left: calc(50% - 5vmax);
    font-size: 6vmax;
    text-align: center;
    text-indent: 1vmax;
    opacity: 0.5;
}

That’s a bunch of instructions for sizing and positioning. I’d explain it, but that would require me to understand it and frankly, I’m not entirely sure I do. But it works. I think.

With a translucent play icon positioned over the thumbnail, all that’s left is to add a :hover style to adjust the opacity:

.videoimglink:hover::before,
.videoimglink:focus::before {
    opacity: 0.75;
}

Wheresoever thou useth :hover, thou shalt also useth :focus.

Okay. It’s good enough. Ship it!

If you embed YouTube videos on your site, and you’d like to make them more performant, check out this custom element that Paul made: Lite YouTube Embed. And here’s a clever technique that uses the srcdoc attribute to get a similar result (but don’t forget to use the youtube-nocookie.com domain).

It was Indie Web Camp Brighton on the weekend. After a day of thought-provoking discussions, I thoroughly enjoyed spending the second day tinkering on my website.

For a while now, I’ve wanted to add maps to my monthly archive pages (to accompany the calendar heatmaps I added at a previous Indie Web Camp). Whenever I post anything to my site—a blog post, a note, a link—it’s timestamped and geotagged. I thought it would be fun to expose that in a glanceable way. A map seems like the right medium for that, but I wanted to avoid the obvious route of dropping a load of pins on a map. Instead I was looking for something more like the maps in Indiana Jones films—a line drawn from place to place to show the movement over time.

I talked to Aaron about this and his advice was that a client-side JavaScript embedded map would be the easiest option. But that seemed like overkill to me. This map didn’t need to be pannable or zoomable; just glanceable. So I decided to see if how far I could get with a static map. I timeboxed two hours for it.

After two hours, I admitted defeat.

I was able to find the kind of static maps I wanted from Mapbox—I’m already using them for my check-ins. I could even add a polyline, which is exactly what I wanted. But instead of passing latitude and longitude co-ordinates for the points on the polyline, the docs explain that I needed to provide …cur ominous thunder and lightning… The Encoded Polyline Algorithm Format.

Go to that link. I’ll wait.

Did you read through the eleven steps of instructions? Did you also think it was a piss take?

1. Take the initial signed value.
2. Multiply it by 1e5.
3. Convert that decimal value to binary.
4. Left-shift the binary value one bit.
5. If the original decimal value is negative, invert this encoding.
6. Break the binary value out into 5-bit chunks.
7. Place the 5-bit chunks into reverse order.
8. OR each value with 0x20 if another bit chunk follows.
9. Convert each value to decimal.
10. Add 63 to each value.
11. Convert each value to its ASCII equivalent.

This was way beyond my brain’s pay grade. But surely someone else had written the code I needed? I did some Duck Duck Going and found a piece of PHP code to do the encoding. It didn’t work. I Ducked Ducked and Went some more. I found a different piece of PHP code. That didn’t work either.

At this point, my allotted time was up. If I wanted to have something to demo by the end of the day, I needed to switch gears. So I did.

I used Leaflet.js to create the maps I wanted using client-side JavaScript. Here’s the JavaScript code I wrote.

It waits until the page has finished loading, then it searches for any instances of the h-geo microformat (a way of encoding latitude and longitude coordinates in HTML). If there are three or more, it generates a script element to pull in the Leaflet library, and a corresponding style element. Then it draws the map with the polyline on it. I ended up using Stamen’s beautiful watercolour map tiles.

That’s what I demoed at the end of the day.

But I wasn’t happy with it.

Sure, it looked good, but displaying the map required requests for a script, a style sheet, and multiple map tiles. I made sure that it didn’t hold up the loading of the rest of the page, but it still felt wasteful.

So after Indie Web Camp, I went back to investigate static maps again. This time I did finally manage to find some PHP code for encoding lat/lon coordinates into a polyline that worked. Finally I was able to construct URLs for a static map image that displays a line connecting multiple points with a line.

I’ve put this maps on any of the archive pages that also have calendar heat maps. Some examples:

• Everything from August 2019
• Notes from July 2018
• Links from June 2017
• Photos from October 2014
• Journal entries from December 2012

If you go back much further than that, the maps start to trail off. That’s because I wasn’t geotagging everything from the start.

I’m pretty happy with the final results. It’s certainly far more responsible from a performance point of view. Oh, and I’ve also got the maps inside a picture element so that I can swap out the tiles if you switch to dark mode.

It’s a shame that I can’t use the lovely Stamen watercolour tiles for these static maps though.

When I was writing about the lie-fi strategy I’ve added to adactio.com, I finished with this thought:

What I’d really like is some way to know—on the client side—whether or not the currently-loaded page came from a cache or from a network. Then I could add some kind of interface element that says, “Hey, this page might be stale—click here if you want to check for a fresher version.”

Trys heard my plea, and came up with a very clever technique to alter the HTML of a page when it’s put into a cache.

It’s a function that reads the response body stream in, returning a new stream. Whilst reading the stream, it searches for the character codes that make up: <html. If it finds them, it tacks on a data-cached attribute.

Nice!

But then I was discussing this issue with Tantek and Aaron late one night after Indie Web Camp Düsseldorf. I realised that I might have another potential solution that doesn’t involve the service worker at all.

Caveat: this will only work for pages that have some kind of server-side generation. This won’t work for static sites.

In my case, pages are generated by PHP. I’m not doing a database lookup every time you request a page—I’ve got a server-side cache of posts, for example—but there is a little bit of assembly done for every request: get the header from here; get the main content from over there; get the footer; put them all together into a single page and serve that up.

This means I can add a timestamp to the page (using PHP). I can mark the moment that it was served up. Then I can use JavaScript on the client side to compare that timestamp to the current time.

I’ve published the code as a gist.

In a script element on each page, I have this bit of coducken:

var serverTimestamp = <?php echo time(); ?>;

Now the JavaScript variable serverTimestamp holds the timestamp that the page was generated. When the page is put in the cache, this won’t change. This number should be the number of seconds since January 1st, 1970 in the UTC timezone (that’s what my server’s timezone is set to).

Starting with JavaScript’s Date object, I use a caravan of methods like toUTCString() and getTime() to end up with a variable called clientTimestamp. This will give the current number of seconds since January 1st, 1970, regardless of whether the page is coming from the server or from the cache.

var localDate = new Date();
var localUTCString = localDate.toUTCString();
var UTCDate = new Date(localUTCString);
var clientTimestamp = UTCDate.getTime() / 1000;

Then I compare the two and see if there’s a discrepency greater than five minutes:

if (clientTimestamp - serverTimestamp > (60 * 5))

If there is, then I inject some markup into the page, telling the reader that this page might be stale:

document.querySelector('main').insertAdjacentHTML('afterbegin',`
  <p class="feedback">
    <button onclick="this.parentNode.remove()">dismiss</button>
    This page might be out of date. You can try <a href="javascript:window.location=window.location.href">refreshing</a>.
  </p>
`);

The reader has the option to refresh the page or dismiss the message.

It’s not foolproof by any means. If the visitor’s computer has their clock set weirdly, then the comparison might return a false positive every time. Still, I thought that using UTC might be a safer bet.

All in all, I think this is a pretty good method for detecting if a page is being served from a cache. Remember, the goal here is not to determine if the user is offline—for that, there’s navigator.onLine.

The upshot is this: if you visit my site with a crappy internet connection (lie-fi), then after three seconds you may be served with a cached version of the page you’re requesting (if you visited that page previously). If that happens, you’ll now also be presented with a little message telling you that the page isn’t fresh. Then it’s up to you whether you want to have another go.

I like the way that this puts control back into the hands of the user.

I’ve been using Mailchimp for years now to send out a weekly newsletter from The Session. But I never visit the Mailchimp website. Instead, I use the API to create a campaign each week, and then send it out. I also use the API whenever a member of The Session updates their email preferences (or changes their details).

I got an email from Mailchimp that their old API was being deprecated and I’d need to update to their more recent one. The code I was using had been happily running for about seven years, but now I’d have to change it.

Luckily, Drew has written a really handy Mailchimp API wrapper for PHP, the language that The Session’s codebase is in. Thanks, Drew! I downloaded that wrapper and updated my code accordingly.

Everything went pretty smoothly. I was able to create campaigns, send campaigns, add new subscribers, and delete subscribers. But I ran into an issue when I wanted to update someone’s email address (on The Session, you can edit your details at any time, including your email address).

Here’s the set up:

use \DrewM\MailChimp\MailChimp;
$MailChimp = new MailChimp('abc123abc123abc123abc123abc123-us1');
$list_id = 'b1234346';
$subscriber_hash = $MailChimp -> subscriberHash('currentemail@example.com');
$endpoint = 'lists/'.$listID.'/members/'.$subscriber_hash;

Now to update details, according to the API, I can use the patch method on that endpoint:

$MailChimp -> patch($endpoint, [
    'email_address' => 'newemail@example.com'
]);

But that doesn’t work. Mailchimp effectively treats email addresses as unique IDs for subscribers. So the only way to change someone’s email address appears to be to delete them, and then subscribe them fresh with the new email address:

$MailChimp -> delete($endpoint);
$newendpoint = 'lists/'.$listID.'/members';
$MailChimp -> post($newendpoint, [
    'email_address' => 'newemail@example.com',
    'status' => 'subscribed'
]);

That’s somewhat annoying, as the previous version of the API allowed email addresses to be updated, but this workaround isn’t too arduous.

Anyway, I figured it share this just in case it was useful for anyone else migrating to the newer API.

Update: Belay that. Turns out that you can update email addresses, but you have to be sure to include the status value:

$MailChimp -> patch($endpoint, [
    'email_address' => 'newemail@example.com',
    'status' => 'subscribed'
]);

Okay, that’s a lot more straightforward. Ignore everything I said.

Quick Note: Setting up a localhost on a Mac | scottohara.me

Okay, I knew about the Python shortcut—I mentioned it in Going Offline—but I had no idea it was so easy to do the same thing for PHP. This is a bit of a revelation for me!

Once in the desired directory, run:

php -S localhost:2222

Now you can go to “localhost:2222” in your browser, and if you have an index.html or .php file in your root directory, you’re in business.

Essential Vanilla JavaScript Functions

The title is overkill, but these functions ported from PHP to JavaScript could be useful (especially for dealing with arrays).

A few technical words about Upsideclown, and some thoughts about audiences and the web (17 Aug., 2017, at Interconnected)

Matt writes about the pleasure of independent publishing on the web today:

It feels transgressive to have a website in 2017. Something about having a domain name and about coding HTML which is against the grain now. It’s something big companies do, not small groups. We’re supposed to put our content on Facebook or Medium, or keep our publishing to an email newsletter. But a website?

But he points out a tension between the longevity that you get from hosting the canonical content yourself, and the lack of unified analytics when you syndicate that content elsewhere.

There’s no simple online tool that lets me add up how many people have read a particular story on Upsideclown via the website, the RSS feed, and the email newsletter. Why not? If I add syndication to Facebook, Google, and Apple, I’m even more at sea.

bastianallgeier/letter: Letter is a simple, highly customizable tool to create letters in your browser.

A nice little use of print (and screen) styles from Bastian—compose letters in a web browser.

Instead of messing around in Word, Pages or even Indesign, you can write your letters in the browser, export them as HTML or PDF (via Apple Preview).

VerbalExpressions/JSVerbalExpressions

Regular expressions are my kryptonite so I can definitely imagine using the PHP port of this plain English syntax.

When I’m grilling outside, I cook on a gas barbecue. There are quite a few people who would take issue with this. Charcoal is clearly better, they claim. And they’re right. But the thing is, I can fire up my gas barbecue quickly and just get down to cooking.

When I’m programming on the server, I code in PHP. There are quite a few people who would take issue with this. Any other language is clearly better, they claim. And they’re right. But the thing is, I can fire up my text editor quickly and just get down to coding.

PHP is the right tool for the job (for all the wrong reasons) - Sam says you should read this

I think there’s a lot of truth to this. By any objective measurement, PHP is clearly inferior to just about every other programming language out there …but its preinstalled out-of-the-box nature means it’s the path of least resistance.

bramus/mixed-content-scan

A really handy command-line tool that scans your site for mixed content — very useful if you’re making the switch from http to https.

Unfashionably profitable

Rachel talks about some of the old-fashioned technologies and business practices driving Perch.

This reminds of a talk by Marco Arment at Webstock a few years back when he described the advantages of not using cutting-edge technologies: most of the time, “boring” well-established technologies are simply more stable.

Craft a better web.

A new PHP-based content management system. It uses Twig for the templating, which I like.

I’m on a workshopping roll. Fresh from running my Responsive Enhancement workshop in Belfast, I’m now heading to Düsseldorf for Beyond Tellerand where I’ll be running the workshop on Sunday (and if you can’t make it, don’t forget that you can book the workshop for your own workplace too).

As part of the process of building a responsive site from the content out rather than the canvas in, I talk about beginning with the individual components divorced from any layout context. Or, as Mark puts it, “start with the bits.”

That’s the way I’ve been starting most of my projects lately: beginning with the atomic units of content and styling them first before even thinking about layout. This ensures that those styles are extremely robust—because they don’t depend on any particular context, they can be safely dropped into any part of a page.

I’ve been calling this initial collection of markup snippets a pattern primer. To help create the pattern primer, I’ve written a little bit of PHP to automatically generate a page of patterns from a folder of HTML snippets.

In my workshop I keep promising to put that script on Github. I finally got around to doing that and you can find it at github.com/adactio/Pattern-Primer.

Take a look at an example pattern primer to get an idea of what a handy deliverable this can be if you’re handing off to other developers. It also acts like a page of unit tests for CSS—whenever you’ve been messing around in the stylesheet you can refresh the page to quickly check to see if anything looks screwed up.

Grab the code; improve upon it; share your changes.

Responsive HTML images

If you’re trying to retrofit an existing desktop-centric site for small screens, this server-side image-resizing technique might be useful but is definitely not the right tool for a content-out, small-screen-first approach.

Markdownify: The HTML to Markdown converter for PHP

This could be handy for the editing process in my home-grown blogging system: a PHP script to convert HTML back to Markdown.