Tag Archives: rollback

Andy Fragen 5:21 pm on April 19, 2024
Tags: , , , , rollback   

Merge Proposal: Rollback Auto-Update

Background

The biggest risk for a site owner when updating plugins is encountering a PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher fatal error that crashes their website. While CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. updates are protected by automatic rollbacks since WordPress 3.7 (#22704), no such protection for plugins was added. Although fatal error protection and recovery mode were added in WordPress 5.2, it requires manual intervention from an administrator, and ideally WordPress should be able to recover on its own in a similar way that Core does. The Upgrade/Install team began exploring rollbacks for pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party updates.

Rollbacks for plugin updates comprises three features:

  1. move_dir() – Introduced in WordPress 6.2 (#57375)
  2. Rollback for plugin/theme update failures when updating manually – Introduced in WordPress 6.3 (#51857)
  3. Rollback for plugin auto-updates when failures are encountered – Covered by this proposal (#58281)

Some background references

Overview

When active plugins are updated, they are briefly deactivated before the new version is installed and reactivated immediately after. Since WordPress 6.3, when an administrator is manually updating plugins, the plugin will not be reactivated if the update causes a PHP fatal error. During an auto-update, this reactivation check does not occur and the next time the site runs users will see the white screen of death (WSOD).

To further protect websites and increase confidence in automatic plugin updates within WordPress, the Rollback Auto-Update feature pluginFeature Plugin A plugin that was created with the intention of eventually being proposed for inclusion in WordPress Core. See Features as Plugins. aims to detect PHP fatal errors during automatic plugin updates, and subsequently rolls back to the previously installed version.

This proposal is to merge the changes required to also perform rollbacks when fatal errors occur during attempted plugin auto-updates by default.

Implementation

Rollback Auto-Update performs a loopback request to the homepage to check for PHP fatal errors, using an approach similar to the Plugin or Theme File Editors. If a PHP fatal error is encountered, an error handler logs the specific message and the previously installed version of the plugin is restored. When a plugin rollback occurs, a notification will be sent to the site’s administration email (stored in the admin_email option) notifying them of the failed update and rollback.

The current implementation attempts to detect a PHP fatal error during an automatic update by using a loopback request to the homepage. If the loopback returns an error, a PHP fatal error in the active plugin is assumed and the update will be reverted for safety.

After the problematic plugin is rolled back, the auto-updating process will continue for any other core, plugin, or theme updates that were in queue. When the next check for auto-updates occurs, WordPress will attempt to update the same plugin again.

Previously, maintenance mode was only enabled during installation of an update. However, testing established that disabling maintenance mode during the rest of the process means that active visits to the site can trigger errors. This can have side effects, such as deactivating plugins, etc. Rollback Auto-Update enables maintenance mode for the duration of all automatic updates. While maintenance mode is enabled for longer, this is relative to the number of updates being performed at that given time – usually a very low number – and helps improve stability of automatic updates.

At the time of publishing, this code is being tested on 6,000+ sites running the Rollback Update Failure feature plugin, which has contained the related code since v7.0.0 was released on 10/12/2023.

For easier testing of the feature within wordpress-develop, a merge PR (Core-5287) has been created.

Due to limitations in the ability to modify wp_is_maintenance_mode() in the feature plugin, the PR is slightly different. The feature plugin is available for historical reference. All pre-merge testing should be done using the PR. Some contributors have also been running the PR on sites for a few months.

Example of email text sent to site’s administration email.

Howdy! Plugins failed to update on your site at https://test.xxxxx.net.

Please check your site now. It’s possible that everything is working. If there are updates available, you should update.

The following plugins failed to update. If there was a fatal error in the update, the previously installed version has been restored.

  • This Plugin Should Not Be Used (from version 0.1 to 0.2) : https://wordpress.org/plugins/this-plugin-should-not-be-used/

To manage plugins on your site, visit the Plugins page: https://test.xxxxx.net/wp-admin/plugins.php

If you experience any issues or need support, the volunteers in the WordPress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ support forums may be able to help.
https://wordpress.org/support/forums/

The WordPress Team

Testing

There are no known issues directly related to Rollback Auto-Update that don’t currently exist in Core.

I (@afragen) have been testing using the test plugin. The plugin is on a test site, active, and set to auto-update. I have been running like this since the beginning of the year using the PR and on other sites for several years using the feature plugin.

  1. Install the PR into WP 6.5.x or trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision..
  2. Install version 0.1 of the test plugin.
  3. Activate the test plugin and enable auto-updates.

The WordPress.org update APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. will serve the version 0.2 version of the plugin, which will cause a PHP fatal error. To confirm a rollback is successful, data is written to the error.log at every point in the auto-update process, creating an audit trail the user can use to discern the flow and results of rolling back an auto-update. This logging is only intended for testing purposes.

FAQ

Stay up to date with development on the PR. Please comment on the GitHub PR if any problems are discovered. 

  • What happens if loopback requests aren’t working?
    • As demonstrated by the Site Health message for loopback requests that aren’t working: “Loopback requests are used to run scheduled events, and are also used by the built-in editors for themes and plugins to verify code stability.” 
    • Auto-updates shouldn’t run if loopback requests aren’t working. If a loopback request fails due to an HTTPHTTP HTTP is an acronym for Hyper Text Transfer Protocol. HTTP is the underlying protocol used by the World Wide Web and this protocol defines how messages are formatted and transmitted, and what actions Web servers and browsers should take in response to various commands. error, Rollback Auto-Update will consider it as a PHP fatal error detected, and revert any plugin updates.

As a project, it’s important to continuously evaluate ways to make site management easier for the large majority of users and site owners. Providing safer plugin auto-updates is just one way for WordPress itself to handle problems that may require technical expertise seamlessly for end users.

All feedback will be collected and addressed over the next 2-3 weeks, with the goal of committing to trunk after all feedback is addressed to ensure that the feature gets plenty of testing through the nightly builds early in the WordPress 6.6 release cycle.

Props: @costdev and @desrosj for review and editing.

#6-6, #feature-projects, #feature-autoupdates, #merge-proposals, #rollback

Andy Fragen 11:53 pm on June 26, 2022
Tags: rollback   

Rollback Feature: Testing Call to Action

While it is a rare occurrence, updating plugins and themes can fail in such a way that leaves the old pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party/theme non-functional, and may leave users with a broken site.  #51857 intends to add the ability for coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. to “rollback” to the previously installed version of a plugin/theme when such a failure occurs.

Over 19 months of development and testing has gone into this feature, primarily by @costdev, @pbiron, and myself (@afragen). Several different solutions and they have led us to the current PR2225.

It was determined that copying the current plugin to an alternate location and in the event of an update failure, copying it back into wp-content/plugins, would be the least resource intensive method. This does require one additional plugin copying operation and two if there is a update failure.

rename() was essential to the rollback feature as many reported issues were timeout issues on some resource starved hosts, #54166 as an example. Using rename() allows us to vastly improve the performance of the current method, rather than using copy_dir() as a recursive file copy. It was thought that this recursive file copy, on some systems, resulted in timeout issues for large plugins.

Do you use a VirtualBox-based environment?

Please follow these steps before continuing.

Call to Action

Big Need:

Broad testing and feedback is needed on many different web hosting platforms, including inexpensive shared hosting, managed hosting, and more.

How do I test Rollback?

Do not test on a production siteProduction Site A production site is a live site online meant to be viewed by your visitors, as opposed to a site that is staged for development or testing..

But do test on a local environment, hosted staging or test environment, or cloud staging or test environment.

  1. Here are some large plugins used for testing: akismet, jetpack, mailpoet, woocommerce, wpforms-lite, wordpress-seo
    • WP-CLIWP-CLI WP-CLI is the Command Line Interface for WordPress, used to do administrative and development tasks in a programmatic way. The project page is http://wp-cli.org/ https://make.wordpress.org/cli/: wp plugin install akismet jetpack mailpoet woocommerce wpforms-lite wordpress-seo
  2. Do this from the plugin’s page on https://wordpress.org/plugins by navigating to the “Development” tab, clicking “Advanced” to the right, and downloading an older version from the dropdown at the bottom of the page. You can also install the current version then modify the version in the plugin’s main file to decrement the version number.
  3. Install the WordPress Beta Tester plugin, set to Bleeding edgebleeding edge The latest revision of the software, generally in development and often unstable. Also known as trunk. and Nightlies. Go to Dashboard > Updates and click the Update to latest 6.2 nightly button or use WP-CLI, wp core update --version=trunk --force
  4. Install the Rollback feature plugin or test using the PR2225 in WordPress/wordpress-develop.
  5. Please make a note of the time required to perform plugin updates. Your phone’s stopwatch function may be the easiest method to do this.

Testing a single plugin update:

  1. Navigate to Plugins > Installed Plugins.
  2. Click “Update Now” located within the plugin row.

Testing bulk plugin updates via “Plugins”:

  1. Navigate to Plugins > Installed Plugins.
  2. Select another two plugins, select “Update” from the Bulk Actions dropdown, and click “Apply”.

Testing bulk plugin updates via “Dashboard”:

  1. Navigate to Dashboard > Updates
  2. Tick all plugins with an available update.
  3. Select “Update” from the Bulk Actions dropdown, and click “Apply”.

Testing updates via WP-CLI (if already familiar).

Validation of successful updates

This requires activating all the testing plugins on your testing site. Unsuccessful updates should show PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher Errors or PHP Fatal Errors.

  1. Activate each of the plugins that were updated.
  2. In WP Adminadmin (and super admin), navigate to each plugin’s menu pages.
  3. Navigate the frontend of your test site.
  4. Navigate to your wp-content/temp-backup/plugins folder. It should be empty.

Forcing an update failure

Use the following filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. to force an update failure. This will reinstall the previously active plugin/theme.

add_filter( 'upgrader_install_package_result', function() {
  return new WP_Error( 'simulated_error', 'Simulated Error' );
});

Testing update failures

When testing for failures on the bulk update in update-core.php you must use the PR. There is a modification in the PR that stops WP_Upgrader::unpack_package() from deleting the items in the temp-backup directory.

Needed Feedback?


How long did the update(s) take?

  • Your phone’s stopwatch may be the easiest option for logging the time to update.
  • Please test without the Rollback plugin or PR active for a baseline.
  • Reinstall the earlier versions as before.
  • Test the updates again with the Rollback plugin or the PR active.
  • Please provide the list of updated plugins/themes, the timings, and your environment details.
    • web host
    • PHP version
    • nginxNGINX NGINX is open source software for web serving, reverse proxying, caching, load balancing, media streaming, and more. It started out as a web server designed for maximum performance and stability. In addition to its HTTP server capabilities, NGINX can also function as a proxy server for email (IMAP, POP3, and SMTP) and a reverse proxy and load balancer for HTTP, TCP, and UDP servers. https://www.nginx.com/., apacheApache Apache is the most widely used web server software. Developed and maintained by Apache Software Foundation. Apache is an Open Source software available for free., etc
    • WordPress version
  • Continue

Did you receive any “Failed to update” errors?

  • Yes:
    • Please provide environment details.
    • Please provide the name of the plugin(s) that failed.
    • Was this a single plugin update or a bulk plugin update?
      • Bulk plugin update:
        • Where was the failed plugin in the list of plugins to be updated?
        • Were you on the Dashboard > Updates page, the Plugins > Installed plugins page, or using WP-CLI?
  • No: Continue

Did you receive any Fatal Errors when browsing your site after the update?

  • Yes:
    • Please provide your environment details.
    • Please provide the name of the plugin(s) that resulted in Fatal Errors.
    • Please provide the Fatal Errors that were displayed.
    • Was this a single plugin update or a bulk plugin update?
      • Bulk plugin update:
        • Where was the failed plugin in the list of plugins to be updated?
        • Were you on the Dashboard > Updates page, the Plugins > Installed plugins page, or WP-CLI?
  • No: Continue

Was the wp-content/temp-backup/plugins folder empty?

  • Yes: Thank you! Please report your results, and environment details, in the comments below.
  • No:
    • Please provide your environment details.
    • Please provide the name of the plugin(s) that remain in wp-content/temp-backup-plugins.
    • Was this a single plugin update or a bulk plugin update?
      • Bulk plugin update:
        • Where was the failed plugin in the list of plugins to be updated?
        • Were you on the Dashboard > Updates page, the Plugins > Installed plugins page, or WP-CLI?

Where do I report issues?

Please report test results or issues as comments on this post. Please list your environment in your results.

Editing assistance from birds and squirrels, also @ironprogrammer, @davidbaumwald, @hellofromtonya

#rollback

Andy Fragen 6:49 pm on February 19, 2021
Tags: rollback   

Feature Plugin: Rollback Update Failure

This feature pluginFeature Plugin A plugin that was created with the intention of eventually being proposed for inclusion in WordPress Core. See Features as Plugins. is an offshoot of Matt’s 9 Projects for 2019. Specifically it’s a follow-up to auto-updates for plugins and themes. Our goal is to provide a safety mechanism of sorts should an update, including auto-updates, fail potentially leaving the user’s site in an unstable state.

This is a feature plugin based on the PR for #51857. We are working towards inclusion in WordPress 5.8. The general overview is to provide a mechanism whereby a failed pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party or theme update does not leave the site in an unstable state. This part of the project is not about ensuring that a successful update does not cause any issues.

Some of the issues of failed updates include:

  • Having a plugin folder content be deleted and the plugin no longer active.
  • Having a plugin not completely update and result in a PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher fatal message or WSOD

While these are not the only results of failed updates, they seem to consistute the majority of reported issues.

The assumption is that most of the errors in large plugins/themes occur during the copy_dir() part of WP_Upgrader::install_package(). TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress. ticketticket Created for both bug reports and feature development on the bug tracker. #52342 brought more error reporting to copy_dir() and Trac ticket #52831 provides a filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. hook in order to process the rollback in the event of a plugin/theme update failure. As of WordPress 5.7-beta1 both of these tickets are in coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress..

It is during the WP_Upgrader::install_package() that the currently installed plugin/theme is deleted in anticipation of copying the new update into that location. Having an empty plugin/theme folder or an incompletely copied update seems to be the most common issue.

Rollback Update Failure Feature Plugin is available for feedback and testing. Contributions from the WordPress community are welcome on the plugin’s GitHub repository.

Testing

There was much discussion regarding the thought that adding additional IO processes for the zip and unzip process could result in server timeout issues on resource starved shared hosts. Activating the feature plugin will result in the creation of a ZIP file of the installed plugin/theme being updated every time an update is performed. The unzip only occurs during testing or if a WP_Error is returned from WP_Upgrader::install_package(). Any issues would only happen during a plugin or theme update.

For the sake of testing assume any server timeout occurring during the update process might be releated to the additional IO processes creating the zipfile. Please report these in GitHub Issues and report your server details. ( Host, RAM, OS, etc. )

There will be messaging in the event of an error and successful or unsuccessful rollback.

To simulate a failure, use the filter add_filter( 'rollback_update_testing', '__return_true' );

Alternatively you can install the Rollback Update Testing plugin, activating it as needed. If you have GitHub Updater installed, you can easily install this Gist from the Install Plugin tab. Select Gist as the Remote Repository Host.

Next Steps

The Rollback Update Failure feature plugin is an early step towards inclusion in WordPress Core. We need your help. Simply installing and activating the plugin will help test whether or not the additional server IO processes may cause issue with resource starved shared hosting.

Thanks @audrasjb for reviewing.

#rollback