Tag Archives: Testing

Javier Casares 16:34 on 2024-03-20
Tags: , , Testing,   

Proposal: PHPUnit Test Runner with Multi-PHP and Multi-Environment

This past weekend, the Clodfest Hackathon took place and the HostingHosting A web hosting service is a type of Internet hosting service that allows individuals and organizations to make their website accessible via the World Wide Web. team proposed a project to improve the tools.

Thanks to Pascal (@swissspidy), Cesar, and myself (@javiercasares), we have been able to make progress on some elements that had been in the backlog for over 4 years.

An important detail is that all the changes made are backward compatible so that the default configuration should not affect any of the new features, especially considering GitHubGitHub GitHub is a website that offers online implementation of git repositories that can can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. Actions.

What has been achieved?

The first step was to include support for Multi-PHPPHP PHP (PHP: Hypertext Preprocessor) is a general-purpose scripting language especially suited to web development. PHP code is usually processed on a web server by a PHP interpreter. On a web server, the result of the interpreted and executed PHP code would form the whole or part of an HTTP response.. This way, a user could submit different reports of the same commit but with different versions of PHP installed.

The second step was to create Multi-Environment. This allows a user to establish different servers with their respective configurations and PHP versions. This allows, for example, having an environment for “shared hosting”, another for “vps”, and another for “cloud”. This is in addition to the Multi-PHP from before.

The third step was to support All-Commits. The tool previously only executed the latest available commit. With this new change, a list of the 10 latest commits will be made, and all of them will be processed in case many commits are sent in a short time. Usually, the tool can take between 5 and 30 minutes per process, and sometimes commits that were never tested in all environments could be lost.

All these changes are accompanied by their respective improvements in the PHPUnit Test Reporter, the pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory or can be cost-based plugin from a third-party. that collects and displays the information.

From now on, the initial list will be a summary of the different environments and how many are working and failing. Once inside, we can see the list of all environments and PHP combinations. This is the visualization for developers: one commit, many results.

Furthermore, there is the option for different bots (i.e., different providers) to have their list and on the same screen, they can see the list of the latest tests and thus easily detect when or in what change an error occurred. This is the visualization for providers: one provider, many results.

Where can the changes be seen?

They are not yet approved because they need code review, documentation, and validation by more people. The focus of the Hackathon was to include the functionality, and it was done “too quickly,” so there is duplicated code and improvements can be made. It works, yes, but it’s not pretty.

The PRs with the changes are at:

If you want to contribute, please review and test the code to validate that everything works correctly.

Next steps

In addition to giving a code review, looking for possible errors, and proposing some improvements, we also intend to review the documentation on how to install and maintain the system automatically.

In parallel, we will review with the #core team that everything is working fine and that the GitHub Actions do not give errors, even with the possibility of running different tests depending on the PHP version.

#php, #testing, #tools

Javier Casares 07:07 on 2024-01-31
Tags: Testing,   

Hosting Tools office hours

When it comes to WordPress hostingHosting A web hosting service is a type of Internet hosting service that allows individuals and organizations to make their website accessible via the World Wide Web., it’s essential that everything works smoothly, regardless of who’s providing the hosting. The WordPress Hosting Team uses some cool tools to make sure of this. They follow guidelines from the WordPress CoreCore Core is the set of software required to run WordPress. The Core Team builds WordPress. Team, which you can find here: WordPress Core Team’s PHPUnit Testing Guide. These tests are super helpful for checking if hosting services are ready for new WordPress updates and if they’ll work well for everyone.

There are two main tools they use: the PHPUnit Test Runner repo and the PHPUnit Test Reporter repo.

The Runner repo is like the engine that runs tests on the hosting service to make sure WordPress will work well on it. It’s like giving the hosting a test drive to catch any problems early.

Then there’s the Reporter repo, which works like a dashboard on WordPress.org. It collects all the test results and shows them in a way that’s easy to understand. This helps everyone see how different hosting services are doing and ensures they’re all ready for the latest WordPress version.

By using these tools, hosting providers can make sure they’re always in sync with WordPress updates. This means fewer surprises and issues for website owners, and a smoother experience for everyone who uses WordPress.

Tools Office hours

Future meetings will be these for the first trimester:

Why everyone plays a role in WordPress Hosting Compatibility

When we talk about making WordPress work smoothly on different hosting services, it’s not just a one-person job. Developers, testers, and hosting providers all have super important roles to play. Here’s why involving everyone matters, and how the “Five for the Future” initiative makes it all possible.

Developers: They’re the builders. Developers write the code that makes WordPress and all its features possible. When it comes to hosting compatibility, they need to make sure their code works well in various hosting environments. This means writing clean, efficient code and being ready to tackle any hosting-specific challenges that come up.

Testers: These are the detectives in the process. Testers dive into new and existing features to locate any bugs or issues. Their job is to think like the end-user and try out every possible scenario to ensure everything works as it should. When they test WordPress on different hosting services, they can spot potential problems before they affect users.

Hosting Providers: These folks offer the space and resources for WordPress sites to live on the internet. They need to stay on top of WordPress updates and ensure their services are optimized for it. By being involved in testing and compatibility checks, hosting providers can promise a smooth, trouble-free experience for website owners.

Now, how does all this teamwork happen? That’s where “Five for the Future” comes in. This initiative encourages companies and individuals in the WordPress community to dedicate 5% of their resources to WordPress development and projects. It’s a way to give back and ensure the platform keeps growing and improving.

Get ready: Join Our Office Hours!

Exciting times ahead! We’re gearing up for an upcoming Hackathon, and we want to make sure everyone’s ready to hit the ground running. To help with this, we’re setting up Office Hours over the following weeks. These sessions are all about getting you prepped and polished for the big event, whether you’re a developer, tester, or hosting provider.

Why Office Hours? Think of Office Hours as your personal prep zone. It’s a space where you can:

  • Ask Questions: Stuck on something? Not sure how to tackle a challenge? This is your chance to ask the experts and get those A-ha! moments.
  • Learn Best Practices: We’ll share tips, tricks, and best practices to make your projects shine. From coding standards to testing techniques, you’ll get the inside scoop on how to excel.
  • Collaborate: Meet other participants, swap ideas, and maybe even form your dream team for the Hackathon. Collaboration is the name of the game.
  • Get Feedback: Have a project idea or a piece of code you’re not sure about? Get constructive feedback to help you refine and improve.

Who Should Join? Everyone who’s planning to be part of the testing! Whether you’re a seasoned developer, a keen tester, or a hosting provider looking to make your services even more WordPress-friendly, these Office Hours are for you.

How to Prepare: To make the most of these sessions, come with your questions, project ideas, and anything you’ve already started working on. If you’re new to the scene, that’s okay too! Just bring your enthusiasm and your willingness to learn.

Cloudfest Hackathon

The WordPress Tools for Hosting Providers project has been selected to participate on March 16-18, 2024 in the CloudFest Hackathon.

For this reason, we have 5 goals for that date.

The first is to leave a stable, functional, and documented version of the test-runner tool so that everyone works on that version. There is a PR #199 that establishes those changes and that base. It has been thoroughly tested and should serve as a basis so that the errors we have previously encountered are not there.

The second is to do the same work that has been done on the test-runner but on the test-reporter so that we also have a more or less stable version to work on the pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory or can be cost-based plugin from a third-party..

The third is to leave documentation of the entire operation of the tools and how to set them up in a development environment, as well as how they can be tested (each tool will need its explanation, since one is best to have a very “hosting” environment, and for the plugin, to have a WordPress development environment of its own).

The fourth, which involves everyone, is to define a list of “what things we want the tools to have.” In principle, there should be two lists, which must complement each other. One that executes the test-runner and allows executing “things”, and the other of the test-reporter that collects all that and displays it on the screen, in addition to planning warnings.

The fifth and last is to say that those who want to come to hack a bit, give a read to Come Hack with Us. We look forward to seeing you March 16-18, 2024, especially to have a great time!

#testing, #tools

Amy Kamala 18:17 on 2022-10-19
Tags: Testing   

Call For Action – Testing Rollback Feature

Hello WordPress Hosts!

The CoreCore Core is the set of software required to run WordPress. The Core Team builds WordPress. Development Team is requesting assistance with testing a new Rollback Feature in WordPress Core. As always, your testing contributions help keep the development process informed and are valuable to the WordPress Open SourceOpen Source Open Source denotes software for which the original source code is made freely available and may be redistributed and modified. Open Source **must be** delivered via a licensing model, see GPL. Project! Thank you all in advance for participating in testing this feature!

This new feature offers the option to “rollback” to a previously installed version of plugins or themes that have thrown an error during a version update. Users may experience such an error as a WSOD (“white screen of death”), a visible PHPPHP PHP (PHP: Hypertext Preprocessor) is a general-purpose scripting language especially suited to web development. PHP code is usually processed on a web server by a PHP interpreter. On a web server, the result of the interpreted and executed PHP code would form the whole or part of an HTTP response. Error on the front-end or in another form during updates that can prevent a site from continuing to function without assistance from a Developer or Technical Support. This rollback feature will provide an option within WordPress Core itself for the user to set pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory or can be cost-based plugin from a third-party. and themeTheme A theme dictates the style and function of your WordPress website. Child Themes derive from the main parent theme. files back to the previously working version.

The feature uses a rename() function to copy plugin/theme files into another directory and then copy previously working files back into core directories, rather than using the less performant copy_dir() function which recursively copies files but is believed to result in timeouts on some systems.

How do I test Rollback?

Do not test on a production siteProduction Site A production site is a live site online meant to be viewed by your visitors, as opposed to a site that is staged for development or testing..

But do test on a local, staging or test environment, or spin up a cloud staging or test environment.

  1. Here are some large plugins used for testing: akismet, jetpack, mailpoet, woocommerce, wpforms-lite, wordpress-seo
    • WP-CLIWP-CLI WP CLI is the Command Line Interface for WordPress, used to do administrative and development tasks in a programmatic way.: wp plugin install akismet jetpack mailpoet woocommerce wpforms-lite wordpress-seo
  2. Do this from the plugin’s page on https://wordpress.org/plugins by navigating to the “Development” tab, clicking “Advanced” to the right, and downloading an older version from the dropdown at the bottom of the page. You can also install the current version then modify the version in the plugin’s main file to decrement the version number.
  3. Install the WordPress Beta Tester plugin, set to Bleeding edge and Nightlies. Go to Dashboard > Updates and click the Update to latest 6.1 nightly button.
  4. Install the Rollback feature plugin or test using the PR2225 in WordPress/wordpress-develop.
  5. Please make a note of the time required to perform plugin updates. Your phone’s stopwatch function may be the easiest method to do this.

Testing a single plugin update:

  1. Navigate to Plugins > Installed Plugins.
  2. Click “Update Now” located within the plugin row.

Testing bulk plugin updates via “Plugins”:

  1. Navigate to Plugins > Installed Plugins.
  2. Select another two plugins, select “Update” from the Bulk Actions dropdown, and click “Apply”.

Testing bulk plugin updates via “Dashboard”:

  1. Navigate to Dashboard > Updates
  2. Tick all plugins with an available update.
  3. Select “Update” from the Bulk Actions dropdown, and click “Apply”.

Testing updates via WP-CLI (if already familiar).

Validation of successful updates

This requires activating all the testing plugins on your testing site. Unsuccessful updates should show PHP Errors or PHP Fatal Errors.

  1. Activate each of the plugins that were updated.
  2. In WP Admin, navigate to each plugin’s menu pages.
  3. Navigate the frontend of your test site.
  4. Navigate to your wp-content/temp-backup/plugins folder. It should be empty.

Forcing an update failure

Use the following filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. to force an update failure. This will reinstall the previously active plugin/theme.

add_filter( 'upgrader_install_package_result', function() {
  return new WP_Error( 'simulated_error', 'Simulated Error' );
});

Testing update failures

When testing for failures on the bulk update in update-core.php you must use the PR. There is a modification in the PR that stops WP_Upgrader::unpack_package() from deleting the items in the temp-backup directory.

For More Information please see this original Call to Action from the Core Team.

The more testing that is done on a wider variety of hostingHosting A web hosting service is a type of Internet hosting service that allows individuals and organizations to make their website accessible via the World Wide Web. environments, the more likely WordPress and its new features are to perform on the wide range of hosting platforms available to users. As such, testing provides value to the software, users and hosts alike. Thank you again for your testing contributions!

#testing

Daniel Bachhuber 04:57 on 2017-08-25
Tags: , , Testing   

Announcing Beta Period for Distributed Host Testing

Just over a year ago, Aaron Jorbin suggested a simple idea:

If WordPress really wants to do quality automated testing, we need to rely on the people hostingHosting A web hosting service is a type of Internet hosting service that allows individuals and organizations to make their website accessible via the World Wide Web. sites to test vs their stack. To do that, CoreCore Core is the set of software required to run WordPress. The Core Team builds WordPress. needs to provide an infrastructure that both encourages and enables easy automated testing.

Today, we’re happy to announce the beginning of a betaBeta Beta is the software development phase following alpha. A Beta phase generally begins when the software is feature complete but likely to contain a number of known or unknown bugs. Software in the beta phase will generally have many more bugs in it than completed software, speed or performance issues, and may still cause crashes or data loss. The focus of beta testing is reducing impacts to users, often incorporating usability testing. period for exactly this: a framework for any hosting company to run the WordPress PHPUnit test suite on their infrastructure and report the results back to WordPress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/.

At a high level, this framework is two parts:

  1. The phpunit-test-runner, which prepares the environment, runs the test suite, and reports the results back to WordPress.org.
  2. The phpunit-test-reporter, which receives the results, stores them in the databaseDatabase A database is an organized collection of data. Access to this data is usually provided by a "database management system" (DBMS) consisting of an integrated set of computer software that allows users to interact with one or more databases and provides access to all of the data contained in the database. Because of the close relationship between them, the term "database" is often used casually to refer to both a database and the DBMS used to manipulate it., and displays them in an accessible manner.

We’d love to see dozens of hosting companies participate in this program. Check out Getting Started for an overview on how you can set it up. Then, stop by the #hosting-community channel in WordPress.org Slack with any questions you might have.

Thanks to DreamHost and WP Engine for volunteering the engineering effort to make this project possible.

#hosting-community, #php, #testing

Jason Stallings 15:06 on 2017-02-16
Tags: , Testing   

Dive Into Distributed Unit Tests

Recently, I have spent some time looking into using pre-built tools for distributed unit tests. What I found was that all of them were built to fetch the test results. What we need, for this project, is to have the results sent to us. At this point it seems like a custom WordPress solution will be the best route.

There was some discussion in #hosting-community about how the tests would be triggered. Some hosts have scripts that run nightly, so slipping tests into that process would make implementation easy. However, using a nightly solution would make it more difficult to determine exactly which commit caused an issue.

If we move forward with running tests on a per-commit basis, do hosts watch for commits and then report the results? Or do we pingPing The act of sending a very small amount of data to an end point. Ping is used in computer science to illicit a response from a target server to test it’s connection. Ping is also a term used by Slack users to @ someone or send them a direct message (DM). Users might say something along the lines of “Ping me when the meeting starts.” hosts? It seems like pinging hosts would be ideal, but it might not be an option for all hosts. It would be really nice to get some feedback here, so your thoughts would be appreciated.

The first step is to start on the method used to report results. This will be a WordPress based application, with a REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/.. Hosts will submit the results with the commit SHA to be aggregated. I’m going to dig into this over the next several weeks and will share a GitHubGitHub GitHub is a website that offers online implementation of git repositories that can can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. link when I have something up and running.

#php, #testing