class-wp-hook.php - apply_filters() infinite loop

[frettled]

I just saw nearly 60 million error log entries (17 GB) due to this bug.

In line 303 of class-wp-hook.php, there is a piece of code that will cause an infinite loop:

} while ( false !== next( $this->iterations[ $nesting_level ] ) );

The problem is that

$this->iterations[ $nesting_level ]

can be null.

Suggested fix:

} while ( ! is_null( $this->iterations[ $nesting_level ] ) && false !== next( $this->iterations[ $nesting_level ] ) );

I also urge developers to look for similar "false !== next()" constructs in code, as they will lead to infinite loops.

[dhanendran]

Hi @frettled

$this->iterations[ $nesting_level ] can't be null.

$this->iterations[ $nesting_level ] = array_keys( $this->callbacks );

array_keys() will return empty array, if it don't have any values in $this->callbacks.

[dd32]

Hi @frettled and welcome to Trac.

Are you able to offer any extra specific details about how this scenario was triggered? For example, the hook name which was executing and anything which was hooked onto it, or all plugins in use on the site?

Protecting against this is something we should do, however in order to do that effectively, we need to determine how and why it happens to test it.
I've tried a few random combinations and I can't get it in a situation where $this->iterations[ $nesting_level ] is unset.

[frettled]

Replying to dd32:

Hi @frettled and welcome to Trac.

Are you able to offer any extra specific details about how this scenario was triggered? For example, the hook name which was executing and anything which was hooked onto it, or all plugins in use on the site?

This specific bug had referer: hxxp://.../wp-includes/admin.php?page=mediafromftp-settings which leads me to guess that there was something about the plugin mediafromftp that initialized the problem.

Protecting against this is something we should do, however in order to do that effectively, we need to determine how and why it happens to test it.
I've tried a few random combinations and I can't get it in a situation where $this->iterations[ $nesting_level ] is unset.

I have a total of three error messages repeating infinitely, until the process is killed by a monitoring script:

1) line 287: current() expects parameter 1 to be array, null given in .../wp-includes/class-wp-hook.php on line 287

2) line 289: Invalid argument supplied for foreach() in .../wp-includes/class-wp-hook.php on line 289

3) line 303: next() expects parameter 1 to be array, null given in .../wp-includes/class-wp-hook.php on line 303

Digging into the logs for non-matching errors from the same website, I see another error which may be related:

array_keys() expects parameter 1 to be array, integer given in ../wp-includes/class-wp-hook.php on line 283

On this line, it's extracting from $this->callbacks.

So perhaps that's a better place for a check.

[dd32]

Thanks for the extra details @frettled!

I'm unable to reproduce this with the plugin mentioned, and can't find any code in any version of it which could trigger it (as far as I could tell).

However with the additional error messages I was able to come up with some code which reproduces it, although any code doing something like this couldn't be expected to work.. I'm hoping it might help someone else see why this case is being triggered.

function test_func() {}
add_action( 'test', 'test_func' );

global $wp_filter;
$wp_filter['test']->callbacks = 123;

do_action( 'test' );

If you can check to see if ->callbacks or $wp_filter is used within any custom plugins it could help explain what's happening here.

@pento - I'm assuming WP_Hook->callbacks is only public for remove_filter()? or was there something else that it's used for?

[pento]

Replying to dd32:

@pento - I'm assuming WP_Hook->callbacks is only public for remove_filter()? or was there something else that it's used for?

It's also there for legacy access - if some code needs to modify $wp_filter before the helpers are available, or if it needs to look at all of the filters (filter debug plugins, for example).

It's ​recommended as an option for those who need access.

[frettled]

Replying to dd32:

Thanks for the extra details @frettled!

You're welcome :)

I'm unable to reproduce this with the plugin mentioned, and can't find any code in any version of it which could trigger it (as far as I could tell).

My guess is that there is something that crashes in the plugin itself.

Perhaps there is a failing POST request, or some other error condition.

It doesn't matter much, because plugin and theme code are outside of the WP core's control anyway, so checking the input from plugins is pretty vital.

[frettled]

I've acquired another error message that appeared shortly before this started bugging out:

PHP Fatal error:  Maximum execution time of 30 seconds exceeded in .../wp-includes/meta.php on line 507, referer: hxxp://.../wp-cron.php?doing_wp_cron=148...

Apart from that, there are apparently no other errors for the same website within the 30 seconds preceding this error (or the other errors, which occurred instantly after this one).

[wordp2509]

Exactly same problem here... Error.log becomes Gigabytes in no time.
Same set of 3 messages repeating itself endlessly.
Even after removing all plugins it did not stop.
Re-installing Wordpress 4.7 did stop it and even with all plugins reinstalled it did not came back.
So indeed something triggers this, and then it does not stop until re-install of 4.7

So any update of wp4.7 comming?
Thanks

[wordp2509]

(using WP 4.7 Multisite)

[dd32]

@wordp2509 Thanks for the feedback - Can you let us know if you were using a specific theme or had one of the drop ins active? (dropins are the .php files in wp-content/.

There's nothing in WordPress itself which can explain the scenario you've run into - where a reinstall would correct the issue.

[dd32]

Incase someone else comes across this - If possible, I'd love to help debug your site, or at least get a copy of the wp-content directory (..minus uploads) at the time of the issue happening, to see if I can reproduce it and work out what's happening.

Just email me, (edit: Email removed, this offer has expired) and I'll take a look for you.

[frettled]

Replying to dd32:

Incase someone else comes across this - If possible, I'd love to help debug your site, or at least get a copy of the wp-content directory (..minus uploads) at the time of the issue happening, to see if I can reproduce it and work out what's happening.

Just email me, and I'll take a look for you.

Why not simply add a test at line 283 to fix the problem once and for all?

I can understand the sentiment that you want plugin authors to fix their plugins, but you will never have complete control over plugin authors, only the core WP code. The fix belongs in core WP, not elsewhere.

[dd32]

Replying to frettled:

Why not simply add a test at line 283 to fix the problem once and for all?

I can understand the sentiment that you want plugin authors to fix their plugins, but you will never have complete control over plugin authors, only the core WP code. The fix belongs in core WP, not elsewhere.

Honestly, because it doesn't actually fix the underlying problem.

The change suggested simply papers over it, ignores that it ever happened. That value should never not be an array, if it is, something has broken somewhere, and we need to find it and fix it.
It could mean that filters are not being applied correctly, or in the correct order under certain circumstances. Right now we have no idea what the actual implications of this issue are - other than an infinite loop, so far experienced by only two known people.

Multiple people have looked at this and haven't been able to reproduce or otherwise determine how or why it's broken.

The code I posted in comment#4 duplicated the issues, but wasn't a realistic thing that any plugin would ever have done, either deliberately, or accidentally. It also would have never worked under 4.6, and more than likely caused WordPress to explode then, so It's much more likely to be a bug in one of our loops in WP_Hook somewhere.

The WordPress filters and actions API is called thousands (sometimes tens of thousand) a page load, any small bugs there can have performance implications, or security implications if certain filters get bypassed accidentally, etc.

[dd32]

I should add to that, that if a certain combination of things that a plugin does causes this - then yes, we'll totally fix it in core to make those plugins work as expected. We just need to know what those things are that the plugin is doing to trigger it.

[frettled]

Replying to dd32:

Honestly, because it doesn't actually fix the underlying problem.

On the contrary, this is the underlying problem.

The change suggested simply papers over it, ignores that it ever happened. That value should never not be an array, if it is, something has broken somewhere, and we need to find it and fix it.

I agree that this should be fixed, too, but that's a superficial problem.

Superficial problems shouldn't result in infinite loops.

It could mean that filters are not being applied correctly, or in the correct order under certain circumstances. Right now we have no idea what the actual implications of this issue are - other than an infinite loop, so far experienced by only two known people.

I've seen this dozens of times, for different WP installations. Please don't mistake the lack of reports for a lack of a problem.

I apologize for not reporting this earlier, although please trust me when I say that I could have reported this a long time ago, and frequently, and kept nagging about it for months and years. :)

Multiple people have looked at this and haven't been able to reproduce or otherwise determine how or why it's broken.

But we know that it's broken. Shouldn't that be sufficient?

The code I posted in comment#4 duplicated the issues, but wasn't a realistic thing that any plugin would ever have done, either deliberately, or accidentally. It also would have never worked under 4.6, and more than likely caused WordPress to explode then, so It's much more likely to be a bug in one of our loops in WP_Hook somewhere.

The error message in comment#7 seems relevant: termination of a too-long-running execution by PHP itself.

My guess is that this is reproducible to a much greater extent if you set max_execution_time to e.g. 1 second, and run the test code in a VM that has an execution cap of say 10% or something like that.

And this would probably reveal all sorts of interesting problems.

However, none of this seem to be valid reasons for avoiding an infinite loop when possible.

[wordp2509]

Replying to dd32:

@wordp2509 Thanks for the feedback - Can you let us know if you were using a specific theme or had one of the drop ins active? (dropins are the .php files in wp-content/.

There's nothing in WordPress itself which can explain the scenario you've run into - where a reinstall would correct the issue.

Hello, Thanks for asking...
I am using Theme: Storefront designed by WooCommerce, with 4 Storefront plugins.
In the root of WP-Content I have 2 .php files: advanced-cache.php and index.php

Since re-install of WP 4.7 the loop has not been triggered yet.

[dd32]

Replying to frettled:

I've seen this dozens of times, for different WP installations. Please don't mistake the lack of reports for a lack of a problem.

If you could provide any common details about installations which are experiencing the issue, it'll help greatly.

I apologize for not reporting this earlier, although please trust me when I say that I could have reported this a long time ago, and frequently, and kept nagging about it for months and years. :)

In 4.7 the code for apply_filters() changed significantly, it was almost a rewrite in many respects - That's why we're interested here, the code in question simply did not exist in it's current form in the past (although previously it was a similar next() loop, but based on data that could change or vanish - unlike in 4.7)

The change you've proposed will not be merged as-is, without a reproducible test case or understanding the circumstances under which it can be triggered.
I'm happy to help you debug a site which experiences the issue in order to track down what's happening, just email me with details.

I've milestoned this for 4.7.1 for tracking purposes and investigation, but will not be in a release until the above conditions are met.

[dd32]

Bumping to 4.7.2 pending a testable patch.

[frettled]

Here is another incident with an infinite loop, from a different site.

This site runs WP 4.7.1.

A plugin (wordpress-backup-to-dropbox) gets interrupted by PHP's maximum execution time limit, and class-wp-hook.php goes bananas, just like plugin.php used to do in 4.6.1 (which, BTW, has had multiple incidents for multiple sites since the last time, but because the code has been fixed in 4.7, I haven't bothered reporting it as 4.6 won't be fixed, right?)

The plugin version is 4.5.3, the most recent is plugin version 4.7. Plugin version 4.7 introduces no code changes, so that's irrelevant.

<pre>
PHP Fatal error: Maximum execution time of 30 seconds exceeded in [...]/wp-content/plugins/wordpress-backup-to-dropbox/Classes/Processed/Files.php on line 40, referer: ​http://[...]/wp-cron.php?doing_wp_cron=[...]
PHP Warning: array_keys() expects parameter 1 to be array, integer given in [...]/wp-includes/class-wp-hook.php on line 283, referer: ​http://[...]/wp-cron.php?doing_wp_cron=[...]
PHP Warning: current() expects parameter 1 to be array, null given in [...]/wp-includes/class-wp-hook.php on line 287, referer: ​http://[...]/wp-cron.php?doing_wp_cron=[...]
PHP Warning: Invalid argument supplied for foreach() in [...]/wp-includes/class-wp-hook.php on line 289, referer: ​http://[...]/wp-cron.php?doing_wp_cron=[...]
PHP Warning: next() expects parameter 1 to be array, null given in [...]/wp-includes/class-wp-hook.php on line 303, referer: ​http://[...]/wp-cron.php?doing_wp_cron=[...]
PHP Warning: current() expects parameter 1 to be array, null given in [...]/wp-includes/class-wp-hook.php on line 287, referer: ​http://[...]/wp-cron.php?doing_wp_cron=[...]
PHP Warning: Invalid argument supplied for foreach() in [...]/wp-includes/class-wp-hook.php on line 289, referer: ​http://[...]/wp-cron.php?doing_wp_cron=[...]
PHP Warning: next() expects parameter 1 to be array, null given in [...]/wp-includes/class-wp-hook.php on line 303, referer: ​http://[...]/wp-cron.php?doing_wp_cron=[...]
</pre>

As previously noted by me, this is testable and reproducible.

An alternative for reproducing is to make a dummy plugin that simply executes for longer than the regular max execution time, and then gets terminated by PHP.

But better yet, ensure that no loops depend on mistaken assumptions about guaranteed nice shutdown of plugins. Plugins may crash due to external circumstances. This particular "external" circumstance is PHP's execution time cap, but when running PHP under suphp or FastCGI or uWSGI or something else, there may be other things going on.

[gitlost]

​Installing if possible the Zend extension Xdebug and enabling zend_extension=xdebug.so in your "php.ini" file so that you get stack traces on PHP notices would go a long way to solving this mystery. (Perhaps something to do with do_action( 'shutdown' ); ???) What PHP version(s) you're using would also be useful info.

[frettled]

@gitlost I've got thousands of WP installations that are monitored, none of them mine, and installing a zend extension in all of them to catch this kind of problem seems a bit excessive. Additionally, I don't think I've got access to enough disk space to store that.

The PHP version in question is the latest 5.5 with security patches backported from 5.6, running under suphp.

This also seems largely irrelevant, though, because we already know where the problem is and how to fix it: don't run infinite loops. Never!

[gitlost]

xdebug.so is only ~1.2M so disk space shouldn't be an issue! Also it's per-server, not per-WP install. Also if the error is reproducible for you then you'd only have it install it once you encounter it. Anyway it's an option to consider next time it happens. It would I think really help to understand the issue. However...

The PHP version in question is the latest 5.5 with security patches backported from 5.6

So you're running some distro's LTS build of PHP? Bit of a red flag, but probably nothing...though it probably complicates using Xdebug.

we already know where the problem is and how to fix it: don't run infinite loops. Never!

It's been said before but I'll repeat it:

<?php
$frettld_doesnt_see = true;
while ( $frettld_doesnt_see ) {
        echo "There is no infinite loop unless WP_Hook is in some unstable condition."
        echo "How does WP_Hook get into some unstable condition?"
        echo "Nobody knows."
}

I've been able to semi-reliably put WP_Hook into an unstable condition by getting the max execution time to trigger during the ksort() in WP_Hook::add_filter() (using PHP 7.0.13):

set_time_limit( 1 );
for ( $i = 0; $i < 100000; $i++ ) {
        add_action( 'shutdown', '__return_true', $i );
}

which sometimes leaves the $this->callbacks array with duplicate entries, which then causes PHP warnings (but not infinite) about undefined offsets and invalid arguments in the inner loop of WP_Hook::add_filters() when it's being run in shutdown_action_hook().

It's an unrealistic scenario of course but it does demonstrate that WP_Hook can be put into an unstable condition "naturally", and that calling complicated code like WP_Hook in the shutdown routine could cause problems. So it might be worth doing the "shutdown" filters in some fail-safe manner, perhaps something like the opposite of WP_Hook::build_preinitialized_hooks...

[frettled]

Replying to gitlost:

It's been said before but I'll repeat it:

<?php
$frettld_doesnt_see = true;
while ( $frettld_doesnt_see ) {
        echo "There is no infinite loop unless WP_Hook is in some unstable condition."
        echo "How does WP_Hook get into some unstable condition?"
        echo "Nobody knows."
}

Jeez.

Thanks for antagonizing me, and for making this a really nice and hostile enviroment for reports.

I'll be sure to let others know how you treat people here.

[jnylen0]

Remember we all want the same thing here: fast and stable software to run our websites.

The change you've proposed will not be merged as-is, without a reproducible test case or understanding the circumstances under which it can be triggered.

The reason for this is to ensure that we don't break millions of people's sites by shipping a change that's not ready.

Bumping to Future Release, that could be changed again if there is some further movement.

[crootof]

@frettled Not terribly helpful but I've had this issue on 2-3 sites so far in the last few weeks. Log files increase in size rapidly and can cause issues. Have yet to determine the cause but confirming another instance. RHEL 6.8, Apache 2, PHP 5.6.30, WP 4.7 + 4.7.2.

[ogrodas]

I have the same issue:
Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-117-generic x86_64)
PHP 5.5.9-1ubuntu4.21
wordpress 4.7.4–nb_NO
Akismet Anti-Spam 3.3.1
CW Image Optimizer 1.1.10
Easy Updates Manager 6.3.0
Google Analytics for WordPress by MonsterInsights 6.1.7
iThemes Security 6.2.1
Jetpack av WordPress.com 4.9
W3 Total Cache 0.9.5.4
WordPress Backup to Dropbox 4.7.1

I think the first error related to the Wordpress Backup to dropbox job creates the loop

[Sun May 07 11:01:41.522249 2017] [:error] [pid 28580] [client 185.35.185.46:53026] PHP Fatal error: Maximum execution time of 600 seconds exceeded in /var/www/nytnaturen.no/wordpress/wp-content/plugins/wordpress-backup-to-dropbox/Classes/Processed/Files.php on line 40, referer: ​https://nytnaturen.no/wp-cron.php?doing_wp_cron=1494146920.7897720336914062500000
[Sun May 07 11:01:41.522886 2017] [:error] [pid 28580] [client 185.35.185.46:53026] PHP Warning: array_keys() expects parameter 1 to be array, integer given in /var/www/nytnaturen.no/wordpress/wp-includes/class-wp-hook.php on line 283, referer: ​https://nytnaturen.no/wp-cron.php?doing_wp_cron=1494146920.7897720336914062500000
[Sun May 07 11:01:41.522911 2017] [:error] [pid 28580] [client 185.35.185.46:53026] PHP Warning: current() expects parameter 1 to be array, null given in /var/www/nytnaturen.no/wordpress/wp-includes/class-wp-hook.php on line 287, referer: ​https://nytnaturen.no/wp-cron.php?doing_wp_cron=1494146920.7897720336914062500000
[Sun May 07 11:01:41.522925 2017] [:error] [pid 28580] [client 185.35.185.46:53026] PHP Warning: Invalid argument supplied for foreach() in /var/www/nytnaturen.no/wordpress/wp-includes/class-wp-hook.php on line 289, referer: ​https://nytnaturen.no/wp-cron.php?doing_wp_cron=1494146920.7897720336914062500000
[Sun May 07 11:01:41.522937 2017] [:error] [pid 28580] [client 185.35.185.46:53026] PHP Warning: next() expects parameter 1 to be array, null given in /var/www/nytnaturen.no/wordpress/wp-includes/class-wp-hook.php on line 303, referer: ​https://nytnaturen.no/wp-cron.php?doing_wp_cron=1494146920.7897720336914062500000
[Sun May 07 11:01:41.522948 2017] [:error] [pid 28580] [client 185.35.185.46:53026] PHP Warning: current() expects parameter 1 to be array, null given in /var/www/nytnaturen.no/wordpress/wp-includes/class-wp-hook.php on line 287, referer: ​https://nytnaturen.no/wp-cron.php?doing_wp_cron=1494146920.7897720336914062500000
[Sun May 07 11:01:41.522959 2017] [:error] [pid 28580] [client 185.35.185.46:53026] PHP Warning: Invalid argument supplied for foreach() in /var/www/nytnaturen.no/wordpress/wp-includes/class-wp-hook.php on line 289, referer: ​https://nytnaturen.no/wp-cron.php?doing_wp_cron=1494146920.7897720336914062500000

[mlmoreno]

Me also, i have the same issue:
Apache 2.4.17
Php 5.5.30
MySQL 5.5.31
WordPress 4.7.4
Akismet 3.3.1
AsynCRONous bbPress Subscriptions 2.3
BackWPup 3.3.7
BadgeOS 1.4.8.2
BadgeOS Community Add-On 1.2.3
BadgeOS Congratulations Modal Add-On 1.0.1
BadgeOS Sensei Add-On 1.0.4
bbPress 2.5.12
Bowe Codes 2.1
Buddypress 2.8.2
BuddyPress for Sensei 1.2.0
BuddyPress Forum Notifier 1.4
Collapse-O-Matic 1.7.4
Contact Form 7 4.7
Cookie Law Info 1.5.3
Google XML Sitemaps 4.0.8
iThemes Security 6.2.1
Loco Translate 2.0.13
Sensei 1.9.14
Sensei Certificates 1.0.16
Sensei Course Progress 1.0.6
Sensei Media Attachments 1.0.0
Sensei Share Your Grade 1.0.3
Social Share WordPress Plugin - AccessPress Social Share 4.3.3
Thrive Leads 1.95.17
W3 Total Cache 0.9.5.4
WC Marketplace 2.6.5
WooCommerce 3.0.5
WooCommerce Direct Checkout 1.1.2
WooCommerce Stripe Gateway 3.1.7
WP-Mail-SMTP 0.10.1
WPFront User Role Editor 2.13.1
Yoast SEO 4.7

The error that creates the loop is in the iThemes Security plugin

[Sun May 07 12:57:00.000219 2017] [proxy_fcgi:error] [pid 30181:tid 140115322074880] [client 79.144.20.183:63887] AH01071: Got error 'PHP message: PHP Fatal error: Maximum execution time of 300 seconds exceeded in xxx/nubemia.es/public_html/wp-content/plugins/better-wp-security/core/modules/file-change/scanner.php on line 542
PHP message: PHP Warning: array_keys() expects parameter 1 to be array, null given in xxx/nubemia.es/public_html/wp-includes/class-wp-hook.php on line 283
PHP message: PHP Warning: current() expects parameter 1 to be array, null given in xxx/nubemia.es/public_html/wp-includes/class-wp-hook.php on line 287
PHP message: PHP Warning: Invalid argument supplied for foreach() in xxx/nubemia.es/public_html/wp-includes/class-wp-hook.php on line 289
PHP message: PHP Warning: next() expects parameter 1 to be array, null given in xxx/nubemia.es/public_html/wp-includes/class-wp-hook.php on line 303

My hosting provider has also suspended my website, is there any alternative solution to avoid the loop?

Thanks so much

[mlmoreno]

When I have deactivated the plugin that triggers the loop (Maximum execution time of 300 seconds exceeded) the web has returned to work.
I think that it may be possible to reproduce the error by forcing a maximum execution time exceeded in a plugin.
The truth is that the error exists and that is a time bomb.

[juanjovv]

Hello all and sorry about my English.

I have the same error too, twice in last 10 days.

Next a link to pastebin of error log, for your information.

​https://pastebin.com/i4jiwUXN

This is on Wordpress 4.7.4
PHP 5.6.30

I dont know if you could need more info about.

My hosting provider has extended max execution time, if I have it again, ill tell you.

[x2048]

I can confirm that this happens with WordPress 4.8, PHP 7.0.

The issue does indeed seem to be with iThemes Security, it still reproduces with the latest version of the plugin.

Steps to reproduce:

Be a logged-in user
Don't log in for a couple days
Attempt to open the site
Error 502 (due to infinite loop in an action/filter and subsequent memory exhaustion)

Logged-out users can still see the site.
Opening the site in incognito mode and logging-in works.
Clearing cookies and logging-in works.

[x2048]

To whom it may concern:
iThemes claim that v6.3.0 fixes the issue:

​https://wordpress.org/plugins/better-wp-security/#developers
"Bug Fix: Fixed an infinite loop that could occur when expiring a cookie and Hide Backend is enabled."

[lkraav]

Yes, I reported this bug to iThemes, and research showed it was all about how their Hide Backend feature used wp_clear_auth_cookie() in a way that became incompatible with specifically WP 4.8, resulting in an infinite loop.

About the ticket topic though: ever since the WP_Hook rewrite/-factor was introduced, everything has worked smoothly.

[dingdang]

WP Version: 4.8.1

Probably caused by: ​https://wordpress.org/plugins/wp-statistics/ (latest version)

• Had 500K records in its table, causing (overloading and) process terminations by the web server:

PHP Fatal error: Maximum execution time of 30 seconds exceeded

• PHP running as CGI

• generated 120GB of error_log, repeating the following 3 lines

PHP Warning: next() expects parameter 1 to be array, null given in .../wp-includes/class-wp-hook.php on line 303, referer: .../wp-admin/index.php
PHP Warning: current() expects parameter 1 to be array, null given in .../wp-includes/class-wp-hook.php on line 287, referer: .../wp-admin/index.php
PHP Warning: Invalid argument supplied for foreach() in .../wp-includes/class-wp-hook.php on line 289, referer: .../wp-admin/index.php

[santrix]

This code in class-wp-hook.php is a major issue

} while ( false !== next( $this->iterations[ $nesting_level ] ) );

I work for a large host and several plugins can cause this to trap itself in an endless loop. The solution offered by frettled seems reasonable, and does prevent it. I might add that in configurations of Apache 2.4 where Mod Security is deployed, this kind of thing will also cause a memory leak leading to rapid destabilisation of a server (tested using fcgi, lsapi, suphp, cgi).

This type of thing needs to be fixed sooner than later throughout the core.

[dd32]

Replying to santrix:

This type of thing needs to be fixed sooner than later throughout the core.

Testing to provide a reproducible test case which can be actioned would be appreciated.
More bug reports saying it's been run into, or providing PHP Warning notices from logs does nothing to advance this ticket.
If a test case and example code which can be used to reliably test this without breaking existing code can be provided, there'll be some further movement here. I've personally looked at a few sites, most of whom "fixed themselves" before I could actually inspect any issues.

[dd32]

#42319 was marked as a duplicate.

[Jonathan.Moore@…]

Defensive coding is needed, it shouldn't be possible for some plugin interaction to produce:
Invalid argument supplied for foreach() in wp-includes/class-wp-hook.php on line 277
etc

In the case I'm attempting to debug it seems to occur with certain combinations of plugins, combined with a cron and/or other load (not reproducible interactively). So no, I can't provide a reproducible test case either despite much trying, I would love to be able to do so.

In these cases php-fpm spins up to full cpu and starts eating memory until out of memory or execution time exceeded or both.
(increasing memory or execution time doesn't solve - better maybe to reduce and fail earier whatever infinite loop is happening). If there's a possibility core itself is producing this infinite loop as per the thread title [whatever 'bad' plugin combination may have caused it], that possibility should be closed.

[breakoftheec]

Same issue here.

2Gigs of error logs and counting, in less than 20 hours.

We cannot reproduce the error on a staging site, and I have no idea how to at least trace the "offending" plugin.

PHP message: PHP Warning:  Invalid argument supplied for foreach() in /home/***************/wp-includes/class-wp-hook.php on line 277

PHP message: PHP Warning:  next() expects parameter 1 to be array, null given in /home/***************/wp-includes/class-wp-hook.php on line 291

PHP message: PHP Warning:  current() expects parameter 1 to be array, null given in /***************/wp-includes/class-wp-hook.php on line 275

[microbit]

And me too...

Invalid argument supplied for foreach() in /home/**/public_html/wp-includes/class-wp-hook.php on line 277
next() expects parameter 1 to be array, null given in /home/**/public_html/wp-includes/class-wp-hook.php on line 291
current() expects parameter 1 to be array, null given in /home/**/public_html/wp-includes/class-wp-hook.php on line 275

[dd32]

With no further details two years on since I last looked at this, here's my best idea of what's happening:

• A long running process is triggering a PHP timeout
• The timeout occurs when WP_Hook is in a specific state
• PHP's Garbage collector starts cleaning up memory, which ultimately results in either $this->callbacks or $this->iterations being null (although sometimes also being set to an int, which I suspect could also mean this is down to corrupt memory).
  • This is possibly triggered by $nesting_level = $this->nesting_level++; allocating memory, which then removes the memory used by $this->callbacks - which explains why the PHP Warnings start at line 271 rather than being prevented by line 265
• PHP attempts to cleanly shut down but allows a short grace period, which ultimately results in this loop running endlessly during that period

Although IMHO it's a bad change, I'm going to support the suggested fix from the original ticket, even with the potential performance aspects of it:

} while ( ! is_null( $this->iterations[ $nesting_level ] ) && false !== next( $this->iterations[ $nesting_level ] ) );

That'll still produce several warnings, but won't be stuck in the endless loop.

The alternative is to add something on line 273 prior to the loop, however, I suspect that if my above thoughts are correct, it's possible to trigger the issue mid-hook-loop in addition to prior to the loop (but the proof of that would be buried in the GB's of logs that this creates in those obscure scenario's).

I'm not going to reassign the milestone, in the hope that another committer will come along and own the fix and tests needed for it.

(edit: Strange observation: This shouldn't be possible to trigger on PHP 5.2, as next() will return false on the invalid inputs there, and exit the loop)

[getithappening]

Hi, I've just come across this forum while trying to work out why my WP error log is filling over 9 GB every day and cutting off my email.

Here are some of the error messages I'm getting:

PHP Warning: array_keys() expects parameter 1 to be array, integer given in /home/*/public_html/wp-includes/class-wp-hook.php on line 271
PHP Warning: current() expects parameter 1 to be array, null given in /home/*/public_html/wp-includes/class-wp-hook.php on line 275
PHP Warning: Invalid argument supplied for foreach() in /home/*/public_html/wp-includes/class-wp-hook.php on line 277
PHP Warning: next() expects parameter 1 to be array, null given in /home/*/public_html/wp-includes/class-wp-hook.php on line 291

There doesn't seem to be a solid fix for this, but I can't keep deleting my error log every day. I have other websites that don't have these issues, and the lines of code listed here that have errors are identical to the lines of code on the websites that do not have errors. So I'm at a loss.

Any help would be greatly appreciated.

Thanks,
Steve

[mordisko]

Hello,

we are a hosting company and this behavior is happening on hundreds of our customer's sites every day. We are forced to delete the logs several times every day because some recent event triggered a lot of these failures. We'd simply like to check in and ratify that this is happening.

What we've got so far:

• error_logs are filled with warnings [1]. The size of these logs is prohibitively high.
• Most of the entries on the logs are grouped by the thousands in the same timestamps (that means hour, minute and second).

We did try to search for similar things amongst the installations and we did not come to a full, solid conclusion.

Most of the sites are WP 4.0.9, a few of them are 5.0.x. Most of them are using the plugins "akismet" and "wordpress-seo". PHP versions are also varied, being most of them 5.6.

If there's anything we can do, please, let us know.

Kind regards,

---

[1]:

[21-Dec-2018 06:55:25 UTC] PHP Warning: next() expects parameter 1 to be array, null given in /var/www/wp-includes/class-wp-hook.php on line 291

[21-Dec-2018 06:55:25 UTC] PHP Warning: current() expects parameter 1 to be array, null given in /var/www/wp-includes/class-wp-hook.php on line 275

[21-Dec-2018 06:55:25 UTC] PHP Warning: Invalid argument supplied for foreach() in /var/www/wp-includes/class-wp-hook.php on line 277

[21-Dec-2018 06:55:25 UTC] PHP Warning: next() expects parameter 1 to be array, null given in /var/www/wp-includes/class-wp-hook.php on line 291

[21-Dec-2018 06:55:25 UTC] PHP Warning: current() expects parameter 1 to be array, null given in /var/www/wp-includes/class-wp-hook.php on line 275

[21-Dec-2018 06:55:25 UTC] PHP Warning: Invalid argument supplied for foreach() in /var/www/wp-includes/class-wp-hook.php on line 277

[21-Dec-2018 06:55:25 UTC] PHP Warning: next() expects parameter 1 to be array, null given in /var/www/wp-includes/class-wp-hook.php on line 291

[21-Dec-2018 06:55:25 UTC] PHP Warning: current() expects parameter 1 to be array, null given in /var/www/wp-includes/class-wp-hook.php on line 275

[21-Dec-2018 06:55:25 UTC] PHP Warning: Invalid argument supplied for foreach() in /var/www/wp-includes/class-wp-hook.php on line 277

[...]

[frettled]

Replying to mordisko:

If there's anything we can do, please, let us know.

As I see it, there are two reasonably easy work-arounds:

1) Hot-patch all Wordpress installations that you come across with the original, suggested patch.
2) Disable logging of PHP warnings for Wordpress sites - which may be sabotaged when other things reenable logging.

There is also a slightly more difficult work-around:

3) Use an ELK stack (or similar), filter out warnings like these before they are stored to the log server. That is more CPU intensive, but should be workable, and you can probably change webserver log rotation to be by file size.

Either work-around is more workable than waiting for a fix. It's been two years, this fix is *never* going to make it to older supported versions of Wordpress, and if it makes it to e.g WP 5.4 sometime in the future, you will still be looking at WP 3.7 through 5.3 making the same mistake.

[pinchies]

I have the same error on my personal blog. On my log, it looks like it only started in October.

• WordPress 4.9.8
• PHP 5.3

[08-Oct-2018 03:40:32 UTC] PHP Warning: array_keys() expects parameter 1 to be array, integer given in /home/*/public_html/wp-includes/class-wp-hook.php on line 271

[08-Oct-2018 03:40:32 UTC] PHP Warning: current() expects parameter 1 to be array, null given in /home/*/public_html/wp-includes/class-wp-hook.php on line 275

[08-Oct-2018 03:40:32 UTC] PHP Warning: Invalid argument supplied for foreach() in /home/*/public_html/wp-includes/class-wp-hook.php on line 277

(home directory masked for privacy)

Error file was 788MB, but hey, who's counting? It only took my website completely offline after the disk filled up, nothing serious. 😑

Fortunately, its only a blog and not a mission critical website. I'm happy to provide a zip or even temporary login details to anyone who wants to have a poke if that might help things be diagnosed more rapidly. You can contact me at my username at google's email service dot company.

[dangergirl42]

hi there,

we are getting the same messages and it lead me to this thread:

[10-Jan-2019 04:32:53 UTC] PHP Warning: next() expects parameter 1 to be array, null given in /home/argylelo/public_html/wp-includes/class-wp-hook.php on line 291
[10-Jan-2019 04:32:53 UTC] PHP Warning: current() expects parameter 1 to be array, null given in /home/argylelo/public_html/wp-includes/class-wp-hook.php on line 275
[10-Jan-2019 04:32:53 UTC] PHP Warning: Invalid argument supplied for foreach() in /home/argylelo/public_html/wp-includes/class-wp-hook.php on line 277

Just to clarify before I have a go - I change this on line 291:

} while ( false !== next( $this->iterations[ $nesting_level ] ) );

to

} while ( ! is_null( $this->iterations[ $nesting_level ] ) && false !== next( $this->iterations[ $nesting_level ] ) );

Is that right?
And can I change it back if it does not work, or do you change the way WordPress functions just by trying this? - sorry I am not clear on all this stuff and want to check first...

I've tried disabling all the plugins etc, but the log keeps growing to around 49GB per day regardless. PHP version is 5.6.39 - WordPress is 5.0.3–en_AU. not sure when it started but I noticed the size a few days ago when the website had no space or memory left to run. I/O Usage and CP usage consistently in red, appreciate assistance thank you!

[frettled]

Replying to dd32:

(edit: Strange observation: This shouldn't be possible to trigger on PHP 5.2, as next() will return false on the invalid inputs there, and exit the loop)

No, that's not how next() works in PHP.

Here's a very simple test for you:

<?php
$var = NULL;
$n = 0;
while ($n < 10 && false !== next($var)) {
    $n++;
    echo $n;
}

This produces an infinite loop in PHP 5.x and 7.x, only by the kindness of the check for $n, your screen's scrollback won't fill infinitely.

My patch works. If it breaks something that you avoid an infinite loop instead of having a log-filling, CPU-gobbling infinite loop, that's perfectly fine. It's better that the code crashes.

If Wordpress has some sort of error handler that can be invoked in the case that the "impossible" (yet prevalent) NULL condition occurs, invoking it at that time seems like a good idea.

[trongn]

Replying to frettled:

Hi,

I tried to apply your fix however the infinitive loop still came back with a slightly different error:
PHP Warning: next() expects parameter 1 to be array, boolean given, ...
PHP Warning: current() expects parameter 1 to be array, ...

So i have modified the fix to use is_array instead of !is_null and it covers both NULL and boolean issues. I have 2 different versions of Wordpress to manage and there are different php files to modify based on the error details in error_log file

Wordpress v4.9.9 (recent version, and fix all similar while loops)
Fix class-wp-hook.php in wp-includes folder (fix all similar while loops)

} while ( is_array( $this->iterations[ $nesting_level ] ) && false !== next( $this->iterations[ $nesting_level ] ) );

Wordpress v4.6.1
Fix plugin.php in wp-includes folder (fix all similar while loops)

} while (is_array($wp_filter[$tag]) && next($wp_filter[$tag]) !== false );

I hope this will help.
Cheers

[jdw0203]

Hi,
I'm managing at least 4 sites which are all affected by this issue. Each of the 4 sites are wordpress, running Divi theme with Woocommerce and a number of other plugins installed. All are almost identical in terms of the plugins used on each. All are sitting on shared hosting servers.

The first 6 lines of the error log are:

[24-Jan-2019 16:43:20 UTC] PHP Warning:  array_keys() expects parameter 1 to be array, integer given in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 271
[24-Jan-2019 16:43:20 UTC] PHP Warning:  current() expects parameter 1 to be array, null given in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 275
[24-Jan-2019 16:43:20 UTC] PHP Warning:  Invalid argument supplied for foreach() in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 277
[24-Jan-2019 16:43:20 UTC] PHP Warning:  next() expects parameter 1 to be array, null given in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 291
[24-Jan-2019 16:43:20 UTC] PHP Warning:  current() expects parameter 1 to be array, null given in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 275
[24-Jan-2019 16:43:20 UTC] PHP Warning:  Invalid argument supplied for foreach() in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 277

The last 3 lines shown above infinitely repeat themselves until the error log grows to a large file size, which then maxes out the disk space available on each hosting service.

I've been experiencing this with all 4 sites since early December, when the last major Wordpress update rollout occurred.

Is anyone willing and able to help debug this issue?

Desperately seeking assistance with this pain in the behind issue.

Cheers

[trongn]

Hi,

I believe fixing all relevant loops in class-wp-hook.php as below will help. Please read my previous comment for more details.

} while ( is_array( $this->iterations[ $nesting_level ] ) && false !== next( $this->iterations[ $nesting_level ] ) );

Cheers

Replying to jdw0203:

Hi,
I'm managing at least 4 sites which are all affected by this issue. Each of the 4 sites are wordpress, running Divi theme with Woocommerce and a number of other plugins installed. All are almost identical in terms of the plugins used on each. All are sitting on shared hosting servers.

The first 6 lines of the error log are:

[24-Jan-2019 16:43:20 UTC] PHP Warning:  array_keys() expects parameter 1 to be array, integer given in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 271
[24-Jan-2019 16:43:20 UTC] PHP Warning:  current() expects parameter 1 to be array, null given in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 275
[24-Jan-2019 16:43:20 UTC] PHP Warning:  Invalid argument supplied for foreach() in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 277
[24-Jan-2019 16:43:20 UTC] PHP Warning:  next() expects parameter 1 to be array, null given in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 291
[24-Jan-2019 16:43:20 UTC] PHP Warning:  current() expects parameter 1 to be array, null given in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 275
[24-Jan-2019 16:43:20 UTC] PHP Warning:  Invalid argument supplied for foreach() in /home/mysite/public_html/wp-includes/class-wp-hook.php on line 277

The last 3 lines shown above infinitely repeat themselves until the error log grows to a large file size, which then maxes out the disk space available on each hosting service.

I've been experiencing this with all 4 sites since early December, when the last major Wordpress update rollout occurred.

Is anyone willing and able to help debug this issue?

Desperately seeking assistance with this pain in the behind issue.

Cheers

[mordisko]

This is still a problem.

As a hosting provider we have hundreds of customers affected by this. The main problem is that an endless loop of errors generates thousands of gigabytes of error logs, which causes customers' quota to fill, which in turn ends up affecting more services than the WordPress itself.

I understand the sentiment of trying to fully fix the underlying problem instead of applying a patch to prevent it from ever happening, but I do think that there has to be some middle ground here. How does having an endless loop help to find the issue? doesn't it make sense to stop the loop at some point, register or throw an error and move on?

So far we have been unable to reproduce the error (and we devoted a lot of time to this) and the fact that this has been opened for quite a while now is forcing us to stop logging php errors altogether, which is a useful default feature for our end customers.

We have been trying to diagnose this for a while now, but so far we haven't been able to find any patterns on plugins, themes or WP/php versions.

It would be good to have a solution for this as it seems kind of strange to allow something like this to ever happen.

[pcamepc]

Hi,

Does your websites runs on cPanel by any chance ?

We are currently experiencing the issue on websites that haven't been modified for a few months. The only change was a cPanel update made today on our server :

Change Log for 80.0.22
Entry: 2019-07-16, 01:30 (UTC)
[TSR-2019-0004]

I'm thinking that maybe the update interrupted a process and started the infinite loop. Like @dd32 suggested :

• A long running process is triggering a PHP timeout
• The timeout occurs when WP_Hook is in a specific state
• PHP's Garbage collector starts cleaning up memory, which ultimately results in either $this->callbacks or $this->iterations being null (although sometimes also being set to an int, which I suspect could also mean this is down to corrupt memory).
• This is possibly triggered by $nesting_level = $this->nesting_level++; allocating memory, which then removes the memory used by $this->callbacks - which explains why the PHP Warnings start at line 271 rather than being prevented by line 265
• PHP attempts to cleanly shut down but allows a short grace period, which ultimately results in this loop running endlessly during that period

Maybe a cPanel update could trigger this issue ?

Thanks

Replying to mordisko:

This is still a problem.

As a hosting provider we have hundreds of customers affected by this. The main problem is that an endless loop of errors generates thousands of gigabytes of error logs, which causes customers' quota to fill, which in turn ends up affecting more services than the WordPress itself.

I understand the sentiment of trying to fully fix the underlying problem instead of applying a patch to prevent it from ever happening, but I do think that there has to be some middle ground here. How does having an endless loop help to find the issue? doesn't it make sense to stop the loop at some point, register or throw an error and move on?

So far we have been unable to reproduce the error (and we devoted a lot of time to this) and the fact that this has been opened for quite a while now is forcing us to stop logging php errors altogether, which is a useful default feature for our end customers.

We have been trying to diagnose this for a while now, but so far we haven't been able to find any patterns on plugins, themes or WP/php versions.

It would be good to have a solution for this as it seems kind of strange to allow something like this to ever happen.

[mordisko]

Yes, they do.

That hypothesis seems plausible because earlier today I've managed to attach a GDB to a faulty process that was running and the back trace showed that the interruption was launched after a garbage collector call.

I've just left the office but I'll post the trace tomorrow in case it helps.

[mordisko]

This is a backtrace that I've managed to get from a faulty system process that was stuck in one of those endless loops. As some of you mentioned, there seems to be a garbage collector call which might be the reason as to why some variables are misbehaving:

#0  0x00007f0ba5f2aeb0 in __open_nocancel () from /lib64/libc.so.6
#1  0x000000000042fabf in php_log_err_with_severity ()
#2  0x000000000042fea3 in php_error_cb ()
#3  0x00007f0b99405113 in soap_error_handler () from /opt/alt/php72/usr/lib64/php/modules/soap.so
#4  0x0000000000431dc1 in zend_error_noreturn ()
#5  0x00000000004324ef in zend_internal_type_error ()
#6  0x0000000000432914 in zend_wrong_parameter_type_error ()
#7  0x000000000076ba5a in execute_ex ()
#8  0x00000000006bb47b in zend_call_function ()
#9  0x00000000006bb9b5 in _call_user_function_ex ()
#10 0x00000000005f5c85 in user_shutdown_function_call ()
#11 0x00000000006ddbab in zend_hash_apply ()
#12 0x00000000005f9ab6 in php_call_shutdown_functions ()
#13 0x00000000006663dd in php_request_shutdown ()
#14 0x0000000000775928 in lsapi_sigterm ()
#15 0x00000000006f745d in zend_signal_handler ()
#16 0x00000000006f75ab in zend_signal_handler_defer ()
#17 <signal handler called>
#18 0x00000000006f436e in gc_scan ()
#19 0x00000000006f4d99 in zend_gc_collect_cycles ()
#20 0x00000000006f49ae in gc_possible_root ()
#21 0x000000000073b079 in ZEND_ASSIGN_SPEC_CV_VAR_RETVAL_USED_HANDLER ()
#22 0x000000000076f515 in execute_ex ()
#23 0x0000000000774643 in zend_execute ()
#24 0x00000000006cb653 in zend_execute_scripts ()
#25 0x00000000006676b0 in php_execute_script ()
#26 0x0000000000776c9b in lsapi_module_main ()
#27 0x0000000000776f46 in processReq ()
#28 0x0000000000436b1a in main ()

Also, here's the process' strace, although is not very interesting, as it's constantly writing the log due to the loop described in this ticket:

open("error_log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 5
write(5, "[16-Jul-2019 15:13:02 UTC] PHP Warning:  current() expects parameter 1 to be array, null given in /home/USERNAME/public_html/wp-includes/class-wp-hook.php on line 275\n", 167) = 167
close(5)                                = 0
open("error_log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 5
write(5, "[16-Jul-2019 15:13:02 UTC] PHP Warning:  Invalid argument supplied for foreach() in /home/USERNAME/public_html/wp-includes/class-wp-hook.php on line 277\n", 153) = 153
close(5)                                = 0
open("error_log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 5
write(5, "[16-Jul-2019 15:13:02 UTC] PHP Warning:  next() expects parameter 1 to be array, null given in /home/USERNAME/public_html/wp-includes/class-wp-hook.php on line 291\n", 164) = 164
close(5)                                = 0
open("error_log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 5
write(5, "[16-Jul-2019 15:13:02 UTC] PHP Warning:  current() expects parameter 1 to be array, null given in /home/USERNAME/public_html/wp-includes/class-wp-hook.php on line 275\n", 167) = 167
close(5)                                = 0
open("error_log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 5
write(5, "[16-Jul-2019 15:13:02 UTC] PHP Warning:  Invalid argument supplied for foreach() in /home/USERNAME/public_html/wp-includes/class-wp-hook.php on line 277\n", 153) = 153
close(5)                                = 0
open("error_log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 5
write(5, "[16-Jul-2019 15:13:02 UTC] PHP Warning:  next() expects parameter 1 to be array, null given in /home/USERNAME/public_html/wp-includes/class-wp-hook.php on line 291\n", 164) = 164
close(5)                                = 0
open("error_log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 5
write(5, "[16-Jul-2019 15:13:02 UTC] PHP Warning:  current() expects parameter 1 to be array, null given in /home/USERNAME/public_html/wp-includes/class-wp-hook.php on line 275\n", 167) = 167
close(5)                                = 0
open("error_log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 5
write(5, "[16-Jul-2019 15:13:02 UTC] PHP Warning:  Invalid argument supplied for foreach() in /home/USERNAME/public_html/wp-includes/class-wp-hook.php on line 277\n", 153) = 153
close(5)                                = 0
open("error_log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 5
write(5, "[16-Jul-2019 15:13:02 UTC] PHP Warning:  next() expects parameter 1 to be array, null given in /home/USERNAME/public_html/wp-includes/class-wp-hook.php on line 291\n", 164) = 164
close(5)

[jacobcolton]

Is anyone who is experiencing this running lsphp / Litespeed on cPanel as we are seeing this but we seem to be only be seeing it on servers running LiteSpeed and cPanel, we have tried upgrading to the latest version of LiteSpeed and we are now reviewing.

[mordisko]

We do. Are you using CloudLinux?

Replying to jacobcolton:

Is anyone who is experiencing this running lsphp / Litespeed on cPanel as we are seeing this but we seem to be only be seeing it on servers running LiteSpeed and cPanel, we have tried upgrading to the latest version of LiteSpeed and we are now reviewing.

[jacobcolton]

Some are and some aren't.

[zhenmue]

Well, we are a hosting provider with at least 2-3 thousand Wordpress Sites.

During the past months we have seen this specific issue very random trough our clients.. maximu 1 or 2 every 3-6 months, nothing that big.

But in the last 3 days?, we have seen it in at least 30+ clients, trough different servers, different wordpress versions, different php versions, etc.

All Servers running CloudLinux, Cpanel, LiteSpeed.

It's a difficult issue, since it's not just filling the error_log up to the Disk space of the plan, but also maxing out the Disk I/O of the plan, and the RAM.

We have been able to "fix" (not really a fix) the majority, just tweaking the error_Reporting value, but this is not a real solution.

This Bug affects at least from what we have seen:
Wordpress 4.9 5.0 5.1 5.2
PHP 5.6 7.0 7.1 7.2
Cpanel 80+
LiteSpeed Latest
CloudLinux latest

[jacobcolton]

I do have a ticket open with cPanel about this topic at the moment and they are actively working with us to debug this. I will update this thread a soon as any relevant information becomes apparent. # 12873113 if useful for anyone.

[mrseiko]

We have about 60 servers, some of them getting hit pretty hard with this.

They are all CloudLinux 7 / cPanel 80.0.22 / Litespeed
What I've been seeing is that the ones affected seem to be Litespeed 5.3.8 build 2

Been updating to Litespeed 5.3.8 build 5, manually resting Litespeed and deleting error_log files.
Haven't seen issues on the updated servers yet, but I'll keep an eye.

[jacobcolton]

So I upgraded LiteSpeed on all our servers yesterday and didn't make any difference. cPanel believe this is a LiteSpeed issue and they have passed the case over to them to review.

[artaweb]

Ok! Now this is turning into a real problem here.

We have 109 baremetal servers that each host at least 450 wordpress websites. The error logs fills our disks in matter of seconds. Its intense and we can't find the problem. Contacted cPanel they said they are looking into it, wordpress does not reply and no one else knows anything. We are just deleting error_log files to overcome the problem until we figure out what is causing the issue.

Our setup on all servers are:
cPanel on CloudLinux and Litespeed web server.

I would appreciate if anyone got a way to fix this update this thread.

[jacobcolton]

Literally just had this email from LiteSpeed:

Hi Jacob,

After investigating this, it seems to be an issue with the latest "php-litespeed" package, or more specifically the LSAPI version that is used within the package.

In LSAPI 7.4 a feature got introduced to gracefully clean up temp files when PHP processes got killed during script execution - this feature got introduced to resolve an issue that resulted in big "temp" files in certain cases.

Sadly this feature also seems to have introduced another bug that causes PHP processes to hang in some cases, but in other cases, it would result in the large error_log files you're experiencing.

We're aware of the issue and are looking into a fix for this, however, I do currently not have an ETA when this fix will be published.

There is a workaround that can be used temporarily until we have a fix and this fix has been pushed by both cPanel and CloudLinux (depending on who maintains the EA4 packages on your systems).

The workaround currently is to do a rollback of the packages to the point where it uses LSAPI 7.3, this can be done using yum history undo, which allows doing a rollback of whole transactions performed by YUM.

On the system with IP xx.xx.xx.xx, to do the rollback you have to do:

$ yum history undo 440

The number 440 is the transaction ID from YUM, this ID can be found by doing yum history, you'll have to look for an update performed somewhere between the 15th of July and now that contains quite a few package updates (Altered), on this system the number is 91, but on systems with more PHP versions you'll likely see a higher number.

To be sure you're doing a rollback of the correct transaction ID, you can do yum history info ID to get info about the packages affected.

Using yum history undo will give you a list of packages that will be affected where you have to either accept or deny this change.

Additionally to prevent EA4 from automatically updating the packages again, you have to ensure that RPMUP in /etc/cpupdate.conf are set as following:

RPMUP=never

If you do this change from WHM, you have to go to Server Configuration -> Update Preferences, and find "Operating System Package Updates" and set this to "Never Update".

I've yet to find another workaround that is less annoying than this one, but at least this gets the systems to a state where you don't have to sit and monitor the disk space constantly.

When I have an update with an ETA, or if we find another way to make a workaround that is simpler, I'll be sure to update the ticket.

Best Regards,
LiteSpeed Technologies

[zhenmue]

So we can force a sooner real fix, we should open tickets reporting the bug to litespeed, so they can see it affects many customers.

[artaweb]

That isn't a fix. Its just gonna complicate things. I am dealing with 30 TB of error_log on almost 50K wordpress installs which majority have this problem. I guess a ticket bomb for the issue could force them to pull the update faster.

[zhenmue]

New update from LiteSpeed:

=update=
The issue is due to newly added logic to gracefully cleanup a crashed/killed PHP process in LiteSpeed SAPI 7.4.3 .
It will take some time to find a proper fix, and take time for CloudLinux team to update the RPM package.

In the mean time, we have released 5.3.8 build 6, which will greatly reduce the chance to trigger this problem, please give it a try.

/usr/local/lsws/admin/misc/lsup.sh -f -v 5.3.8

=update=

According to the changelog of the latest build it says:

[Bug fix] Temporarily stop PHP processes with SIGKILL as a workaround for problems caused by clean shutdown logic added to PHP LiteSpeed SAPI v7.4.3.

I hope this does helps.

But also, does someone see this issue without litespeed? or is 100% related to it?, would be good to know.

[mortezafarsiabi]

i have this issue
its abouts litespeed cash plugin, de active it till new update

[jacobcolton]

Has anyone tried 5.3.8 build 6, does it work?

[mortezafarsiabi]

i de active lite speed cash and it was good for hours its running again
nobody find any solution?

[zhenmue]

Using the Build 6 of LSWS, haven't seen this issue again since the update, almost 48 hours free issues.

Replying to mortezafarsiabi:

i de active lite speed cash and it was good for hours its running again
nobody find any solution?

Read comment 70.

[artaweb]

Upgraded to build 6.

I do not confirm that the issue is resolved but I don't see any sign of the heavy logging.

Here is a SSH command for those who need to delete the error_log file globally.

Safe to use:

find /home/*/public_html/ -name error_log -delete

(Note: This command deletes all the error_log files that is located inside every public_html folder for all accounts on the server inside the drive /home/)

[webdesignbymark]

This is happening on my Litespeed servers as well as Apache. I have websites hosted on Veerotech, which is Litespeed and on Siteground which is not. Doesn't matter, same problem and no resolution.

[jacobcolton]

To update you all, I have been informed the following:

---
cPanel has released an update for the LSAPI update as of 24th July, additionally, CloudLinux has released it as a beta for their alt-php (If using PHP Selector): ​https://www.cloudlinux.com/cloudlinux-os-blog/entry/beta-alt-php-updated-1-148 and I believe it will take roughly a week for that to go from their beta repository to their production repository.

CloudLinux still has to publish the update for their EasyApache 4 repositories, but I believe that will be done somewhere today, and then this will take roughly a week as well. They might, however, push these out with a bit more urgency.

---

So hopefully in the next few days, the fix will be in production and resolve this.

[webdesignbymark]

This is NOT just a Litespeed issue.