Cannot save new custom template when name contains none-Latin characters

[mburridge]

When you try to save a new custom template where the template name contains one or more ​non-latin characters you get an error. The error appears to be related to the REST endpoint.

See ​GitHub issue #41214.

Too reproduce:

1. Select: Appearance -> Editor in the WP Admin
2. Click the WordPress logo in top left, and select Templates
3. Click Add New button (top right)
4. Select Custom template and give it a name with at least one non-latin character
5. Save the new template
6. Confirm that you see the error in the editor and in the console.

[mburridge]

#57328 was marked as a duplicate.

There is an issue where when a page has a slug that has nonlatin characters when a template is created, the user cannot make any changes, nor delete the template. This pr solves that problem.

Trac ticket: 57329

[antonyagrios]

Minor explanation of the introduced regex:

Before it was \w: This actually matches the following: a-z, A-Z, 0-9 and _
Changing this to \pL0-9_, we match the same as before, plus all the unicode characters for all the languages mentioned [here](​https://www.php.net/regexp.reference.unicode).

I also kept the 0-9_ to match what we had before when we used \w.

One thing I'm uncertain about is whether \p is safe to use on all platforms. It seems ​support was added in PHP 5.1, so I imagine we're _probably_ fine? It'd be good to have some confirmation of this.

This is what I thought as well. That is because it was introduced in PHP 5.1 would be safe. I also noticed that the tests we are running have 5.6 as the older target. So my guess is that we are good here. Also, we have some PHP compatibility tests in place, which, I assume as I haven't read them, there are there to check code compatibility.

@kozer When you have a moment, can you update the pull request description?

It also looks like you need to resolve any merge conflicts.

This is a great improvement. Thank you, @kozer, @danielbachhuber!

P.S. I think we also need to update slug pattern matcher in schema - ​https://github.com/WordPress/wordpress-develop/blob/6.1/src/wp-includes/rest-api/endpoints/class-wp-rest-templates-controller.php#L837

This is a great improvement. Thank you, @kozer, @danielbachhuber!

P.S. I think we also need to update slug pattern matcher in schema - ​https://github.com/WordPress/wordpress-develop/blob/6.1/src/wp-includes/rest-api/endpoints/class-wp-rest-templates-controller.php#L837

Done! Thanks for mentioning that.

@kozer Looks like there's one last test that might need to be updated here, too.

@kozer Looks like there's one last test that might need to be updated here, too.

@danielbachhuber , It seems there is a check that's failing in all of those checks:
Ensure version-controlled files are not modified or deleted
However wp-api-generated.js is auto-generated when tests are running, and indeed there are changes that have occurred.
Do I need to version the file somehow? Do something else? The tests are passing and this isn't an implication locally.

It seems there is a check that's failing in all of those checks:
Ensure version-controlled files are not modified or deleted
However wp-api-generated.js is auto-generated when tests are running, and indeed there are changes that have occurred.
Do I need to version the file somehow? Do something else? The tests are passing and this isn't an implication locally.

@kozer The specific test that fails is git diff --exit-code:

It fails because git is dirty when you run the entire test suite:

$ git status
## fix_template_api_non_latin
 M tests/qunit/fixtures/wp-api-generated.js

If you run composer test -- --filter test_build_wp_api_client_fixtures locally, the file is regenerated and you can commit the diff.

It seems there is a check that's failing in all of those checks:
Ensure version-controlled files are not modified or deleted
However wp-api-generated.js is auto-generated when tests are running, and indeed there are changes that have occurred.
Do I need to version the file somehow? Do something else? The tests are passing and this isn't an implication locally.

@kozer The specific test that fails is git diff --exit-code:

It fails because git is dirty when you run the entire test suite:

$ git status
## fix_template_api_non_latin
 M tests/qunit/fixtures/wp-api-generated.js

If you run composer test -- --filter test_build_wp_api_client_fixtures locally, the file is regenerated and you can commit the diff.

@danielbachhuber I run this locally, and the file is now regenerated. However, I see some changes that I don't know if they should be there.
Can you help me with that?

However, I see some changes that I don't know if they should be there.
Can you help me with that?

@kozer What are the changes?

However, I see some changes that I don't know if they should be there.
Can you help me with that?

@kozer What are the changes?

For example ​this change or ​this

I assume that only the regexes should be changed.

@kozer Yep, that's correct.

For example ​this change or ​this

What did you find when you looked around for additional context?

What did you find when you looked around for additional context?

@danielbachhuber I didn't look around for additional context, tbh. And this is because even with those changes, the tests are still failing.
Originally I had only the regexes committed. And then I took those changes back, tests kept failing, and then tested the theory you proposed (which introduced the extra changes)
I'm not even sure if we should commit any changes to this file, as it seems that it changes, every time the tests run.

I didn't look around for additional context, tbh. And this is because even with those changes, the tests are still failing.
Originally I had only the regexes committed. And then I took those changes back, tests kept failing, and then tested the theory you proposed (which introduced the extra changes)

@kozer Thanks for looking into it! No further action needed on your part. I'll see if I can track it down sometime next week and then land it once I do.

[danielbachhuber]

In 55294:

REST API: Support non-Latin characters in template route regex.

Non-Latin characters are URL-encoded (e.g. %cf%84%ce%b5%cf%83%cf%84). Matching % in the route ensures templates with non-Latin titles can be properly saved.

Props antonyagrios, mburridge.
Fixes #57329.

I didn't look around for additional context, tbh. And this is because even with those changes, the tests are still failing.

Originally I had only the regexes committed. And then I took those changes back, tests kept failing, and then tested the theory you proposed (which introduced the extra changes)

@kozer As it turns out, the generated wp-api-generated.js is slightly different if you run the full test site (vs the individual test). I updated it with 2e58f622d58fe6782cf2ab47afdc88721f2a2885

Pull request is shipped with https://core.trac.wordpress.org/changeset/55294

I didn't look around for additional context, tbh. And this is because even with those changes, the tests are still failing.
Originally I had only the regexes committed. And then I took those changes back, tests kept failing, and then tested the theory you proposed (which introduced the extra changes)

@kozer As it turns out, the generated wp-api-generated.js is slightly different if you run the full test site (vs the individual test). I updated it with ​2e58f62

Thanks for sticking with it!

Pull request is shipped with https://core.trac.wordpress.org/changeset/55294

Can you provide me the exact command you run to update that file, for reference?

@kozer, I think it's npm run test:php without the --fiter or a --group.

Moves test files from ​https://github.com/WordPress/wordpress-develop/pull/3819 to Latin character filenames. Non-Latin characters can break SVN checkouts (e.g. ​https://github.com/wp-cli/scaffold-command/actions/runs/4202081401/jobs/7293427452)

Trac ticket: https://core.trac.wordpress.org/ticket/57329

[danielbachhuber]

In 55362:

REST API: Only use Latin characters in test filenames.

Non-Latin characters can break SVN checkout in some environments.

Follow up from [55294].

See #57329, #57761.

Shipped in https://core.trac.wordpress.org/changeset/55362