[Experiment] Try running performance tests w/ Playwright #34699
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Size Change: 0 B Total Size: 1.06 MB ℹ️ View Unchanged
|
WunderBart
added
[Type] Technical Prototype
Yeah, that's the important bit. The other one which would be harder to check is whether the auto-waits could "hide" performance impact. Meaning, if something is slower but both branches auto-waits, could this lead to numbers being "normalized" in both cases? I don't think the ones relying on "trace" would be but the ones relying on timers (load I think) could. |
Not sure what you mean, sorry. How could something that is slower/faster than the comparison branch be obscured by the auto-waiting mechanism? |
WunderBart
force-pushed
the
try/playwright
branch
from
8e63353
to
3cdb35f
Compare
|
@WunderBart hey! I'm from the Playwright team, saw that you are considering migrating to Playwright. Awesome work so far! If you have any questions, don't hesitate to reach out to me. In general we recommend our own test-runner (which is quite similar to the Jest test-runner but written from scratch) since it contains a lot of end-to-end testing utils, tooling, concurrency, reporting, assertions, artifacts etc. and an extensive configuration support. In your scenario it might make sense to do it in iterations and migrate to it in a later standpoint. |
Thanks so much for reaching out, @mxschmitt! It's all still very much in a POC phase. Having said that, I'm pretty optimistic about the migration as more and more folks seem to be on board with it. Regarding the first-party test-runner, I'd really like to see it utilized in Gutenberg eventually. I'm not yet fully familiarized with all the custom e2e things that are being currently provided for Puppeteer though, so I can't say how feasible the migration to Playwright test-runner would be. I'd love to hear what some more experienced folks have to say about it, though! 🙏 /cc @gziolo @youknowriad @tellthemachines @kevin940726 |
It's not available directly via Page with Playwright
WunderBart
force-pushed
the
try/playwright-performance
branch
from
d498111
to
258ea78
Compare
|
I'd say it's pretty feasible. However, I don't see any reasons to migrate to that now since that Jest seems to work just fine too and people are already familiar with it. We can definitely re-consider it in a later standpoint as you said though 👍 . Thanks for the offer, @mxschmitt, let's keep in touch |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Base PR: #34089
Following up on concerns about performance metrics, I've migrated (all) the performance tests to Playwright and compared them to ones ran with Puppeteer. It looks like Playwright is showing just a tad better numbers in terms of speed and stability, but the important part is that those metrics do not differ significantly, I guess? Pinging @youknowriad and @tellthemachines for some feedback on these numbers 🙏
You can run migrated performance tests locally by checking out this branch and running
npm run test-performance.