29. 12. 2023
Development, NetEye
In order to visually test NetEye on browsers at each stage of development and before each release, we have a continuous integration (CI) step dedicated to Selenium testing.
Selenium is a longtime open-source tool for browser automation; it’s a widely-used tool with a huge community of users. The framework is capable of automating and controlling web browsers and interacting with UI elements, and it’s the most popular framework in the IT industry today. It supports just about all major browsers and popular programming languages.
Over time, we’ve developed a large number of tests for each NetEye module. To improve the quality of the code, we also created a common library that contains the most often used functions and all the logic that’s shared among different modules. And in order to save time during a package build we parallelize tests, which helped us move from about an hour dedicated just to Selenium testing to only 15 minutes.
After all this time working with Selenium, we decided to look around for a possible fresh alternative. Playwright caught our attention.
Playwright is a relatively new open-source tool for browser automation, with its first version released by Microsoft in 2020. It’s designed for end-to-end automated testing of web apps. Like Selenium, it’s cross-platform, cross-browser, and cross-language.
In Selenium, WebDriver does not track the active, real-time state of the DOM, which can lead to some flaky tests. In fact, we developed different custom functions that use other types of wait, like implicit waits and explicit waits, and retry, until the searched element gets correctly rendered in the DOM.
Playwright comes with a built-in auto-wait strategy. It performs a range of actionability checks on the elements before proceeding to ensure these actions behave as expected. It auto-waits for all the relevant checks to pass and only then performs the requested action. If the required checks don’t pass within the given timeout, the action fails with the TimeoutError. This helps Playwright eliminate that flakiness.
Both Selenium and Playwright support parallel test execution, but with Playwright, it comes out of the box. Moreover, with Playwright, you can have test scenarios that span multiple tabs, multiple origins, and multiple users. You can also create scenarios with different contexts for different users and run them against your server, all in a single test.
Screenshots and recordings help debug test failures, especially when running in headless mode in a CI environment. Unlike with Playwright where as I said it’s shipped with these capabilities out of the box, with Selenium instead we had to set up a third-party app for checking test execution and failures.
In terms of speed, Selenium is considered to be slower than Playwright. The former is more suitable for small to average-sized scraping projects, as more computing power will significantly reduce speed.
Our journey with Playwright has just started. Our first step was to create a common library where we defined the most-used functions, as for example the login in NetEye that every test should execute before running the test itself. This library also takes care of the installation and configuration of the Playwright environment in a NetEye container.
We are currently working on migrating our Selenium tests. It will take a while for sure, in particular because we have to translate the tests applying new best practices and this sometimes also means rethinking the underlying logic.
One of the principal issues we’re currently facing is test isolation: in fact for running multiple tests in parallel on the same container, we have to work at ensuring that one test doesn’t affect the other ones; we have to use proper locators and sometimes avoid particular actions that can cause an unexpected scenario for those other tests.
There’s a long road ahead before we can completely remove Selenium from our testing environment …
If you are interested in finding out if Playwright is a valid alternative to Selenium and if it will help you save time and achieve stability, stay tuned, new blog post(s) will follow!
These Solutions are Engineered by Humans
Did you find this article interesting? Does it match your skill set? Programming is at the heart of how we develop customized solutions. In fact, we’re currently hiring for roles just like this and others here at Würth Phoenix.
Valentina Da Rold
Hi, I'm Valentina and I'm a Frontend Developer at Wuerth Phoenix. I started out my career applying my Cryptography skills to coding, but really quickly fell in love with the web. I have been making websites and applications since 2012 and I still can't get enough of it. Along the way I found a passion for front-end development, and I use this passion to create interfaces that solve problems. When I'm not creating beautiful solutions, I enjoy cooking or doing sport, while listening to beautiful music.
Author
Latest posts by Valentina Da Rold
19. 12. 2023
Development, Events, NetEye
We spent a (Vue)day in Verona