vgagaleski Imagine a test automation framework so powerful and versatile that it ignites a deep interest, making you eager to explore and understand every aspect of it. That’s exactly what Playwright did for me. In a world filled with countless testing tools, Playwright stands out as a true game-changer, revolutionizing how we approach test automation.
The landscape of test automation
Four years ago, while exploring various test automation frameworks, the landscape looked quite different. Selenium was the industry standard, heavily used by every team that wanted to do UI automation. Meanwhile, Cypress was just beginning to make waves in the testing community, offering a fresh approach to automation testing. As I explored various options, I came across an exciting new project in browser automation: Playwright. To understand its significance, it’s important to look at its roots.
At the moment, Playwright is maintained and developed by Microsoft, but its story begins with Puppeteer, Google’s JavaScript library which provides a high-level API to control Chrome/Chromium over the DevTools Protocol. Puppeteer was my go-to tool for several projects at that moment, particularly in the realm of web performance automation. In fact, I developed an extensive repository of recipes specifically for analyzing frontend web performance using Puppeteer. While Puppeteer was powerful, it had limitations which were stopping it from wider adoption.
Personally, the Puppeteer experience proved invaluable when I discovered Playwright. During my initial Playwright PoC, the transition felt natural and intuitive. I quickly recognized familiar concepts and patterns, which significantly eased my learning curve. Playwright was developed by a team of engineers who had previously worked on Puppeteer, but was designed to address limitations and extend its capabilities.
In this blog, I’ll dive deeper into these features and show you how Playwright has transformed my approach to test automation. Whether you’re a QA professional or just starting your automation journey, you’ll see why Playwright might be the last test automation framework you’ll ever need.
Why Playwright?
The standout characteristic of Playwright, in my view, is the amount of built-in tools that make writing and maintaining test cases incredibly straightforward. Playwright provides everything you need out-of-the-box, making it easy to get started and equally easy to keep things running smoothly over time.
I highly recommend using Playwright with TypeScript, as it’s the native and most efficient way to unlock Playwright’s full capabilities. Playwright has TypeScript built-in, which makes for a seamless and powerful development experience. However, this highly depends of your environment and tech-stack.Playwright is compatible with any IDE, but it truly shines in Visual Studio Code. The standard setup uses npm, yarn, or pnpm, but if you’re just starting out and prefer a more visual approach, there’s a Playwright VS Code extension that lets you complete the entire setup through a user-friendly UI. Whether you choose to install via the UI or the command line, Playwright will be up and running within minutes. You’ll even get a basic example test that runs across Chromium, Firefox, and WebKit, with built-in support for parallel runs on multiple workers. Once the test completes, Playwright automatically generates an HTML report showing results per browser, including screenshots, videos of failed tests, and robust test tracing for easier debugging.
And this is just the beginning! I haven’t even touched on the full feature set, but it’s already clear that starting with Playwright is a great experience. To explain why I’m not considering any other test automation frameworks and why I believe every team should consider Playwright for test automation, I’ll break down a few key criteria that everyone should evaluate when selecting a test automation framework.
1. Writing stable and reliable tests quickly
In today’s fast-paced development environment, writing stable and reliable tests quickly is essential for maintaining high-quality software. In an Agile environment, where sprints are packed with new features and often testing is deprioritized, we need to have the best tools to support the development and make sure proper automation testing is added. With Playwright’s robust features, such as its auto-waiting mechanism and user-centric locators, developers can minimize test flakiness and ensure that tests remain reliable even as applications evolve. This efficiency not only boosts confidence in the code but also accelerates the delivery of new features to users.
Auto-waiting mechanism
Playwright performs a range of actionability checks on elements to ensure actions behave as expected. Before executing any action, like clicking or typing, it automatically waits for all necessary conditions, such as element visibility, attachment to the DOM, and stability, to be met. Only when these checks pass does Playwright perform the action, if they do not pass within the specified timeout, the action fails with a TimeoutError. This auto-waiting approach significantly reduces flakiness compared to other frameworks by handling timing issues internally, leading to more stable and reliable tests without the need for explicit waits.
Accessible Locators First
A common pain point in UI automation is finding and interacting with the right elements on a page in a way that is both stable and straightforward. Playwright’s accessible locators first approach prioritizes accessible, human-readable locators, which are more resilient to UI changes over time. Moving away from fragile CSS and XPath selectors, Playwright prioritizes accessibility-focused locators like getByRole, getByLabel, and getByText. This approach not only makes tests more stable and resistant to UI changes but also encourages developers to build more accessible applications.
// Traditional locators
await page.click('#login-btn')
await page.fill('.username-input', 'user')
await page.click('//button[@class="submit"]')
await page.fill('[data-testid="password"]', 'pass123')// Accessible locators
await page.getByRole('button', { name: 'Login' }).click()
await page.getByLabel('Username').fill('user')
await page.getByText('Forgot Password?').click()
await page.getByRole('textbox', { name: 'Email' }).fill('[email protected]')This way, we are writing tests which are as close as possible to how a real user would use the application, as real users don’t see xPaths and testIds, but they do see buttons, labels, inputs with text etc.
Code generation
The built-in codegen tool allows you to record and generate test scripts by interacting with your application naturally. It serves as an excellent learning tool for understanding Playwright’s locator strategies and provides a quick starting point for complex test scenarios.
Fixtures
Test fixtures in Playwright enable consistent and reusable test setup code across your test suites. This feature helps reduce boilerplate code and ensures your tests start from a known, reliable state, making them isolated and easier to maintain.
I will be writing a full extended blog about fixtures, so stay tuned for that one.
2. CICD integration and running in parallel
Incorporating Playwright tests into your CI/CD pipeline is essential to ensure that new code changes are thoroughly tested and any issues are identified before they reach production. Playwright’s powerful integration with CI/CD tools and its ability to parallelize tests across multiple devices and browsers make it an excellent choice for fast and reliable testing at scale. Here’s a closer look at the critical components for setting up an effective CI/CD test integration using GitHub Actions, along with tips for handling parallel execution, multi-browser testing, and leveraging workers and shards.
Workers and shards
When running large test suites, it’s essential to maximize speed and efficiency. Playwright is built for this, offering horizontal and vertical parallel execution. You can easily setup workers, only by passing the cli flag or adding workers number as part of the config, and you can run 4, 6, 8 … tests in parallel based on how powerful your machine is. If you want to go one step further, you can setup shards, which generally run as part of CICD setup. Each shard is a separate container that allows you to run different tests. This means that with few lines of code you can execute X shards with Y workers which is X multiplied by Y number of tests.
Browsers and devices
One of the standout features of Playwright is its native support for testing across all major browsers - Chromium, Firefox, WebKit and Edge. This cross-browser compatibility is key for ensuring your application functions smoothly regardless of the user’s browser. You can specify the browsers and devices directly in your test command or define them in your configuration file.
// Playwright browser and device configuration
import { defineConfig, devices } from '@playwright/test'
export default defineConfig({
projects: [
/* Test against desktop browsers */
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] }
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] }
},
/* Test against mobile viewports */
{
name: 'Mobile Chrome',
use: { ...devices['Pixel 5'] }
},
{
name: 'Mobile Safari',
use: { ...devices['iPhone 12'] }
}
]
})GitHub Actions setup
Playwright integrates with a lot of CI/CD tools, but I would always recommend GitHub Actions, as that is where you will get the most out of it for the shortest amount of setup time. GitHub Actions offers reusable workflows that can run on PR, main, trigger, you name it, and out-of-the-box parallelization leveraging workers and shards using GitHub Actions matrix, which is perfect for fast test execution. Setting up a Playwright job in GitHub Actions is as simple as adding a .yml configuration file to your project:
# .github/workflows/playwright-tests.yml
name: Playwright Tests
on:
pull_request:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
browser: [chromium, firefox, webkit]
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Install dependencies
run: npm install
- name: Run Playwright tests
run: npx playwright test --project=${{ matrix.browser }}3. Test reporting and debugging
Effective test reporting that provides clear reasons for failures with flexible tools for debugging is a crucial asset on your automation journey. Teams should have reporting and tools in place that will help them understand test failures fast and act swiftly to address them. Playwright comes with a lot of features that provide extensive test reporting and support for test debugging.
Playwright test reports
Playwright provides several reporters, but the HTML test report shows clear view of your test execution results and helps identify issues quickly. It automatically organizes your test runs, showing passed, failed, flaky and skipped tests along with their execution time. The report includes detailed error messages and stack traces when tests fail. Additionally, you can set Playwright to automatically capture videos, screenshots, and traces when tests fail. Videos show the exact sequence of actions that led to a failure, while screenshots capture the page state at the moment of failure.
// Playwright configuration for test artifacts
export default defineConfig({
use: {
video: 'retain-on-failure', // Save videos for failed tests
screenshot: 'only-on-failure', // Capture screenshots on failure
trace: 'retain-on-failure' // Generate traces for debugging
}
})Trace files are the most powerful debugging tool in the HTML report. They provide the most detailed insight, offering a timeline view of the test execution including network requests, console logs, and source location. You can even test out different locators in the trace viewer. These artifacts are invaluable when debugging issues, especially in CI environments where you can’t reproduce the problem locally, and the best thing about all of this is that you get them out-of-the-box, only by providing a few lines of code in the config.
Debugging
Playwright includes several tools to help you identify and fix test issues effectively. You can pause test execution at any point to inspect the current state of the page and step through your test code. There is a Playwright Test VSCode extension that allows you to debug the tests directly from VSCode. You can also debug tests in UI mode which offers an interactive way to develop and debug your tests. You can watch tests execute in real-time and see exactly what’s happening at each step. The time-travel feature lets you move back and forth through test execution, examining the page state at any point. This makes it particularly useful when writing new tests or investigating failing ones.
Conclusion
My experience as an SDET working with Playwright for over the past few years has been nothing short of transformative. Playwright’s extensive feature set, from auto-waiting mechanisms and accessible locators to powerful debugging and reporting tools, has allowed me to build reliable and efficient test automation frameworks. However, this didn’t happen over night, it was a process of continuous evaluation and iteration.
As I continue to deepen my expertise with Playwright on a daily basis, I look forward to sharing more insights and practical tips in future blogs. My goal is to help others leverage the power of Playwright to elevate their automation practices, enhance test reliability, and ensure faster, more efficient testing. Stay tuned for more blogs as I explore and share ways to make the most out of this great test automation framework!
For more details on Playwright, you can always refer refer to the official documentation which was used for my examples as well.