Success with UI Automation

UI Automation is hard to create to make reliable, consistent testing to validate functionality. Especially for products that have existed for years. I have been a product for many years where we moved from minimal automated tests to completely relying on tests. I’ve also witnessed a journey that isn’t going very well and we need to “right the ship”.

I believe that test testing pyramid is correct as shown in The testing pyramid . We need more unit tests than integration tests than UI tests. As you more up the pyramid, tests become more brittle, take longer to run and delay feedback.

New Projects

New projects should be easier. Invest in automated tests and Clean Architecture from the start.

• You goal is to have as few as possible, but confidence.
  • always ask, can I cover this with a unit or integration test instead?
• Start with UI automation for happy paths only.
  • Build in good practices (automation ids, loading indicators, speedy UI).
    • I’ve written about this in the past
• Use the correct tool
  • use Cypress , Playwright or Puppeteer or another modern UI automation tool.

Existing Legacy Product

If you have an “old” product (or one without tests), then you are in for a challenge. I’m glad you’ve recognized the need for and value of automated tests. They should increase quality, increase velocity, increase confidence, and reduce the time spent for manual regression test (hopefully eliminate them).

• Consider bringing in outside help to get the base framework going.
  • make sure they don’t do all the work and have all the knowledge when they leave
  • leverage their experience on avoiding 🐲 and known problems, have them teach your employees and work together
• collaboration between developers and others writing the tests
  • your application wasn’t created with automation in mind. You’ll need to add identifiers and loading dialogs (shown or hidden) to let the automation key off of.
  • let developers help guide towards maintainability
  • have a goal where developers can create tests and/or work very closely with others before the work is considered done.
• Do not measure solely on tests created. Measure on reliable, fast tests that cover the manual regression tests that have to be ran before every release.
  • A few good tests are much better than 100s that are flaky and require investigation time
• Selenium and Selenide if you have CRM or other difficult UIs
  • have good DOM polling for elements (avoid Thread.Sleep). Selenide should help with this
• Page Objects
• Do Not use Thread.Sleep!!
  • use waitFor or add code the is continually checking the DOM for the loading indicator to be gone and the element is visible and ready to be clicked
• Look for ways to call an API to setup scenarios, don’t use the UI to setup scenarios
• Use the API and database calls to verify data
• Consider your data and plan for parallel tests
  • Use a new database with base data for each test run if possible
  • create the scenarios at the beginning of tests and cleanup afterwards (or use a snapshot and replace)

Watch out for dragons! 🐲

Taking on UI Automation is like getting a free puppy or a garden. It’s a lot of work and continual work. Some of this will include creating the tests, getting them running in builds, making them fast and reliable to make them valuable (you will want to run tests in parallel as soon as you have more than a few dozen), diagnosing failures (was it a timing issue or a functional failure that was caught), adding and updating tests as the application changes, watching out for duplication and what tests can be moved down the pyramid.

If you don’t slay these dragons, I’m afraid that you’ll end up with a lot of code and tests that no one trusts, is out of date.

"It's better to have a small set of highly valuable tests than a large number of mediocre ones." ~ Vladimir Khorikov from [his email](https://www.getdrip.com/deliveries/1dttpsi6hu3htm2e77d3?view_in_browser=true) and [EnterpriseCraftsmanship.com](https://enterprisecraftsmanship.com/)

Anti-Patterns

Here’s a summary of anti-patterns that I’ve recognize over the years. Most of them are already listed above.

• Measure success on the number of tests created
• using Thread.Sleep
• not creating maintainable and reusable code
• using long xpath and other complicated ways to find DOM elements
• avoiding getting tests running in a build until later
• sectioning off the team that is creating the tests from those creating code for the application
  • not involving the developers (create product teams that have all members working together, not siloed teams)
• Not planning for daily time needed to diagnose test failures
• Not continually improving the tests
• Not moving to “automating under the skin” and repeating the same UI setup steps over and over
• not identifying UI tests that can be replaced with unit and integration tests
  • You want a pyramid, not an ice-cream cone