image
Arinze Obikili
Test Automation

6 Test Automation Strategies You Want To Define Before Writing A Script

In today’s software development world, test automation is not just a nice to have, it’s become a must have. Software architecture has increasingly become more complex, and so the need to not just test more but test quickly has become very essential to any software organization.

It is important to note that while test automation is key to any software development environment, it does not (and should not) entirely replace manual testing, and the “eye test” (risk based testing) is still as valuable as ever. Some might argue, it’s even more valuable now.

With that, I figured my first official blog post should touch on the best techniques and strategies an organization should utilize to get the best out of software test automation.

Dedicate A Team

While it’s true that a good automation engineer should also be a good manual tester, when it comes to defining the automation strategy for an organization, it’s key to have a core dedicated team of automation engineers. Test automation is a full-time task, and having engineers juggle between manual and automation testing is a risk. This might not be a possibility for all, but as much as possible it’s important to have at least a dedicated core of automation engineers focused on automation tasks.

Design A Framework That Is Scalable

How successful your automation effort at your organization is depends heavily on how you start. Designing a framework that is scalable and suitable for the application (or suite of applications) is absolutely paramount. Decisions about test tools, version control, reusability of artifacts should all factor into this design.

The last thing you’ll want to happen is going down the wrong path for months/years, and having to restart your effort due to an unsuitable decision about an automation tool, framework or development strategy.

Automate Below GUI Level As Much As You Can

GUI automation while very necessary can be both expensive to execute (time) and expensive to maintain when the UI elements change often. For web based applications, as much as possible develop a framework that utilizes API or webservice automation.

The key reason here is execution time, and speed to fail. The time it takes to execute an API and determine if the application has a server error is a fraction of the time it’ll take to launch a browser, navigate to the application URL, and then validate the same error.

Once again, this is an ‘as much as you can’ situation and does not completely eliminate the need for GUI automation, but a fair bet is having a large percentage of scenario based testing done below the GUI level. Be it API/Webservices, or command line executions for desktop applications.

Make Sure You Have A Business Level GUI Smoke Test

You can call it a smoke test, sanity test or build verification test (BVT). Whatever your organization calls it, be sure to have this test (or suite of tests) automated. This is a high level end to end test that can validate at any time that your latest build is a safe build and the integrated components are not broken. In a high delivery development environment, having this test (or suite of tests) is key to maintaining a solid environment with little to no manual effort.

Decide Your Execution Strategy

Congratulations! You’ve written all these automated tests, now you have to decide how, when and where they’ll be executed. Defining an execution strategy is highly significant to getting the benefits of your test automation effort.

  • How – Determine if your tests (and which tests) will be executed through a manual trigger, on schedule, or through a CI/CD pipeline (Bamboo, Jenkins come to mind)
  • When – How often are these tests executed? Which tests are executed when? Your BVT will want to be run daily or after every build, and others suites (including the full regression) can be run nightly or on demand. Depending on your application, these decisions will have to made earlier to ensure your development strategy is in sync.
  • Where – What suite of tests will be executed against which environments. Some tests can be run in Dev, others in QA and another group in a UAT environment.

Don’t Forget To Collect Metrics For Analytics

To be able to determine how effective your automated tests are, what the ROI on your automation investment is and track results trends/environment stability, you will have to determine a strategy to collect metrics and measure KPIs. Are you looking to show your pass/fail rate overtime? your median execution time? or defect leakage prevention? Your strategy to collecting metrics will enable your organization to measure KPIs and make decisions based on proper analytics, and not speculative thoughts.

Conclusion

These strategies of automation can save your organization plenty of time, money and headaches if done correctly early. Technology is evolving at a rapid pace, and keeping your strategies in focus will be key to ensuring that decisions are made with an end goal in mind. While the tools we use might (or will) change, the strategy should stay consistent.