Questions To Ask (And Answer) Before Setting Up A Test Automation Framework

So you’ve been tasked with designing and setting up a test automation framework for your domain or development group. At this point your biggest job is to begin to consider what framework works best for the software application that’s being designed & built. As well as what’s sustainable over the long term.

Picking a framework on the surface seems pretty simple. “Let’s just go with Selenium Webdriver in Java. That seems to be the most popular” or maybe it’s “Let’s just get this automated testing tool. It’s only $XXX per license.

Here are a few questions and things to consider before selecting a framework with our without a particular language:

  • What is the tech stack that the application (or suite of applications) is being built on?
  • Is this a web app, a desktop app, mobile app or an entire backend system with little to no user interface?
  • If there’s a UI, what’s front end built on? Angular? React? And what’s the backend? Rest APIs? Web services? Web sockets?
  • What’s the overall technical strength and expertise of your team? Is your team a big python team or a node.js team? This will be key in being able to decide on a framework that multiple team members can use with little to no upskilling.
  • Where will these scripts be executed? Locally? On a VM via Jenkins? or on one of the cloud based automated testing platforms (i.e. SauceLabs or BrowserStack)?
  • What framework is ubiquitous in the market? Will you get community support when you encounter issues? Stack overflow a great resource, but if your framework is very nice and tiny piece of the market, then it’ll be pretty difficult to find solutions to issues that you’ll surely encounter.
  • What’s the overall strategy for test automation? What model will your team be writing their test automation code? Depending of if you’re in agile shop or waterfall, the framework could change. Is test automation part of the sprint or outside of it?
  • Would you rather go for a testing like UFT or SoapUI? If so, are you ready to embrace the risk of being wedded to the tool for a while as well as the costs?

I said a few right? But there’s obviously quite a bit to consider before embarking on that journey. You don’t want to get halfway through only to discover that maybe the framework you selected doesn’t work for your team. Then we get into the 2–3 framework territory. Yikes… Not good.

Are you building a framework in your domain or group? What are some of the questions you’ve had to consider.