CAST 2012 Enforced Randomization – Increasing the value of test automation

Posted on Posted in Software Testing, Test Automation, Training / Conferences, Web Testing

Last week I was in San Jose for CAST 2012 to give talks on two interesting topics. My first talk was around the limitations of acceptance criteria and the second talk was focused on how we can increase the value of automation by introducing randomization where it makes sense.

During my conversation with participants around randomization, many people suggested that I should write a blog about this – and so here it goes.

Pretty much all the test automation projects have four main elements.

  • Navigation – How do you reach to the screen/page which to test
  • Data – How many variations of data are possible
  • Sequence – In which sequence data can be used on the page
  • Operation – What are the different ways in which operations can be performed.

For example, if we take the example of registration page for any popular site such as Twitter, this is how these four elements can be used

  • People can reach on registration page in many different ways – from the header, footer, promotional pages, etc.
  • On the registration page, the user can use the valid first name, password and email address.
  • The registration form can be filled in any sequence – top to bottom, bottom to top, random order.
  • And form data can be saved in many different ways – with the mouse, keyboard or direct post.

With test automation, it is possible to build randomization in all of these activities and ensure that test runs are identical to each other. Randomization has potential to increase the probability of uncovering interesting information. Testing, in my opinion, is a sampling exercise and with randomization, we can enhance our sample size substantially.

Since this talk was recorded, pretty much everything I would like to tell/explain is already explained in the following video – so I suggest you watch this video.

By the way, if you are interested in model-based testing – you might find these posts fromHarry Robinson interesting.

The key point I wanted to make in the talk is – Manual testing does not guarantee variability and automated testing does not have to be repetitive. I finished my talk with a bunch of questions which I will repeat here as well –

Why is randomization not the mainstream in test automation projects? Is it our bias that test should always fail in a predictable way? Is it a barrier in our mind that randomization in test automation is complex?

I will let you decide. Please leave your comments if you have used randomization or have any questions on my talk or this topic.

Please follow and like us: