Exercise for Checking Effectiveness of Tests

Most teams often write Test to complete the coverage and no one is really bothered whats the effectiveness of a particular test case.  If you are facing such scenarios then i suggest an exercise Test the Tests.

Objective: To find out the effectiveness of Test Net created by whole team ( Dev and Testers)

What do we do:

  1. Bring Entire Team in one room after the demo
  2. Re-occurring in every Sprint preferably after every Sprint Demo
  3. In Sprint planning, team should own tasks to create Test Pyramid for each Stories or at-least together for all Stories of Sprint
  4. Introduce bugs in the system intentionally ( like introduce null pointers, third party system down, css style path incorrect, validation errors etc)
  5. Team should be given time so that they can tell which test case will fail, The recommendation will be in two levels:
    1. Which type(unit/Integration/Acceptance) of test case will fail for one particular bug introduced in Step 2
    2. Which particular test case should fail
  6. Now System should be build which includes running Automated unit and Acceptance Test cases
  7. Test Case reports should be analyzed to check if it matches with the suggestions given by the team or not
  8. Intention is not to malign anyone but to sensitise everyone of the effectiveness of work that we are doing

Take-Aways:

  1. Test Effectiveness:
    1. Test Leaks: No bug should go undetected in all 3 layers of Tests
    2. Duplication: Test case failing at two levels for same bug ( Acceptance test case also covering a condition already covered by Unit Test)
    3. Wrong Level of Test Case fail: instead of unit test case, Acceptance test case is failing means we need to write more unit tests
  2. Test Awareness:
    1. If the suggestions of the Team matches with the report, then teams awareness is good about test strategy
    2. If the suggestions of the team does not match the report, then team should be sensitised in that meeting about requirement traceability matrix
  3. Test Pyramid:
    1. Test Pyramid of that Sprint should be checked for if its really a pyramid or inverted cone or hour Glass

Benefits:

  • This one hour or two-hour meeting every Sprint can really help sanitise every team member about the benefits of following Test Pyramid.
  • Value Add vs Cost of each test case will go up drastically if this practice is implemented rigorously
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s