How to choose between different test types with SpecFlow, Cucumber or other BDD acceptance test framework?

I am looking at SpecFlow examples, and it's MVC sample contains several alternatives for testing:

  • Acceptance tests based on validating results generated by controllers;
  • Integration tests using MvcIntegrationTestFramework;
  • Automated acceptance tests using Selenium;
  • Manual acceptance tests when tester is prompted to manually validate results.

I must say I am quite impressed with how well SpecFlow examples are written (and I managed to run them within minutes after download, just had to configure a database and install Selenium Remote Control server). Looking at the test alternatives I can see that most of them complement each other rather than being an alternative. I can think of the following combinations of these tests:

  • Controllers are tested in TDD style rather than using SpecFlow (I believe Given/When/Then type of tests should be applied on higher, end-to-end level; they should provide good code coverage for respective components;
  • MvcIntegrationTestFramework is useful when running integration tests during development sessions, these tests are also part of daily builds;
  • Although Selenium-based tests are automated, they are slow and are mainly to be started during QA sessions, to quickly validate that there are no broken logic in pages and site workflow;
  • Manual acceptance tests when tester is prompted to confirm result validity are mainly to verify page look and feel.

If you use SpecFlow, Cucumber or other BDD acceptance test framework in you Web development, can you please share your practices regarding choosing between different test types.

Thanks in advance.

7
задан John Farrell 23 October 2010 в 15:45
поделиться