Definition

An acceptance test is a formal description of the act of a software product, usually expressed as an example.

A number of different documentation and approaches have been proposed for such examples or scenarios. In many cases the goal is the possibility to automate the execution of such tests by a software tool, either ad-hoc to the development team or off the shelf.

Similar to a unit test, an acceptance test usually has a binary result, pass or fail. A failure suggests, (but it does not prove) the presence of a bug in the product.

Teams mature in their practice of agile use acceptance tests as the principal form of functional specification and the only formal expression of business requirements. Other teams use acceptance tests as a complement to specification documents containing uses cases or more narrative text.

Also Known As

The terms “functional test“, “acceptance test” and “customer test” are used more or less mutually. A more specific term “story test”, referring to user stories is also used (as in the phrase “story test driven development”.)

Expected Benefits

Acceptance testing has expected benefits, complementing those which can be obtained from unit tests:

  • encouraging closer collaboration between developers on the one hand and customers, users or domain experts on the other, as they demand that business requirements should be expressed
  • providing a clear and unambiguous “contract” between customers and developers; a product which passes acceptance tests will be considered adequate (though customers and developers might refine existing tests or suggest new ones as necessary)
  • decreasing the chance and severity both of new defects and regressions (defects impairing functionality previously reviewed and declared acceptable)

Common Pitfalls

Expressing acceptance tests in an overly technical manner

The main audience for acceptance tests are customers and domain experts, and they find tests that have details(of implementation) difficult to understand and review.

To avoid those tests from being overly concerned with technical implementation, involve customers and/or domain experts in the creation and discussion of acceptance tests.

Acceptance tests that are extremely focused on technical implementation also run the risk of failing due to minor changes which in reality do not have any impact on the product’s behavior.

For example, if an acceptance test references the label for a text field, and that label changes, the acceptance test fails even though the actual functioning of the product is not impacted.

 Potential Costs

Automated acceptance tests are not universally viewed as a net benefit.

unlike automated unit tests and  that generates some discussion after experts such as Jim Shore or Brian Marick questioned whether the following costs were outweighed by the benefits of the practice:

  • many teams report that the establishment of automated acceptance tests requires significant effort.
  • sometimes because of the “fragile” test issue, teams find the maintenance of automated acceptance tests are overwhelming.
  • the first generation of tools in the Fit/FitNesse tradition resulted in acceptance tests that customers or domain experts could not understand.