Testing is performing all of the following:
• Providing software with inputs (a “workload”)
• Executing a piece of software
• Monitoring software state and/or outputs for expected properties, such as:
– Conformance to requirements
– Preservation of invariants (e.g., never applies brakes and throttle together)
– Match to expected output values
– Lack of “surprises” such as system crashes or unspecified behaviours
General idea is attempting to find “bugs” by executing a program.

Testing terminology
Workload:
• “Inputs applied to software under test”
• Each test is performed with a specific workload Behaviour:
• “Observed outputs of software under test”
• Sometimes special outputs are added to improve observability of software (e.g., “test points” added to see internal states) Oracle:
• “A model that perfectly predicts correct behaviour”
• Matching behaviour with oracle output tells if test passed or failed
• Oracles come in many forms:
– Human observer
– Different version of same program
– Hand-created script of predicted values based on workload
– List of invariants that should hold true over behaviour

What is a bug?
A “bug” can be one or more of the following:
• Failure to provide required behaviour
• Providing an incorrect behaviour
• Providing an undocumented behaviour or behaviour that is not required
• Failure to conform to a design constraint (e.g., timing, safety invariant)
• Omission or defect in requirements/specification
• Instance in which software performs as designed, but it’s the “wrong” outcome
• Any “reasonable” complaint from a customer
• … other variations on this theme …

The goal of most testing is to attempt to find bugs in this expanded
Sense.
Types of testing
Smoke testing
• Exploratory testing
• Black box testing
• White box testing

Testing situations:
• Unit test
• Subsystem test
• System