Black box testing techniques are used fortesting the software application. Boundary Value Analysis (BVA) Equivalence Class Partitioning. Decision Table based testing.
Black Box Testing
Black Box Testing is testing without knowledge of the internal workings of the item being tested. For example, when black box testing is applied to software engineering, the tester would only know the “legal” inputs and what the expected outputs should be, but not how the program actually arrives at those outputs. It is because of this that black box testing can be considered testing with respect to the specifications, no other knowledge of the program is necessary. For this reason, the tester and the programmer can be independent of one another, avoiding programmer bias toward his own work. For this testing, test groups are often used,
Though centered around the knowledge of user requirements, black box tests do not necessarily involve the participation of users. Among the most important black box tests that do not involve users are functionality testing, volume tests, stress tests, recovery testing, and benchmarks . Additionally, there are two types of black box test that involve users, i.e. field and laboratory tests. In the following the most important aspects of these black box tests will be described briefly
Black box testing - without user involvement
The so-called “functionality testing” is central to most testing exercises. Its primary objective is to assess whether the program does what it is supposed to do, i.e. what is specified in the requirements. There are different approaches to functionality testing. One is the testing of each program feature or function in sequence. The other is to test module by module, i.e. each function where it is called first.
The objective of volume tests is to find the limitations of the software by processing a huge amount of data. A volume test can uncover problems that are related to the efficiency of a system, e.g. incorrect buffer sizes, a consumption of too much memory space, or only show that an error message would be needed telling the user that the system cannot process the given amount of data.
During a stress test, the system has to process a huge amount of data or perform many function calls within a short period of time. A typical example could be to perform the same function from all workstations connected in a LAN within a short period of time (e.g. sending e-mails, or, in the NLP area, to modify a term bank via different terminals simultaneously).
The aim of recovery testing is to make sure to which extent data can be recovered after a system breakdown. Does the system provide possibilities to recover all of the data or part of it? How much can be recovered and how? Is the recovered data still correct and consistent? Particularly for software that needs high reliability standards, recovery testing is very important.
The notion of benchmark tests involves the testing of program efficiency. The efficiency of a piece of software strongly depends on the hardware environment and therefore benchmark tests always consider the soft/hardware combination. Whereas for most software engineers benchmark tests are concerned with the quantitative measurement of specific operations, some also consider user tests that compare the efficiency of different software systems as benchmark tests. In the context of this document, however, benchmark tests only denote operations that are independent of personal variables.
Black box testing - with user involvement
For tests involving users, methodological considerations are rare in SE literature. Rather, one may find practical test reports that distinguish roughly between field and laboratory tests. In the following only a rough description of field and laboratory tests will be given. E.g. Scenario Tests. The term “scenario” has entered software evaluation in the early 1990s . A scenario test is a test case which aims at a realistic user background for the evaluation of software as it was defined and performed It is an instance of black box testing where the major objective is to assess the suitability of a software product for every-day routines. In short it involves putting the system into its intended use by its envisaged type of user, performing a standardised task.
In field tests users are observed while using the software system at their normal working place. Apart from general usability-related aspects, field tests are particularly useful for assessing the interoperability of the software system, i.e. how the technical integration of the system works. Moreover, field tests are the only real means to elucidate problems of the organisational integration of the software system into existing procedures. Particularly in the NLP environment this problem has frequently been underestimated. A typical
example of the organisational problem of implementing a translation memory is the language service of a big automobile manufacturer, where the major implementation problem is not the technical environment, but the fact that many clients still submit their orders as print-out, that neither source texts nor target texts are properly organised and stored and, last but not least, individual translators are not too motivated to change their working habits.
Laboratory tests are mostly performed to assess the general usability of the system. Due to the high laboratory equipment costs laboratory tests are mostly only performed at big software houses such as IBM or Microsoft. Since laboratory tests provide testers with many technical possibilities, data collection and analysis are easier than for field tests
Black box testing should make use of randomly generated inputs (only a test range should be specified by the tester), to eliminate any guess work by the tester as to the methods of the function
Data outside of the specified input range should be tested to check the robustness of the program
Boundary cases should be tested (top and bottom of specified range) to make sure the highest and lowest allowable inputs produce proper output
The number zero should be tested when numerical data is to be input
Stress testing should be performed (try to overload the program with inputs to see where it reaches its maximum capacity), especially with real time systems
Crash testing should be performed to see what it takes to bring the system down
Test monitoring tools should be used whenever possible to track which tests have already been performed and the outputs of these tests to avoid repetition and to aid in the software maintenance
Other functional testing techniques include: transaction testing, syntax testing, domain testing, logic testing, and state testing.
Finite state machine models can be used as a guide to design functional tests
According to Beizer the following is a general order by which tests should be designed:
1. Clean tests against requirements.
2. Additional structural tests for branch coverage, as needed.
3. Additional tests for data-flow coverage as needed.
4. Domain tests not covered by the above.
5. Special techniques as appropriate–syntax, loop, state, etc.
6. Any dirty tests not covered by the above.
Black box testing Methods
1.Graph-based Testing Methods
Black-box methods based on the nature of the relationships (links) among the program objects (nodes), test cases are designed to traverse the entire graph
Transaction flow testing (nodes represent steps in some transaction and links represent logical connections between steps that need to be validated)
Finite state modeling (nodes represent user observable states of the software and links represent transitions between states)
Data flow modeling (nodes are data objects and links are transformations from one data object to another)
Timing modeling (nodes are program objects and links are sequential connections between these objects, link weights are required execution times)
Equivalence PartitioningBlack-box technique that divides the input domain into classes of data from which test cases can be derived
An ideal test case uncovers a class of errors that might require many arbitrary test cases to be executed before a general error is observed
Equivalence class guidelines:
1. If input condition specifies a range, one valid and two invalid equivalence classes are defined
2. If an input condition requires a specific value, one valid and two invalid equivalence classes are defined
3. If an input condition specifies a member of a set, one valid and one invalid equivalence class is defined
4. If an input condition is Boolean, one valid and one invalid equivalence class is defined
Boundary Value AnalysisBlack-box technique that focuses on the boundaries of the input domain rather than its center
1. If input condition specifies a range bounded by values a and b, test cases should include a and b, values just above and just below a and b
2. If an input condition specifies and number of values, test cases should be exercise the minimum and maximum numbers, as well as values just above and just below the minimum and maximum values
3. Apply guidelines 1 and 2 to output conditions, test cases should be designed to produce the minimum and maxim output reports
4. If internal program data structures have boundaries (e.g. size limitations), be certain to test the boundaries
Black-box testing for safety critical systems in which independently developed implementations of redundant systems are tested for conformance to specifications
Often equivalence class partitioning is used to develop a common set of test cases for each implementation
Orthogonal Array Testing
Black-box technique that enables the design of a reasonably small set of test cases that provide maximum test coverage
Focus is on categories of faulty logic likely to be present in the software component (without examining the code)
Priorities for assessing tests using an orthogonal array
1. Detect and isolate all single mode faults
2. Detect all double mode faults
3. Multimode faults
Graphical user interfaces
Documentation and help facilities
1. Task testing (test each time dependent task independently)
2. Behavioral testing (simulate system response to external events)
3. Intertask testing (check communications errors among tasks)
4. System testing (check interaction of integrated system software and hardware)
Advantages of Black Box Testing
More effective on larger units of code than glass box testing
Tester needs no knowledge of implementation, including specific programming languages
Tester and programmer are independent of each other
Tests are done from a user’s point of view
Will help to expose any ambiguities or inconsistencies in the specifications
Test cases can be designed as soon as the specifications are complete
Disadvantages of Black Box Testing
Only a small number of possible inputs can actually be tested, to test every possible input stream would take nearly forever
Without clear and concise specifications, test cases are hard to design
There may be unnecessary repetition of test inputs if the tester is not informed of test cases the programmer has already tried
May leave many program paths untested
Cannot be directed toward specific segments of code which may be very complex (and therefore more error prone)
Most testing related research has been directed toward glass box testing