|
|
In this laboratory assignment we will consider the following:
A. What is software testing?
B. Unit Testing: Structural Testing
C. Unit Testing: Black Box Testing
A previous lab dealt with debugging, the process of "localizing, analyzing, and removing" errors in a program. In this lab, we will again examine errors, but we will examine them from a different perspective. We will not be concerned with removing errors but with attempting to show that errors exist. Software Testing is the process of showing that there are errors in a software system. But, as Professor Edsger Dijkstra says, "Testing can never show the absence of errors, only that there are errors. Software testing should span the entire life-cycle of a software system from analysis through maintenance." The focus in this lab will be on the formal testing methods that are used after implementation to show that the system has errors (bugs).
Testing is a critical part of quality software development. Inadequate testing embarrassed NASA when the Hubble Space Telescope and Mars Observer failed to operate properly. Unfortunately, examples of poor testing are abundant.
There are several reasons for inadequate testing but NO EXCUSES. Often by the time the software is running, the project is so far behind schedule that there is little time for testing. Additionally, it is often the developer (programmer) who does the testing. No one wants to find errors in his or her own work. Besides, the programmer knows it works since he or she wrote it. Companies like Microsoft and NASA's Jet Propulsion Laboratory have separate independent test groups. The job of a group of testers is to show that software has errors. If they don't find errors, they fail. This attitude characterizes a good tester but this attitude is rarely found in a programmer testing his or her own code.
In the testing phase, there are several levels of testing: unit
testing, subsystem testing, system testing, regression testing. We will
emphasize unit testing in this lab.
Unit testing (testing one function or procedure) is at the lowest level of the testing phase. Unit testing is usually done by the programmer who wrote the unit or module and makes use of the structure of the code. For example, consider the following example:
if (grossPay < 30000.00)
action 1;
else
action 2;
To test this statement, the tester would input test data for grossPay less than 30000 and other test data for grossPay greater than 30000 and the boundary case grossPay = 30000. Because the tester can see the internal structure, such testing is called structural testing. It is also referred to as white-box (or glass-box) testing since the tester can see inside the unit (i.e., see the code). There are three types of structural or glass box testing: statement, branch, and path coverage.
In statement coverage, test cases are input that cause the execution of each statement in the unit. If some statement is not executed during testing, its reliability must be suspect. Branch testing is implemented by running test cases that exercise each branch of an "if" statement. Branch coverage is a stronger testing technique than statement coverage. In a procedure with one entry and one exit, branch coverage guarantees statement coverage.
Path testing insures that all paths through the code are executed. A path is a list of instructions necessary to go from the entry to the exit statement in the unit. The main disadvantage of path testing is that a unit that contains loops may have an "infinite" number of paths. For example, once through the loop and out or twice through the loop and out, etc.
lines 1-2-8
List the other two paths through the function on your answer sheet.
//Calculate and return the appropriate csci course to recommend
//based on a student's ACT and math background. An ACT of less
//than 18 or lack of Algebra II keeps a student from taking a csci
//course and 0 is returned.
int csci(int act, // Student ACT score
int math) // HS Math: 0 = noAlg, 1 = algl, 2 = alg2
// Return csci course to take
{
int take;
if ((act < 18) && (0 < act)) //line 1
take = 9; //line 2
else //line 3
if (math < alg2) //line 4
take = 0; //line 5
else //line 6
take = 117; //line 7
return take; //line 8
}
Look at the previous csci( ) function and note that the input data ACT = 12 and math = 2 causes the execution of one PATH in the function (lines 1,2, and 8). The number of paths through a unit is determined by the control statements such as "if', "while", "do", "switch", and so on. To test a unit usually requires a test driver program. For example, to input the test data to the csci() function above would require another procedure or function (probably a small main program) to call the function and pass it the different values of the arguments. The output should be printed for the tester to inspect. The following is a simple test driver for the csci( ) function:
// testCsci.cc : Test driver for csci function.
#include <iostream>
using namespace std;
const int noalg=0;
const int alg1=1;
const int alg2=2;
int csci(...) // code for csci function goes here
{
...
}
int main(void)
{
int answer;
// First test case
cout << "Test 1: act= " << 18 << " math = " << 2;
answer = csci(18,2);
cout << " csci = " << answer;
// Second test case
}
As mentioned in the lab on debugging, many programming errors occur in conditional expressions. Values at the boundaries of conditionals should be tested. The boundary value of the conditional ACT < 18 is 18. The boundary values for the loop "for( i=0; i < NumberStudents; i++)" are i = 0 and i = NumberStudents.
Boundary values may also include very small or very large values for the
inputs. For example, in testing a program that reads a file, you should
try a file with no data in it, a file with one item, and a very large file.
Another type of testing is called black box testing. The tester cannot examine the structure of the code. The requirements of the program, input, and expected output are used to make the test cases. Black box testing focuses on functionality, or what the system should do. One type of black box testing is input partitioning or equivalence partitioning. Information about the input is used to divide the input domain into valid and invalid groups of input values. In the csci( ) function, we could divide the ACT input values into: less than 0, the value 0, between 0 and 18, the value 18, and greater than 18. We could divide the math input values into: less than 0, 0, 1, 2, and greater than 2.
InlabD is a run-unit which is designed to calculate and print the two roots of a quadratic equation of the form
a x2 + b x + c = 0
where a, b, and c are integers. The program will prompt for the
values for a, b, and c. As you remember from mathematics, a formula called
the quadratic equation calculates the two roots, if a is not 0:
Let's test this run-unit using black-box techniques, but first think about values of a, b, and c that should be valid and invalid and values that yield different types of roots: repeated real, distinct real, and non-real(complex).
When testing uncovers an error, someone, usually the programmer, must debug the software. Then, the system must be tested again. Research has shown that correcting an error introduces additional errors into the program 20-40% of the time. That is, while fixing one error we create another error (or perhaps we don't fix the one already identified). Testing is never finished. Test drivers and test data should be kept so they can be rerun after debugging. This type of testing is called regression testing.
Imagine you wrote the software for an X-Ray machine that would administer radiation treatment to you. When would you stop testing? If a company tests too long, its competitors could announce a similar product and capture the market (you could lose your job). Determining how much testing is enough is a difficult task. Currently, much research is being conducted in this area of computing which is called software reliability.