[Colloquium] today, March 9, 2005 David Coppit, William & Mary College

[Nita Yack]

David Coppit, Assistant Professor

William & Mary College

     Exhaustive Testing Revisited: A New Look at an Old Idea
     March 9, 2005, 2:30 pm - 4:00 pm
     Ryerson 251

Speaker's Homepage:

     http://www.cs.wm.edu/~coppit/

Abstract:

     Exhaustive testing of all possible inputs is an effective software 
verification method. However, it was long ago dismissed as infeasible 
in practice due to the large, sometimes infinite, input spaces of most 
programs. Instead, researchers have focused on the development of test 
selection criteria that seek to identify a small set of representative 
test cases, based on certain assumptions about the nature of the 
domain, input, or implementation. As effective as such criteria are, 
they fail for an important class of faults: those that violate the 
assumptions of the test selection method. As a result, faults involving 
rare and unexpected combinations of events elude pre-deployment 
testing, and become failures in the field.

     In this talk, we will present recent research in bounded exhaustive 
testing (BET), where the software is automatically tested for all 
inputs within some bound. Research to date indicates that the approach 
is feasible for small systems having simple inputs. However, the 
feasibility and utility of applying BET to systems of significant size, 
even with state of the art technology, remains unknown. We will present 
results from an experiment evaluating the feasibility of employing BET 
on an industrial system. We discuss the key scalability impediments, 
and techniques we developed to address them. Using our techniques, we 
were able to test over 250,000 structurally complex inputs, revealing 
several previously unknown specification and implementation faults. We 
will also discuss more recent work that seeks to provide testers with 
more efficient, easy-to-use tools for employing BET in practice.
Nita Yack
Departmental Administrator
Computer Science Department
1100 E. 58th Street - Room 151
Chicago, IL 60637
(773) 702-6019