Jeff Tian's Research in Statistical Testing

The Cases for Statistical Testing

There are several important reasons for usage-based statistical testing:

User-focus: With the focus on target users and customers in today's competitive market, testing based on their actual usage scenarios and frequencies helps us ensure satisfactory performance and reliability to them.
Large size and high complexity of modern software systems favor selective instead of exhaustive testing. Usage-based statistical testing provides a good way to prioritize and perform such selective testing.
Varied application domains and diverse usage scenarios/sequences/patterns imply that specific characteristics of these domains and specific usages need to be captured in various operational profiles (part of the usage-based statistical testing strategy, see below) to ensure satisfactory performance and reliability of software under all these situations.

For user-oriented (or user-centered), large systems that satisfies varied needs of a large populations, usage-based statistical testing is typically preferred to traditional product-centered testing techniques. One excellent example where such usage-based statistical testing is appropriate is the Internet as a whole as well as various web applications.

UMM for Usage-based Statistical Testing and Applications to Web Testing

We have developed an approach using UMMs, or unified Markov models, to support usage-based statistical testing and reliability analysis. In UMMs, usage scenarios are represented by state transition diagrams, with associated likelihood for individual transitions represented by the transition probabilities, to form a set of hierarchical Markov chains. Much of our work in this areas is conducted and validated using web applications.

This work is described in the following papers (you can also see the abstracts for some of these and other recent papers):

C. Kallepalli and J. Tian, "Measuring and Modeling Usage and Reliability for Statistical Web Testing", IEEE Trans. on Software Engineering, Vol.27, No.11, pp.1023-1036, Nov., 2001.
J. Tian and A. Nguyen, "Statistical Web Testing and Quality Assurance", 9th International Conference on Software Quality, pp. 263-274, Cambridge, MA, Oct., 1999.
J. Tian and E. Lin, "Unified Markov Models for Software Testing, Performance Evaluation, and Reliability Analysis", 4th ISSAT International Conference on Reliability and Quality in Design, Seattle, Washington, August 12-14, 1998.

Test Activity Characterization and Operational Profile Development

My research in this area includes: methods to capture and model customer operational profiles, examining model assumptions in practical software testing environments, and case studies on various systems, particularly to web sites.

This research is summarized in the following papers (see also abstracts for these and other recent papers):

C. Kallepalli and J. Tian, "Measuring and Modeling Usage and Reliability for Statistical Web Testing", IEEE Trans. on Software Engineering, Vol.27, No.11, pp.1023-1036, Nov., 2001.
K. Chruscielski and J. Tian. "An Operational Profile for the Cartridge Support Software", 8th Int. Symp. on Software Reliability Engineering, pp.203-212, Albuquerque, New Mexico, Nov., 1997.
(Received ISSRE'97 "best paper" award.)

Prepared by Jeff Tian (tian@engr.smu.edu).
Last update: March 1, 2003.