CS 5314/7314 (Fall 2023) Contents Description
Software Testing and Quality Assurance
Prof. Jeff Tian, CS Dept., SMU, Dallas, TX 75275
Phone: (214)768-2861
E-mail: tian@smu.edu
Webpage: s2.smu.edu/~tian/class/7314.23f
Contents Description
Software quality assurance (SQA or simply QA)
includes testing and various other activities aimed at
ensuring that appropriate functionalities have been
implemented correctly and efficiently in the software systems
or software-related products or services
to satisfy the requirements, needs and wants, and expectations
of their target customers and users.
One important feature of this course is the user-centered
perspective of software quality, testing, and QA.
User requirements for new systems and usage models of existing
systems or alternative systems will be the starting point
and focus of our course.
Quality metrics meaningful to the users will be used throughout
the QA and SQE (Software Quality Engineering) process.
Another important feature of this course is the focus on the
"prioritized" (or "risk-based") approach to testing and quality assurance.
For any realistic system today,
exhaustive testing and similar quality assurance approaches
would be infeasible because of the increasing size and complexity.
We will focus on the "kay" areas/goals/components/etc.,
related to likely operations, defect, usage frequencies, etc., for
our testing and QA effort.
We will devote slightly more than half of the class
time to topics related to software testing,
or executing the software in order to observe its behavior to ensure
that it conforms to our expectations or
to identify behavior deviations and possible
underlying problems for correction.
We will emphasis formal/systematic testing techniques
and their applications, including:
- black-box or functional testing,
with a focus on covering (or thoroughly testing) external functions
or services required or expected by target customers and users;
- clear/transparent/white-box or structural testing;
with a focus on covering (or verifying/testing) implementation details,
including internal components and structures;
- usage-based statistical testing (UBST),
with a focus on adequately testing operations or services
important to and commonly used by target customers and users
to ensure overall product or system reliability
(a quality view most meaningful to customers and users)
after product delivery or system deployment.
These testing techniques are organized by their underlying models:
-
Testing techniques based on
"flat" structures, such as lists,
(top-down) partitions and (bottom-up) equivalent classes,
include
t1) basic partition testing (PT),
t2) input domain boundary testing (BT),
and
t3) Musa's operational profiles (Musa OP) or usage model for UBST.
-
Testing techniques based on
finite-state machines (FSM) and related variations
include
t4) basic FSM testing to,
t5) control flow testing (CFT),
t6) data dependency analysis and data flow testing (DFT),
and
t7) Markov operational profiles (Markov OP) for UBST.
These models will help us prepare, perform,
and perfect (manage/improve) testing to achieve functional
or structural coverage or to mimic realistic usage scenarios
in a systematic way to ensure overall product/system
quality and reliability.
Specific testing techniques to be covered:
Besides the
formal/systematic ones mentioned above
(t1-t7: PT, BT, Musa-OP/UBST, FSM, CFT, DFT, Markov-OP/UBST),
we will also briefly cover
various forms of informal or ad-hoc testing,
checklists, fault injection and mutation testing.
Specialized testing techniques for specific purposes
and/or applicable to different application domains,
such as usability testing, Web testing,
Cloud testing, embedded system testing,
performance and stress testing, etc.
will also be briefly covered.
Test activities, management, automation, and related issues,
such as team organization,
testing process,
people's roles and responsibilities,
user involvement (user-centered testing/QA),
test automation tools,
test integration, etc.,
will be briefly discussed.
Applicability and effectiveness of specific testing techniques
in different sub-phases and for different purposes will
also be briefly discussed.
The rest of the class time,
beyond various testing-related topics mentioned above,
will be devoted to the following topics:
-
General concepts about quality,
quality assurance (QA),
and software quality engineering (SQE).
Concepts related to quality will be covered,
including reliability, safety, security,
usability, dependability, maintainability, availability, etc.,
primarily from a user's perspective.
QA activities and SQE process will also be discussed,
in the context of overall software development and maintenance processes,
with customer involvement.
-
Other QA alternatives beyond testing,
including, activities and related techniques
for defect prevention and process improvement,
inspection/review/walkthrough,
formal verification (and its prerequisite, formal specification, briefly),
fault tolerance and safety assurance.
A good understanding of each alternative's relative strengths
and weaknesses will help us construct and carry out
a comprehensive QA strategy that more often than not will
include and integrate several such alternatives.
-
Quality assessment and improvement, including,
overall strategy for quantifiable quality improvement,
measurement and feedback mechanisms,
quality models and measurements,
defect classification and analysis,
risk identification and analysis,
and software reliability engineering.
These activities will help us monitor, assess, improve
the overall SQE process to achieve our quality and reliability goals
effectively and efficiently.
Prepared by Jeff Tian
(tian@smu.edu).
Posted: Aug. 22, 2023.
Last update: Aug. 22, 2023.