Other Useful Books

Books on Testing, general

1. Boris Beizer, "Software Testing Techniques, 2nd edition", International Thomson Computer Press, 1990.
   - A classic on software testing techniques, suitable for in-depth discussions of specific testing models and related techniques. Most relevant to various coverage-based testing topics.

2. Paul Ammann and Jeff Offutt, "Introduction to Software Testing", Cambridge University Press, 2008.
   - A good alternative to Beizer book above, and actually more suitable as a textbook instead of a reference book due to its genesis. Coverage criteria play a central role in this textbook, and therefore it is more relevant to various coverage-based testing topics we "cover" in our class.

3. Paul C. Jorgensen, "Software Testing: A Craftsman's Approach", 3rd edition, Auerbach Publications, 2008.
   - Another comprehensive alternative to the books above. A feature I particularly liked is the parts on pre-req (Part I: A Mathematical Context) and some recent development (Part VI: Millennium Testing, which includes exploratory testing, use-case-based testing, and test-driven development).

4. Rex Black, "Pragmatic Software Testing", Wiley, 2007.
   - A good companion to the other Rex Black book below, and a more "pragmatic" alternative to the two books above, still with good coverage of "technical" aspects often missing in books claim to be "practical" or "pragmatic".

5. James A. Whittaker, "How to Break Software", Addison-Wesley, 2002.
   - According to the message on the back cover, this is "a very applied and non-rigid approach", test "on the fly", and a "nose for where bugs are hiding". Should be enjoyable reading for many, especially as an alternative/companion to more traditional testing books (either the type focuses on "systematic" testing techniques or on formalized processes).

6. William E. Perry, "Effective Methods for Software Testing, 3rd Edition", Wiley, 2006.
   - "Includes Complete Guidelines and Checklists" was printed on the title page, and it is truly "complete" in just under 1000 pages. Besides the core "7-step testing process", there is also good coverage about testing competencies, environments, specializations, and agility.

Books on Testing, more specialized

1. Robert Binder, "Testing Object-Oriented Systems: Models, Patterns, and Tools", Addison-Wesley, 1999.
   - Very comprehensive, with 1100+ pages (longest book on our list). Good reference, not just limited to OO testing.

2. John D. McGregor and David A. Sykes, "A Practical Guide to Testing Object-Oriented Systems", Addison-Wesley, 2001.
   - Another book on OO testing, with an emphasis on being "practical", and organized around OO systems (classes, interactions, hierarchies, distributed objects, and then systems).

3. Ken Beck, "Test-Driven Development", Addison-Wesley, 2003.
   - If you are into agile development or extreme programming (XP), this should be part of the basic resources for you.

4. Rex Black, "Critical Testing Processes" Addison-Wesley, 2004.
   - More of an operational/managerial perspective on testing, breaking down testing into four major steps (4Ps: plan/prepare/perform/perfect). Works well in combination with our main textbook.

5. Gerald D. Everett and Raymond McLeod Jr., "Software Testing: Testing Across the Entire Software Development Life Cycle", Wiley, 2007.
   - Life cycle perspective: Fitting testing into the overall software development life cycle. -- Just as the subtitle accurately indicated. More suitable for managers and some practitioners.

6. Bart Broekman and Edwin Notenboom, "Testing Embedded Software", Addison-Wesley, 2003.
   - Well, it might be suitable students in industries dealing with embedded systems, such as aerospace, automotive, etc. Very much domain-specific, complement the technical focus in our main textbook.

7. Lydia Ash, "Web Testing Companion: The Insider's Guide to Efficient and Effective Tests", John Wiley and Sons, 2003.
   - As the title implies, it's about Web testing. Much more domain-specific than technically focused. A nice reference for Web testers and researchers (well, our team did benefit from this book in our research on Web testing and quality/reliability assurance).

8. Jeffrey Rubin and Dana Chisnell, "Handbook of Usability Testing: How to Plan, Design and Conduct Effective Tests, 2nd Edition", Wiley, 2008.
   - Actually, it is also closely related to another SMU course, CSE 8317 User Interface Design. Usability testing is an integral part of design for usability. This book covers the whole spectrum of activities related to this.

9. Elfriede Dustin, Jeff Rashka, and John Paul, "Automated Software Testing: Introduction, Management, and Performance", Addison-Wesley, 1999.
   - "Automated" must involve tools, and the chapter on tool evaluation/selection, and the survey of tools in the appendix are particularly useful.

Books on Testing and Beyond

1. Boris Beizer, "Software System Testing and Quality Assurance", Van Nostrand Reinhold, 1984.
   - Older book than the testing techniques book by the same author, but more comprehensive, with the final two chapters on QA, not just testing.

2. Mauro Pezze and Michal Young, "Software Testing and Analysis: Process, Principles, and Techniques", Wiley, 2009.
   - Comprehensive coverage of the three things they mentioned about testing, but presented in the altered order of principles, techniques (and models), and process. Nice reference, but related topics might thread through several different parts.

3. Robert Culbertson, Chris Brown, and Gary Cobb, "Rapid Testing", Prentice-Hall, 2002.
   - More comprehensive than the title implies, particularly in the "static testing" part, which includes metrics, formal reviews, checklists, audits, inspections, formal verification, theorem proofs, requirements traceability, symbolic execution, etc. covered in "other QA" and elsewhere of our main textbook. Emphasizes streamlining of test planning, execution, and reporting.

4. William E. Lewis, Software Testing and Continuous Quality Improvement", Auerbach, 2000.
   - If you are interested in testing in software maintenance environment, this is a unique book for you, although it also covers testing in the "development" lifecycle and other topics.

Books on QA and other Quality Related Topics, not Limited to Testing

1. Shari Lawrence Pfleeger, Les Hatton and Charles C. Howell, "Solid Software", Prentice-Hall, 2004
   - The topics covered resemble the QA part (Ch. 1, 2, 6, 7, 13-17) in this class, but from the perspective of safety-critical systems point of view, i.e., quality assurance for safety-critical systems.

2. John Musa, "Software Reliability Engineering", McGraw-Hill, 1998. or
   John Musa, "Software Reliability Engineering: More Reliable Software Faster and Cheaper, 2nd Edition", (order info at http://members.aol.com/JohnDMusa), AuthorHouse.
   - The original title "Software Reliability Engineered Testing" relates more to this class: A book on statistical testing with Musa's operational profiles as the usage models to achieve the goals of "More Reliable Software Faster and Cheaper. Most relevant to our usage-based statistical testing topics.

3. Michael R. Lyu, editor, "Handbook of Software Reliability Engineering", McGraw-Hill, 1995.
   The entire book is now online
   - Besides general coverage on software reliability engineering, this books also include issues such as defect analysis (ODC chapter by Chillarege), metrics for reliability, operational profile (Chapter by Musa) and usage-based statistical testing, etc.

4. Stephen H. Kan, "Metrics and Models in Software Quality Engineering, 2/e", Addison-Wesley, 2002. ISBN 0-201-72915-6.
   - A lot of models and practical information from a former IBMer. Most comprehensive book of its kind.

Other References (Papers)

• Panagiotis Louridas, "JUnit: Unit Testing and Coding in Tandem", IEEE Software, Vol.22, No.4, pp.12-15, July/Aug., 2005.
  - A nice short survey about JUnit.

• Other information about JUnit: search the web for many useful sites. (e.g., www.junit.org, junit.sourceforge.net, junit.netbeans.org, etc.)

• Panos Louridas, "Test Management", IEEE Software, Vol.28, No.5, pp.86-91, Sept./Oct. 2011.
  - Including interesting figures about testing effort % (larger system, higher %), developer-to-tester ratios (high-assurance, more testing), test measurement, and test case management tools.

• There are various research topics and related publications from our research group at SMU, including testing, quality (with a primary focus on reliability, safety, and more recently on usability), and applications in various types of systems (commercial, Web-based, Cloud, telecommunication, Net-centric, etc.). For more information, please visit Jeff Tian's home page. A couple of papers most closely related to test model construction and maintenance are listed below:
  1. J. Tian, A. Bokhary and S. Alanazy, "A Comprehensive Framework for Measuring and Improving API Usability", 8th Annual Conf. on Computational Science & Computational Intelligence (CSCI'21), Las Vegas, NV, Dec., 2021,
  2. Y. Tian, J. Tian and N. Li, "Cloud Reliability and Efficiency Improvement via Failure Risk Based Proactive Actions", Journal of Systems and Software, Vol.163, May, 2020.
  3. E. Abuta and J. Tian, "Defect Classication and Analysis in a Small Company", Int. Conf. on Computer Applications in Industry and Engineering (CAINE'2018), New Orleans, LA, Oct., 2018,
  4. G. Karami and J. Tian, "Applying Task Models from Human Computer Interaction to Support and Improve Usage Based Statistical Testing for Web Applications", Int. J. of Computers and Their Applications, Vol.25, No.2, pp.64-75, June, 2018.
  5. G. Karami and J. Tian, "Maintaining accurate web usage models using updates from activity diagrams", Information and Software Technology, Vol.96, pp.68-77, April, 2018.
     - Closely related to our Markov-OP/UBST topic. Available on Canvas.
  6. G. Karami and J. Tian, "Improving Web Application Reliability and Testing Using Accurate Usage Models", In R. Lee, editor, Software Engineering Research, Management and Applications. SERA 2017. Studies in Computational Intelligence, Vol. 722, pp.75-92, Springer-Verlag, 2018.
  7. J. Tian and L. Ma, "Web Testing for Reliability Improvement", in Advances in Computers, Vol.67, pp.177-224, Academic Press, 2006.
     - Summary of Web testing work at our research group. Closely related to our UBST and specialized testing topics. Available on Canvas.
  8. Z. Li and J. Tian, "Testing the Suitability of Markov Chains as Web Usage Models", 27th Annual International Software and Applications Conference, pp.356-361, Dallas, Texas, Nov., 2003.
  9. C. Kallepalli and J. Tian, "Measuring and Modeling Usage and Reliability for Statistical Web Testing", IEEE Trans. on Software Engineering, Vol.27, No.11, pp.1023-1036, Nov., 2001.
  10. J. Tian and E. Lin, "Unified Markov Models for Software Testing, Performance Evaluation, and Reliability Analysis", 4th ISSAT International Conference on Reliability and Quality in Design, Seattle, Washington, August 12-14, 1998.
  11. K. Chruscielski and J. Tian, "An Operational Profile for the Cartridge Support Software", 8th Int. Symp. on Software Reliability Engineering, pp.203-212, Albuquerque, New Mexico, Nov., 1997.
      (Received ISSRE'97 "best paper" award)