CSE 7314/5314 (Fall 2015): Course Project
Assignment
Your project is a major part of your CSE 7314/5314. It will
consist of the following parts or stages:
-
A project proposal:
due on 9/30/15 for on-campus students and 10/2/15 for distance students.
Due on 9/1/15 for executive class SEC'11,
with informal proposal presentation
(no slides required, verbal description would be enough) during
class on 8/29/15.
-
A project summary
due on 11/11/15 for on-campus students and 11/13/15 for distance students;
or an optional (formal) project presentation
to be scheduled for the last 3-4 classes.
Notice:
This part is required for graduate students (those enrolled in CSE 7314) only,
and optional for undergraduate students (those enrolled in CSE 5314).
All executive students in SEC'11 will be required to give
a project presentation (but no summary will be required),
to be given in class on 10/10/15.
-
A final project report:
due on 11/18/15 for on-campus students and 11/20/15 for distance students.
Due on 10/16/15 for executive class at AATC.
Notice:
There is also slightly different requirements for graduate
and undergraduate students.
It can be either
an individual project or a small group project.
The details are given below.
Acceptable project types
There are two types of basic choices for your course project,
one focusing on testing, and anther focusing on other QA techniques:
-
An application of some specific testing techniques/models
to some programs/products/applications/services
you are developing/testing/maintaining/supporting/using/considering.
For example, you may choose to construct control flow and data flow models
to test a module you are developing at your work
from a designer/developer's perspective,
or to construct a finite state machine to capture the expected behavior
of a software or application and testing it accordingly from a user's
perspective,
or augment your FSM with quantified usage information into a Markov
chain operational profile (OP) to perform usage-based statistical
testing from a collective users' perspective.
Another example is the development, validation, and usage
of an operational profile for a large software systems you are
working on and collect testing data to quantify its reliability.
It's generally a good idea to consider multiple
(I'd say two as a minimum for an individual project)
testing techniques and actually
use a couple of them in your project to get a hands-on feeling
of how different techniques and models work in practical applications.
Pay special attention to the evaluation of your testing/analysis results.
Be prepared to answer this type of questions:
- How do you know if the testing technique works?
Consider the applicability, effectiveness, and cost.
- What's the basis for comparison (baseline or status quo)?
- What about some other testing techniques that might be appropriate?
Your answer can be based on either the practical evidence
(executing several types of testing cases and observing the results)
or based on logical arguments
(suitability of certain testing technique on certain types of products),
or both.
- Graduate students are expected to study relevant material from Part IV of
our textbook to prepare them for the analyses above.
- Undergraduate students can choose to perform only a high-level analysis
to answer the above questions without going into details.
-
You may choose to focus on performing various other quality assurance
activities and document the results,
while performing limited/small-scale testing.
Possibilities include comprehensive inspection,
defect prevention and process improvement,
formal verification, and fault tolerance.
You may also collect inspection/testing/QA records for a product at your work,
and construct quality models to analyze the results,
to assess the effectiveness of your inspection/testing/QA techniques,
or to identify high-defect modules for focused
quality improvement actions.
If you choose this second type of project,
I still want you to perform at least some small scale/amount of testing
because it is such an important part of our course.
For the QA part,
make sure you are focusing on quality that can be quantified and analyzed.
For example, some defect analysis, both the root cause analysis type
and statistical analysis, or even some classified defect data analysis
using schemes such as ODC/af,
and some reliability analysis would be appropriate.
However,
a process definition/improvement initiative with only a
logical argument for its superiority is not suitable for this class.
Again, the primary difference in the project requirements for undergraduate
and graduate students is in the result analysis part, similar to that
for testing-focused project discussed above.
Where to Find Something to Test or to Perform QA on?
-
Well, I mentioned about your work,
a product or a system, or a sub-part of it,
that you are involved as a architect/designer/developer/tester/user/manager/etc.
at your work.
However, if you don't feel comfortable taking this as your class
project or talking about it (without mentioning product name and
raw data) in class, you need to find something else to work on.
Notice:
Your project report must contain testing related technical
information, such as test models, test cases, and test results,
although you don't have to give detailed product information.
As a rule of thumb, if you feel uncomfortable with the material
you have to put into this report, find something else to work on.
Check with your manager about it if you are not sure.
-
Many students used their past projects from work,
from their former college/graduate school classes,
from their hobby (e.g., game programming),
or from their role as users of some specific
packages/software/web-site/services/etc.
Apply the same sensitivity rule as for your current project above.
-
You can get it from someone/somewhere else.
Open source software and programs are a wonderful source for many
student projects in the past as well.
In addition,
many web-based services/systems/applications can be a good source
for your project.
Project proposals
Your project proposal should be around 3-4 double spaced pages
in length, and should include the following information:
- a project title (be as specific/informative as possible),
- a one-paragraph abstract or an executive summary,
at the beginning of the proposal,
that summarize the whole project,
- introduction: clearly identify the problem that you are going to address,
- brief background information (about 1/2 or 1 page),
- the solution strategy you intend to use
(which testing technique? what kind of analyses? etc.),
- expected results,
- analysis of result to be performed,
- followup actions,
- a rough schedule,
- indicate whether you'll be making a project presentation,
and, if possible, your preferred presentation date.
In case of a group project, please also pay attention to the following:
-
Please provide information regarding each team member's
roles and responsibilities.
-
The amount of work proposed for a group project should be appropriate
for the group size. As a general rule of thumb, if something can be
comfortably done by a single student, it is not suitable as a group
project.
-
Similarly, more diversity is also expected for a group project.
-
Unless there is a compelling reason,
group size of 3 or more is discouraged due to the expectation that
everyone in the team will be performing hands-on testing
activities using multiple testing techniques.
-
You only need to submit one proposal and one report for the project,
which must follow the same instruction as the individual projects.
-
The grade will be assigned to the project,
not to the individual members of the team.
So, if there are some issues, work them out within your team.
Please keep in mind that by the time you submit your project proposal,
we have only covered less than half of the class material,
although an overview of the whole course was given at the beginning of
the semester.
Therefore,
you may make certain modifications to the things you propose,
but the basic framework should be there in your proposal.
Once I have reviewed your proposal and provided my feedback,
you need to address the issues I raised in your final project report.
However, in most of the cases,
you do NOT need to submit a revised proposal.
In the rare case that your proposal is "unacceptable",
I'll explicitly ask you to re-do/re-submit your proposal.
Project summary vs presentation for on-campus and distance students
(
Notice that all students in SEC'11 will be required to give a
project presentation.
)
All the graduate students are required to submit a project summary,
in presentation slide format (2-5 slides),
suitable for in-class presentation by the professor
(or by the students),
if you are not doing a full presentation (see below) in class.
You summary should focus on the main results from your project
for us to get the basic picture.
Here is a template for your project summary.
You may choose to do a (full) project presentation.
In that case, you don't need to submit the project summary.
Each presentation should last about 10-15 minutes,
with appropriate numbers of slides.
You need to highlight the problem/solution-strategy/results/analysis
for us to get the basic picture,
but not necessarily all the details,
which would require much more than 15 minutes.
One common mistake in the past is too much background information
but not enough testing/QA-specific technical information.
Also, avoid detailed description of the test cases and activities performed.
As I stated earlier, this part is optional for undergraduate students.
Project report
The project report should be treated as a term paper,
around 15 double-spaced pages in length,
but no longer than 20 pages, for graduate students.
The report should clearly and comprehensively states the background,
problem, strategy, activities, results, result analysis,
lessons learned, followup actions, and a high level summary
(and an abstract at the beginning).
Additional material, such as graphs, models, test cases, etc. produced,
information sources and raw data,
customer surveys, etc., can be included in the appendix and clearly
marked as such (so it will not be counted towards your 20 page quota).
For undergraduate students, you report should be similar, except the
analysis part (see information in the "Proposal" section above).
As a result, your report is expected to be around 10-12 pages.
But, if you choose to, you can follow the instruction for graduate students
as well.
The format/sequence should be similar to what I listed for
the project proposal,
but, of course, with more results
(what you did and discovered, not just what you proposed to do).
If you are not familiar with technical reports or papers,
please consult technical/professional journals and conferences
proceedings published by ACM, IEEE (particularly its Computer Society),
and/or other reputable organizations.
Several common mistakes to avoid:
-
It is supposed to be a "report",
not a set of "presentation slides".
So, limit your use of lists/bullets, and put most of the material/discussions
in paragraphs.
Similarly,
only figures and tables without corresponding discussions
do not make a good report.
-
Your project report must contain testing related technical
information, such as important test models, test cases,
test results, and analysis of the results.
In addition,
you need to describe/discuss this information unless it is
clearly self-explanatory.
-
On the other hand,
you shouldn't include all the graphs, models, test cases, etc. produced
for the project in the report text itself.
As I mentioned above, they can be included in the appendix, if you desire,
together with other material, such as raw data,
customer surveys, etc.
Prepared by Jeff Tian
(tian@lyle.smu.edu).
Initially posted: Aug. 14, 2015.
Last update: Sept. 12, 2015.