CMU CS 15-675 Architectures for Software Systems Spring 1998
COTS Evaluation
with answers
Garlan & Kazman Questions on Readings for Lecture 23 Due: Wed April 8, 1998
The papers:
[Kon96]: A Case Study in Applying a Systematic Method for COTS Selection.
[MT97b]: IusWare
[CW98]: A Basis for Evaluation of Commercial Software
Hints:
The three papers assigned all approach the problem of how to evaluate products that you purchase, rather than build, so called COTS (commercial off the shelf). The issues that are identified are: these products are complex, they live within complex environments, the products change over time, the evaluation criteria may not be well understood or well specified, the interaction of the criteria may not be understood, and so forth. And yet choices must still be made. These papers describe ways of approaching the problem of COTS selection so that the choices can be informed (if not optimal) ones.
Questions:
1) Briefly explain why the weighted scoring method doesn’t work well for COTS evaluation?
The results are often (wrongly) interpreted as having meaningful scales
Assigning of weights for the criteria is difficult when the # of criteria is high
It is difficult to define a set of criteria and weights so that they are independent or adjusted for overlap.
2) What do Carney and Wallnau identify as the dimensions of variability in COTS software evaluation? Give a brief explanation of each.
Modified: 5/21/98