CMU CS 15-675 Architectures for Software Systems Spring 1998

COTS Evaluation

with answers

Garlan & Kazman Questions on Readings for Lecture 23 Due: Wed April 8, 1998

The papers:

[Kon96]: A Case Study in Applying a Systematic Method for COTS Selection.

[MT97b]: IusWare

[CW98]: A Basis for Evaluation of Commercial Software

Hints:

The three papers assigned all approach the problem of how to evaluate products that you purchase, rather than build, so called COTS (commercial off the shelf). The issues that are identified are: these products are complex, they live within complex environments, the products change over time, the evaluation criteria may not be well understood or well specified, the interaction of the criteria may not be understood, and so forth. And yet choices must still be made. These papers describe ways of approaching the problem of COTS selection so that the choices can be informed (if not optimal) ones.

Questions:

1) Briefly explain why the weighted scoring method doesn’t work well for COTS evaluation?

The results are often (wrongly) interpreted as having meaningful scales

Assigning of weights for the criteria is difficult when the # of criteria is high

It is difficult to define a set of criteria and weights so that they are independent or adjusted for overlap.

 

2) What do Carney and Wallnau identify as the dimensions of variability in COTS software evaluation? Give a brief explanation of each.

  1. Formal: this describes the characteristics of the product or products being evaluated.
  2. Contexture: this describes the characteristics of the environment in which the products will be situated.
  3. Evaluative: this describes the characteristics of the processes, techniques, and goals of the evaluation.

 


Back

Heather L. Marko

Modified: 5/21/98