Assigning Metrics for Optimization

Evaluation Measures Each evaluation measure (EM) is a category by which an option is ranked/graded –Example: A car can be ranked by mileage, top speed, number of gears, and seating capacity. A metric (or value function) must be established for each EM

Value Functions A single value metric must be specified for each EM: –Sometimes, lower ratings are preferred Example: lower cost –Sometimes, higher ratings are preferred Example: higher mileage –Choose the one that is applicable to the case. The scale of each metric must then be normalized to a range of 1.

Choosing a Value Function for an Evaluation Measure There are two types of value functions: –Piecewise linear Used for arbitrary scales of performance, attractiveness, etc. Few data points are available Good for expressing discontinuity –Exponential Used for stress, deflection, and other physical or concrete factors. Many data points can be obtained Suitable for incorporating a risk factor

Example: Piecewise Linear Value Function The x-axis shows the grading scale for productivity (chosen arbitrarily) The y-axis shows the relative value of each grade In this case, an improvement from a grade of -1 to 0 is as much as an improvement from 0 to 2.

Risk Tolerance How daring is the decision maker? –Risk tolerant: report a better score than is calculated by the metric (positive  –Risk averse: report a worse score than is calculated (negative  –Risk neutral: report the actual score (  The risk level  can be –Calculated using the method in the next slide –Estimated by looking at the different graphs shown in the following slides or created by a simulation.

Technical Method for Solving ρ ρ > 0.1 * Range of Measurement Find mid-value score (i.e. value = 0.5) Assign value of 0.5 to a specific score Solve numerically or use a table –Calculate normalized mid-value (range of scores  1) –Find normalized ρ –De-normalize ρ by multiplying by the range

Exponential Value Function Higher scores are better Risk tolerance  –  > 0: risk tolerant –  < 0: risk averse

Exponential Value Function (cont.) Lower scores are better

Final Evaluation Combine value function grades with their respective weights to calculate the final grade/value

Bicycle Example In our example, the EMs are: –Cost Measured in “dollars” Lower score is preferred –Weight Measured in “pounds” Lower score is preferred –Lifetime Measured in “months” Higher score is preferred

Bicycle Example (cont). We use the exponential value function. –Use a risk-averse outlook for personal safety, e.g. set ρ = -5. The exponential value function has been built into an MS Excel module and will be explained/utilized in later lectures.