How-To
Implement a Multi-Objective Decision Analysis (MODA) Approach
This How-To Guide provides a list of the steps involved in implementing a MODA approach. Agencies can use this How-To Guide to determine if they have considered all the necessary steps in setting up their approach for prioritizing projects or investments. Note that this guide is a summary of the materials presented in the final report of NCHRP Project 08-103. This report has additional details on each of the elements described here.
Establish the Scope
- Determine which assets to include. Specify the asset classes to consider as part of the analysis. Often a cross-asset resource allocation approach will focus initially on pavements and bridges, but may extend to other asset classes as well, such as drainage assets, traffic and safety assets, and facilities.
- Determine which investments to include. A cross-resource allocation decision-making process should include investments in existing assets, such as preservation, rehabilitation and replacement or reconstruction actions. The process may include other types of investments, such as improvements in safety or mobility, as well.
- Determine the investment period. It is also important to determine the time frame for investments being considered. Often the process is defined to prioritize investments over a single one or two-year decision period, but it may be defined to include investments over multiple periods.
- Decide how the approach relates to the existing business process. Every organization has some sort of process for making decisions about it investments in its assets. In this step one must consider the existing process and how an improved cross-asset resource allocation process will be integrated into it. For instance, the process might entail replacing one or more steps in the existing process with a more formal approach to identifying investment needs and prioritizing potential investments.
- Decide how the results will be used. One must decide how the results of the process are intended to be used. Will they help establish the level of investments in different assets or types of investments? Or provide an initial set of priorities for decision-makers to review? Or help document the final selection of specific candidate investments through a formalized process?
Define Goals and Objectives
- Review existing agency documents describing agency mission. Review the organization’s strategic plan, long-range transportation plan, and other planning document. Typically investment goals are defined in areas such as: mobility; preservation; safety; security; resilience; environment; community; economic development; accessibility; and environmental justice.
- Define goals. Determine the goals that will be addressed through the cross-asset investments. A cross-asset resource allocation process is often focused on a subset of the organization’s goals, such as improving safety. However, investments in existing assets may yield progress towards achieving other goals as well.
- Define objectives supporting each goal. For each goal under consideration, one must determine the objectives supporting it. For instance, if the goal is to preserve existing assets, the specific objectives might be to improve pavement and bridge conditions. Or if the goal is to improve safety, the objective may be to reduce the number of fatal crashes.
Select Performance Measures and Evaluation Criteria
- Select measures supporting each objective. One must identify specific performance measures that relate to each objective. The measures, once quantified, should demonstrate whether the organization is making progress towards meeting its goals and objectives. The final report for NCHRP Project 08-103 provides examples of measures used for cross-asset resource allocation.
- Determine how to quantify each measure. Once measures have been selected one must determine how best to quantify them. Ideally, a measure should be based on quantitative data, such as a measure of asset condition. However, where it is not feasible to obtain quantitative data, it may be necessary to use a qualitative evaluation of the improvement resulting from an investment, such as a five-point scale based on expert judgment.
- Consider how measures scale based on project size. It is important to consider how the measures selected will vary with the size of a project. Often, quantitative measures–such as deck area of bridges in good condition, reduced number of crashes, or hours of delay–scale naturally with the size of a project. Where qualitative measures are used, the issue of scaling is particularly important, and it may be necessary to adjust how a measure is defined to account for the scale of a project. If two projects yield the same result in terms of some measure, then most structured approaches will higher prioritize the project that provides the same result at a lower cost.
- Normalize the measure for comparison between projects. Once performance measures are defined, they must be normalized to a defined scale. Most structured processes requires normalizing measures on a scale from 0 to 100 percent, where 100 percent is the greatest achievement possible towards a given objective.
Assess Data and Analytical Capabilities
- Determine how to measure and predict each measure. Review available data to determine how to quantify each measure using a combination of existing data and predictive models.
- Revisit the analysis scope and measures. Revise the scope of analysis and measures as needed to reflect any issues revealed in existing data. For instance, if needed data are not available and cannot be easily predicted, it may be necessary to revise the process to handle selected types of investments separately.
- Collect additional data. Collect any additional data required to support the process. This might include collecting more data on assets that might be improved through a proposed investment, or collecting additional data on past projects to better predict the impact of future investments.
Prototype the Approach
- Collect data for sample projects. Test the process on a set of sample projects. The test project should be representative of the full set of assets, investment types, and objectives included as part of the process.
- Calculate project performance. Walk through the process of predicting performance for test projects using the previously-defined measures.
- Review and revise the approach. Review and revise the approach as needed. This may include revisions to the scope of the process, reconsidering goals and objectives, making adjustments to performance measures, and/or changing how measures are scaled and normalized.
- Document the approach and assumptions. Carefully document the approach and supporting assumptions to aid in the remaining steps of the process.
Set Weights on Goals and Objectives
- Determine weighting approach. Decide how the goals and objectives will be weighted. The final report for NCHRP 08-103 describes different options for weighting measures, including using a panel of experts to perform a pairwise comparison of different goals, voting on weights using the Delphi Method, or using Data Envelopment Analysis to establish weights programmatically using a data-driven approach.
- Use the approach to set weights. Set weights on each goal and objective. This may require conducting a workshop with a set of experts to perform pairwise comparisons or vote on weights.
Apply the Model
- Identify candidate investments. Use the cross-asset allocation approach to prioritize investments. Potential or candidate investments are identified that will be prioritized as part of the process.
- Calculate measures for each candidate. For each candidate investment, calculate the measures that will result from the investment using the established approach.
- Prioritize candidates. Follow the previously-established approach to prioritize the candidate investments.
- Use priorities to support resource allocation. Use information on priorities to aid the resource allocation process. The process may result in a score for each of a set of candidate projects that is then used by decision makers when deciding which investments to pursue. Or the process may include performing an optimization to determine which investments would maximize performance, subject to budget and other constraints.
- Update key assumptions and parameters as needed. Carefully document the approach and supporting assumptions to aid in performing the remaining steps of the process.
Communicate the Results
- Document approach. Document the results of the cross-asset resource allocation approach, including the priorities on different investments generated using the approach.
- Document key assumptions and parameters. Record information on the parameters used in the analysis. This should include documentation on how different objectives were weighed, and the weights that were established.
- Make the results available to stakeholders. Share results of the prioritization process with key stakeholders.