6.1 Monitoring Performance Measures

Guide Home / 6. Monitoring and Adjustment

Section 6.1

Monitoring Performance Measures

Performance measures are used by transportation agencies to align agency investment decisions with organizational objectives, such as asset condition or system reliability, and to monitor progress towards achieving agency goals. In TAM, asset performance is most commonly defined in terms of asset condition, but performance can also be represented by operational considerations, such as safety or traffic reliability.


Section 6.1

Monitoring Performance Measures

Performance measures are used by transportation agencies to align agency investment decisions with organizational objectives, such as asset condition or system reliability, and to monitor progress towards achieving agency goals. In TAM, asset performance is most commonly defined in terms of asset condition, but performance can also be represented by operational considerations, such as safety or traffic reliability.


6.1.1

Selecting and Using Performance Measures


This section discusses the importance of using performance data to make decisions. It highlights the role of performance measures and identifies how they are used to establish achievable performance targets. A more detailed discussion of Transportation Performance Management can be found in Chapter 2.


Performance Management Framework

As discussed in Chapter 2, transportation agencies have embraced the use of performance data to drive investment decisions. A performance-based management approach enables agencies to select and deliver the most effective set of projects for achieving strategic objectives, while also improving internal and external transparency and accountability.

A typical performance management framework includes:

  • A clear idea of the agency’s strategic objectives.
  • The use of performance measures to assess performance.
  • Methods to evaluate and monitor performance results.
  • The evaluation of factors with capacity to improve long-term performance.
  • The allocation of funding to achieve agency objectives.
  • Ongoing processes to monitor and report progress.

A fundamental component of the framework is the use of performance measures to evaluate system performance and the importance of establishing business processes to evaluate, monitor, and use the data to influence agency decisions. These are achieved by aligning decisions at all levels of the organization with the agency’s strategic objectives and ensuring that the right performance measures are being used to drive decisions. This alignment helps to ensure that resource allocation decisions and the day-to-day activities of agency personnel support the agency’s priorities and the interests of external stakeholders.

The existence of a regular, ongoing process to monitor and report results is critical to identifying and implementing improvements to system performance or to further the effectiveness of the performance management process. The continual monitoring and update of a performance management framework is reflected in Figure 6.1, which illustrates inputs to performance targets and how ongoing monitoring and adjustments are fed back into the framework to adjust future targets. The surveys conducted regularly to support a pavement, bridge or maintenance management system are examples of the types of performance monitoring activities fundamental to an effective performance management organization.

Agencies with a performance management framework in place have benefited from:

  • Maintaining a clear and unified focus for making agency decisions based on agency priorities, public input and available resources.
  • Using available funding more effectively to preserve or improve system performance while lowering life cycle costs.
  • Allocating available resources based on analysis of past performance and expected conditions to address areas most in need of attention.
  • Having the data to confidently defend funding requests or explain the impact of reduced budgets.
  • Building a transparent and accountable organization by communicating the basis for making resource decisions.
  • Meeting legislative requirements.

Figure 6.1 Monitoring and Adjustment in the TPM Process



TIP
It is important to select performance measures that are meaningful to the agency and that can directly inform decisions. This may vary depending on the agency context, culture, and TAM maturity.

Arizona DOT

In 2001, during the development of a long-range transportation plan (LRTP), the Arizona DOT took a strategic approach to how investments should be made. Under the new approach, Arizona DOT established the following three investment categories:

  • Preservation, including activities that preserve existing transportation infrastructure.
  • Modernization, including improvements that upgrade the efficiency, functionality, and safety without adding capacity.
  • Expansion, including improvements that add transportation capacity by adding new facilities or services.

To implement the new initiative, the Arizona DOT developed a report titled “Linking the Long-Range Transportation Plan and Construction Program” or” P2P Link” that applied financial constraints to the long-term vision. Through a collaborative process that involved a consultant, local and regional governments, and transit agencies, the Arizona DOT published an implementation plan for putting the P2P Link into practice. The resulting process includes scoring projects based on both a technical and policy score that are added together to determine a project’s ranking. The technical score is generated by the asset owner based on an analysis of the data while the policy score is determined based on each project’s contribution to LRTP goals and performance measures. The process helps to ensure that projects are ranked in accordance with the agency’s strategic objectives using only the most meaningful criteria in a transparent and defensible way.

Arizona DOT’s Link Between Strategic Objectives and Investment Decisions

Source: ADOT. 2014. Linking the Long-Range Plan and Construction Program P2P Link Methodologies & Implementation Plan.
https://azdot.gov/sites/default/files/2019/08/p2p-methodologies-implementation.pdf

Performance Measures

Performance measures are used within a performance management framework to allocate resources and provide feedback on the effectiveness of the activities in achieving overall objectives. Performance measures are indicators used for evaluating strategies and tracking progress. A performance measure can be an indication of asset condition, such as a pavement condition rating, or an indication of an operational characteristic, such as the annual number of fatalities on a facility.

The most effective performance measures drive decisions that are important to the success of the program. For example, maintenance departments may use performance measures that track actual expenditures to planned expenditures to ensure that available funding is directed towards the highest-priority items, as shown in the Colorado DOT practice example.

It is also important that the measures drive the desired performance within an organization. For instance, a performance requirement that measures whether pavement or bridge designs are submitted on time might cause incomplete or incorrect submittals to meet a deadline, leading to an increase in construction modifications. A more effective measure might focus on a minimal number of design modifications during the construction phase of a project.

Effective performance measures should also primarily be outcome-based rather than output-based, meaning that they focus on the result or impact of an activity rather than the inputs that went into the activity. Several examples of outcome- and output-based measures are shown in the sidebar on Page 6-8. Outcome-based measures are generally preferred because they indicate the effect on the traveling public resulting from the actions taken, so they usually relate to user priorities such as the length of time for a road to be cleared after a snow event or the absence of litter and graffiti. They are developed based on a description of what an agency wants to achieve as a result of the actions undertaken. Outcome-based measures are commonly used for managing ancillary assets such as drainage assets and signs. For instance, the performance of drainage assets might be reported in terms of the percent of pipes/culverts greater than 50 percent filled or otherwise deficient and the performance of signs might be reported in terms of the percent of signs viewable at night.

Output-based measures, on the other hand, track the resources used to achieve the outcome, such as the number of hours of labor used or the number of light-bulbs changed in a month. While the data is important information for managing resources, it does not necessarily drive outcomes that would matter to the public. For instance, travelers on a highway are much more interested in knowing when the road will be cleared of snow than how much overtime went into the operation.

When possible, agencies should use performance measures that are leading measures rather than lagging measures to influence future decisions. A leading measure uses changes in performance to provide insights into potential changes that might influence a future decision one way or another. For example, knowledge that a ramp meter has exceeded the manufacturer’s suggested service life might drive a decision to replace that meter. Similarly, increases in equipment downtime might indicate risks due to an aging fleet are growing or that planned operational activities will not be performed as planned. A lagging measure, on the other hand, looks back on the results of past investment strategies after the decisions have been made. Because a lagging measure is recorded after the fact, there is a delay (lag) in the agency’s ability to adjust its practices and improve performance. Bridge and pavement condition measures are examples of lagging measures because the reported conditions reflect the impact of decisions made several years in the past. Lagging measures are commonly used to evaluate a program’s effectiveness or to verify that actual investments achieved projected results.

In transportation, an agency might have a lagging measure for tracking complaints responded to within a 48-hour window. The measure provides an indication of the public’s satisfaction with the road network and is easy to monitor and report. However, if an agency really wants to effect change, it might develop leading measures to track the percent of complaints not worked on within a two-hour window or the percent of complaints that can’t be resolved by the initial point of contact and must be passed to someone else. Focusing on these types of measures could drive agency decisions to ensure complaints are being worked on quickly and are being assigned to the right people. General characteristics of effective performance measures are presented in Table 6.1.

North Carolina DOT

The North Carolina DOT authorizes its divisions to determine how funding will be used for maintenance activities and uses performance data to assist with this activity. Each year, Division Engineers submit annual plans detailing what work will be accomplished; these plans are reviewed quarterly with the Chief Engineer to discuss actual versus planned work. Their accomplishments are also displayed in a dashboard for internal use, as shown in the following image. Public-facing dashboards are also available showing overall conditions and performance trends. The Division Engineers are also held accountable for their performance, since their planned and actual performance data are incorporated into their annual evaluations.



Source: Leading Management Practices in Determining Funding Levels for Maintenance and Preservation. Scan Team Report, NCHRP Project 20-68A, Scan 14-01, National Cooperative Highway Research Program, May 2016.

Use of Performance Measures

Performance measures are used to:

  • Connect agency policies and objectives to investment decisions.
  • Establish desired and targeted levels of service that consider past performance, current and future demand, stakeholder priorities, and anticipated funding.
  • Align agency policies, investments, and day-to-day practices in a meaningful and easily understood manner.
  • Prioritize investment needs.
  • Monitor and report progress towards desired objectives to both internal and external stakeholders in a consistent, cost-effective, and transparent manner as illustrated in practice examples from the Washington State, North Carolina, and Virginia DOTs.

TIP
Outcome-based measures better relate to performance characteristics noticed by the public and other stakeholders than output-based measures.


Table 6.1 - Desired Performance Measure Characteristics

Desired CharacteristicsRationale/Purpose
Measurable with available tools/dataMay require no additional cost for data collection
ForecastableEnables data-driven target setting based on future conditions
Clear to the public and lawmakersAllows performance story-telling to customers and policymakers
Agency has influence over resultMeasures agency activities rather than impact of external factors
Source: Federal Highway Administration, 2017. TPM Guidebook.

Washington DOT

The Washington DOT uses its Maintenance Accountability Process (MAP) to comprehensively manage maintenance budgets and to communicate the impacts of policy and budget to both internal and external stakeholders. Field condition surveys are conducted annually to assess the condition of 14 assets on the highway system such as signs and signals, ITS assets, tunnels, and highway lighting. For each asset, a level of service target is established, based on expected funding levels and importance of the asset to the agency’s strategic objectives. The targeted and actual performance is summarized on a statewide basis and presented to the legislature, media, internal stakeholders, and other DOTs in a format similar to what is shown in the figure (https://www.wsdot.wa.gov/NR/rdonlyres/8EC689DF-9894-43A8-AA0F-92F49AC374F5/0/MAPservicelevelreport.pdf). In 2018, Washington State DOT achieved 77 percent of its highway maintenance targets. Targets that were not achieved are shown as red bullseyes and areas where the targets were exceeded include a checkmark with the bullseye. The results illustrate where additional investment is needed on a statewide basis and provides a basis for setting maintenance priorities during the year.

Targeted and Actual Performance Results Used to Set Maintenance Priorities

Source: WSDOT. 2017. Multimodal Asset Performance Report. Washington State DOT. https://wsdot.wa.gov/publications/fulltext/graynotebook/Multimodal/AssetPerformanceReport_2017.pdf

Washington DOT

To support accountability, credibility, and transparency, the Washington State DOT publishes its quarterly performance report, referred to as The Gray Notebook. Each edition of the Gray Notebook presents updates on multimodal systems' and programs' key functions and analysis of performance in strategic goal areas based on information reported to the Performance Management and Strategic Management offices of the Transportation Safety and Systems Analysis Division. Washington State DOT also publishes its Gray Notebook Lite, which highlights key metrics referenced in the Gray Notebook in a format for quick reading. Examples from each of these documents are presented in the figures.

The Gray Notebook and the Gray Notebook Lite

Source: WSDOT. 2019. https://www.wsdot.wa.gov/Accountability/GrayNotebook/

Virginia DOT

Performance dashboards are also a popular way to present progress, using color-coded indicators similar to those on the dash of an automobile. An example of the interactive dashboard available from the Virginia DOT is shown in the figure. The screen reports performance in seven areas (performance, safety, condition, finance, management, projects, and citizen survey results) and the needles indicate whether the performance is within targeted ranges. Hyperlinks are available in each area if a user wants to explore historical trends or explore performance objectives in more detail.

Virginia DOT's Performance Dashboard

Source: Virginia DOT. 2019. http://dashboard.virginiadot.org/

Future Directions in Performance Measures

As agencies advance the maturity of their practices and move towards investment decisions across assets and modes (as discussed in Chapter 5), there is increasing interest in the use of leading measures and asset performance measures other than asset condition.

Asset management plans document the processes and investment strategies developed by an agency to manage its infrastructure assets. These asset management plans support an agency’s performance-based planning and programming processes for making long-term investment decisions and feed shorter-term project and treatment selection activities. Together, these activities ensure the investment decisions of an agency are aligned with performance objectives and goals.

Examples of these types of measures include:

  • Financial Measures – Internationally, financial performance measures have been used successfully to express whether the level of investment has been adequate to offset the rate of asset deterioration or depreciation. For example, the Queensland Department of Infrastructure and Planning uses an Asset Sustainability Ratio defined as the capital expenditure being made on asset renewals (e.g., improvements) divided by the depreciation expense (discussed further in Chapter 4). If the ratio is less than 100 percent, the level of investment is not adequately replacing the depreciation occurring each year. Queensland also uses an Asset Consumption Ratio comparing the current value of the depreciable assets to their replacement value in order to show the aged condition of the assets.
  • Life Cycle Measures – A life cycle performance measure is a relatively new leading measure, promoting the selection of sound, long-term strategies best able to maximize performance at the lowest possible cost. There are several life cycle performance measures under consideration by the FHWA, including the Remaining Service Interval (RSI), which is being validated under a research project. The RSI is based on identifying a structured sequence of the type and timing of various repair and replacement actions needed to achieve a desired LOS over a long timeframe at the minimum practicable cost. The results of the RSI evaluation may be used to generate a Life Cycle Impact Factor, summarizing the difference in life cycle costs associated with the various strategies being considered.
  • Sustainability Measures – With an increased focus on identifying long-term sustainable solutions to transportation system needs, agencies may seek to develop new sustainability performance measures in order to properly indicate the impact a proposed solution may have on environmental conditions. The use of a recycling measure for gauging the amount of recycled material used in road construction is an example of this type of measure, as are measures for monitoring carbon dioxide emissions.

TIP
Making performance measures publicly available through reports, scorecards, or dashboards increases transparency into agency operations, which can serve as motivation to improve staff’s desire to meet the standards established, thereby increasing the chance of success.

North Carolina DOT

The North Carolina DOT has an interactive Organizational Performance Scorecard that provides an online indicator of the Department’s success at meeting targets in the following six core goal areas:

  • Make Transportation Safer.
  • Provide Great Customer Service.
  • Deliver and Maintain Infrastructure Effectively and Efficiently.
  • Improve Reliability and Connectivity of Transportation Systems.
  • Promote Economic Growth Through Better Use of Infrastructure.
  • Make NCDOT a Great Place to Work.

An example of how the information is shown; it presents the target for an overall infrastructure health index and the most recent results. As shown by the red “x” in the box on the far right, NCDOT is not currently meeting its target of a health index of 80 percent or more.

North Carolina DOT’s Organizational Performance Scorecard Website – Excerpt

Source: NCDOT. 2019.https://www.ncdot.gov/about-us/our-mission/Performance/Pages/default.aspx

6.1.2

Evaluating the Effectiveness of Performance Measures


Because of the important role performance measures have in supporting performance-based decisions, agencies should use care in selecting measures that drive the right types of results. This section introduces several approaches to evaluate the effectiveness of an agency’s performance measures.


Assessment

In its handbook for agency executives, AASHTO suggests an assessment of performance measures should consider the following:

  • Is the number of performance numbers reasonable? – An agency should retain performance measures addressing critical areas of importance that are maintainable with time. The Maryland and New Mexico DOTs have approximately 80 measures reviewed on a regular basis, but the Florida and Pennsylvania DOTs use approximately 15 to 20 measures to review strategic performance. Some agencies identify a small number (< 10) of KPIs selected from the pool of operational and tactical measures that best reflect an agency’s progress toward achieving its overall goals.
  • Are the measures meaningful? – Some agencies choose only to use easily measured performance activities because the information is easy to obtain. However, other measures may do a better job of driving good decision making.
  • Does the level of detail in data collection match the level of detail required to drive decisions? – Agencies should balance data availability with the analytic rigor used to make decisions. For instance, if pavement markings are replaced every year, it is not necessary to collect retro-reflectivity information annually. Similarly, collecting data on one lane of a two-lane highway may be enough for approximating the condition across the full width of the roadway.
  • Do they support the right decisions? – The performance measures should drive decisions in support of strategic objectives. For example, a performance measure based on the amount of overtime incurred after a snow event is less effective than one able to monitor the number of hours until the roads are cleared.
  • Are existing data sources reliable? – In most situations, existing data can provide the information needed for performance management, but it must be reliable and maintained regularly to be useful.

An assessment of performance measures can be important, since many organizations find that over time, the number of performance measures they are managing can become unwieldy.

Pennsylvania DOT

After using performance measures for years, the Pennsylvania DOT recognized that the number of measures being used had increased to a level that was difficult to manage. In 2011, the Pennsylvania DOT conducted an assessment of their performance measures using the following series of questions to guide their decisions as to which measures to keep, which to change, or which to delete:

  • Who is using the measure?
  • What exactly is being measured?
  • Why is this particular measure needed?
  • Whose performance is being measured?
  • Is the performance goal defined?
  • Does a similar measure already exist?
  • Is the existing measure meeting the needs and intent or should it be modified?

If a measure was needed where no measure exists, the following additional questions were used:

  • Does the measure affect continuous improvement?
  • Is the data for the measure updated as frequently as needed? Should it be updated monthly, quarterly, or yearly?
  • Is the measure easy to quantify?
  • Is the measure easy to understand?
  • Is it clear who owns the measure?
  • Does the measure provide a means of comparison?
  • Have unintended consequences been investigated?
  • Can the unintended consequences be successfully mitigated?

The process has helped to ensure that the agency is focused on the right measures to drive desired results and behaviors. The analysis found several issues that could be addressed, including eliminating duplicate or overly complicated measures, modifying measures that were driving unintended consequences, and resolving data quality issues.

SMART Evaluation

As discussed earlier, performance measures are used to set desired or targeted levels of service. Targets may be short-term, such as the 2- and 4-year targets state DOTs are required to submit to FHWA, or they may be long-term targets, such as the desired State of Good Repair (SOGR) serving as the basis for an agency’s TAMP.

Performance targets are evaluated using the “SMART” method, which evaluates whether targets are:

  • Specific. The performance is explicitly described.
  • Measurable. Progress towards the target can be monitored in a consistent manner.
  • Achievable. The target considers past performance, expected changes in demand, available resources and other considerations that make it realistic.
  • Relevant (also referenced as results-oriented). The target should be meaningful to the agency and drive the right outcomes.
  • Time-related (also referenced as timely or time-bound). There is a stated timeframe for achieving the target.

Nevada DOT

The Nevada DOT recognized that although performance measures were being reported regularly, they were not driving agency policies or decisions. The assessment evaluated the performance measures being used in each of the five key performance areas shown in the figure as well as the organizational culture to support performance management.

The study recommended improvements to emphasize the importance of messaging in order to advance the agency’s performance management culture, extend the performance culture beyond the headquarters office to field staff, and develop job performance plans emphasizing accountability at the division, office and unit levels. The study also recommended the periodic review of performance measures to ensure their continued relevance to agency business processes.

Nevada DOT’s five key performance areas and measures



Source: Nevada DOT. 2017. Adapting a Culture for Performance Management at the Nevada Department of Transportation.

Benchmarking

In simple terms, benchmarking is a process of comparing performance and practice among similar organizations as part of an agency’s continuous improvement activities. Benchmarking provides an opportunity to learn about approaches used by high-performing organizations to uncover noteworthy practices, inform target-setting activities, or to foster innovation and improvement within an agency. Benchmarking should focus on improvement and lessons learned rather than as a way to penalize underperformers.

As mentioned in Chapter 1, AASHTO has developed a comparative benchmarking tool for enabling state DOTs to compare performance outcomes and practices with peer agencies as part of their continuous improvement activities (http://benchmarking.tpm-portal.com/). This includes a peer selection tool, so agencies can compare practices to peers with similar characteristics. It also features a performance comparison tool with a number of chart options enabling agencies to compare results. For instance, an agency may elect to compare pavement smoothness characteristics with a neighboring state. There is also a portal to facilitate the exchange of practices among registered DOT users through a Notable Practice Narrative.

An example from the AASHTO TPM Portal showing a comparison of bridge deck percentage determined to be structurally deficient is shown in Figure 6.2. Similar comparisons are available for safety, environmental, and non-motorized (bicycle and pedestrian) performance measures. For transit agencies, Transit Cooperative Research Program (TCRP) Report 141, A Methodology for Performance Measurement and Peer Comparison in the Public Transportation Agency, provides specific guidance for comparing performance with other agencies.

Figure 6.2 Example Performance Comparison from the AASHTO TPM Portal



Source: TPM Portal. 2019. http://benchmarking.tpm-portal.com/compare/bridge-condition/deficient-bridges

Audits

Internationally, ISO standards include the conduct of periodic internal audits to help an agency evaluate whether its asset management program and components meet the agency’s needs, adhere to best practices and are being used to support decisions. In addition, agencies use auditing for service providers to confirm contract compliance in situations where road network maintenance and management activities have been outsourced.

TIP
The AASHTO TPM Benchmarking Tool (http://benchmarking.tpm-portal.com/) was designed to assist state DOTs with benchmarking TPM data, providing a data source and comparison tools.


Arizona DOT

In 2001, during the development of a long-range transportation plan (LRTP), the Arizona DOT took a strategic approach to how investments should be made. Under the new approach, Arizona DOT established the following three investment categories:

  • Preservation, including activities that preserve existing transportation infrastructure.
  • Modernization, including improvements that upgrade the efficiency, functionality, and safety without adding capacity.
  • Expansion, including improvements that add transportation capacity by adding new facilities or services.

To implement the new initiative, the Arizona DOT developed a report titled “Linking the Long-Range Transportation Plan and Construction Program” or” P2P Link” that applied financial constraints to the long-term vision. Through a collaborative process that involved a consultant, local and regional governments, and transit agencies, the Arizona DOT published an implementation plan for putting the P2P Link into practice. The resulting process includes scoring projects based on both a technical and policy score that are added together to determine a project’s ranking. The technical score is generated by the asset owner based on an analysis of the data while the policy score is determined based on each project’s contribution to LRTP goals and performance measures. The process helps to ensure that projects are ranked in accordance with the agency’s strategic objectives using only the most meaningful criteria in a transparent and defensible way.

Arizona DOT’s Link Between Strategic Objectives and Investment Decisions

Source: ADOT. 2014. Linking the Long-Range Plan and Construction Program P2P Link Methodologies & Implementation Plan.
https://azdot.gov/sites/default/files/2019/08/p2p-methodologies-implementation.pdf

North Carolina DOT

The North Carolina DOT authorizes its divisions to determine how funding will be used for maintenance activities and uses performance data to assist with this activity. Each year, Division Engineers submit annual plans detailing what work will be accomplished; these plans are reviewed quarterly with the Chief Engineer to discuss actual versus planned work. Their accomplishments are also displayed in a dashboard for internal use, as shown in the following image. Public-facing dashboards are also available showing overall conditions and performance trends. The Division Engineers are also held accountable for their performance, since their planned and actual performance data are incorporated into their annual evaluations.



Source: Leading Management Practices in Determining Funding Levels for Maintenance and Preservation. Scan Team Report, NCHRP Project 20-68A, Scan 14-01, National Cooperative Highway Research Program, May 2016.

Washington DOT

The Washington DOT uses its Maintenance Accountability Process (MAP) to comprehensively manage maintenance budgets and to communicate the impacts of policy and budget to both internal and external stakeholders. Field condition surveys are conducted annually to assess the condition of 14 assets on the highway system such as signs and signals, ITS assets, tunnels, and highway lighting. For each asset, a level of service target is established, based on expected funding levels and importance of the asset to the agency’s strategic objectives. The targeted and actual performance is summarized on a statewide basis and presented to the legislature, media, internal stakeholders, and other DOTs in a format similar to what is shown in the figure (https://www.wsdot.wa.gov/NR/rdonlyres/8EC689DF-9894-43A8-AA0F-92F49AC374F5/0/MAPservicelevelreport.pdf). In 2018, Washington State DOT achieved 77 percent of its highway maintenance targets. Targets that were not achieved are shown as red bullseyes and areas where the targets were exceeded include a checkmark with the bullseye. The results illustrate where additional investment is needed on a statewide basis and provides a basis for setting maintenance priorities during the year.

Targeted and Actual Performance Results Used to Set Maintenance Priorities

Source: WSDOT. 2017. Multimodal Asset Performance Report. Washington State DOT. https://wsdot.wa.gov/publications/fulltext/graynotebook/Multimodal/AssetPerformanceReport_2017.pdf

Washington DOT

To support accountability, credibility, and transparency, the Washington State DOT publishes its quarterly performance report, referred to as The Gray Notebook. Each edition of the Gray Notebook presents updates on multimodal systems' and programs' key functions and analysis of performance in strategic goal areas based on information reported to the Performance Management and Strategic Management offices of the Transportation Safety and Systems Analysis Division. Washington State DOT also publishes its Gray Notebook Lite, which highlights key metrics referenced in the Gray Notebook in a format for quick reading. Examples from each of these documents are presented in the figures.

The Gray Notebook and the Gray Notebook Lite

Source: WSDOT. 2019. https://www.wsdot.wa.gov/Accountability/GrayNotebook/

Virginia DOT

Performance dashboards are also a popular way to present progress, using color-coded indicators similar to those on the dash of an automobile. An example of the interactive dashboard available from the Virginia DOT is shown in the figure. The screen reports performance in seven areas (performance, safety, condition, finance, management, projects, and citizen survey results) and the needles indicate whether the performance is within targeted ranges. Hyperlinks are available in each area if a user wants to explore historical trends or explore performance objectives in more detail.

Virginia DOT's Performance Dashboard

Source: Virginia DOT. 2019. http://dashboard.virginiadot.org/

North Carolina DOT

The North Carolina DOT has an interactive Organizational Performance Scorecard that provides an online indicator of the Department’s success at meeting targets in the following six core goal areas:

  • Make Transportation Safer.
  • Provide Great Customer Service.
  • Deliver and Maintain Infrastructure Effectively and Efficiently.
  • Improve Reliability and Connectivity of Transportation Systems.
  • Promote Economic Growth Through Better Use of Infrastructure.
  • Make NCDOT a Great Place to Work.

An example of how the information is shown; it presents the target for an overall infrastructure health index and the most recent results. As shown by the red “x” in the box on the far right, NCDOT is not currently meeting its target of a health index of 80 percent or more.

North Carolina DOT’s Organizational Performance Scorecard Website – Excerpt

Source: NCDOT. 2019.https://www.ncdot.gov/about-us/our-mission/Performance/Pages/default.aspx

Pennsylvania DOT

After using performance measures for years, the Pennsylvania DOT recognized that the number of measures being used had increased to a level that was difficult to manage. In 2011, the Pennsylvania DOT conducted an assessment of their performance measures using the following series of questions to guide their decisions as to which measures to keep, which to change, or which to delete:

  • Who is using the measure?
  • What exactly is being measured?
  • Why is this particular measure needed?
  • Whose performance is being measured?
  • Is the performance goal defined?
  • Does a similar measure already exist?
  • Is the existing measure meeting the needs and intent or should it be modified?

If a measure was needed where no measure exists, the following additional questions were used:

  • Does the measure affect continuous improvement?
  • Is the data for the measure updated as frequently as needed? Should it be updated monthly, quarterly, or yearly?
  • Is the measure easy to quantify?
  • Is the measure easy to understand?
  • Is it clear who owns the measure?
  • Does the measure provide a means of comparison?
  • Have unintended consequences been investigated?
  • Can the unintended consequences be successfully mitigated?

The process has helped to ensure that the agency is focused on the right measures to drive desired results and behaviors. The analysis found several issues that could be addressed, including eliminating duplicate or overly complicated measures, modifying measures that were driving unintended consequences, and resolving data quality issues.

Nevada DOT

The Nevada DOT recognized that although performance measures were being reported regularly, they were not driving agency policies or decisions. The assessment evaluated the performance measures being used in each of the five key performance areas shown in the figure as well as the organizational culture to support performance management.

The study recommended improvements to emphasize the importance of messaging in order to advance the agency’s performance management culture, extend the performance culture beyond the headquarters office to field staff, and develop job performance plans emphasizing accountability at the division, office and unit levels. The study also recommended the periodic review of performance measures to ensure their continued relevance to agency business processes.

Nevada DOT’s five key performance areas and measures



Source: Nevada DOT. 2017. Adapting a Culture for Performance Management at the Nevada Department of Transportation.