IT Dashboard still inaccurate, says GAO
The IT Dashboard still does not live up to its billing as a real-time indicator of agency information technology programs' status, says the Government Accountability Office.
In a report dated March 15, the GAO finds data quality issues with the website, which the Office of Management and Budget says it uses to select IT projects for "TechStat" reviews, which are 60-minute meetings meant to analyze project shortcoming and develop remedies. Such reviews have resulted in a $3 billion cost avoidance, OMB says.
The report assigns blame for the lack of data quality to OMB and to the agencies reporting data into it. The formula OMB uses to characterize a project's cost and schedule status on the dashboard draws on ongoing as well as completed activities, meaning that recent developments can be masked by historical data. For example, in September 2010, the IT Dashboard reported that the Federal Aviation Administration's En Route Automation Modernization effort was 21 days over schedule--when in fact it was 256 days overdue, the GAO report says.
Surprisingly, however, the GAO says the OMB is more often harsher in its overall assessment of projects than the facts warrant. In a three-month comparison of the dashboard's traffic light ranking to agency earned value management system and other data, the GAO finds that dashboard is prone to assigning a red or yellow status when a yellow or green would be correct.
In oral comments to GAO auditors, OMB officials from the office of e-government and information technology--of which Vivek Kundra is the administrator, although he prefers the title "federal chief information officer"--said they disagreed with the report's methodology.
EVM data is significantly variable in quality across agencies, and in any case not a sound basis for measuring the accuracy of the dashboard traffic light score, OMB officials told GAO auditors--arguments that the report rejects as essentially moot. GAO auditors took steps to ensure they obtained reliable EVM data and in any case drew on other data sources as well, the report says.
Auditors also acknowledge that while the dashboard is meant to characterize the status of all projects, not just developmental ones for which there exist EVM data and therefore OMB uses its own cost and schedule formula to determine the traffic light score of a project, the report only examines development projects in the first place. "We did this because earned value management is a proven technique for providing objective quantitative data on program performance, and alternative approaches do not always provide a comparable substitute for such data," the report says. As such, EVM is a valid data source for use in making a comparison of the IT dashboard to reality.
The report also notes that agency data posted onto the dashboard often lags behind a project's true status or is inaccurate. For example, the Social Security Administration submitted start dates for activities for one project even though the actual start date was later. SSA officials said they did so due to internal processes that require start dates to correspond to the beginning of the fiscal year, which is Oct. 1.
This is the GAO's second report examining IT Dashboard accuracy; OMB also had many objections to the first one, released in July 2010.
- download the report, GAO-11-262 (.pdf)