OMB IT Dashboard contains inaccurate information, says GAO

The IT Dashboard, a much-vaunted Office of Management and Budget website displaying evaluations of federal information technology projects, contains inaccurate cost and schedule data, according to the Government Accountability Office.

The GAO report, dated July 16 and released publically July 20, found a primary reason for the inaccuracies has been that while the dashboard is meant for use as a near-real time reflection of agency performance, the information OMB has required agencies to submit into the dashboard has been up to almost two years old.

"From the very first day the Obama administration took over, we're very focused on making sure that we're delivering results for the American people," said Federal CIO Vivek Kundra, the main force behind the dashboard, during a recent press availability with reporters. "We launched the federal IT dashboard to shine light on over $76 billion worth of IT investments," he added.

When GAO examined the dashboard's cost and schedule performance data on eight major IT projects in five of the largest cabinet departments, however, it found errors. Not dramatic ones, wrote report author David Powner, GAO director of information technology management issues, but of a magnitude large enough to nudge a project's rating from "green" status to "yellow" or vice-versa on the dashboard traffic light scoring system. GAO undertook research of the IT dashboard from January to July 2010.

Kundra has said he relies on dashboard information to select projects for review. The dashboard has been a source of outdated data because OMB has calculated project ratings according to agency performance as measured against completed project milestones, without taking into consideration performance during milestone completion, the GAO found. In one case, a Defense Department effort known as the Joint Precision Approach and Landing System, the last reported completed milestone was in July 2008, meaning that OMB's cost assessment of the project has been 645 days out of date. OMB has told agencies that it wants more detailed work breakdown schedules--which contain more milestones against which to measure performance--but OMB guidance has been unclear, the GAO report states.

Moreover, some agencies have disregarded the OMB guidance--specifically, the departments of Agriculture, Defense and Energy, according to the GAO.

In addition to inaccurate cost and schedule data, the GAO also found inaccuracies with the dashboard's data about the number of times agency projects have been rebaselined. Rebaselining occurs when actual current cost and schedule performance is at wide variance from original projections. According to the dashboard, the Energy Department's Integrated Management Navigation System and Sequoia Platform had each been rebaselined four times. In fact, they had never been rebaselined, but appeared so on the dashboard because OMB considered any correction to dashboard information, including typo fixes, as a rebaselining, the GAO report states. OMB officials told GAO that they designed the dashboard that way in order to hold agencies "accountable for the information they report."

In his official response, Kundra took several exceptions to the report, arguing, for example, that GAO's use of the word "inaccurate" as applied to the IT dashboard "could be easily construed by the public to mean that the [ratings] calculation was defective or not performing as designed." In an unusual report section dedicated to comments on comments, Powner allowed that the cost and schedule calculations "are performing as planned" but that nonetheless they do "not factor in the performance of ongoing milestones, which we and OMB agrees is an area for improvement." OMB told GAO that it will start factoring in current performance starting in July, changes that Powner wrote had yet to be completed as of July 16. Kundra did announce on July 14 that the IT dashboard is now available in a mobile device version, however.

Kundra also questioned whether GAO's analysis of cost and performance data itself is accurate, stating that "a direct comparison of [Earned Value Management] reports against IT dashboard calculations is therefore not appropriate." Powner, in his response to the response, agreed that agency EVM data--which GAO utilized--only provides data on the contracted development phases of a project, but that the report made an accurate analysis by focusing on dashboard information about projects in development and primarily carried out by contractors. "As stated in the report, we found that the [IT dashboard] ratings were not always accurate," Powner wrote.

For more:
- read the report, GAO 10-701 (.pdf)
- go to the federal IT dashboard
- read a July 14 blog post by Kundra announcing the mobile version of the dashboard

Related Articles:
OMB throws down review power over federal IT
AUDIO - Roger Baker and Vivek Kundra announce cancellation of VA financial management project
OMB mixes up cookie policy