IRS IT programs' cost and schedule metrics unreliable, says GAO
Metrics used by the Internal Revenue Service to ensure information technology programs meet cost and schedule estimates aren't consistent, says an April 17 Government Accountability Office report (.pdf).
The IRS claims that 12 of 20 major IT investments were within 10 percent of cost and schedule estimates or significantly below cost between October 2011 and October 2012. Of the other eight investments, three were significantly above cost and five were significantly behind schedule, and five of the eight had significant variances for one month.
But because those conclusions are based on inconsistent metrics, auditors say they aren't necessarily accurate. Schedule variances can only be determined from reliable cost and schedule estimates and consistent processes for comparing actual and projected amounts--two things IRS is lacking, says the report.
The agency used the GAO's cost guide and schedule guide, which outline best practices for developing estimates, for several programs, says the report. But in other cases IRS determined variances using projected information for in-process activities, but the agency's guidance did not specify how projected amounts should be determined.
According to GAO, a reliable cost estimate should be comprehensive, well-documented, accurate, and credible. Auditors found that six investments of seven in a sample at least minimally met all four characteristics of a reliable cost estimate. All seven investments in the sample at least minimally met the criteria for a well-constructed and controlled schedule estimate, according to GAO.
Report authors recommend the IRS improve the reliability of reported cost and schedule variance information by ensuring each investment has a reliable cost estimate, revising schedules so they are well-constructed and controlled, updating project performance information 60 days after completion of an activity, and crafting best practices guidance.
This guidance should outline how to evaluate critical paths for projected schedules, use earned value management data, compare completed work to the remaining budget, assess resources needed to complete remaining work and estimate future conditions, write auditors.
In a written response to the report, the IRS agreed with all the recommendations, except that it use earned value management data as a best practice to determine projected cost and schedule amounts. The agency said the cost and burden to use earned value management data outweigh the value added. Auditors, however, disagree with the agency's stance.
- download the report, GAO-13-401 (.pdf)