Report questions FBI's ability to implement agile development for Sentinel
The report (.pdf), commissioned by the FBI and written by the Software Engineering Institute at Carnegie Mellon University--obtained by FierceGovernmentIT through a Freedom of Information Act request--calls the switch to agile development a "positive step toward improvement over the prior development approach."
"While an Agile approach cannot solve all the program's problems, it is likely to help," report authors added while also listing several concerns about the bureau's ability to implement agile processes.
An October Justice Department inspector general report revealed that the FBI made plans to adopt agile development, lessen its reliance on Sentinel prime contractor Lockheed Martin (NYSE: LMT), and drastically cut personnel. The FBI said it can finish Sentinel for $20 million and within 12 months--despite having already spent nearly 90 percent of its $451 million budget, and despite also being two years behind schedule and having completed, at best, two of the four phases of Sentinel's development.
The Sentinel agile development plans reviewed by the SEI, dubbed by the FBI as the "Path Forward"--although authors wrote that little of FBI plans are written down--called for a mix of agile techniques, including Scrum and eXtreme Programming.
FBI officials told SEI authors that Sentinel will be completed by two teams of approximately 25 personnel each: A developer group located inside an FBI building (the SEI report doesn't specify which one) and a quality assurance group located "in another office nearby." Developers are meant to produce new functionality in two-week sprints and the personnel a mix of government and contractor employees.
The SEI review found that overall objectives were clearly defined, as were cost and schedule objectives, and that the bureau laudably intended to build on existing assets as much as possible.
But, it appeared to the authors that the FBI planned to designate only one ScrumMaster and "an expectation of shared authority was rarely in evidence."
A cornerstone of Scrum is to allow considerable autonomous decision making by teams of five to 10 people. Hierarchical decision making by the program manager--the FBI chief technology officer--might be at first necessary "given the aggressive schedule," report authors wrote.
"Our concern, however, is that the shift toward a self-organized, self-empowered posture will not truly materialize," they added. They also noted that the usual agile tactic is for roles to be shared and for all team members to be physically co-located.
SEI authors also found FBI officials lacked detail on key agile processes, such as daily meetings, sprint retrospectives, continuous integration and test driven development. Nor was it clear to SEI personnel how the user community would interface with Sentinel agile teams. The FBI also appeared to take a predetermined fixed time/fixed scope approach to deliverables.
"While some of these above conditions might occur in any given Agile project, in the aggregate, they suggest that the key principles that underlie an Agile approach have been only partially embraced," report authors wrote.
Report authors refused to make a prediction of failure or success due to all the unknowns, which also include user community reaction to a "an 80 percent solution." However, unless the FBI remedies those weaknesses, it will surely face grater difficulties than it would otherwise, the authors concluded.
- download a copy of the Sept. 2010 SEI report