Prepared for the Subcommittee on Commerce, Justice, State and the Judiciary U.S. Senate Committee on Appropriations
On Thursday, February 3 at 2:00 p.m., ET, Arnold Punaro, SAIC Executive Vice President and General Manager – Washington Operations, was to testify before the Senate Appropriations Subcommittee on Commerce, Justice, State and Judiciary. Before Punaro was able to make his opening statement and provide his full testimony, the hearing was recessed. Punaro's testimony was to address SAIC's performance on the FBI's Trilogy Virtual Case File system. Subcommittee Chairman Judd Gregg, R-N.H. promised to hold future hearings on the subject, which will give SAIC a chance to make its case.
Chairman Gregg and Senator Leahy:
It is a privilege to appear before you today to testify concerning our portion of the work on the Trilogy Project for the Federal Bureau of Investigation. Mr. Chairman, I ask consent that my entire statement be entered into the record and with your permission I am prepared to summarize.
I. Introduction and context
At the outset, let us say clearly that SAIC understands and appreciates the overwhelming demands and difficulties that the FBI has faced since the attacks of September, 11. While we disagree with the Bureau over aspects of the Trilogy program history, we have only the greatest respect for the dedication with which the Bureau has pursued its mission of defending our nation under the enormous, and sometimes conflicting, pressures that surfaced in the aftermath of the terrorist attacks.
SAIC, with 45,000 employees, is the largest privately owned research and engineering firm and one of the largest government contactors in the nation. As employee owners, we have prided ourselves since our founding 36 years ago on our ability to assist the U.S. Government on programs of national importance. Our dedication to work that matters is further reflected in an aggressive and pervasive ethics program. How our company operates and how we are perceived are matters of vital, personal interest to each and every employee. We have grown to become a very successful and sought after company by providing quality products and creating satisfied customers.
In that respect, let me mention several major, illustrative software engineering projects successfully designed and deployed for the FBI to illustrate the work we've done.
In sum, SAIC comes to this issue with a record of outstanding achievement in challenging projects, including specifically for the Federal Bureau of Investigation. We point this out not to boast, but to provide the context for considering some of the issues that have marked the public discussion of Trilogy and the manner in which SAIC has performed on this contract.
The Results and the Reasons
Currently, the contract has a negotiated value of $130.3 million and a funded value of $123 million. To date, SAIC has been paid $115.2 million. We expect to be paid the funded value of $123 million at completion. In conjunction with this work effort, the company has invested $3.9 million of its own money to support the Trilogy program.
Aerospace Corporation
Before presenting SAIC's testimony about the course of its work on Trilogy in detail, I want to speak briefly to the report by the Aerospace Corporation. While we have not been given a copy of this report, we were allowed to read a copy last week at the FBI. We appreciate that opportunity. Aerospace Corporation did not inform us, nor attempt to discuss in any way its findings-a lapse we find both inexplicable and contrary to the practices of inspectors general, the General Accounting Office, and other scientific groups, who find that comments from those reviewed contribute to a more balanced and useful report.
The Aerospace Corporation produced a report on the wrong software while failing to concentrate on central issues that determine system performance.
Had they asked us for comment, we could have told them they examined the wrong software. Mr. Chairman, I mean that in a literal sense. Aerospace Corporation explicitly evaluated a snapshot in time of the software as if it were a finished product when in reality, as everyone should have known, it was still being developed. The Aerospace Corporation says it found "evidence of incompleteness" and "failure to optimize." This is hardly unexpected in a work in progress that was still months away from its delivery date. In academic terms, it was as if we had been assigned a paper due December but then graded it the previous summer.
The product we presented to the FBI in December 2004 is not the product evaluated by Aerospace Corporation. VCF IOC was rigorously tested and accepted by the FBI after meeting 100 percent of its requirements.
Because the software evaluated was different from the software delivered, SAIC believes that the Aerospace Corporation report is not an adequate basis for deciding on a future course of action concerning VCF.
This is not to say we accept Aerospace Corporation's judgments about the product that was evaluated. We emphatically do not. The Aerospace Corporation is a national asset in its realm of expertise: aerospace. The Trilogy project is something else, altogether. We respectfully-but strongly-urge this subcommittee to consider that Aerospace Corporation did not bring a sufficient understanding of the uniqueness, complexity, and scope of the FBI undertaking to evaluate our software product.
Central to the Aerospace report is criticism of requirements documentation. Time and again, in the Aerospace report we reviewed, we saw instances where criticisms about requirements were based not on the substance of the requirements or whether or not the product satisfied the requirements, but rather on ancillary data such as syntax in documentation. How well the product satisfied requirements was not a part of their evaluation. Based on examination of the documentation they concluded they were not assured the product would meet requirements and went no farther.
In particular, SAIC categorically rejects the assertion that its work lacked engineering discipline, an assertion that appears without support in the document we read. This kind of assertion, without rigorous-or even specific-support should be unacceptable in an endeavor of this importance. For instance, Aerospace Corporation did not look at the software development folders, which are key documents on how the code was designed and written. These comprise the "Bible" for software developers. In a football analogy, it was as if Aerospace Corporation was asked to scout another team which had made available its playbook. They didn't bother to read it. In fact, they scouted the wrong team.
Even so, Mr. Chairman, we would welcome the opportunity, late though it may be, to discuss the findings with Aerospace Corporation. It could only benefit the FBI, which is our aim here.
II. SAIC's participation in Trilogy
The FBI's Trilogy program is a massive, multi-part, multi-contractor program for broad-based modernization and improvement of its information technology. In June 2001, SAIC was competitively awarded a cost-plus-award fee developmental contract for the Trilogy User Application Component (UAC). This is an appropriate contract type because the project involved first working with the customer to develop and agree on what was needed (the requirements) and then execute the agreed tasks. The complexity and uniqueness of the missions of the Bureau also argued for this approach. Some of the public discussion of the Trilogy contract has been conducted as if the required tasks were well known at the start, and easily achievable. At no point in time has either condition existed.
At the time of award in June 2001, the contract scope for SAIC called for development of a web front-end to the existing legacy applications used to manage case information When this effort was complete, SAIC was to define an Enterprise Case Management System. This was a measured low-risk approach building on existing, or legacy, systems within the Bureau.
The attack of 9/11
The September 11, 2001, attacks had as profound an affect on this project as it did elsewhere in the nation. Following 9/11, the Bureau faced enormous and sometimes conflicting pressures. Prior to the attack, the Bureau was dealing with revelations that a spy, Robert Hansen, had plundered FBI secrets. Security and integrity of information is a fundamental issue for the FBI. After the attack, it faced three often conflicting demands:
Thus, the FBI faces a task of great difficulty and complexity in building an information technology system that simultaneously meets all three imperatives.
Trilogy after 9/11
Following the attack, the Bureau fundamentally reexamined the project. The earlier, measured approach of June 2001 called for improving legacy systems. In the wake of the attack, the FBI correctly determined that the legacy applications should be replaced to make the Bureau more effective in responding to terrorists' threats as well as to improve the efficiency of the continuing criminal investigative mission.
In the months following 9/11, the Bureau conducted an independent review of available Commercial Off-The-Shelf (COTS) systems and Government developed systems, and determined they could not satisfy the requirements. Therefore, SAIC was tasked to in February 2002 to develop the replacement for the legacy systems using the original contract. The SAIC UAC contract was restructured to incorporate an aggressive development plan first conceived in February 2002. This became the electronic Virtual Case File (VCF) contract. Thus, the FBI shelved 6 months of work that no longer fit the post 9-11 world, and directed SAIC take on a much more ambitious, high risk project.
The Trilogy VCF was a large and complex enterprise-level undertaking. There are no other criminal investigative management systems of this scale in the world. In terms of size, the VCF DELIVERY 1 system was to manage millions of case files on Day One with an annual growth of hundreds of thousands of cases per year. At start-up, the VCF DELIVERY 1 system was to store and index more than hundreds of millions of documents in a wide variety of formats. The VCF DELIVERY 1 system would support 30,000 users geographically dispersed across the United States and other countries. FBI agents, analysts, and support personnel would rely on the VCF DELIVERY 1 to conduct nearly all the business functions that support the criminal investigative process. The VCF DELIVERY 1 was also to provide hundreds of interfaces to legacy systems. The VCF DELIVERY 1 system would manage this workload while providing a 3-second response to users as well as high system availability. This would not be an ordinary case file management system.
The VCF was intended, in sum, to provide the next generation system supporting the FBI's case file management concept. It would be, as the Justice Department Inspector General has reported, "the first real change in the FBI's workflow and processes since the 1950's". The VCF would move the FBI from its slow, paper-based processes into the twenty-first century with electronic work flow. VCF, it was envisioned, would support real-time coordination among agents, allow secure access to, and reporting of case information for all those authorized to receive it, regardless of organization or location. VCF would support a dispersed community of users in creating, accessing, and managing centrally stored electronic case file information. It would provide the foundation upon which the FBI could migrate its disconnected business processes into an integrated and seamless work environment.
Following the 9/11 attacks, time was of the essence. SAIC was asked to devise an approach to deliver VCF in record time-on an even more aggressive schedule. The new challenge was to define, develop, and deploy a bureau-wide enterprise-level case management system in just 22 months. Without defined requirements or an enterprise architecture for the FBI IT systems, this was a high risk approach that reflected the post 9/11 atmosphere. Here is where SAIC made honest mistakes. We should have made known that this approach was too ambitious..." (End excerpt. Link follows.) Arnold Punaro's Record Testimony
"Before Punaro was able to make his opening statement and provide his full testimony, the hearing was recessed. Punaro's testimony was to address SAIC's performance on the FBI's Trilogy Virtual Case File system. Subcommittee Chairman Judd Gregg, R-N.H. promised to hold future hearings on the subject, which will give SAIC a chance to make its case."