Performance Pays

In the past, agency acquisition officials faced a daunting task if they needed to fill a new call center with state-of-the-art servers and associated software, networking gear and storage devices.

They’d meticulously specify hardware and software components, sometimes down to parts numbers and performance characteristics, while also setting implementation milestones — perhaps even telling potential contractors how the new system would plug into existing operations.

But performance-based acquisitions promise a new approach. The PBA strategy charges the acquisitions people with defining performing levels for the new call center and lets contractors figure out how to achieve the desired results.

This makes intuitive sense, but widespread PBA use remains sketchy, even given Office of Management and Budget mandates stretching back to 1991 and the later 2001 President’s Management Agenda directive that targeted PBA use for 50 percent of relevant acquisitions (a number later reduced to 40 percent).

But new research suggests that even the lower targets are tough to meet. In addition to cultural clashes that force acquisitions officers to learn new work methods and forge new relationships with operations staff, agencies also often sidetrack PBAs because of difficulties tied to estimating return on investments over a contract’s life.

According to OMB, agencies wrote 45 percent of their eligible service contracts as PBAs last year. But a study sent this month to the president and Congress concludes that agencies may be over- reporting PBA reliance.

“We found a large number of [agencies] have erroneously reported their use of performance-based contracts,” says Carl DeMaio, president of the Performance Institute of Washington and co-chairman of the Performance-Based Service Acquisition Working Group of the Federal Acquisition Advisory Panel, which wrote the new report. “They are calling contracts performance-based when in fact they are not.”

DeMaio estimates that only about 20 percent of federal contracts truly use performance-based acquisition techniques.
The problem stems from continuing confusion about the use of such acquisitions. “Federal agencies are very unclear as to what a PBA is and how to use it,” DeMaio says. One difficult area is how to define and collect performance measurements. “Agencies tend to measure too many things, and that creates its own costs for contract administration,” he adds.

That’s bad news for PBA proponents who believe the methodology offers the best hope for simultaneously delivering technology and services that meet agency needs, while also assuring that government spends taxpayers’ dollars wisely. “PBA grew out of the $800 hammer” scandals of years past, says Mike Cameron, a PBA best-
practices specialist at the consulting
firm Booz Allen Hamilton of McLean, Va.

Watch Your Step

Stumbling blocks occur over ROI and performance measurements, he says. “If you don’t think beyond the procurement to consider how you are going to ensure that the delivery occurs in a manner that satisfies your needs, then the wheels are going to fall off before you’ve ever received the first bid,” Cameron cautions.

The Coast Guard is among the handful of agencies observers cite for having found success with its PBA efforts. The key for estimating ROI often lies in what happens after the agency and vendor sign the initial contract, says Scott Palmer, chief of the Coast Guard’s Major Systems Contracting Division. “Any return on investment calculation that relies purely on the initial contract cost is a poor one,” he warns.

Realistic ROI estimates should include using the initial contract value as the starting point and then add costs for training and performance monitoring through the contract’s term. Both these cost areas are fluid, as agencies and contractors change how they meet a project’s objectives, so they are not static expenses an agency can easily define from the start, Palmer says.

“Midway through a contract, a contractor or one of our representatives may say, ‘Here’s what’s working; here’s what’s not,’ ” he says. “Once you implement a change
in the interest of performance, it can be
difficult to capture the costs associated with that. You are going to incur some costs to put the change to work and then that’s got to get pumped into your ROI formula.”

Best Buys

Building an accurate ROI model starts with forming an integrated project team, which might consist of a program manager, contracting officer, budget and legal personnel, end users and quality assurance specialists. The team’s job is to define project needs and tools for measuring success.

“Once we get the parties at the table, it’s a fairly simple exercise to get the structure we need,” Cameron says. “The hard part is overcoming a culture that says the [procurement and operations] sides of the house don’t talk with each other very often.”

DeMaio adds that outside acquisition advisers should be brought in to facilitate this process for any “high-risk, high-cost or high-impact” procurement. “Hire an adviser to help you look at your alternatives and define your needs,” he says, adding that the outside team needs to have market experience in the procurement’s focus area, a component of the market research stage that PBA experts consider a planning best practice.

PBA training is also essential for both acquisitions personnel and the operations staff who must carry the new responsibility of defining project results. “You cannot leave the program side out of the training equation,” DeMaio says. “They are the ones that know their problems. They are the ones that must define the needs and the requirements. The acquisition workforce is there to help keep things legal and help everyone through the process.”

Even with a full working group, the acquisition team might continue to struggle with creating accurate ROI models that don’t underestimate full lifecycle costs, Palmer acknowledges. But they should stick with it, he adds.

“That fact paints us as being overly optimistic with ROI because we are not capturing all the costs,” Palmer says. “But I would also submit that large organizations may also be overly conservative when we are estimating some of the benefits, such as end-user productivity.”