Why don’t governments hedge against their Big IT investments? And what if they did?

Back in November, John Ivison covered the saga of the Government of Canada’s Benefits Delivery Modernization (“BDM”) project for the National Post. Its already-enormous price tag has ballooned four-fold, to almost $8 billion:

Benefits Delivery Modernization (BDM) … will move old age security (OAS), employment insurance (EI) and the Canada Pension Plan (CPP) — programs that last year delivered $150 billion in benefits to 10 million Canadians — into the cloud.

The project was announced in 2017 with an estimated price tag of $1.75 billion and an end date of 2030. Last year, a review by consultants PwC said initial cost estimates for software and implementation were well below the industry average and timelines would be longer than first thought. The new price tag was estimated at $3.4 billion (or higher) and the completion date was forecast to be 2034.

Sources suggest ESDC is set to go back to the Treasury Board Secretariat, which controls the public purse on the project, with a new price estimate of almost $8 billion.

What John couldn’t put on display is how little BDM has accomplished in the six years since it was launched. But there is one easy metric for the hundreds of millions of dollars spent to date: the number of Canadians the project has demonstrably helped. I say it’s easy because, as far as I can tell from all the reports and audits and dashboards about BDM, that number is zero. At least it was when I left in late 2021.

But I don’t want to talk about why BDM’s price tag is so huge. (Well, not today. But let’s come back to that.) I want to talk about why it’s the only option on the table.

—-

Imagine you’re a senior executive at a big company, and you’ve been asked to lead the purchase of an asset worth $100 million. Surely, for an asset so valuable, you’d take out an insurance policy on it, to manage the risk of the potential loss of it or of its value. And if you couldn’t buy insurance, you’d find some other way to make sure the company managed that risk — probably by making a “hedge” investment to mitigate the risk of the asset losing value.

You probably see where I’m going with this. Our governments purchase IT assets for $50M, $100M, even $1B or more, all the time, with no insurance policy and no hedge. And the list of government IT project failures at that scale is legion. If the Standish Report is even close to accurate, it’s entirely predictable that a sizable percentage of big projects will fail, and the bigger the project, the more likely it is to fail.

The U.S. government alone spends nearly $100B each year on federal government IT, and sends roughly that same amount to the states to spend on their own IT implementing federal policies. Yet no department or jurisdiction is required to hedge their bets against any of this spending, despite what we know about the failure rate of such spending.

Why?

Well, you can’t buy Government IT insurance; no insurer would ever issue a policy against a government IT project given (among other reasons) the data on success rates. But why don’t governments hedge against the potential failure of their technology solutions?

Well, for one thing, governments — and politics — generally abhor redundancy. The organizational structure of bureaucracies all but demands that the authority to solve any given problem rest with one and only one part of the organization. Explicit, strategic, intentional internal competition, a common tool in large private enterprises, is usually viewed in government environments (not to mention the halls of budget-setting legislatures) as wastefully redundant, and any internally competitive budgeting would be an easy target for cuts.

(Governments do occasionally introduce internal competition, but it’s almost always veiled, and often set up to fail. In 2018, the Government of Canada instigated a “Next Generation Pay” program to run alongside their long-failing “Phoenix” pay system, ostensibly to replace the latter while it limped along. In reality, the move created an excruciating slow-motion internecine competition between the two systems’ teams that persisted for years (and, in all, cost over $2 billion). “Digital Service” teams and “Offices of Innovation” are also sometimes thrust (or thrust themselves) into a competitive posture on a project, sometimes helpfully, sometimes disruptively, but almost never with the right combination of authorities and budget.)

But internal politics are only symptoms. Hedging just isn’t how government IT budgets are set up to work. To make a long story short (though it’s a good topic for another day), budgets for large IT projects are often driven not by how much they should cost, but by how much companies say their technology solutions will cost. I’ve seen governments literally ask companies for their proposals, budget accordingly, and then bid out to those same companies.

So it’s not just that there’s “only enough room in this town” for one owner of any given problem space; it’s also that there’s only budget for one solution path — that is, until the project starts to fail, and the team has to go back to the well for more money. At that point, the budget process almost always succumbs to the sunk cost fallacy, and forks over the additional funds. This is how a $364M project becomes a $647M-and-still-rising project, not counting the tears, desperation, and myriad other ancillary quantifiable and unquantifiable costs that don’t show up in an IT budget.


This, of course, is all bonkers.


I’ll save “how did we get here” for another day. But I believe it’s entirely possible for governments to hedge against large budgets for most of its biggest IT projects.

So, in the spirit of “no bad ideas,” here’s a proposal: What if a government were to require that any IT project budgeted for (or expected to cost) more than $50M redirect a tiny slice of that project budget — say, 3%, or $1.5M of $50M — into a hedge investment, in the form of paying for a small agile product team to do what small agile product teams do: conduct design research, prototype quickly, iterate, learn, and document what they’re learning.

Call it “agile insurance” — agile as a hedge against the prospect of big-dollar project failure.

One year later, compare the progress of the two teams. One of two scenarios might be playing out. Maybe the big-budget solution is going swimmingly and demonstrating real progress toward meeting real users’ needs. (Experienced folks, suppress your jeers.) If so, the small hedge team could fold up its tent after documenting its learnings — learnings which, no doubt, will be useful in some ways to the main project, and probably well worth the small incremental price paid for their work.

But maybe the big-budget solution isn’t working out so well. If that’s the case, you’re not hopelessly stuck with a failing project in a sunk cost spiral. You have a second team that’s been researching with users, testing, and learning all along, and is likely to be able to make recommendations about how to either (1) constructively pivot the big project and proceed with greater chance of success than before; (2) switch the project’s main track to invest in and build on the agile team’s work; or (3) cut bait altogether instead of succumbing to the sunk cost fallacy, and start over, with important learnings in hand (which is what should have happened first anyway).

There are complications and difficulties, to be sure: Both teams vying for the attention and time of the policy owners and other key stakeholders could be taxing; the external optics of internal competition and redundancy can be tricky; finding the skilled labor for the agile teams is a persistent problem for many civil services_. _But other large organizations frequently find ways around these problems, and neither are they insurmountable in government.

On its face, the budget math seems to work out strongly in favor of running an experiment in agile hedging. Say a government decided to hedge against ten $50M projects, at a total cost of $15M against $500M in investments. If the hedge exercises were to stop just one bad project from proceeding to waste another $50M the following year, the entire experiment would already have paid for itself more than three-fold (minus any overhead associated with operating the hedge program). And if just two or three of the ten hedge teams help their big-budget counterparts into better understanding of users and better project-wide decision-making, the savings — and benefits to users — stack up quickly.

On top of all the project efficacy and efficiency reasons for agile hedging, there’s another potential benefit as well: this kind of experiment is a vector for introducing an empowered agile delivery team into environments where none may yet exist. And that’s almost sure to be a healthy outcome, regardless of whether agile hedging turns out to succeed.

My colleagues at the Canadian Digital Service ran with this idea and drafted the beginnings of a policy proposal. In every government, someone has the means to attach hedging as a condition to various streams of funding; the US government’s TMF and OMB’s budget authority, the Government of Canada’s Treasury Board and Enterprise Architecture Review Board, and the UK’s digital and technology spend control levers, all come to mind.

Will someone — an executive or elected official who’s leery of a Big IT project budget — try this experiment and attach hedge requirements? If it’s been done, I’d love to hear all about it from you.