Data Quality Demands Programme Discipline
The most consequential transformation programme in any large organisation is almost certainly the one that nobody wants to sponsor, because it has the word ‘data’ in the title and the word ‘quality’ immediately after it.
Executive Summary
Data quality is the transformation that every organisation needs and almost none wants to fund properly. It lacks the glamour of digital, the urgency of regulatory compliance, and the executive appeal of customer experience. Yet the failure of data quality underpins the failure of nearly everything else: the analytics that cannot be trusted, the migrations that overrun, the regulatory submissions that require manual reconciliation, the customer journeys that break at the seams between systems. This essay argues that data quality improvement, when it is to succeed at enterprise scale, must be treated as a programme — with the governance structures, funding models, and senior sponsorship that the word implies — rather than being delegated to operational teams as an unfunded mandate. The pattern I have observed across sectors is that organisations which treat data quality as a technical hygiene task fail repeatedly, while the rare organisations that treat it as a programme of structural change achieve durable results.
The Invisibility Problem
Data quality occupies a peculiar position in the landscape of enterprise change. Everyone agrees it matters. Every transformation business case assumes it. And almost no one wants to lead the programme that delivers it.
The reason is partly cultural and partly structural. Culturally, data quality lacks narrative power. A digital transformation has a vision — new channels, new capabilities, new customer propositions. A regulatory programme has urgency — deadlines, penalties, board-level scrutiny. A data quality programme has neither. Its vision is that things will stop going wrong, which is a difficult story to tell in an investment committee. Its urgency is chronic rather than acute; the costs of poor data quality accumulate gradually, distributed across dozens of processes, and are rarely attributed to a single root cause.
Structurally, data quality suffers from the same ownership vacuum that plagues master data management. The data is created in one part of the organisation, consumed in another, and degraded somewhere in between. No single function bears the full cost of poor quality, and therefore no single function has a compelling incentive to fund the remedy. The result is that data quality improvement is everyone’s second priority and no one’s first.
Why ‘Programme’ Is the Right Word
The distinction between a programme and a project is not merely semantic. A project delivers a defined output within a defined scope and timeline. A programme delivers a sustained capability through the co-ordination of multiple workstreams, typically spanning organisational boundaries and requiring ongoing governance beyond the life of any individual project.
Data quality improvement at enterprise scale is unambiguously a programme, for three reasons.
It Spans Organisational Boundaries
Data quality problems are never contained within a single system or a single team. A customer record that is created inaccurately in the onboarding system propagates errors into billing, into marketing, into regulatory reporting, and into analytics. Fixing the problem at the point of creation requires changing the behaviour of the onboarding team. Fixing the propagation requires changing integrations, business rules, and exception-handling processes in downstream systems. No single project can span this scope, and no single project sponsor has the authority to mandate changes across all the affected functions.
It Requires Sustained Governance
Data quality is not a state that can be achieved and then maintained passively. Data degrades continuously — through system changes, process changes, staff turnover, new data sources, and the simple accumulation of exceptions that no one resolves. A programme that improves data quality to an acceptable standard and then disbands will see that standard erode within twelve to eighteen months. The governance mechanisms — stewardship, monitoring, exception management, standard enforcement — must persist beyond the programme, and establishing them as durable operational capabilities is itself a programme-level undertaking.
It Depends on Behavioural Change
The most intractable data quality problems are not technical. They are behavioural. The call centre agent who enters a placeholder postcode because the customer is impatient. The product team that creates a new SKU without checking whether the item already exists under a different code. The finance team that maintains a parallel spreadsheet because they do not trust the system of record. These behaviours are rational responses to local incentives, and changing them requires changing the incentives — which means changing processes, performance metrics, training, and sometimes organisational structures. This is change management at scale, and it is programme work.
The Anatomy of Failure
The pattern I have observed in failed data quality initiatives is remarkably consistent. It proceeds through several predictable stages.
Stage One: The Trigger
Something makes the cost of poor data quality suddenly visible. A regulatory submission fails validation. A migration discovers that forty per cent of customer records cannot be matched. An analytics team publishes a report that is visibly wrong, and the error is traced to source data. The trigger creates a moment of executive attention, and a data quality initiative is approved.
Stage Two: The Technical Response
The initiative is framed as a technical problem. A data quality tool is procured. Profiling is conducted. The worst data quality issues are identified and quantified. Remediation rules are developed. A cleansing exercise is undertaken, targeting the most visible problems — duplicates, missing values, format inconsistencies, orphaned records.
Stage Three: Early Success
The cleansing exercise delivers measurable improvement. Quality scores rise. The immediate pain point that triggered the initiative is resolved. The executive sponsor declares progress. The initiative produces a set of dashboards showing data quality metrics by domain.
Stage Four: The Plateau
The easy wins are exhausted. The remaining quality issues are harder — they involve semantic inconsistencies, business rule conflicts between systems, and root causes that are embedded in upstream processes. Resolving them requires co-operation from business units that did not sponsor the initiative and do not see data quality as their problem. The initiative team escalates, but the governance mechanisms to compel co-operation do not exist, or exist on paper without real authority.
Stage Five: Decay
The initiative loses momentum. The executive sponsor moves on. The budget is reduced. The data quality tool continues to run, producing dashboards that fewer and fewer people review. New data quality issues are introduced faster than old ones are resolved, because the root causes were never addressed. Within two years, the organisation is back where it started, and someone proposes a new data quality initiative.
The cycle of trigger, technical response, early success, plateau, and decay is not a failure of execution. It is the predictable outcome of treating a programme-level challenge as a project-level task.
What Programme Discipline Actually Means
If data quality improvement is to escape this cycle, it must be constituted as a programme with the structural characteristics that the word implies. In my experience, the following elements are not optional.
Executive Sponsorship with Teeth
The programme sponsor must be senior enough to compel co-operation across business units, and must be willing to use that authority. In practice, this means the CFO, the COO, or a chief data officer with genuine line authority — not an advisory role. The sponsor’s commitment must be visible and sustained; data quality programmes that lose their sponsor invariably stall within six months.
A Funding Model That Reflects Reality
Data quality improvement requires two distinct funding streams: programme funding for the remediation and capability-building work, and operational funding for the ongoing stewardship and governance. The programme funding can be time-limited, but the operational funding must be permanent. Organisations that fund only the programme and assume that operational disciplines will be absorbed into business-as-usual are guaranteeing the decay stage described above.
Root-Cause Orientation
The programme must be oriented toward root causes, not symptoms. Cleansing data that is continuously re-contaminated by broken processes is an expensive treadmill. Every quality issue identified must be traced back to its point of origin — the process, the system, the behaviour that creates the defect — and the remediation must address the origin, not just the downstream manifestation.
This is where the programme encounters its greatest resistance, because root-cause remediation often requires changes that business units do not want to make. Adding validation to a data entry process slows down the process. Enforcing referential integrity between systems constrains how those systems can be changed independently. Requiring a product team to check for duplicates before creating a new record adds a step to their workflow. Each of these changes has a local cost borne by the team that makes the change, and a distributed benefit enjoyed by teams elsewhere in the organisation. Without programme-level authority to mandate the change, rational self-interest will prevent it.
Measurement That Drives Action
Data quality metrics are useful only if they are connected to consequences. The programme must establish not just what is measured, but what happens when a metric breaches its threshold. Who is notified? What decision is triggered? What is the escalation path? A quality dashboard that is reviewed monthly in a governance meeting and generates no action is a reporting mechanism, not a management mechanism.
The most effective model I have observed is one where data quality metrics are incorporated into the operational KPIs of the business units that create the data. When a business unit’s data quality score affects its performance assessment, the incentive to maintain quality is embedded in the structure, not dependent on goodwill.
The GDPR Catalyst
The introduction of GDPR in May 2018 created an unexpected tailwind for data quality programmes. The regulation’s requirements around data accuracy, data minimisation, and the rights of data subjects created a regulatory imperative for capabilities that data quality practitioners had been advocating for years: reliable data inventories, accurate linkage of records to individuals, the ability to identify and correct inaccurate data, and the ability to demonstrate the provenance and processing history of personal data.
For organisations that already had mature data quality programmes, GDPR was an accelerant. It provided additional funding, additional executive attention, and — crucially — a regulatory mechanism to compel business units to co-operate with data quality requirements. For organisations that did not, GDPR often triggered a compliance-focused response that addressed the regulatory requirements without building the underlying data quality capability.
“The most consequential transformation programme in any large organisation is almost certainly the one that nobody wants to sponsor, because it has the word ‘data’ in the title and the word ‘quality’ immediately after it.”
The pattern that is emerging, a year after the regulation took effect, is that the organisations which used GDPR as a catalyst for genuine data quality investment are now in a measurably stronger position — not just for compliance, but for every initiative that depends on trustworthy data. Those that treated GDPR as a standalone compliance exercise are discovering that their compliance artefacts — data inventories, processing records, consent databases — are already degrading, because they were never connected to a sustainable data quality discipline.
The Programme Manager’s Honest Assessment
Data quality programmes are, in my experience, among the most difficult to lead. The work is unglamorous. The benefits are indirect and often attributed to other programmes. The stakeholders are diffuse and often resistant. The governance mechanisms required are politically contentious. The funding model is structurally misaligned with how most organisations budget for change. And the timescale is long — genuine, durable improvement in enterprise data quality takes three to five years, which is longer than most programme sponsors, most CIOs, and most executive committees are willing to commit to a single initiative.
But the alternative — continuing to treat data quality as a background task, funded sporadically and owned by no one — is more expensive. It is more expensive in direct costs: the manual reconciliation, the failed migrations, the unreliable analytics, the regulatory remediation. And it is more expensive in opportunity costs: the transformation programmes that underperform because they cannot trust their data, the customer initiatives that break at the seams, the strategic decisions made on foundations that no one has verified.
The most boring transformation is, in the end, the most consequential. The organisations that recognise this — and fund, govern, and sustain it accordingly — will outperform those that do not. The evidence for this is already visible, for anyone willing to look at it honestly.