Governed data

Every system of record holds part of the answer to a business question, and none of them holds all of it.

Take a plain question. Which customers are paying late and also have open quality complaints on goods shipped last quarter? Receivables sit in the finance system. Complaints sit in a service platform. Shipments sit in the ERP or a logistics tool. The customer record sits in a CRM. Each of these systems is correct about its own domain.

None of this is a design fault. Each system was built to run one process well, with its own identifiers, its own calendar and its own idea of what a customer is. The question crosses boundaries that the systems were never asked to cross, and no vendor's roadmap will change that, because the boundary is between your systems and not inside any one of them.

So the work falls to people. Someone exports from each system, reconciles the results in a spreadsheet and assembles the answer. The pattern has three predictable consequences.

• Timing. The assembly happens periodically, so the answer arrives after the decision needed it.
• Copies. Each function builds its own version, with slightly different filters, dates and exclusions. The review meeting then debates the data and not the decision.
• Dependence. The knowledge of how to assemble the answer lives in one or two people, and it leaves when they do.

There are three broad ways to bring the parts together. You can copy everything into a central store, such as a warehouse or lake. You can query the systems where they are and join the results at question time. Or you can combine the two, copying what must be copied and querying the rest. Later modules in this track cover the trade-offs. For now, the point is that the choice of technology comes second. The first task is to describe the question precisely.

A usable description has four parts. Name the question in one sentence. List the systems that hold each part of the answer. Identify the key that joins each pair of systems. State who is allowed to see the result. If you cannot name the join key between two systems, you have found the real project, and it is a matter of identity and not of tooling. Customer numbers that differ between billing and service, or part numbers that were reissued after an acquisition, will defeat any platform you buy.

You can recognise the problem without any technical review. Two people ask the same question and receive different answers. A month-end close needs several days of reconciliation. The count of active customers depends on which team you ask. Any of these signals a question that no system owns.

It helps to say what the problem is not. It is not a shortage of data, and it is rarely a shortage of analysts. Most organisations hold more than enough data. They lack an agreed route from the question to the records that answer it.

In practice: Pick one recurring question that currently takes a person more than an hour to assemble. Write down the systems, the join keys and the people who may see the result. Then ask who owns the answer today. If the honest reply is that nobody does, you have identified your first candidate for a governed layer.

Next: A2, Reading systems in place avoids migration, and it costs load, latency and discipline.

A dashboard displays a figure. Only an agreed definition, a traceable lineage and a quality rule can settle an argument about it.

Ask three teams for last month's revenue. Finance may report recognised revenue. Sales may report orders booked. Operations may report goods shipped. All three are defensible and all three are labelled revenue. The meeting that follows spends its time on which number is right, and the decision waits.

Building another dashboard does not help, because the dashboard inherits whichever definition its author chose. It adds a fourth presentation of the same disagreement. The remedy lies underneath the dashboard, in three things that are easy to name and hard to maintain.

A definition is a business decision, and it needs a named owner. It states what is included, what is excluded, the date basis and the currency treatment. It is published where people can read it, and it is applied in the layer that computes the measure. If each report re-implements the definition, the definitions will diverge within months.

Definitions also change. When finance revises how a measure treats returns, the history before and after the change is not comparable unless the definition is versioned and the change is recorded. A figure without its definition version is a figure that cannot be compared with last year.

Lineage means that a figure resolves to the source system, the object, the row and the version it came from. This is what turns an analytical answer into one an auditor will accept. Without it, a challenge to a number is settled by whoever argues most confidently.

Quality rules work best when written in business language and applied as the data arrives, with failures reported as exceptions. Silent correction is a hazard. If a system quietly fixes a bad record, the fix is invisible, the cause is never addressed and the number cannot be reproduced from the source. A visible exception can be assigned to someone and closed.

There is a cultural point as well. Agreeing a definition forces a conversation that many organisations have avoided for years. The conversation is uncomfortable, and it is the actual work. The technology only records and enforces the outcome.

In practice: Choose the three measures that appear most often in your leadership meetings. For each, find out how many distinct definitions are in use, and who could decide between them. Publish the chosen definition with a version and an owner before anyone builds a further report.

Next: A4, Entitlement belongs at query time, because the front door is not the only way in.

A system that can only tell you how things stand now cannot account for what it told someone last March.

An auditor, a regulator or a customer in dispute asks a specific question. What did the system show, and on what basis, on a named date? Answering requires more than a backup. It requires that several different things were kept, and that they can be put back together.

Start with the two kinds of time. Valid time is when a fact was true in the world. Transaction time is when the system learned about it. Suppose an invoice for a February sale was corrected in April. The February report, as it was produced in March, used the original invoice. A restated report uses the corrected one. Both are legitimate, and a system must be able to produce either on request. Tracking both clocks is often called bitemporal storage, and most operational systems do not do it, because they overwrite.

Five things must be retained to reconstruct a decision.

• The data. Prior values, not only current ones, with the time each was recorded.
• The definitions. The version of each measure that applied on the date, since definitions change.
• The policy. The entitlement rules in force, so you can show what the person was allowed to see.
• The evidence. For an AI answer, the records and passages retrieved, not merely the final text.
• The model and its settings. Which model ran, with which prompt and configuration, because a different version may answer differently.

Retention alone is insufficient if the record can be changed afterwards. A tamper-evident log, in which each entry carries a hash of the one before it, makes deletion or reordering detectable. The test is not whether logs exist. It is whether you can hand an auditor an account of what happened and have it hold up.

Cost needs attention. Keeping every version of every record is expensive, so decide which data and which decisions carry this obligation. Statutory retention rules often give a starting point. Tiered storage, where older data sits on cheaper media but remains directly queryable, avoids the restore step that turns an audit request into a project.

Few vendors can demonstrate this end to end. EigenForge's Kyros platform, powered by Kaman and in production, produces a sealed record per agent action and can reconstruct a decision as of its date, and the same questions are fair to put to any alternative.

In practice: Choose one decision your organisation made six months ago on the basis of a report or an AI answer. Try to reproduce exactly what was shown then, with the same data, definitions and access rights. Where you fail, note which of the five items was missing. That list is your retention requirement.

Next: B1, Pilots demonstrate well because the conditions that cause failure are absent.