Identity & Grain Contracts¶

Grain answers what one row represents. Identity answers what makes that financial thing the same thing again.

Those concepts overlap, but they are not synonyms. A table can have a valid physical primary key and still have the wrong economic grain. A deterministic ID can be stable while a downstream aggregation violates that grain.

Analytical grain catalog¶

Deterministic identity namespaces¶

Canonical serialization¶

The identity implementation normalizes values before hashing:

match val:
    case None:
        return "<NULL>"
    case bool():
        return "true" if val else "false"
    case datetime():
        return val.isoformat()
    case date():
        return val.strftime("%Y-%m-%d")

It also normalizes numeric values through Decimal, Unicode strings through NFC, and strips surrounding whitespace.

Known namespaces use contract-defined field order. Unknown/new namespaces fall back to sorted field names so identity remains deterministic rather than caller-insertion-order dependent.

The payload is serialized as:

[namespace, "v2", [[key, value], ...]]

then SHA-256 hashed and truncated to a 16-hex-character suffix prefixed by the namespace.

The v2 marker is a technical identity protocol version, not an application release number.

Grain is not primary-key syntax¶

Consider gold.Investment_By_ISIN with grain Date-ISIN.

The grain tells downstream analytics that one row represents one instrument on one valuation date. Whether the physical table declares a composite SQL primary key is secondary to that semantic contract.

Join discipline¶

Before joining two datasets, ask:

1. What is the grain on each side?
2. Is the join one-to-one, one-to-many, or many-to-many at those grains?
3. Which side may legally duplicate the other?
4. Does the join preserve the intended output grain?

Many-to-many joins in financial data deserve suspicion until proven necessary.

Identity continuity through rebuilds¶

Deterministic IDs allow analytical rebuilds to recreate the same entity identity when the same economic evidence and namespace contract are present.

That supports:

• replay comparison;
• lineage continuity;
• reconciliation;
• stable downstream relationships.

Changing identity fields/order/normalization is therefore a contract migration, not a refactor detail.

Related specifications¶

• Deterministic Identity & Replay
• Investment Data Model
• Data Contract & Schema Evolution