Public data APIs facts: what an empty field or result means

Once the license question is settled and the data is flowing, a second set of problems begins, and none is about access. The hardest one is the empty cell. Registry data looks tabular but behaves like prose, and a blank in it is a sentence with the verb missing.

An empty field and an empty result set look alike on screen and mean different things. Reading either one correctly decides whether a pipeline stays honest, and each misreading below has cost somebody a rebuild.

What to take away

What a blank field can mean

A blank can mean the office does not collect the field for this entity type, the company did not answer, the field exists but is not exposed by this channel, or your request failed and returned an empty payload. Store which one you are looking at, or store nothing.

The fourth row is the dangerous one. A failure stored as a blank looks identical to a genuine absence, and it spreads into every report built on the table afterwards.

Why the same field is blank in one source and not another

Optionality varies by jurisdiction and by entity type. A field that is always present in your first source will be sometimes absent in your fourth, and a schema built from the first source will reject the fourth. Build for absence from the start.

The interface and the channel can also disagree. An API frequently exposes fewer fields than the web page, and occasionally more. Check field by field before promising anything, because the gap is not documented as a gap. Where a register documents its interface, as Companies House does in its public API specifications, the written field list is the baseline to compare the web page against.

Two fields deserve extra care because their blanks are rarely blanks at all. Identifiers are strings, and leading zeros, prefixes and separators are part of them. Parsing one as a number corrupts it, and the entity number lookup pillar explains why jurisdiction plus identifier is the only pair that means anything.

Addresses are what a filer typed, and they are frequently a registered agent's address serving thousands of entities. A filled address field is therefore a fact about a service provider as often as about the company.

No answer is not no match

A source that returns an empty result set under rate limiting will fill your table with clean-looking nil results for entities that exist. The pipeline has to make the two outcomes distinct and refuse to store the second as the first.

Test the failure on purpose. Throttle a request, break a key, and watch what lands in the table. If an error and a genuine no-match produce the same row, the pipeline cannot be trusted with a nil.

Treat a nil result the way a human searcher should: as a fact about one query, one index and one day. The reasoning in entity number lookup search applies to a script as much as to a person at a search box.

How much of the world the source covers

Coverage is the biggest silent gap. Federal filing systems are richer and more queryable than most state offices, and the SEC even documents its own application programming interfaces for exactly that purpose.

They include the submissions history by filer and the XBRL data from financial statements, and they hold only the small subset of companies that file with the securities regulator. A product built on federal data works beautifully in a demo and fails on the user's actual counterparty. An empty result for a private company in a federal feed is a coverage fact, not an entity fact.

Test coverage before you commit to a source. Take twenty counterparties your users actually have, look for each one by hand, and count the hits. It is a morning's work and it settles the question that a coverage claim on a marketing page never will.

Dating an empty value

An empty is only as good as its date. Every value is a statement by one office at one time, and a blank carries the same dates as a filled field.

Dates come in at least three kinds: the date a document was filed, the date it takes effect, and the date you retrieved the value. Merging any two of them produces a false precision that will mislead whoever queries it later. A record's own update date is not your retrieval date either. A status read today from a record last touched in March is a March fact, and both dates belong in the row.

Status is the field that moves in silence. Almost every field changes only when somebody files something, but status can change because a deadline passed and the office acted. Read registration status before deciding what a blank or a stale status means.

What to keep when the answer is nothing

Keep four things with every empty, so that it can be explained later:

The record shape that holds all of this is set out in company verification data, and the checks for when to run a status again are in which fields go stale.

Common questions

How do I test coverage before committing to a source?

Take twenty counterparties your users actually have, look for each one by hand, and count the hits. It is a morning's work and it settles the question that a coverage claim on a marketing page never will.

The API and the website disagree about a field. Which is right?

Ask the office. In the meantime record which channel each value came from, because a value with no channel attached cannot be reconciled once you have two of them.

Is it worth storing the raw response?

Yes. When a field's meaning turns out to differ from your assumption, the raw payload is the only way to repair history rather than re-fetch it.

Can I normalize status into active and inactive?

You can, and you will lose the difference between an entity that never filed, one that was administratively dissolved, and one that dissolved voluntarily. Keep the original string in its own column so the loss is recoverable.