Maintenance
Part of Public data APIs: questions before you build
Can a public data APIs search stay accurate after normalizing the query?
Can a public data APIs search stay accurate after normalizing the query? Yes—if you record match mode, keep the raw query, and log every outcome.
A search you built on top of registry data is a promise to your users about what "not found" means. Most such promises are made accidentally, by whoever wired the query, and they are usually wrong. The source matched on prefixes; your interface implied a full-text search. The source was throttled; your interface said nothing was there.
This page is about a public data APIs search over registry sources: how matching differs from what a user expects, how to normalize a query without destroying it, and how to present a result set honestly.
Yes — a search built on public data APIs can stay accurate after the query is normalized, but only when three things are recorded: the source's match mode, the raw query beside the normalized one, and the outcome of every call. Strip any of the three and the normalized query is not so much inaccurate as unaccountable, because nobody can tell whether a nil result means the register has no such entity or the query never had a chance.
What to take away
- The source's matching behavior, not yours, decides what your search means. Find out what it is before you build a box.
- Normalize the query, keep the original, and show the user which one produced the result.
- A nil result and a failed request must never look the same to a user or to a downstream system.
Establish the source's matching behavior first
Registry search engines use exact, starts-with, and contains matching, sometimes different ones on different fields, and frequently a different one in an interface than in the search page a person would use. A state office's business search — the Delaware Division of Corporations' entity search, say, or a secretary of state's business filings portal — can also behave differently from an aggregator such as OpenCorporates that re-serves the same records under its own match rules. The difference is rarely documented as a difference.
Five probes settle it for any source, and they take minutes:
- Exact stored form.Search a name you know is registered, typed exactly as the register writes it. If that returns the row, the source can match on the stored string.
- Legal-form suffix removed.Run the same name with Inc., LLC, or Co. taken off. It reveals whether the index strips legal-form tokens or treats them as part of the name.
- Punctuation and ampersand.Run it once with commas and periods removed, then once with the ampersand written out as "and." It reveals which characters the index treats as significant.
- Case and spacing.Run it in lower case, then with doubled spaces. It reveals whether case and whitespace are folded before matching.
- Identifier plus filters.Look the entity up by number with the jurisdiction attached, then repeat with the status filter cleared. It reveals whether an identifier needs a jurisdiction and whether the interface applies a filter you never asked for.
| Probe | What it reveals |
|---|
Five Probes for Source Matching
- Known name missing first word
- Name with and without entity suffix
- Known dissolved entity
- Known trade name
- Identifier with and without prefix
Record the answers with the source. Every later result on that source is interpretable, and the same probes rerun after a platform change catch a silent behavior shift.
Normalize, but keep the original
Users type names the way they appear on invoices. The register holds them the way a filer typed them. Bridging the two means a normalization step, and normalization is where searches quietly become wrong.
Safe vs Unsafe Normalization
Safe
- Whitespace
- Trim, collapse doubles
- Case
- Fold if insensitive
- Suffix
- Offer two searches
- Ampersands
- Leave as typed
- Punctuation
- Keep intact
- Noise words
- Keep them
Unsafe
- Whitespace
- Silent stripping
- Case
- Assume always
- Suffix
- Drop silently
- Ampersands
- Expand unannounced
- Punctuation
- Strip quietly
- Noise words
- Drop by judgment
What normalization is: folding case to a single form for comparison, collapsing runs of whitespace, standardizing the separators between tokens, deciding explicitly whether legal-form tokens and suffixes stay in the match key or come out, and deciding what an ampersand becomes. Each of those is a choice you make and record, not a cleaning step you run.
What is not: stripping punctuation silently, expanding or contracting ampersands without saying so, or dropping words you judged to be noise. The business name search pillar is about how much meaning hides in exactly those characters.
Keep the raw query alongside the normalized one, run more than one form where the source is exact-matching, and show the user which form produced each result. That last part is what makes a surprising result explicable rather than suspicious.
An illustration of the whole problem. A user types Harbor & Vale, Inc. The register stores HARBOR & VALE INC. Fold the case, collapse the whitespace, and drop the comma and the period, and the form harbor & vale inc matches on an exact-matching index. Expand the ampersand into "and" and that form returns nothing. Run both forms, keep the raw query, and tell the user which one produced the row.
Search by identifier where you can
Identifier search is exact, unambiguous, and immune to every name problem above. Prefer it whenever a number is available, store identifiers as text with prefixes and leading zeros intact, and remember that a number means nothing without its jurisdiction, which is the point entity number lookup keeps returning to.
Identifier Search Handling
- Prefer identifier when a number exists
- Store as text, keep prefixes and zeros
- Always attach the jurisdiction
- Try a small set of variants
- Report which variant matched
Where your users supply numbers from documents, expect the numbers to arrive mangled by spreadsheets and by transcription. A search interface that tries a small set of variants and reports which one matched saves an enormous amount of support time.
Present results honestly
Show the source and the retrieval date on every row. A value is a statement by one office at one moment, and a result set with no provenance invites the reader to assume currency it does not have.
Presenting Results Honestly
- Show source and retrieval date per row
- Distinguish no match from no answer
- Never merge jurisdictions into one row
- Show status string as written
Distinguish no match from no answer. If a request failed, was throttled, or timed out, say so in the result rather than returning an empty list. A user who sees "nothing found" when the truth is "we could not ask" has been misinformed by your interface, not by the register.
Do not merge jurisdictions into one row. The same company holds a different identifier in every state it registered in, and merging them loses the jurisdiction that gives each record meaning. Show them as separate registrations of one entity, in the shape state entity directories describes.
Show the status string as the office wrote it. Any mapping you apply is your vocabulary, not the office's, and the original belongs beside it for the reasons registration status sets out.
Federal sources search differently
A federal filing system is usually far more queryable than a state office. EDGAR full-text search, run by the SEC, searches the contents of filings rather than matching a name against an index of entities, and the EDGAR application programming interfaces are documented separately from the web interface.
Federal vs State Search
Federal
- Query power
- Full-text, APIs
- Coverage
- Only filers
- Demo look
- Excellent
- Private counterparty
- Often missed
State
- Query power
- Name index
- Coverage
- All registrations
- Demo look
- Ordinary
- Private counterparty
- Usually found
Where a jurisdiction publishes an open-data catalog — the federal catalog at Data.gov, or a state's own portal — its own guidance on APIs describes the same distinction from the publisher's side.
The catch is coverage, not capability. Federal filing systems hold only the companies that file with the relevant regulator, a small fraction of registered businesses.
A search that reaches only federal sources will look excellent in a demonstration and fail on an ordinary private counterparty.
Logging a search so it can be defended
Store the raw query, the normalized query, the source, the filters in force, the timestamp, and the outcome. A nil result is a statement about one index on one day under one query, and without the query recorded it is reproducible by nobody, including the person who ran it.
Where the search feeds a decision, that log is what lets a colleague repeat it and get the same answer, and it is the same discipline the wider public data APIs pillar applies to every value the pipeline holds.
Common questions
Should I offer fuzzy matching?
Only if you can show the user what was matched and why. Fuzzy matching that silently returns near-misses turns a legal-name search into a suggestion engine, and the user cannot tell which they received.
How many query variants should I run per search?
Enough to cover the suffix and the punctuation, which is usually two or three. Beyond that you are generating result noise, and the better move is to let the user narrow by formation date, city, or entity type.
Can I cache search results?
Cache the fact of a search, freely. Cache the status field for a decision, never. The distinction is between remembering what you asked and pretending you asked recently.
A user insists the company exists and our search finds nothing. What now?
Check the probes first: match type, status filter, and index. Then check the jurisdiction. In most cases the register is behaving exactly as documented and the query never had a chance of matching.
