Large Documents
Long inputs are split, extracted and merged for you. This page says what to expect and how the result is checked.
You run a long input the same way as a short one (Running extractions). Every field: API reference.
Extraction Strategies
2kw.ai picks the strategy from the size of the input. You send the same request in every case.
| Strategy | What happens |
|---|---|
| Single-shot | Short inputs are processed in one call |
| Chunked | Longer inputs are split into parts that are extracted one after the other. The results are merged, deduplicated and validated, and each part knows what the earlier parts found, so an item is not extracted twice |
| Async | Very long inputs are processed in the background; poll for the result |
Estimate the tokens first to see which strategy an input will take: the response names it in strategy (SINGLE_SHOT, CHUNKED or ASYNC). Merging, deduplication and validation need no configuration and work with any schema.
Deduplication
When an extraction is chunked, the same item can appear in more than one part. 2kw.ai groups items that share their identifier fields (like sourceFile, partNumber or name) into one: gaps are filled, exact duplicates removed and sequential numbering fixed.
The resultMetadata.dedup object tells you what happened:
| Field | Description |
|---|---|
applied | Whether dedup was needed |
itemsRemoved | Number of exact duplicates removed |
itemsMerged | Number of partial items merged into one |
resequenced | Whether sequential numbering was fixed |
originalItemCount | Items before dedup |
finalItemCount | Items after dedup |
Dedup just works for most schemas
If your array items have clearly named identifier fields (sourceFile, partId, name, etc.), dedup handles everything automatically. For optional fields, prefer nullable types ("type": ["string", "null"]) over defaulting to "": a null reads as "not found in this part", an empty string as an answer.
Key Field Auto-Detection
Items are matched by their identifier fields. When keyFields is not set in your extraction config, 2kw.ai takes the fields whose names read like identifiers: id, name or key, or names that end in id, name, file, key, code or number, or begin with source or file. The same fields are weighted higher in grounding scores.
If that picks the wrong fields, or misses yours, set keyFields explicitly in your extraction config.
Post-Merge Validation
After deduplication, a chunked extraction goes through a validation pass that catches two types of issues:
Hallucination removal. Models sometimes invent items near the edge of a part, for example a file name that does not exist in the source. The validator checks each item's literal reference fields (like sourceFile) against the input. If a filename appears nowhere in the source, the item is removed.
Required field restoration. When results are merged, required top-level fields (like date or inquiryNumber) can be dropped. The validator checks the schema's required array and inserts null for any missing required field, so the absence is explicit rather than silent.
The resultMetadata.postValidation object tells you what happened:
| Field | Description |
|---|---|
applied | Whether post-merge validation ran |
hallucinatedItemsRemoved | Number of items removed because their identifier wasn't found in the source |
requiredFieldsRestored | Number of required fields that were missing and filled with null |
removedItemKeys | The identifier values of removed items (for debugging) |
Hallucination detection uses literal fields
By default, only file-reference fields (sourceFile, fileName, etc.) are checked against the source text. These fields should appear verbatim in the input. Derived values like summaries or computed measurements are never checked — they wouldn't be found via text matching even when correct.
To mark additional fields as literal references for post-merge validation, use the literal field hint in your extraction config. Grounding scoring ignores the hint, so a literal field scores 0.5 as unverifiable; do not set it on key fields:
"fieldHints": {
"sourceFile": { "scoring": "literal" },
"documentId": { "scoring": "literal" }
}