Knowledge Graph Ingestion Prioritizes Curation, Identity Decisions

Building a robust knowledge graph involves more than just extracting entities and relationships; the crucial step is deciding their identity and how they connect. This research highlights that identity decisions are fundamentally destructive if incorrect, as merging records erroneously can lead to irreversible data corruption. The paper describes a specific ingestion and ontology-tagging layer used in a production knowledge graph containing over half a million entities from government documents. It introduces a "record-identity ladder" that determines sameness based on identifiers, names, and context, rather than just name similarity. A coarser canonical-name key is then applied for automatic merging of records with exact matches. The authors advocate for a policy where entity resolution only flags candidates for human review, rather than automating merges. This approach stems from an incident where an erroneous merge corrupted records, underscoring the need for careful curation before connecting data to prevent undetectable errors and maintain data integrity.