Named Entity Recognition (NER) and Entity Linking are two complementary NLP tasks that form the foundation of knowledge graph construction. NER identifies and classifies mentions of real-world entities in unstructured text—people, organizations, locations, dates, products—while entity linking resolves those mentions to specific entries in a knowledge base, handling the ambiguity inherent in natural language.
Consider the sentence: “Apple announced that Tim Cook will visit the new Paris store.” NER identifies three entities: [Apple]ORG, [Tim Cook]PERSON, [Paris]LOCATION. But which “Apple”? Which “Paris”? Entity linking disambiguates: “Apple” maps to Apple Inc. (Q312 in Wikidata), not the fruit. “Paris” maps to Paris, France (Q90), not Paris, Texas or Paris Hilton.
For AI engineers building knowledge graphs and GraphRAG systems, NER and entity linking are the critical first steps in the pipeline. They transform unstructured documents into structured knowledge—extracting the nodes that will populate your graph and resolving aliases so that “NYC”, “New York”, and “the Big Apple” all point to the same entity.
NER is a sequence labeling task: given a sequence of tokens, assign each token an entity label or mark it as non-entity. The standard labeling scheme is BIO (Beginning, Inside, Outside):
Token: Jim bought 300 shares of Acme Corp. in 2006
Label: B-PER O O O O B-ORG I-ORG O B-DATE
B- = Beginning of entity
I- = Inside (continuation) of entity
O = Outside (not an entity)Common entity types in standard NER models include:
The evolution of NER methods mirrors the broader NLP landscape:
Rule-based systems: Pattern matching with regular expressions and gazetteers (dictionaries of known entities). Fast and precise for narrow domains but brittle and expensive to maintain. Still useful for structured data like phone numbers, email addresses, and product codes.
Statistical models (CRF): Conditional Random Fields model the probability of label sequences given input features (word shape, capitalization, part-of-speech tags). The classic approach before deep learning, still found in production systems like early spaCy models.
Deep learning (BiLSTM-CRF): Bidirectional LSTMs capture contextual information in both directions, with a CRF layer on top for structured prediction. Significantly improved accuracy over statistical methods, especially for rare entities.
Models like BERT, RoBERTa, and DeBERTa have become the standard for NER. Fine-tuned on labeled datasets like CoNLL-2003 or OntoNotes, they achieve F1 scores above 93% on standard benchmarks. Their bidirectional attention mechanism captures long-range context, handling ambiguous entities far better than previous approaches.
LLM-based extraction: Modern LLMs (GPT-4, Claude, Gemini) can perform NER via structured prompting without any fine-tuning. This is particularly powerful for domain-specific entity types that don’t exist in standard NER models:
# LLM-based NER via structured prompting
prompt = """Extract all named entities from the text below.
Return JSON with entity types: HOTEL, COMPANY, CITY, AMENITY.
Text: "The Marriott Marquis in Times Square features a revolving
restaurant managed by Marriott International."
Output:
{
"entities": [
{"text": "Marriott Marquis", "type": "HOTEL", "start": 4, "end": 20},
{"text": "Times Square", "type": "LOCATION", "start": 24, "end": 36},
{"text": "Marriott International", "type": "COMPANY", "start": 80, "end": 102}
]
}"""Entity linking (also called entity disambiguation or entity resolution) maps the surface forms detected by NER to unique entries in a knowledge base. This is a three-step process:
A critical aspect of entity linking is NIL prediction—recognizing when a mention refers to an entity not yet in the knowledge base. For domain-specific applications, many entities won’t exist in general knowledge bases like Wikidata. The system must decide whether to link to an existing entry or flag as a new entity requiring creation.
Beyond linking to external knowledge bases, entity resolution handles the problem of recognizing that different text mentions refer to the same entity within and across documents:
spaCy provides a production-ready NER pipeline with entity linking capabilities:
import spacy
# Load a transformer-based model
nlp = spacy.load("en_core_web_trf")
text = "Google announced that Sundar Pichai will open a new office in Berlin."
doc = nlp(text)
# Extract named entities
for ent in doc.ents:
print(f"{ent.text:<20} {ent.label_:<10} [{ent.start_char}:{ent.end_char}]")
# Output:
# Google ORG [0:6]
# Sundar Pichai PERSON [22:35]
# Berlin GPE [64:70]
# For entity linking, spaCy provides a pipeline component:
# nlp.add_pipe("entity_linker", config={"kb": knowledge_base})
# This maps detected entities to your custom knowledge baseIn a GraphRAG construction pipeline, NER and entity linking feed directly into graph building:
The quality of your knowledge graph is directly bounded by the quality of your NER and entity linking. False positives create noise nodes; missed entities create gaps; failed disambiguation creates duplicate nodes that fragment your graph.
NER and entity linking are the essential first steps in building any knowledge graph from unstructured text. When processing thousands of hotel descriptions, property documents, or customer reviews, NER identifies every mention of hotels, amenities, locations, and companies. Entity linking then ensures that “the Ritz-Carlton”, “Ritz Carlton”, and “RC Downtown” all map to the same node, creating a clean, deduplicated graph.
Search engines use NER to understand query intent. When a user searches “best restaurants near Hilton Chicago”, NER identifies “Hilton Chicago” as a hotel entity, enabling the system to look up its exact location rather than doing a fuzzy text match. Entity linking ensures the correct Hilton property is identified, not a different Hilton in a different city.
Enterprise document processing pipelines use NER to extract structured data from contracts, invoices, reports, and emails. Extracting company names, dates, monetary amounts, and clause references from legal documents—and linking them to the correct entities in a CRM or ERP system—automates what was previously hours of manual data entry.
Media companies and content platforms use NER to automatically tag articles, videos, and social posts with relevant entities. This enables entity-based navigation (“show me all content mentioning Tesla”), trend analysis, and content recommendation. Entity linking ensures consistent tagging even when authors use different spellings or abbreviations.
Financial institutions use NER and entity linking to monitor news feeds and regulatory filings for mentions of entities in their portfolio or sanctions lists. Entity linking is critical here: “Bank of China” and “BOC” must be resolved to the same entity, and “China Construction Bank” must be correctly distinguished from “China Merchants Bank” despite similar names.