📝 Part-of-Speech (POS) Tagging

Description

< What is it? >

• Meaning: Part-of-Speech (POS) tagging assigns a grammatical category to each word in a sentence, such as noun, verb, adjective, or adverb. It is a token-labeling task.

• Example: “The young cat sleeps peacefully.”

  Word or token	POS tag	Meaning
  The	Determiner (DET)	Introduces the noun
  young	Adjective (ADJ)	Describes the cat
  cat	Noun (NOUN)	Names an animal
  sleeps	Verb (VERB)	Describes an action
  peacefully	Adverb (ADV)	Describes how it sleeps
  .	Punctuation (PUNCT)	Ends the sentence

  These labels follow the Universal Dependencies POS tag set. Other tag sets may use different labels.

Key points

< POS annotations in spaCy >

• Meaning: POS annotations are grammatical labels attached to tokens. POS tagging is the process; POS annotations are the resulting labels.

• Categories: token.pos_ provides broad categories such as determiner, proper noun, and verb. The table below lists the Universal POS categories; spaCy also supports SPACE for whitespace tokens.

  POS tag	Category	Example
  DET	Determiner	the, a, this book
  PROPN	Proper noun	Alice, London, Google
  NOUN	Common noun	cat, book, city
  PRON	Pronoun	I, she, they
  VERB	Verb	run, eat, write
  AUX	Auxiliary verb	She is running
  ADJ	Adjective	a beautiful picture
  ADV	Adverb	runs quickly
  ADP	Adposition, such as a preposition	in, on, under
  CCONJ	Coordinating conjunction	and, or, but
  SCONJ	Subordinating conjunction	because, although
  NUM	Numeral	three, 42
  PART	Particle	to in “to run”
  INTJ	Interjection	wow, oh
  PUNCT	Punctuation	., ,, !
  SYM	Symbol	$, +
  X	Other	A token that cannot be assigned another category

• Example: “Alice reads a book.”

  Alice → PROPN  (proper noun)
  reads → VERB   (verb)
  a     → DET    (determiner)
  book  → NOUN   (common noun)
  .     → PUNCT  (punctuation)

• spaCy attributes: the trailing underscore gives the readable string. Without it, token.pos and token.tag return integer identifiers. See the spaCy Token documentation.

  Attribute	Meaning	Example for cats
  token.pos_	Broad grammatical category	NOUN
  token.tag_	More detailed, language-specific tag	NNS — plural noun
  token.lemma_	Dictionary base form; a separate annotation, not a POS tag	cat

< Context determines the tag >

• Context: the same word can have different grammatical categories depending on its use in a sentence.

• Example: in “I book a flight,” book is a verb. In “I read a book,” book is a noun.

< What is POS tagging used for? >

• Grammatical analysis: tags describe word categories. Finding relationships such as subject and object requires additional syntactic analysis.
• Information extraction: tags can help identify useful patterns, such as an adjective followed by a noun in “young cat.”

Comparison

< POS tagging vs. Named Entity Recognition >

• Different labels: POS tagging identifies grammatical categories, while Named Entity Recognition (NER) identifies entity spans and types such as people, organizations, and locations.

  Task	What it identifies	Example: “Alice works at Google”
  POS tagging	Grammatical categories of words	Alice and Google are proper nouns (PROPN); works is a verb (VERB)
  NER	Named entities and their types	Alice is a person; Google is an organization

Implementation

< Read POS annotations with spaCy >

• Setup: install spaCy and its trained English pipeline.

  python -m pip install spacy
  python -m spacy download en_core_web_sm

• Example: process the complete sentence, then inspect each token's text, broad POS, detailed tag, and lemma.

  import spacy
  
  nlp = spacy.load("en_core_web_sm")
  doc = nlp("The cats were running.")
  
  for token in doc:
      print(token.text, token.pos_, token.tag_, token.lemma_)

  Illustrative output (not executed here; predictions depend on the pipeline):

  The DET DT the
  cats NOUN NNS cat
  were AUX VBD be
  running VERB VBG run
  . PUNCT . .

  For running, VERB is the broad category, VBG identifies a gerund or present participle in the English tag scheme, and run is the lemma.

• Pipeline requirement: spacy.blank("en") alone tokenizes text but does not predict POS tags. A suitable trained pipeline supplies these annotations. POS information also supports rule-based lemmatization. See spaCy's linguistic features guide.

Related ideas

• Named Entity Recognition (NER) labels entity spans in text.
• Classification explains predicting category labels.

Reference

• spaCy: Linguistic features
• spaCy: Token attributes
• Universal POS tags (Universal Dependencies)
• CoNLL-U format explains how POS tags and syntactic annotations are stored.