> ## Documentation Index
> Fetch the complete documentation index at: https://docs.myspellchecker.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Word Validation

> This document describes the word validation system in mySpellChecker, which is Layer 2 of the validation pipeline.

Layer 2 extends syllable validation to handle multi-syllable words, including dictionary lookup, compound validation, OOV recovery, and context-aware ranking.

## Overview

Word validation extends syllable validation to handle multi-syllable words. It includes:

* Dictionary lookup for complete words
* Compound word validation (SymSpell)
* Productive reduplication validation (ReduplicationEngine)
* Compound word synthesis via DP segmentation (CompoundResolver)
* OOV (Out-of-Vocabulary) recovery via morphological analysis
* Context-aware suggestion ranking
* Morpheme-level suggestion correction (MorphemeSuggestionStrategy)
* Colloquial variant detection

## Architecture

```python theme={null}
  +-------------------+
  | Input Text        |
  +---------+---------+
            |
            v
  +-------------------------------+
  | Layer 1: Syllable Validation  |
  |   See syllable-validation.md  |
  +---------------+---------------+
                  |
                  | syllables valid
                  v
  +-------------------------------------------+
  | Layer 2: Word Validation (THIS MODULE)    |
  |                                           |
  |   +---------------------+                 |
  |   | Dictionary lookup   |                 |
  |   +---------+-----------+                 |
  |             |                             |
  |             v                             |
  |   +---------------------+                 |
  |   | Compound validation |  (SymSpell)     |
  |   +---------+-----------+                 |
  |             |                             |
  |             v                             |
  |   +----------------------------+          |
  |   | Reduplication validation   |   |
  |   +---------+------------------+          |
  |             |                             |
  |             v                             |
  |   +----------------------------+          |
  |   | Compound synthesis (DP)    |   |
  |   +---------+------------------+          |
  |             |                             |
  |             v                             |
  |   +-----------------------------+         |
  |   | Context-aware suggestions   |         |
  |   | (incl. morpheme correction) |         |
  |   +-----------------------------+         |
  +-------------------------------------------+
```

## WordValidator

### Initialization

```python theme={null}
from myspellchecker.core.validators import WordValidator
from myspellchecker.core.config import SpellCheckerConfig

validator = WordValidator(
    config=SpellCheckerConfig(),
    segmenter=segmenter,
    word_repository=provider,
    syllable_repository=provider,
    symspell=symspell,
    context_checker=context_checker,       # Optional
    suggestion_strategy=strategy,           # Optional
    reduplication_engine=redup_engine,      # Optional (Phase 1)
    compound_resolver=compound_resolver,   # Optional (Phase 2/3)
)
```

### Factory Method

```python theme={null}
from myspellchecker.core.validators import WordValidator

validator = WordValidator.create(
    word_repository=provider,
    syllable_repository=provider,
    segmenter=segmenter,
    symspell=symspell,
    config=config,
    context_checker=context_checker,
)
```

### Basic Usage

```python theme={null}
# Validate text and get errors
errors = validator.validate("မြန်မာနိုင်ငံသည်")

for error in errors:
    print(f"Error: {error.text} at position {error.position}")
    print(f"Type: {error.error_type}")
    print(f"Suggestions: {error.suggestions[:3]}")
    print(f"Confidence: {error.confidence}")
```

## Validation Process

### Step 1: Word Segmentation

Text is segmented into words using the configured segmenter:

```python theme={null}
words = segmenter.segment_words(text)
# ["မြန်မာ", "နိုင်ငံ", "သည်"]
```

### Step 2: Dictionary Lookup

Each word is checked against the word repository:

```python theme={null}
if word_repository.is_valid_word(word):
    # Valid word - check for colloquial variants
    pass
else:
    # Not found - continue to compound check
    pass
```

### Step 3: Compound Validation

Words not found directly may be valid compounds:

```python theme={null}
# Check if word splits into valid parts with no edits
compound_check = symspell.lookup_compound(word, max_edit_distance=0)

if compound_check and compound_check[0][1] == 0:
    # Valid compound word
    pass
```

### Step 4: Reduplication Validation

Words not found in the dictionary or via compound check may be productive
reduplications of known words:

```python theme={null}
# Check if word is a valid reduplication (e.g., ကောင်းကောင်း from ကောင်း)
if reduplication_engine.analyze(word, dict_check, freq_check, pos_check):
    # Valid reduplication - accept without error
    pass
```

Supported patterns:

* **AA**: Simple repetition (ကောင်းကောင်း "well")
* **AABB**: Each syllable doubles (သေသေချာချာ "carefully")
* **ABAB**: Whole word repeats (ခဏခဏ "frequently")
* **RHYME**: Known rhyme pairs from `grammar/patterns.py`

Safeguards: base must be in dictionary, frequency >= 5, POS must be V/ADJ/ADV/N.

### Step 5: Compound Synthesis

Words not matching any previous check may be valid compounds formed from
known dictionary morphemes:

```python theme={null}
# Check if word splits into valid morphemes (e.g., ကျောင်းသား = ကျောင်း + သား)
if compound_resolver.resolve(word, dict_check, freq_check, pos_check):
    # Valid compound - accept without error
    pass
```

Uses dynamic programming for optimal segmentation. Allowed patterns: N+N, V+V,
N+V, V+N, ADJ+N. Blocked patterns: P+P, P+N, N+P.

Safeguards: all parts in dictionary, frequency >= 10 per morpheme, max 4 parts.

### Step 6: OOV Recovery (Morphology)

For unknown words, attempt morphological analysis:

```python theme={null}
from myspellchecker.text.morphology import analyze_word

# Analyze word structure
analysis = analyze_word(word, dictionary_check=is_valid)

if analysis.root and analysis.suffixes:
    # Recovered root: စား with suffixes: ['ခဲ့', 'သည်']
    # Generate suggestions from similar roots
    pass
```

### Step 7: Suggestion Generation

Suggestions are generated via the unified strategy pipeline:

```python theme={null}
# Uses multiple sources:
# - SymSpell edit distance suggestions
# - Morphology-based suggestions
# - Compound splitting suggestions
# - Morpheme-level corrections (NEW - fixes typos inside compounds/reduplications)
# - Context ranking from N-grams

suggestions = strategy.suggest(word, context)
```

### Step 8: Context Ranking

Suggestions are ranked using bidirectional context:

```python theme={null}
# Context from surrounding words
prev_word = words[i-1] if i > 0 else None
next_word = words[i+1] if i < len(words)-1 else None

# Context-aware suggestions via NgramContextChecker
context_suggestions = context_checker.suggest(
    prev_word=prev_word,
    current_word=word,
    next_word=next_word,
)
```

## OOV Recovery Details

### Morphological Analysis

The morphology module decomposes unknown words:

```python theme={null}
from myspellchecker.text.morphology import analyze_word, WordAnalysis

# Word: စားခဲ့သည် (ate - formal)
analysis = analyze_word("စားခဲ့သည်", dictionary_check=lambda w: w in dict)

# WordAnalysis:
#   original: "စားခဲ့သည်"
#   root: "စား" (eat)
#   suffixes: ["ခဲ့", "သည်"] (past tense, formal ending)
```

### Enhanced Suggestions

OOV recovery improves suggestion quality:

```python theme={null}
# Original word has typo in root
word = "စားခဲ့သည်"  # typo: စား → စာ

# Without morphology: generic edit-distance suggestions
# With morphology:
#   1. Finds root "စာ" (close to "စား")
#   2. Gets suggestions for root: ["စား", "စာ"]
#   3. Reconstructs: "စားခဲ့သည်", "စာခဲ့သည်"
#   4. Better suggestions for the full inflected form
```

## Colloquial Variant Detection

### Configuration

```python theme={null}
from myspellchecker.core.config import SpellCheckerConfig, ValidationConfig

config = SpellCheckerConfig(
    validation=ValidationConfig(
        colloquial_strictness="lenient"  # or "strict", "off"
    )
)
```

### Behavior by Mode

| Mode      | Behavior                                            |
| --------- | --------------------------------------------------- |
| `strict`  | Flag colloquial as error, suggest standard form     |
| `lenient` | Info note with low confidence, not counted as error |
| `off`     | No special handling                                 |

### Examples

```python theme={null}
from myspellchecker.text.phonetic_data import is_colloquial_variant, get_standard_forms

# Common colloquial forms
is_colloquial_variant("ကျနော်")  # True (colloquial for ကျွန်တော်)
get_standard_forms("ကျနော်")     # ["ကျွန်တော်"]

is_colloquial_variant("ကျွနော်")  # True (colloquial for ကျွန်တော်)
```

## Interface Segregation

WordValidator uses narrow repository interfaces:

### WordRepository Interface

```python theme={null}
class WordRepository(Protocol):
    def is_valid_word(self, word: str) -> bool: ...
    def get_word_frequency(self, word: str) -> int: ...
    def get_all_words(self) -> Iterator[Tuple[str, int]]: ...
```

### SyllableRepository Interface

```python theme={null}
class SyllableRepository(Protocol):
    def is_valid_syllable(self, syllable: str) -> bool: ...
    def get_syllable_frequency(self, syllable: str) -> int: ...
```

This design:

* Reduces coupling to full DictionaryProvider
* Makes testing easier with minimal mocks
* Allows different storage backends

## Error Types

WordValidator returns `WordError` objects:

```python theme={null}
from myspellchecker.core.response import WordError

class WordError(Error):
    text: str           # The invalid word
    position: int       # Character position in text
    suggestions: List[str]  # Correction suggestions
    confidence: float   # Error confidence (0.0-1.0)
    error_type: str     # "invalid_word", "colloquial_variant", etc.
    syllable_count: int # Number of syllables in word
```

### Error Type Values

| Type                 | Description                        |
| -------------------- | ---------------------------------- |
| `invalid_word`       | Unknown word, not in dictionary    |
| `colloquial_variant` | Colloquial spelling (strict mode)  |
| `colloquial_info`    | Colloquial spelling (lenient mode) |

## Configuration Options

### Via SpellCheckerConfig

```python theme={null}
from myspellchecker.core.config import SpellCheckerConfig

config = SpellCheckerConfig(
    # Suggestion settings
    max_suggestions=5,
    max_edit_distance=2,

    # Feature flags
    use_phonetic=True,        # Phonetic similarity
    use_context_checker=True, # N-gram context ranking

    # Validation settings
    validation=ValidationConfig(
        word_error_confidence=0.8,
        colloquial_strictness="lenient",
        allow_extended_myanmar=False,

        # Morphological synthesis        use_reduplication_validation=True,       # Accept valid reduplications
        reduplication_min_base_frequency=5,      # Min frequency for base word
        reduplication_cache_size=1024,            # Cache size

        use_compound_synthesis=True,             # Accept valid compounds
        compound_min_morpheme_frequency=10,      # Min frequency per morpheme
        compound_max_parts=4,                    # Max parts in compound
        compound_cache_size=1024,                # Cache size
    )
)
```

## Usage with SpellChecker

### Validation Level

```python theme={null}
from myspellchecker import SpellChecker
from myspellchecker.core.constants import ValidationLevel

checker = SpellChecker()

# Word-level validation (includes syllable + word)
result = checker.check(text, level=ValidationLevel.WORD)

# Syllable-only validation (faster, less coverage)
result = checker.check(text, level=ValidationLevel.SYLLABLE)
```

### Performance Comparison

| Level    | Speed    | Coverage           |
| -------- | -------- | ------------------ |
| SYLLABLE | Fast     | Most errors        |
| WORD     | Moderate | More comprehensive |

See [benchmarks](/development/benchmarks) for measured performance data.

## Suggestion Strategy

### Composite Strategy Pipeline

```python theme={null}
from myspellchecker.algorithms.suggestion_strategy import (
    CompositeSuggestionStrategy,
    SymSpellSuggestionStrategy,
    MorphologySuggestionStrategy,
    CompoundSuggestionStrategy,
)
from myspellchecker.algorithms.morpheme_suggestion_strategy import (
    MorphemeSuggestionStrategy,
)

strategy = CompositeSuggestionStrategy([
    SymSpellSuggestionStrategy(symspell),           # Edit-distance suggestions
    MorphologySuggestionStrategy(symspell, ...),    # OOV recovery
    CompoundSuggestionStrategy(symspell, ...),      # Word splitting/joining
    MorphemeSuggestionStrategy(                     # Morpheme-level correction        compound_resolver=resolver,
        reduplication_engine=engine,
        symspell=symspell,
        dictionary_check=dict_check,
    ),
])

# Unified suggestion with source attribution
result = strategy.suggest(word, context)
# Returns: SuggestionResult with ranked suggestions and sources
```

## Testing

### Unit Test Example

```python theme={null}
from myspellchecker.core.validators import WordValidator
from myspellchecker.providers import MemoryProvider

# Create minimal test provider
provider = MemoryProvider(
    syllables={"မြန်": 100, "မာ": 50},
    words={"မြန်မာ": 80}
)

validator = WordValidator.create(
    word_repository=provider,
    syllable_repository=provider,
    segmenter=MockSegmenter(),
    symspell=MockSymSpell(),
)

# Test validation
errors = validator.validate("မြန်မာ")
assert len(errors) == 0  # Valid word
```

## See Also

* [Syllable Validation](/core/syllable-validation) - Layer 1 validation
* [Context Checking](/features/context-checking) - N-gram context validation
* [SymSpell Algorithm](/algorithms/symspell) - Edit distance suggestions
* [Morphology Analysis](/features/morphology) - Word decomposition
* [Suggestion Strategy](/algorithms/suggestion-strategy) - Unified suggestion pipeline