Confusable Detection

​

Overview

​

Confusable Sources

1. Database (confusable_pairs table): ~21K pairs mined from corpus data during the enrichment pipeline. Covers aspiration swaps, medial confusion, nasal endings, and tone mark variants.
2. YAML (rules/confusable_pairs.yaml): Curated fallback pairs that corpus mining cannot discover.

# Confusable pairs are loaded automatically when using SQLiteProvider
from myspellchecker.providers import SQLiteProvider

provider = SQLiteProvider(database_path="path/to/dictionary.db")
# provider.get_confusable_pairs("ကြောင်း") → {"ကျောင်း", ...}

​

Statistical Confusable Strategy

​

How It Works

1. For each word, look up known confusable variants from the database
2. Compare bidirectional bigram ratios:
   • Left context: P(variant | previous_word) vs P(word | previous_word)
   • Right context: P(next_word | variant) vs P(next_word | word)
3. If the combined ratio exceeds the threshold, flag as confusable error

​

Configuration

from myspellchecker.core.config import SpellCheckerConfig, ValidationConfig

# Statistical confusable is enabled by default when use_context_checker=True
config = SpellCheckerConfig(
    use_context_checker=True
)

​

Parameters

​

Confusable Compound Classifier

​

Features Used

• Word and variant frequencies (log-scaled)
• Bigram probabilities in both directions
• PMI (Pointwise Mutual Information) with neighbors
• POS tag compatibility
• Morphological pattern indicators (title suffixes, compound markers)

​

Configuration

# Requires an ONNX classifier model
# The model path is configured via the builder or config
from myspellchecker.core import SpellCheckerBuilder

checker = (
    SpellCheckerBuilder()
    .with_confusable_classifier("path/to/classifier.onnx")
    .build()
)

​

Confusable Semantic Strategy (MLM)

​

How It Works

1. For each valid word, generate confusable variants (character substitutions, medial swaps)
2. Filter variants to valid dictionary words
3. Use MLM predict_mask() to get logits for both the current word and the best variant
4. If the logit difference exceeds the threshold, flag as confusable error

​

Example

Input:  ကျွန်တော် ကြောင်းကို သွားတယ်။
                  ^^^^^^^^
- "ကြောင်း" (cat) is valid — passes all rule-based checks
- MLM predicts "ကျောင်း" (school) with much higher logit in "went to [X]" context
- logit_diff exceeds threshold → flagged as confusable_error
- Suggestion: ကျောင်း (school)

​

Configuration

from myspellchecker.core.config import SpellCheckerConfig, SemanticConfig

# Requires semantic model to be loaded
config = SpellCheckerConfig(
    semantic=SemanticConfig(
        model_path="path/to/semantic-model",
        enabled=True,
    )
)

​

Parameters

​

Guards and Filters

• Exempt pairs: Known pairs that should not trigger (e.g., particles with overlapping usage)
• Variant blocklist: Specific variants excluded from detection
• Medial-only pairs: Pairs differing only in medial consonants use adjusted thresholds
• Tone-only pairs: Pairs differing only in tone marks use higher thresholds
• DB suppression: Pairs explicitly suppressed in the database

​

Integration with SpellChecker

from myspellchecker.core import SpellCheckerBuilder, ConfigPresets

checker = (
    SpellCheckerBuilder()
    .with_config(ConfigPresets.ACCURATE)
    .build()
)

result = checker.check("ကျွန်တော် ကြောင်းကို သွားတယ်။")

for error in result.errors:
    if error.error_type == "confusable_error":
        print(f"{error.text} → {error.suggestions[0]}")

​

Performance

​

See Also

• Validation Strategies — Full strategy pipeline
• Homophones Detection — Sound-alike word detection
• Context Checking — N-gram context validation
• Semantic Checking — AI-powered MLM validation