Overview

mySpellChecker does not include a bundled dictionary. You build your own from a text corpus. The data pipeline reads raw text, segments it into syllables and words, calculates N-gram probabilities, and packages everything into an optimized SQLite database.

Quick Start

Build a dictionary from the command line:

# Build from a text corpus
myspellchecker build --input corpus.txt --output dict.db

# Build a sample database for testing
myspellchecker build --sample

Or use the Python API for programmatic control:

from myspellchecker.data_pipeline import Pipeline

pipeline = Pipeline()
pipeline.build_database(
    input_files=["corpus.txt"],
    database_path="dict.db",
)

Pipeline Architecture

Dictionary building pipeline: Input Files → Ingestion → Segmentation → Frequency Counting → Packaging → SQLite Dictionary

What the Database Contains

Table	Content	Purpose
`syllables`	Valid syllables + frequencies	Syllable validation
`words`	Words + frequencies + POS tags	Word validation, suggestions
`bigrams`	Word pair probabilities	Context checking (2-gram)
`trigrams`	Word triple probabilities	Context checking (3-gram)
`fourgrams`	4-word sequence probabilities	Context checking (4-gram)
`fivegrams`	5-word sequence probabilities	Context checking (5-gram)
`pos_unigrams/bigrams/trigrams`	POS tag probabilities	Grammar checking
`metadata`	Key-value build metadata	Build info, versioning
`processed_files`	Ingested file paths + timestamps	Incremental updates
`confusable_pairs`	Confusable word pairs + type + overlap	Confusable error detection
`compound_confusions`	Compound vs split-word pairs + PMI	Compound error detection
`collocations`	Word pair co-occurrence + PMI/NPMI	Collocation error detection
`register_tags`	Word register labels (formal/informal)	Register mixing detection

Next Steps

Corpus Format

Supported input formats and requirements

Building Dictionaries

CLI reference, Python API, config options

Optimization

DuckDB acceleration, Cython, parallel workers

Custom Dictionaries

Curated lexicons, domain-specific builds

Getting Started

Dictionary Building

Text Processing

Spelling & Text Checking

Grammar & NER

Performance & APIs

Customization

Integration & Deployment

Help & FAQ

Quick Start

Pipeline Architecture

What the Database Contains

Next Steps

Corpus Format

Building Dictionaries

Optimization

Custom Dictionaries

​Quick Start

​Pipeline Architecture

​What the Database Contains

​Next Steps

Corpus Format

Building Dictionaries

Optimization

Custom Dictionaries

Quick Start

Pipeline Architecture

What the Database Contains

Next Steps