Skip to main content
The DictionaryProvider interface abstracts the storage of vocabulary and frequency data. This allows mySpellChecker to run in different environments (server, desktop, mobile, embedded) by swapping the backend.

Provider Comparison

Types of Providers

1. SQLiteProvider (Default)

  • Storage: Disk-based (.db file).
  • Pros: Low memory footprint. Handles massive datasets (millions of N-grams).
  • Cons: Slightly slower than RAM (but mitigated by caching).
  • Use Case: General purpose, desktop apps, web servers with limited RAM.

2. MemoryProvider

  • Storage: RAM (Python Dictionary).
  • Pros: Fast (hash map lookup).
  • Cons: High memory usage. Long startup time (loading data into RAM).
  • Use Case: High-performance servers where RAM is abundant and latency must be minimized.
Memory Usage:

3. JSONProvider

  • Storage: JSON file.
  • Pros: Human-readable, easy to edit/debug.
  • Cons: Slow to load, memory inefficient for large datasets.
  • Use Case: Unit testing, small custom vocabularies, config files.
JSON Format:

4. CSVProvider

  • Storage: CSV/TSV file.
  • Pros: Easy to export from spreadsheets.
  • Cons: Similar performance issues to JSON for large data.
  • Use Case: Importing word lists from Excel/Sheets.

DictionaryProvider Interface

All providers implement the DictionaryProvider abstract base class:

Configuration

You can switch providers during initialization:

Caching

The SQLiteProvider uses an LRU cache to speed up repeated lookups. Configure via ProviderConfig:

Performance Benchmarks

Database Schema

If you wish to inspect the database directly or build one manually, here is the SQLite schema:

syllables

Stores unique syllables and their frequencies.
  • id: Integer (PK)
  • syllable: Text (Unique)
  • frequency: Integer

words

Stores valid words, frequency data, and POS tags.
  • id: Integer (PK)
  • word: Text (Unique)
  • syllable_count: Integer
  • frequency: Integer
  • pos_tag: Text (Optional, e.g., ‘N’, ‘V’)
  • is_curated: Integer (0 or 1, default 0)
  • inferred_pos: Text (POS tag from inference)
  • inferred_confidence: Real (confidence score)
  • inferred_source: Text (inference method used)

bigrams

Stores 2-word sequences and their probabilities.
  • id: Integer (PK)
  • word1_id: Integer (FK -> words.id)
  • word2_id: Integer (FK -> words.id)
  • probability: Real, P(w2 | w1)
  • count: Integer (Raw frequency)

trigrams

Stores 3-word sequences.
  • id: Integer (PK)
  • word1_id, word2_id, word3_id: Integers (FK -> words.id)
  • probability: Real, P(w3 | w1, w2)
  • count: Integer

processed_files

Tracks ingested files for incremental updates.
  • path: Text (PK)
  • mtime: Real
  • size: Integer