Skip to main content
Utility functions used internally by the data pipeline for pre-flight checks like disk space verification before building dictionaries.

Overview

Functions

check_disk_space

Verify that a disk partition has sufficient free space:

Usage

Behavior

Path Resolution

The function handles various path scenarios:

Error Conditions

Silent Failures

Some errors are logged but don’t raise exceptions:

Integration with Pipeline

Pre-Build Check

With Reporter

InsufficientStorageError

Custom exception for disk space issues:

Handling the Exception

Disk Usage Information

The function uses shutil.disk_usage() internally:

Best Practices

1. Check Before Large Operations

2. Check Multiple Locations

3. Graceful Degradation

See Also