POD5 Format Overview¶

POD5 is the successor to FAST5 for storing Oxford Nanopore sequencing data. It uses Apache Arrow IPC (Feather V2) for efficient columnar storage with custom signal compression.

Design Goals¶

• Efficient access - Columnar format enables reading specific fields without loading entire reads
• Compact storage - VBZ compression achieves 60-80% reduction in signal data size
• Batch processing - Data organized in batches for parallel processing
• Self-describing - Schema embedded in file, extensible metadata

File Structure¶

A POD5 file contains:

1. Signature - Magic bytes identifying the file format
2. Signal Table - Compressed raw signal data (Arrow IPC)
3. Run Info Table - Sequencing run metadata (Arrow IPC)
4. Reads Table - Read records with references to signal (Arrow IPC)
5. Footer - FlatBuffer with table locations and metadata

See Container Structure for detailed layout.

Data Tables¶

Reads Table¶

One row per read containing:

Signal Table¶

Compressed signal chunks:

Run Info Table¶

One row per unique acquisition:

Signal Compression¶

Signal data uses the VBZ codec:

1. Delta encoding - Store differences between samples
2. Zigzag encoding - Map signed to unsigned integers
3. SVB16 - Variable-length encoding (1-2 bytes per value)
4. ZSTD - Final compression

See Compression for algorithm details.

Signal Analysis Algorithms¶

For barcode demultiplexing and signal analysis, see Segmentation Algorithms:

• LLR boundary detection - Find adapter regions using log-likelihood ratio
• T-test segmentation - Extract fingerprints from signal segments
• DTW classification - Dynamic time warping for barcode matching

Arrow Extension Types¶

POD5 uses custom Arrow extension types:

Version History¶

Comparison with FAST5¶