chunk_tool

Splits a file into a directory of randomly-named chunks, and merges those chunks back into the original file. Chunk order is not stored anywhere explicit — it's reconstructed at merge time from a password-derived hash chain, so the chunks on disk carry no visible sequence information. Chunks are also written to disk in a shuffled order, so filesystem metadata (timestamps, inode allocation order) doesn't leak the real sequence either.

Usage

chunk_tool.py split <file> <directory> <size>
chunk_tool.py merge <file> <directory>

• <size> is a number optionally followed by a unit: b, k/kb, m/mb, g/gb, t/tb (e.g. 500b, 50k, 100MB, 1.5gb). No unit means MB (e.g. 50 == 50MB).
• Both commands prompt for a password on stdin (via getpass, so it isn't echoed or stored in shell history).

Examples

# Split a file into ~50MB chunks
./chunk_tool.py split big_file.dat ./chunks 50mb

# Merge the chunks back together
./chunk_tool.py merge restored_file.dat ./chunks

How it works

Each chunk is written under a random UUID filename in the output directory, and the files are created in a shuffled order — not the logical order of the data — so directory listings and filesystem metadata don't betray the sequence.

Every chunk file is fully encrypted with AES-256-GCM. The entire on-disk content — including the ordering headers and length metadata — is ciphertext. Without the password, nothing in the chunk files is readable.

Encryption uses a two-layer key derivation:

1. Master key — one scrypt call per chunk, keyed on the password and the chunk's UUID (the filename). This is the expensive, memory-hard step that makes offline brute-forcing costly.

2. Per-purpose keys — two fast HKDF expansions from the master key:

   • A 32-byte AES-256 key + 12-byte nonce for encrypting/decrypting the entire chunk record (header, length, and data).
   • A 32-byte header + 8-byte length mask for the password-derived hash chain that encodes chunk ordering.

During split, each chunk's plaintext record (32-byte predecessor header + 8-byte XOR-masked length + padded data) is built, then encrypted as a single AES-256-GCM ciphertext with the chunk's UUID as associated data.

During merge, each chunk is first decrypted (the master key only needs the password and the UUID filename — no content dependency), producing the plaintext record. The ordering chain is then reconstructed by computing scrypt-derived chain material from the now-known plaintext content and matching headers, exactly as before. The authenticated encryption also provides tamper detection: any modification to the ciphertext causes decryption to fail immediately.

scrypt is used instead of a plain fast hash deliberately: it's a memory-hard KDF, so each link costs real time and memory to compute. A legitimate merge only pays this cost once per chunk; an attacker without the password guessing their way through candidate passwords pays it for every guess, which makes offline brute-forcing meaningfully more expensive than a single unsalted SHA-256 call would.

Password semantics

The password encrypts everything. Each chunk file is fully encrypted with AES-256-GCM, keyed from the password. Without the password:

• Chunk contents are confidential — the on-disk ciphertext reveals nothing about the original data.
• Chunk ordering is unrecoverable — the hash chain that encodes the sequence is derived from the password.
• Tampering is detected — both the authenticated encryption (GCM tag) and the content-bound hash chain catch any modification.

With the correct password:

• Each chunk decrypts successfully (GCM authentication verifies integrity).
• The hash chain reconstructs the correct order from first to last.
• The original file is reassembled exactly.

With the wrong password (or corrupted/missing chunks), merge fails loudly: decryption raises an error, or the hash-chain matching breaks — either no chunk looks like a valid first chunk, multiple chunks claim the same predecessor, or the reconstructed chain doesn't cover every chunk.

Requirements

Python 3 with the cryptography package (pip install cryptography). Standard library modules: getpass, hashlib, os, random, re, sys, uuid. Requires a Python build whose OpenSSL supports scrypt (true on most modern systems); if hashlib.scrypt raises ValueError: unsupported hash type scrypt, the Python interpreter's OpenSSL is too old.