From Analog to Digital — How Images, Audio, and Video Become Binary
When I first tried to understand how a photograph or a song could be stored as nothing but zeros and ones, it felt like magic. The reality, as I discovered, is a three-step pipeline that applies to every type of media: sampling, quantization, and encoding. These three steps form the backbone of all digital media, from the JPEG images on your phone to the MP3 files in your playlist and the 4K video streaming on your TV.
The key insight is that the real world is continuous — a sound wave has infinite possible values at infinite points in time — but computers work with discrete data. Sampling turns continuous time or space into discrete points. Quantization turns continuous values into discrete levels. Encoding turns those discrete levels into binary. I have walked through this pipeline countless times now, and I still find it elegant that the same three-step recipe applies whether you are digitizing a voice recording or a satellite image.
Key Insight: The analog-to-digital pipeline (sample, quantize, encode) is universal. Every digital representation of a real-world signal follows these exact three steps.
Sampling is the first step, and it is where we decide how often or how densely we capture the original signal. For audio, sampling means measuring the amplitude of a sound wave at regular time intervals. The standard CD-quality rate is 44,100 samples per second — 44.1 kHz. I remember the first time I understood the Nyquist-Shannon theorem: you must sample at least twice the highest frequency present in the signal to reconstruct it perfectly. Human hearing caps out around 20 kHz, so 44.1 kHz gives you just enough margin. If you sample too slowly, high frequencies wrap around and create aliasing — a nasty distortion that sounds like a screechy, metallic artifact in audio.
For images, sampling measures brightness and color at regular spatial intervals — that is what pixels are. A 300 PPI (pixels per inch) scan captures 300 distinct color samples per linear inch. The digital equivalent of aliasing in images is the moire pattern — those weird wavy artifacts you see when photographing a fine striped shirt or a chain-link fence on a low-resolution camera. For video, sampling also happens along the time axis: frames per second (FPS). Standard video uses 24, 30, or 60 fps. The Nyquist theorem applies here too — if you film a fast-spinning fan at 24 fps, it can appear to spin backwards, which is temporal aliasing in action.
Once we have our samples, each one needs to be rounded to the nearest representable value — this is quantization. In audio, quantization is controlled by bit depth. A 16-bit audio recording gives 65,536 possible amplitude levels per sample. Bump that to 24-bit and you get over 16 million levels. The error introduced by rounding the true analog value to the nearest discrete level is called quantization noise. I remember listening to a 16-bit versus an 8-bit version of the same recording and being shocked at the hissy, crunchy sound of the 8-bit version — that is quantization noise you can actually hear.
For images, quantization determines color depth. An 8-bit per channel image gives 256 shades of red, 256 of green, and 256 of blue, for a total of over 16 million colors (true color). Drop to 4-bit per channel and you get only 16 shades each — 4,096 total colors — which produces visible banding in gradients like a sunset sky. I learned this lesson the hard way when I compressed a photo too aggressively for a website and ended up with ugly color contours across what used to be a smooth blue sky. More quantization levels mean less noise and smoother gradations, but every additional bit doubles the file size.
Real-World Trade-Off: Every extra bit of quantization depth doubles the precision but also doubles the storage required. 16-bit audio is 256 times more precise than 8-bit audio — and 256 times larger.
After sampling and quantization, we have a raw stream of numbers. The next step is encoding that stream into a compact binary representation. Compression encoding falls into two camps: lossless and lossy. Lossless compression preserves every single bit of the original data. Common techniques include Run-Length Encoding (RLE), which replaces repeated values with a count-value pair; Huffman coding, which assigns shorter binary codes to more frequent values; LZW; and DEFLATE. You will find lossless compression in PNG images, FLAC audio files, and ZIP archives. I have used all of these — FLAC is my go-to for archiving CDs because I know the data is bit-perfect.
Lossy compression takes a different approach: it discards information that is perceptually less important to humans. JPEG uses the Discrete Cosine Transform (DCT) to separate a block of pixels into frequency components, then quantizes the high-frequency components more aggressively because the human eye is less sensitive to them. MP3 does the same thing with audio using psychoacoustic masking — frequencies that are masked by louder nearby frequencies get discarded or quantized roughly. AAC improves on MP3 with better psychoacoustic modeling. The trade-off is brutal: you can shrink a file by 90% or more, but each compression run loses irreversible detail. I archive my photos as RAW (no compression) and only export lossy JPEGs for sharing.
Video is the heavyweight champion of multimedia binary encoding because it is not just a stream of images — it is a sequence of images plus audio tracks, subtitles, metadata, and timing information all packed together. The raw data is staggering: a single minute of uncompressed 1080p video at 30 fps with 24-bit color runs around 10 GB. That is why video codecs are engineering marvels. The central trick is inter-frame compression: instead of storing every frame as a full image (I-frames, or key frames), the codec stores only the differences between frames using P-frames (predicted) and B-frames (bi-directional).
Modern codecs like H.264 and H.265 use motion compensation algorithms that divide each frame into blocks, then track where each block moves to in the next frame. Instead of storing a whole new frame, the codec stores a motion vector per block plus the residual error. The binary structure of a video stream is layered: Network Abstraction Layer (NAL) units carry the actual compressed data, while Sequence Parameter Sets (SPS) and Picture Parameter Sets (PPS) carry the decoding parameters like resolution, frame rate, and color profile. I have spent hours dissecting video bitstreams with hex editors, and seeing the NAL unit headers — each starting with the magic bytes 00 00 00 01 — is like seeing the skeleton of the video come into view.
Fun Fact: A 90-minute 4K movie without compression would be over 1.5 TB. With H.265 compression, it fits on a 64 GB file. That is a 96% reduction.
A video file is more than just a stream of encoded frames — it is a container that wraps video tracks, audio tracks, subtitles, chapter markers, and metadata into a single binary file. The three most common container formats are MP4, MKV (Matroska), and AVI. Each uses a different internal structure, and understanding the binary markers within them has helped me debug countless playback issues over the years. MP4 is based on the ISO Base Media File Format (ISOBMFF), which organizes everything into boxes (also called atoms).
Each box in an MP4 file starts with a 4-byte size field followed by a 4-byte type code. The type tells the player what the box contains. ftyp (file type box) identifies the file format and compatible brands — a player reads this first to decide if it can handle the file. moov (movie box) holds all the metadata: track information, codec parameters, duration, and the seek table. mdat (media data box) contains the actual audio and video samples. I once fixed a corrupted MP4 by manually editing the moov box offset — the file had the video data but the player could not find the metadata header. Understanding the binary structure turned what seemed like a dead file into a salvageable one.
Open the Binary Code Decoder in a new tab and enter some binary patterns to see the results instantly. All conversions happen in your browser — no data is sent to any server.