Binary Representation in C

Why C is the King of Binary Programming

In my experience, no other language gives you as much direct control over binary data as C. When I need to parse a raw network packet, implement a file system driver, or optimize a protocol buffer layout at the bit level, C is my first choice. C's philosophy is that the programmer knows what they are doing, and it provides unfiltered access to memory, bits, and hardware registers. I have used C to build embedded system firmware, binary protocol analyzers, and custom compression algorithms over the years. The learning curve is steep, but the level of control is unmatched.

Bit Fields in C Structs

C's bit fields allow you to pack multiple integer values into a single machine word with bit-level precision. This is essential for implementing hardware registers, network protocol headers, and any data structure where memory is constrained:

I have used bit fields extensively for memory-mapped I/O in embedded systems. However, I always tell developers new to C that bit fields come with important caveats. The ordering of bits within a word is implementation-defined, meaning code that works on a little-endian x86 processor may behave differently on a big-endian ARM chip. Also, you cannot take the address of a bit field member with the & operator. For portable code, I prefer manual bit manipulation with shift and mask macros over bit fields for cross-platform projects.

Unions for Binary Data Access

Unions are one of C's most powerful tools for binary data manipulation. They let you overlay different data types on the same memory location, providing multiple interpretations of the same binary data:

In my binary protocol parsers, I use unions to overlay structured headers onto raw byte buffers. This gives me direct field access without manual offset calculations or memcpy calls. The technique is especially useful for parsing fixed-size packet headers where you have a known binary layout. Just be mindful of alignment and padding -- compilers may insert padding between struct members to satisfy alignment requirements, so I always use __attribute__((packed)) in GCC or #pragma pack(1) in MSVC for binary-mapped structures.

Pointer-Level Memory Operations

C gives you unrestricted access to memory through pointers, which is both its greatest strength and its most dangerous feature for binary work. I rely on pointer casting for quick type punning and byte-level memory inspection:

Pointer aliasing rules in C are strict, though. The uint32_t* cast above violates strict aliasing if the underlying memory was originally declared as unsigned char[]. Modern compilers with optimizations enabled can produce incorrect code in these cases. I avoid this by using memcpy for type punning in production code, which modern compilers optimize into the same efficient assembly as the pointer cast, but without the undefined behavior.

Binary File I/O with fread and fwrite

C's standard library provides fread() and fwrite() for efficient binary file I/O. These functions work directly on raw memory, making them ideal for reading and writing binary data structures:

When reading binary files in C, I always check the return value of fread() against the expected count. A short read indicates either end-of-file or an I/O error, and I use ferror() to distinguish between the two. For large binary files, I read in chunks rather than loading the entire file into memory. Also, I never assume the file was written on a machine with the same endianness -- a BMP file from Windows on x86 is little-endian, but the same data on a PowerPC Mac would be byte-swapped.

Bit Masking Techniques

Bit masking is the most fundamental skill in C binary programming. I use it constantly for extracting and setting individual bits within integers:

These macros have saved me countless hours of repetitive coding. The BITS() macro in particular is invaluable for extracting bit fields from hardware registers and protocol headers. I always use 1U instead of 1 in shift expressions to avoid signed integer overflow, which is undefined behavior in C. On 32-bit systems, shifting a signed int by 31 or more positions can trigger this UB, while the unsigned version is well-defined.

Integer Memory Layout and Endianness

Understanding how integers are laid out in memory is critical for binary programming in C. The same value can look completely different depending on the host architecture's endianness:

In my network protocol project, I use ntohl() and htonl() from <arpa/inet.h> to convert between network byte order (big-endian) and the host's native byte order. For file formats that use a specific endianness, I write byte-swapping macros that are conditionally compiled based on the target platform. The key rule I follow is: always store data in a well-defined byte order on disk or the wire, and convert to native order when reading into memory.

Best Practices for Binary Programming in C

After years of writing low-level C code, these are the habits I swear by:

• Always use unsigned types for bit manipulation. Signed integer shifts and bitwise operations have implementation-defined behavior for negative values. Stick with uint32_t, uint8_t, and other fixed-width unsigned types from <stdint.h>.
• Prefer memcpy over pointer casts for type punning. Pointer casting violates strict aliasing rules. memcpy produces the same optimized assembly and is standards-compliant.
• Use __attribute__((packed)) for binary-mapped structs. Without packed attributes, compilers insert padding that will break your binary layout when read from a file or network.
• Always check fread() return values. Binary file I/O can fail or read fewer bytes than expected. Never assume the read was successful without checking.
• Be explicit about sizes with sizeof. Don't hardcode sizes like 4 for int. Use sizeof(int) or better, use exact-width types from <stdint.h>.
• Handle endianness at the boundary. Convert from network/file byte order to native order when reading, and convert back when writing. Keep internal data in native order for performance.