Endian Library

Abstract

Boost.Endian provides facilities to manipulate the byte ordering of integers.

• The primary use case is binary I/O of integers for portable exchange with other systems, via either file or network transmission.
   
• A secondary use case is minimizing storage size via integers of sizes and/or alignments not supported by the built-in types. Integers 1, 2, 3, 4, 5, 6, 7, and 8 bytes in length are supported.
   
• Two distinct approaches to byte ordering are provided. Each approach has a long history of successful use, and each approach has use cases where it is superior to the other approach.
   
  • The explicit approach provides conversion functions to reorder bytes. All four combinations of non-modifying or modifying, and unconditional or conditional, functions are provided.
     
  • The implicit approach provides integer types that mimic the built-in integers, implicitly handling all byte reordering.

Introduction to endianness

Consider the following code:

int16_t i = 0x0102;
FILE * file = fopen("test", "wb");   // MUST BE BINARY
fwrite(&i, sizeof(int16_t), 1, file);
fclose(file);

On a modern Apple, Linux, or Windows computer with an Intel CPU, a hex dump of the "test" output file produces:

0201

On a Oracle/Sun Solaris computer with a SPARC CPU, a hex dump of the "test" output file produces:

0102

What's happening here is that Intel CPU's order the bytes of an integer with the least-significant byte first, while SPARC CPU's place the most-significant byte first. Some CPU's, such as the PowerPC, allow the operating system to choose which ordering applies.

Most-significant-byte-first ordering is traditionally called "big endian" ordering and the least-significant-byte-first is traditionally called "little-endian" ordering. The names are derived from Jonathan Swift's satirical novel Gulliver’s Travels, where rival kingdoms opened their soft-boiled eggs at different ends.

See the Wikipedia's Endianness article for an extensive discussion of endianness.

Except for reading an occasional core dump, most programmers can ignore endianness. But when exchanging binary integers with other computer systems, whether by file transfers or over a network, programmers have to deal with endianness.

Introduction to the Boost.Endian library

The Boost.Endian library provides facilities to deal with integer endianness.

The library provides two approaches to dealing with integer endianness:

Endian conversions for native integers - The application uses the built-in integer types, and calls the provided conversion functions to convert byte ordering as needed. Both mutating and non-mutating conversions are supplied, and each comes in unconditional and conditional variants. This approach is simple and usually more efficient, but is less flexible in terms of size and alignment, can be hard-to-manage and error-prone in code with many logical paths involving endianness transitions, and can foster very hard to debug logic errors.

Endian integer types - The application uses the provided endian types which mimic the built-in integer types. For example, big32_t or little64_t. This approach is also simple, but can be less efficient. Types with lengths of 1-8 bytes are supported, rather than just 1, 2, 4, and 8 bytes. Strict alignment requirements are avoided, and this may allow data to be packed more tightly.

Boost Endian is a header-only library.

Acknowledgements

Comments and suggestions were received from Benaka Moorthi, Christopher Kohlhoff, Cliff Green, Gennaro Proto, Giovanni Piero Deretta, dizzy, Jeff Flinn, John Maddock, Kim Barrett, Marsh Ray, Martin Bonner, Matias Capeletto, Neil Mayhew, Phil Endecott, Rene Rivera, Roland Schwarz, Scott McMurray, Sebastian Redl, Tomas Puverle, Vincente Botet, and Yuval Ronen.

Last revised: 04 September, 2011

© Copyright Beman Dawes, 2011

Distributed under the Boost Software License, Version 1.0. See www.boost.org/ LICENSE_1_0.txt