I've now used the std::bitset for generating bit-sequences for the datatypes and then concatenated these bitsets to one big bitset. This then holds the data I need. I just have to check for little/big endian.

A file is a stream of bytes without any implied structure. If you want to load a list of binary blobs then you should store some additional metadata to restore the structure e.g., you could use the netstring format:

#!/usr/bin/env python
blocks = [b'\xa1\r\xa594\x92z\xf8\x16\xaa', b'xfbI\xfdqx|\xcd\xdb\x1b\xb3']

# save blocks
with open('blocks.netstring', 'wb') as output_file:
    for blob in blocks:
        # [len]":"[string]","
        output_file.write(str(len(blob)).encode())
        output_file.write(b":")
        output_file.write(blob)
        output_file.write(b",")

Read them back:

#!/usr/bin/env python3
import re
from mmap import ACCESS_READ, mmap

blocks = []
match_size = re.compile(br'(\d+):').match
with open('blocks.netstring', 'rb') as file, \
     mmap(file.fileno(), 0, access=ACCESS_READ) as mm:
    position = 0
    for m in iter(lambda: match_size(mm, position), None):
        i, size = m.end(), int(m.group(1))
        blocks.append(mm[i:i + size])
        position = i + size + 1 # shift to the next netstring
print(blocks)

As an alternative, you could consider BSON format for your data or ascii armor format.