msgpack document

MessagePack is a efficient format for inter language data exchange.

API reference

msgpack.pack(o, stream, **kwargs)

Pack object o and write it to stream

See Packer for options.

dump() is alias for pack()

msgpack.packb(o, **kwargs)

Pack object o and return packed bytes

See Packer for options.

dumps() is alias for packb()

msgpack.unpack(stream, object_hook=None, list_hook=None, bool use_list=1, encoding=None, unicode_errors='strict', object_pairs_hook=None, ext_hook=ExtType, Py_ssize_t max_str_len=2147483647, Py_ssize_t max_bin_len=2147483647, Py_ssize_t max_array_len=2147483647, Py_ssize_t max_map_len=2147483647, Py_ssize_t max_ext_len=2147483647)

Unpack an object from stream.

Raises ValueError when stream has extra bytes.

See Unpacker for options.

load() is alias for unpack()

msgpack.unpackb(packed, object_hook=None, list_hook=None, bool use_list=1, encoding=None, unicode_errors='strict', object_pairs_hook=None, ext_hook=ExtType, Py_ssize_t max_str_len=2147483647, Py_ssize_t max_bin_len=2147483647, Py_ssize_t max_array_len=2147483647, Py_ssize_t max_map_len=2147483647, Py_ssize_t max_ext_len=2147483647)

Unpack packed_bytes to object. Returns an unpacked object.

Raises ValueError when packed contains extra bytes.

See Unpacker for options.

loads() is alias for unpackb()

class msgpack.Packer(default=None, encoding='utf-8', unicode_errors='strict', use_single_float=False, bool autoreset=1, bool use_bin_type=0, bool strict_types=0)

MessagePack Packer

usage:

packer = Packer()
astream.write(packer.pack(a))
astream.write(packer.pack(b))

Packer’s constructor has some keyword arguments:

Parameters:

• default (callable) – Convert user type to builtin type that Packer supports. See also simplejson’s document.
• encoding (str) – Convert unicode to bytes with this encoding. (default: ‘utf-8’)
• unicode_errors (str) – Error handler for encoding unicode. (default: ‘strict’)
• use_single_float (bool) – Use single precision float type for float. (default: False)
• autoreset (bool) – Reset buffer after each pack and return its content as bytes. (default: True). If set this to false, use bytes() to get content and .reset() to clear buffer.
• use_bin_type (bool) – Use bin type introduced in msgpack spec 2.0 for bytes. It also enables str8 type for unicode.
• strict_types (bool) – If set to true, types will be checked to be exact. Derived classes from serializeable types will not be serialized and will be treated as unsupported type and forwarded to default. Additionally tuples will not be serialized as lists. This is useful when trying to implement accurate serialization for python types.

bytes(self)

Return buffer content.

pack(self, obj)

pack_array_header(self, long long size)

pack_ext_type(self, typecode, data)

pack_map_header(self, long long size)

pack_map_pairs(self, pairs)

Pack pairs as msgpack map type.

pairs should be a sequence of pairs. (len(pairs) and for k, v in pairs: should be supported.)

reset(self)

Clear internal buffer.

class msgpack.Unpacker(file_like=None, Py_ssize_t read_size=0, bool use_list=1, object_hook=None, object_pairs_hook=None, list_hook=None, encoding=None, unicode_errors='strict', int max_buffer_size=0, ext_hook=ExtType, Py_ssize_t max_str_len=2147483647, Py_ssize_t max_bin_len=2147483647, Py_ssize_t max_array_len=2147483647, Py_ssize_t max_map_len=2147483647, Py_ssize_t max_ext_len=2147483647)

Streaming unpacker.

arguments:

Parameters:

• file_like – File-like object having .read(n) method. If specified, unpacker reads serialized data from it and feed() is not usable.
• read_size (int) – Used as file_like.read(read_size). (default: min(1024**2, max_buffer_size))
• use_list (bool) – If true, unpack msgpack array to Python list. Otherwise, unpack to Python tuple. (default: True)
• object_hook (callable) – When specified, it should be callable. Unpacker calls it with a dict argument after unpacking msgpack map. (See also simplejson)
• object_pairs_hook (callable) – When specified, it should be callable. Unpacker calls it with a list of key-value pairs after unpacking msgpack map. (See also simplejson)
• encoding (str) – Encoding used for decoding msgpack raw. If it is None (default), msgpack raw is deserialized to Python bytes.
• unicode_errors (str) – Used for decoding msgpack raw with encoding. (default: ‘strict’)
• max_buffer_size (int) – Limits size of data waiting unpacked. 0 means system’s INT_MAX (default). Raises BufferFull exception when it is insufficient. You should set this parameter when unpacking data from untrusted source.
• max_str_len (int) – Limits max length of str. (default: 2**31-1)
• max_bin_len (int) – Limits max length of bin. (default: 2**31-1)
• max_array_len (int) – Limits max length of array. (default: 2**31-1)
• max_map_len (int) – Limits max length of map. (default: 2**31-1)

example of streaming deserialize from file-like object:

unpacker = Unpacker(file_like)
for o in unpacker:
    process(o)

example of streaming deserialize from socket:

unpacker = Unpacker()
while True:
    buf = sock.recv(1024**2)
    if not buf:
        break
    unpacker.feed(buf)
    for o in unpacker:
        process(o)

feed(self, next_bytes)

Append next_bytes to internal buffer.

read_array_header(self, write_bytes=None)

assuming the next object is an array, return its size n, such that the next n unpack() calls will iterate over its contents.

Raises OutOfData when there are no more bytes to unpack.

read_bytes(self, Py_ssize_t nbytes)

Read a specified number of raw bytes from the stream

read_map_header(self, write_bytes=None)

assuming the next object is a map, return its size n, such that the next n * 2 unpack() calls will iterate over its key-value pairs.

Raises OutOfData when there are no more bytes to unpack.

skip(self, write_bytes=None)

Read and ignore one object, returning None

If write_bytes is not None, it will be called with parts of the raw message as it is unpacked.

Raises OutOfData when there are no more bytes to unpack.

tell(self)

unpack(self, write_bytes=None)

Unpack one object

If write_bytes is not None, it will be called with parts of the raw message as it is unpacked.

Raises OutOfData when there are no more bytes to unpack.

class msgpack.ExtType

ExtType represents ext type in msgpack.

exceptions

These exceptions are accessible via msgpack package. (For example, msgpack.OutOfData is shortcut for msgpack.exceptions.OutOfData)

exception msgpack.exceptions.BufferFull

Bases: msgpack.exceptions.UnpackException

exception msgpack.exceptions.ExtraData(unpacked, extra)

Bases: msgpack.exceptions.UnpackValueError

exception msgpack.exceptions.OutOfData

Bases: msgpack.exceptions.UnpackException

exception msgpack.exceptions.PackException

Bases: Exception

Deprecated. Use Exception instead to catch all exception during packing.

exception msgpack.exceptions.PackOverflowError

Bases: msgpack.exceptions.PackValueError, OverflowError

PackOverflowError is raised when integer value is out of range of msgpack support [-2**31, 2**32).

Deprecated. Use ValueError instead.

exception msgpack.exceptions.PackValueError

Bases: msgpack.exceptions.PackException, ValueError

PackValueError is raised when type of input data is supported but it’s value is unsupported.

Deprecated. Use ValueError instead.

exception msgpack.exceptions.UnpackException

Bases: Exception

Deprecated. Use Exception instead to catch all exception during unpacking.

exception msgpack.exceptions.UnpackValueError

Bases: msgpack.exceptions.UnpackException, ValueError

Deprecated. Use ValueError instead.