Serializers

Serializers can be attached to backends in order to serialize/deserialize data sent and retrieved from the backend. This allows to apply transformations to data in case you want it to be saved in a specific format in your cache backend. For example, imagine you have your Model and want to serialize it to something that Redis can understand (Redis can’t store python objects). This is the task of a serializer.

To use a specific serializer:

>>> from aiocache import Cache
>>> from aiocache.serializers import PickleSerializer
cache = Cache(Cache.MEMORY, serializer=PickleSerializer())

Currently the following are built in:

NullSerializer

class aiocache.serializers.NullSerializer(*args, encoding=<object object>, **kwargs)[source]

This serializer does nothing. Its only recommended to be used by aiocache.SimpleMemoryCache because for other backends it will produce incompatible data unless you work only with str types because it store data as is.

DISCLAIMER: Be careful with mutable types and memory storage. The following behavior is considered normal (same as functools.lru_cache):

cache = Cache()
my_list = [1]
await cache.set("key", my_list)
my_list.append(2)
await cache.get("key")  # Will return [1, 2]
dumps(value)[source]

Returns the same value

loads(value)[source]

Returns the same value

StringSerializer

class aiocache.serializers.StringSerializer(*args, encoding=<object object>, **kwargs)[source]

Converts all input values to str. All return values are also str. Be careful because this means that if you store an int(1), you will get back ‘1’.

The transformation is done by just casting to str in the dumps method.

If you want to keep python types, use PickleSerializer. JsonSerializer may also be useful to keep type of symple python types.

dumps(value)[source]

Serialize the received value casting it to str.

Parameters:

value – obj Anything support cast to str

Returns:

str

loads(value)[source]

Returns value back without transformations

PickleSerializer

class aiocache.serializers.PickleSerializer(*args, protocol=4, **kwargs)[source]

Transform data to bytes using pickle.dumps and pickle.loads to retrieve it back.

DEFAULT_ENCODING: str | None = None
dumps(value)[source]

Serialize the received value using pickle.dumps.

Parameters:

value – obj

Returns:

bytes

loads(value)[source]

Deserialize value using pickle.loads.

Parameters:

value – bytes

Returns:

obj

JsonSerializer

class aiocache.serializers.JsonSerializer(*args, encoding=<object object>, **kwargs)[source]

Transform data to json string with json.dumps and json.loads to retrieve it back. Check https://docs.python.org/3/library/json.html#py-to-json-table for how types are converted.

ujson will be used by default if available. Be careful with differences between built in json module and ujson:

  • ujson dumps supports bytes while json doesn’t

  • ujson and json outputs may differ sometimes

dumps(value)[source]

Serialize the received value using json.dumps.

Parameters:

value – dict

Returns:

str

loads(value)[source]

Deserialize value using json.loads.

Parameters:

value – str

Returns:

output of json.loads.

MsgPackSerializer

class aiocache.serializers.MsgPackSerializer(*args, use_list=True, **kwargs)[source]

Transform data to bytes using msgpack.dumps and msgpack.loads to retrieve it back. You need to have msgpack installed in order to be able to use this serializer.

Parameters:
  • encoding – str. Can be used to change encoding param for msg.loads method. Default is utf-8.

  • use_list – bool. Can be used to change use_list param for msgpack.loads method. Default is True.

dumps(value)[source]

Serialize the received value using msgpack.dumps.

Parameters:

value – obj

Returns:

bytes

loads(value)[source]

Deserialize value using msgpack.loads.

Parameters:

value – bytes

Returns:

obj

In case the current serializers are not covering your needs, you can always define your custom serializer as shown in examples/serializer_class.py:

 1import asyncio
 2import zlib
 3
 4from aiocache import Cache
 5from aiocache.serializers import BaseSerializer
 6
 7
 8class CompressionSerializer(BaseSerializer):
 9
10    # This is needed because zlib works with bytes.
11    # this way the underlying backend knows how to
12    # store/retrieve values
13    DEFAULT_ENCODING = None
14
15    def dumps(self, value):
16        print("I've received:\n{}".format(value))
17        compressed = zlib.compress(value.encode())
18        print("But I'm storing:\n{}".format(compressed))
19        return compressed
20
21    def loads(self, value):
22        print("I've retrieved:\n{}".format(value))
23        decompressed = zlib.decompress(value).decode()
24        print("But I'm returning:\n{}".format(decompressed))
25        return decompressed
26
27
28cache = Cache(Cache.REDIS, serializer=CompressionSerializer(), namespace="main")
29
30
31async def serializer():
32    text = (
33        "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt"
34        "ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation"
35        "ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in"
36        "reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur"
37        "sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit"
38        "anim id est laborum.")
39    await cache.set("key", text)
40    print("-----------------------------------")
41    real_value = await cache.get("key")
42    compressed_value = await cache.raw("get", "main:key")
43    assert len(compressed_value) < len(real_value.encode())
44
45
46async def test_serializer():
47    await serializer()
48    await cache.delete("key")
49    await cache.close()
50
51
52if __name__ == "__main__":
53    asyncio.run(test_serializer())

You can also use marshmallow as your serializer (examples/marshmallow_serializer_class.py):

 1import random
 2import string
 3import asyncio
 4
 5from marshmallow import fields, Schema, post_load
 6
 7from aiocache import Cache
 8from aiocache.serializers import BaseSerializer
 9
10
11class RandomModel:
12    MY_CONSTANT = "CONSTANT"
13
14    def __init__(self, int_type=None, str_type=None, dict_type=None, list_type=None):
15        self.int_type = int_type or random.randint(1, 10)
16        self.str_type = str_type or random.choice(string.ascii_lowercase)
17        self.dict_type = dict_type or {}
18        self.list_type = list_type or []
19
20    def __eq__(self, obj):
21        return self.__dict__ == obj.__dict__
22
23
24class MarshmallowSerializer(Schema, BaseSerializer):  # type: ignore[misc]
25    int_type = fields.Integer()
26    str_type = fields.String()
27    dict_type = fields.Dict()
28    list_type = fields.List(fields.Integer())
29
30    # marshmallow Schema class doesn't play nicely with multiple inheritance and won't call
31    # BaseSerializer.__init__
32    encoding = 'utf-8'
33
34    @post_load
35    def build_my_type(self, data, **kwargs):
36        return RandomModel(**data)
37
38    class Meta:
39        strict = True
40
41
42cache = Cache(serializer=MarshmallowSerializer(), namespace="main")
43
44
45async def serializer():
46    model = RandomModel()
47    await cache.set("key", model)
48
49    result = await cache.get("key")
50
51    assert result.int_type == model.int_type
52    assert result.str_type == model.str_type
53    assert result.dict_type == model.dict_type
54    assert result.list_type == model.list_type
55
56
57async def test_serializer():
58    await serializer()
59    await cache.delete("key")
60
61
62if __name__ == "__main__":
63    asyncio.run(test_serializer())

By default cache backends assume they are working with str types. If your custom implementation transform data to bytes, you will need to set the class attribute encoding to None.