探险新型序列化工具MessagePack

淡淡的烟草味﹌ 2022-05-14 04:38 254阅读 0赞

MessagePack是最近一个比较热门与Json做比较的序列化工具，它的优点是相比于json，序列化速度更快和序列化之后的字节数组更小，正如它的官网[https://msgpack.org/][https_msgpack.org]卖的广告所说

> It’s like JSON.  
> but fast and small.

下面我们以三个方面来对MessagePack做一个初步的探险

## 一.What is MessagePack ##

##### 1.以官网的阐述来表明： #####

> MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it’s faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves.  
> 中文翻译：  
> MessagePack是一种有效的二进制序列化格式。 它允许您在多种语言（如JSON）之间交换数据。 但它更快更小。 小整数被编码为单个字节，典型的短字符串除了字符串本身外只需要一个额外的字节。

可以看出它序列化出来能比json更小的原因是，整数会被编码成一个字节，另外json的字符串的冒号和引号也会被以一种特殊的方式进行压缩，下面的图片可以看出来MessagePack压缩之后的效果：  
![这里写图片描述][70]  
from:[http://indiegamr.com/cut-your-data-exchange-traffic-by-up-to-50-with-one-line-of-code-msgpack-vs-json/][http_indiegamr.com_cut-your-data-exchange-traffic-by-up-to-50-with-one-line-of-code-msgpack-vs-json]

##### 2.验证Message的序列化速度和内容是否更优于JSON #####

package msgpack;
    
    import com.alibaba.fastjson.JSON;
    import com.fasterxml.jackson.databind.ObjectMapper;
    import org.msgpack.jackson.dataformat.MessagePackFactory;
    
    import java.io.IOException;
    
    public class MsgPackMain {
        public static void main(String[] args) throws IOException {
            Person person = new Person("andrew", "24", "man");
            //fastjson
            long jsonStartTime = System.currentTimeMillis();
            String jsonStr = JSON.toJSONString(person);
            System.out.println("json花费时间:" + (System.currentTimeMillis() - jsonStartTime));
            System.out.println("json byteArr size is" + jsonStr.getBytes("UTF-8").length);
    
            //msgPack use jackson
            ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
            long msgPackStartTime = System.currentTimeMillis();
            byte[] msgPackByteArr = objectMapper.writeValueAsBytes(person);
            System.out.println("msgPack花费时间:" + (System.currentTimeMillis() - msgPackStartTime));
            System.out.println("msgPack byteArr size is :" + msgPackByteArr.length);
        }
    }

测试结果：

json花费时间:199
    json byteArr size is40
    msgPack花费时间:72
    msgPack byteArr size is :28

从测试结果可以看出msgPack明显在速度和size方面都远胜于json，而且由于有jackson的依赖库，所以使用和替换jackson-json都非常简单

## 二.How to use ##

下面我们来做一些简单的demo：  
引入需要依赖的maven库:

<dependency>
          <groupId>org.msgpack</groupId>
          <artifactId>jackson-dataformat-msgpack</artifactId>
          <version>${msgpack.version}</version>
    </dependency>

jackson包装的msgPack，使用非常简单，下面来封装简单的MsgPackUtil

package msgpack;
    
    import com.fasterxml.jackson.core.JsonProcessingException;
    import com.fasterxml.jackson.core.type.TypeReference;
    import com.fasterxml.jackson.databind.ObjectMapper;
    import org.msgpack.jackson.dataformat.MessagePackFactory;
    
    import java.io.IOException;
    import java.util.List;
    import java.util.Map;
    
    public class MsgPackUtil { 
    
        private static final ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
    
        public static <T> T readObj(byte[] byteArr, Class<T> clazz) throws IOException {
            return objectMapper.readValue(byteArr, clazz);
        }
    
        public static <T> byte[] writeObj(T t) throws JsonProcessingException {
            return objectMapper.writeValueAsBytes(t);
        }
    
        public static <T> List<T> readList(byte[] listByteArr, TypeReference<List<T>> typeReference) throws IOException {
            return objectMapper.readValue(listByteArr, typeReference);
        }
    
        public static <T,V> Map<T, V> readMap(byte[] mapByteArr, TypeReference<Map<T, V>> typeReference) throws IOException {
            return objectMapper.readValue(mapByteArr, typeReference);
        }
    }

demo代码:

private static void msgPackJackSonDemo() throws IOException {
            //Object
            Person person = new Person("andrew", "24", "man");
            byte[] personByteArr = MsgPackUtil.writeObj(person);
            System.out.println("ObjByteArr size is :" + personByteArr.length);
            //使用readValue时候，需要空的构造方法
            System.out.println(MsgPackUtil.readObj(personByteArr, Person.class));
    
            //list
            List<String> testList = new ArrayList<>(3);
            testList.add("hello");
            testList.add("msgPack");
            testList.add("java");
            byte[] listByteArr = MsgPackUtil.writeObj(testList);
            System.out.println("listByteArr size is :" + listByteArr.length);
            //readList只支持POJO和基本类型List
            System.out.println(JSON.toJSONString(MsgPackUtil.readList(listByteArr, new TypeReference<List<String>>() {})));
    
            //map
            Map<String, String> testMap = new HashMap<>(3);
            testMap.put("hello", "hello");
            testMap.put("msgPack", "msgPack");
            testMap.put("java", "java");
            byte[] mapByteArr = MsgPackUtil.writeObj(testMap);
            System.out.println("mapByteArr size is :" + mapByteArr.length);
            System.out.println(JSON.toJSONString(MsgPackUtil.readMap(mapByteArr, new TypeReference<Map<String, String>>() {})));
        }

## 三.使用场景 ##

1.适用于将数据存储在类似于mc和redis这类的nosql数据库中，因为数据可以序列化的更小更快，那么在网络上的传输消耗会减少，同时存在redis里面的内存占用也会减少。  
2.适用于分布式框架里面多机之间的网络传输，生产上的实现：fluentd [https://github.com/fluent/fluentd][https_github.com_fluent_fluentd]  
3.适用于前后端数据的交互，当前后端完全分离之后，后端可以使用MsgPack代替json以获取更快的速度

## 四.引用的相关资料： ##

[http://indiegamr.com/cut-your-data-exchange-traffic-by-up-to-50-with-one-line-of-code-msgpack-vs-json/][http_indiegamr.com_cut-your-data-exchange-traffic-by-up-to-50-with-one-line-of-code-msgpack-vs-json]  
[https://msgpack.org/][https_msgpack.org]  
[https://github.com/msgpack/msgpack-java/blob/develop/msgpack-jackson/README.md][https_github.com_msgpack_msgpack-java_blob_develop_msgpack-jackson_README.md]

written by 黄文岳
                                                                                    2018.09.02

[https_msgpack.org]: https://msgpack.org/
[70]: /images/20220514/c936a06d02af42c8b38fe1dd8484011a.png
[http_indiegamr.com_cut-your-data-exchange-traffic-by-up-to-50-with-one-line-of-code-msgpack-vs-json]: http://indiegamr.com/cut-your-data-exchange-traffic-by-up-to-50-with-one-line-of-code-msgpack-vs-json/
[https_github.com_fluent_fluentd]: https://github.com/fluent/fluentd
[https_github.com_msgpack_msgpack-java_blob_develop_msgpack-jackson_README.md]: https://github.com/msgpack/msgpack-java/blob/develop/msgpack-jackson/README.md