博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
C#高性能二进制序列化
阅读量:4873 次
发布时间:2019-06-11

本文共 14332 字,大约阅读时间需要 47 分钟。

        二进制序列化可以方便快捷的将对象进行持久化或者网络传输,并且体积小、性能高,应用面甚至还要高于json的序列化;开始之前,先来看看dotcore/dotne自带的二进制序列化:C#中对象序列化和反序列化一般是通过BinaryFormatter类来实现的二进制序列化、反序列化的。

        BinaryFormatter序列化:

1 System.Runtime.Serialization.Formatters.Binary.BinaryFormatter serializer = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();2 3 System.IO.MemoryStream memStream = new System.IO.MemoryStream();4 5 serializer.Serialize(memStream, request);

        BinaryFormatter反序列化:

1  memStream.Position=0; 2  3  System.Runtime.Serialization.Formatters.Binary.BinaryFormatter deserializer = 4  5  new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter(); 6  7  object newobj = deserializer.Deserialize(memStream); 8  9  memStream.Close();10 11  return newobj;

        用着多了就发现BinaryFormatter有很多地方不妥,下面就来数数这个序列化的“三宗罪”:

        1.类名上面要加上[Serializable],不加不给序列化;正常的用法应该是序列化一个对象,不需的地方加上NonSerialized才合理吧;

        2.序列化byte[]结果非常大,使用System.Text.Encoding.UTF8.GetString(bytes)查看下,发现里面有一大堆的元数据;对比看看google的protobuf,pb为什么在网络上应用的越来越多,这和他本身序列化完后体积小有着绝大部门的原因;

        3.序列化对象需要完全一致,连类的命名空间都要相同,这点对于分面式开发的应用来说也是不可接受的;

        既然BinaryFormatter不好用,那就只能动手自行实现一个解决上述问题的二进制序列化方案;首先去掉[Serializable]这个标签,接着主要是分析对象,并定义对象序列化后的数据结构;这里的想法是按长度加内容的方式来定义,举个例子:使用int作为长度,来保存一个int值,序列化完应该是:4,0,0,0,1,0,0,0这样的一组bytes,同理可以将int、short、long、float、double、datetime、enum、array、string、class、generic等按照这个格式进行序列化,这里主要使用的是BitConverter、反射等来实现序列化与反序列化;

        序列化实现如下:

1         public static byte[] Serialize(object param)  2         {  3             List
datas = new List
(); 4 5 var len = 0; 6 7 byte[] data = null; 8 9 if (param == null) 10 { 11 len = 0; 12 } 13 else 14 { 15 if (param is string) 16 { 17 data = Encoding.UTF8.GetBytes((string)param); 18 } 19 else if (param is byte) 20 { 21 data = new byte[] { (byte)param }; 22 } 23 else if (param is bool) 24 { 25 data = BitConverter.GetBytes((bool)param); 26 } 27 else if (param is short) 28 { 29 data = BitConverter.GetBytes((short)param); 30 } 31 else if (param is int) 32 { 33 data = BitConverter.GetBytes((int)param); 34 } 35 else if (param is long) 36 { 37 data = BitConverter.GetBytes((long)param); 38 } 39 else if (param is float) 40 { 41 data = BitConverter.GetBytes((float)param); 42 } 43 else if (param is double) 44 { 45 data = BitConverter.GetBytes((double)param); 46 } 47 else if (param is DateTime) 48 { 49 var str = "wl" + ((DateTime)param).Ticks; 50 data = Encoding.UTF8.GetBytes(str); 51 } 52 else if (param is Enum) 53 { 54 var enumValType = Enum.GetUnderlyingType(param.GetType()); 55 56 if (enumValType == typeof(byte)) 57 { 58 data = new byte[] { (byte)param }; 59 } 60 else if (enumValType == typeof(short)) 61 { 62 data = BitConverter.GetBytes((Int16)param); 63 } 64 else if (enumValType == typeof(int)) 65 { 66 data = BitConverter.GetBytes((Int32)param); 67 } 68 else 69 { 70 data = BitConverter.GetBytes((Int64)param); 71 } 72 } 73 else if (param is byte[]) 74 { 75 data = (byte[])param; 76 } 77 else 78 { 79 var type = param.GetType(); 80 81 82 if (type.IsGenericType || type.IsArray) 83 { 84 if (TypeHelper.DicTypeStrs.Contains(type.Name)) 85 data = SerializeDic((System.Collections.IDictionary)param); 86 else if (TypeHelper.ListTypeStrs.Contains(type.Name) || type.IsArray) 87 data = SerializeList((System.Collections.IEnumerable)param); 88 else 89 data = SerializeClass(param, type); 90 } 91 else if (type.IsClass) 92 { 93 data = SerializeClass(param, type); 94 } 95 96 } 97 if (data != null) 98 len = data.Length; 99 }100 datas.AddRange(BitConverter.GetBytes(len));101 if (len > 0)102 {103 datas.AddRange(data);104 }105 return datas.Count == 0 ? null : datas.ToArray();106 }
View Code

        反序列化实现如下:

1         public static object Deserialize(Type type, byte[] datas, ref int offset)  2         {  3             dynamic obj = null;  4   5             var len = 0;  6   7             byte[] data = null;  8   9             len = BitConverter.ToInt32(datas, offset); 10             offset += 4; 11             if (len > 0) 12             { 13                 data = new byte[len]; 14                 Buffer.BlockCopy(datas, offset, data, 0, len); 15                 offset += len; 16  17                 if (type == typeof(string)) 18                 { 19                     obj = Encoding.UTF8.GetString(data); 20                 } 21                 else if (type == typeof(byte)) 22                 { 23                     obj = (data); 24                 } 25                 else if (type == typeof(bool)) 26                 { 27                     obj = (BitConverter.ToBoolean(data, 0)); 28                 } 29                 else if (type == typeof(short)) 30                 { 31                     obj = (BitConverter.ToInt16(data, 0)); 32                 } 33                 else if (type == typeof(int)) 34                 { 35                     obj = (BitConverter.ToInt32(data, 0)); 36                 } 37                 else if (type == typeof(long)) 38                 { 39                     obj = (BitConverter.ToInt64(data, 0)); 40                 } 41                 else if (type == typeof(float)) 42                 { 43                     obj = (BitConverter.ToSingle(data, 0)); 44                 } 45                 else if (type == typeof(double)) 46                 { 47                     obj = (BitConverter.ToDouble(data, 0)); 48                 } 49                 else if (type == typeof(decimal)) 50                 { 51                     obj = (BitConverter.ToDouble(data, 0)); 52                 } 53                 else if (type == typeof(DateTime)) 54                 { 55                     var dstr = Encoding.UTF8.GetString(data); 56                     var ticks = long.Parse(dstr.Substring(2)); 57                     obj = (new DateTime(ticks)); 58                 } 59                 else if (type.BaseType == typeof(Enum)) 60                 { 61                     var numType = Enum.GetUnderlyingType(type); 62  63                     if (numType == typeof(byte)) 64                     { 65                         obj = Enum.ToObject(type, data[0]); 66                     } 67                     else if (numType == typeof(short)) 68                     { 69                         obj = Enum.ToObject(type, BitConverter.ToInt16(data, 0)); 70                     } 71                     else if (numType == typeof(int)) 72                     { 73                         obj = Enum.ToObject(type, BitConverter.ToInt32(data, 0)); 74                     } 75                     else 76                     { 77                         obj = Enum.ToObject(type, BitConverter.ToInt64(data, 0)); 78                     } 79                 } 80                 else if (type == typeof(byte[])) 81                 { 82                     obj = (byte[])data; 83                 } 84                 else if (type.IsGenericType) 85                 { 86                     if (TypeHelper.ListTypeStrs.Contains(type.Name)) 87                     { 88                         obj = DeserializeList(type, data); 89                     } 90                     else if (TypeHelper.DicTypeStrs.Contains(type.Name)) 91                     { 92                         obj = DeserializeDic(type, data); 93                     } 94                     else 95                     { 96                         obj = DeserializeClass(type, data); 97                     } 98                 } 99                 else if (type.IsClass)100                 {101                     obj = DeserializeClass(type, data);102                 }103                 else if (type.IsArray)104                 {105                     obj = DeserializeArray(type, data);106                 }107                 else108                 {109                     throw new RPCPamarsException("ParamsSerializeUtil.Deserialize 未定义的类型:" + type.ToString());110                 }111 112             }113             return obj;114         }
View Code

        其他详细的代码可以查看

        功能基本实现了,下面对比一下10000次的实体序列化与反序列化测试结果:

        实体代码:

1             var groupInfo = new GroupInfo() 2             { 3                 GroupID = 1, 4                 IsTemporary = false, 5                 Name = "yswenli group", 6                 Created = DateTimeHelper.Now, 7                 Creator = new UserInfo() 8                 { 9 10                     ID = 1,11                     Birthday = DateTimeHelper.Now.AddYears(-100),12                     UserName = "yswenli"13                 },14                 Users = new System.Collections.Generic.List
()15 {16 new UserInfo()17 {18 19 ID = 1,20 Birthday = DateTimeHelper.Now.AddYears(-100),21 UserName = "yswenli"22 }23 }24 };

        测试代码:

1         public static byte[] SerializeBinary(object request) 2         { 3  4             System.Runtime.Serialization.Formatters.Binary.BinaryFormatter serializer = 5  6             new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter(); 7  8             using (System.IO.MemoryStream memStream = new System.IO.MemoryStream()) 9             {10                 serializer.Serialize(memStream, request);11 12                 return memStream.ToArray();13             }14         }15 16 17         public static object DeSerializeBinary(byte[] data)18         {19             using (System.IO.MemoryStream memStream = new System.IO.MemoryStream(data))20             {21                 System.Runtime.Serialization.Formatters.Binary.BinaryFormatter deserializer =22 23                 new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();24 25                 return deserializer.Deserialize(memStream);26             }27         }28 29         static void SerializeTest()30         {31             var groupInfo = new GroupInfo()32             {33                 GroupID = 1,34                 IsTemporary = false,35                 Name = "yswenli group",36                 Created = DateTimeHelper.Now,37                 Creator = new UserInfo()38                 {39 40                     ID = 1,41                     Birthday = DateTimeHelper.Now.AddYears(-100),42                     UserName = "yswenli"43                 },44                 Users = new System.Collections.Generic.List
()45 {46 new UserInfo()47 {48 49 ID = 1,50 Birthday = DateTimeHelper.Now.AddYears(-100),51 UserName = "yswenli"52 }53 }54 };55 56 var count = 100000;57 var len1 = 0;58 var len2 = 0;59 60 Stopwatch sw = new Stopwatch();61 sw.Start();62 63 List
list = new List
();64 for (int i = 0; i < count; i++)65 {66 var bytes = SerializeBinary(groupInfo);67 len1 = bytes.Length;68 list.Add(bytes);69 }70 ConsoleHelper.WriteLine($"BinaryFormatter实体序列化平均:{count * 1000 / sw.ElapsedMilliseconds} 次/秒");71 72 sw.Restart();73 for (int i = 0; i < count; i++)74 {75 var obj = DeSerializeBinary(list[i]);76 }77 ConsoleHelper.WriteLine($"BinaryFormatter实体反序列化平均:{count * 1000 / sw.ElapsedMilliseconds} 次/秒");78 ConsoleHelper.WriteLine($"BinaryFormatter序列化生成bytes大小:{len1 * count * 1.0 / 1024 / 1024} Mb");79 list.Clear();80 sw.Restart();81 82 for (int i = 0; i < count; i++)83 {84 var bytes = RPC.Serialize.ParamsSerializeUtil.Serialize(groupInfo);85 len2 = bytes.Length;86 list.Add(bytes);87 }88 ConsoleHelper.WriteLine($"ParamsSerializeUtil实体序列化平均:{count * 1000 / sw.ElapsedMilliseconds} 次/秒");89 sw.Restart();90 for (int i = 0; i < count; i++)91 {92 int os = 0;93 94 var obj = RPC.Serialize.ParamsSerializeUtil.Deserialize(groupInfo.GetType(), list[i], ref os);95 }96 ConsoleHelper.WriteLine($"ParamsSerializeUtil实体反序列化平均:{count * 1000 / sw.ElapsedMilliseconds} 次/秒");97 ConsoleHelper.WriteLine($"ParamsSerializeUtil序列化生成bytes大小:{len2 * count * 1.0 / 1024 / 1024} Mb");98 sw.Stop();99 }
View Code

        运行结果:

 

 

更多内容,请Fork或Star我的github:

转载于:https://www.cnblogs.com/yswenli/p/9217495.html

你可能感兴趣的文章
unisynedit 在Delphi 2010下的编译问题
查看>>
mssql 动态行转列。
查看>>
工作杂记
查看>>
老话题,权限设计及实现!
查看>>
POJ 2393 贪心 简单题
查看>>
3. Quartz2D 绘制矩形、圆形、弧形
查看>>
latex 学习笔记
查看>>
SQL 数据库 学习 005 学习必备的一些操作 --- 如何新建数据库 如何附加和分离数据库(如何备份还原数据库) 如何删除数据库...
查看>>
[php排错] Forbidden You don't have permission to access / on this server.
查看>>
MVC中的helper标签
查看>>
Spring Cloud Gateway入门
查看>>
XCode 4.2 新功能 - Storyboard
查看>>
Tomcat不保存SESSIONS.ser设置
查看>>
QEMU使用手册 - 2 QEMU计算机系统模拟器
查看>>
VIM技巧之去除代码行号并缩进代码
查看>>
自动化测试用例getText()获取某一个元素的值返回null或空
查看>>
大数智能未来
查看>>
virtualenv和virtualenvwrapper 的安装和使用
查看>>
MAC sublime text 无法自动补齐标签
查看>>
NgBook留言本开发全过程(1)
查看>>