Home » Community » U++ community news and announcements » Log compression
Log compression [message #43836] |
Fri, 31 October 2014 13:43 |
|
mirek
Messages: 13976 Registered: November 2005
|
Ultimate Member |
|
|
Recently I have noticed sort of problem in RPC or SQL logs when large pieces of hex or base64 encoded data are being transfered/logged - it leads to extremely long (one of my production apps can easily produce 2GB log per day) and hard to process logs, while all of that encoded binary data is basically useless in the log.
Thinking about the issue, I have invented a simple heuristics: If any substring of log that does not contain any whitespaces is longer than 200 characters, it gets "compressed", e.g.:
<value>
<string>89504e470d0a....[122379 bytes]....e44ae426082</string>
</value>
This is implemented in simple "CompressLog" funciton in Core and then used in Core/RPC and Sql. Both have compression implicitly ON with option to switch it off.
|
|
|
Goto Forum:
Current Time: Thu May 09 23:53:04 CEST 2024
Total time taken to generate the page: 0.02443 seconds
|