Compression in Zimbra

Compression Types in Zimbra

Photo by Mathyas Kurmann on Unsplash

Http Compression

(https://github.com/Zimbra/zm-mailbox/blob/develop/common/src/java/com/zimbra/common/account/ZAttrProvisioning.java) zimbraHttpCompressionEnabled parameter in “/conf/attrs/zimbra-attrs.xml” file.

From https://wiki.zimbra.com/wiki/Zimbra_Attributes_ZCS7:

<attr id="1467" name="zimbraHttpCompressionEnabled" type="boolean" cardinality="single" optionalIn="globalConfig,server" flags="serverInherited" requiresRestart="mailbox" since="7.2.5">
<globalConfigValue>TRUE</globalConfigValue>
<desc>Whether or not to enable HTTP compression. Defaults to true.</desc>
</attr>

Run this command and see the output (don’t forget to change the url):

sudo -u zimbra /opt/zimbra/bin/zmprov gs zimbrathree.nils.local | grep -i compress

zimbraMailUncompressedCacheMaxBytes → Deprecated since: 6.0.7 (uncompressed files on disk will never exceed zimbraMailFileDescriptorCacheSize.. Orig desc: max number of bytes * stored in the uncompressed blob cache on disk)

zimbraMailUncompressedCacheMaxFiles → Deprecated since: 6.0.7 (The number of * uncompressed files on disk will never exceed * zimbraMailFileDescriptorCacheSize.. Orig desc: max number of files in * the uncompressed blob cache on disk)

Volume Compression

Run this command and see its output:

sudo -u zimbra /opt/zimbra/bin/zmvolume -l

Via Zimbra Admin Screen:

https://www.sbarjatiya.com/notes_wiki/index.php/CentOS_7.x_Configure_or_optimize_new_Zimbra_installation#Enable_compression_of_email_data

Select “Compress Blobs” and click OK.

After that, when you query the config parameter again:

https://forums.zimbra.org/viewtopic.php?t=22625

“When you enable compression existing blobs aren’t touched.”

https://forums.zimbra.org/viewtopic.php?t=25757&start=20

“Blob compression is being done on demand. I you disable Blob compression all new data wil be stored as uncompressed data. When you re-enable compression existing blobs aren’t touched but all new data exeeding “compressionThreshold” (4k default) will be stored as compressed Blob again.
If you want to re-compress all existing data on a message store there are several ways to do this. e.g. moving the Message store by enabling HSM.”

https://zimbra.github.io/adminguide/8.8.10/index.html

“If Compress Blobs is enabled (YES), the disk space used is decreased, but memory requirements for the server increases.”

Via Admin Soap API:

You can set Volume parameters via Admin Soap API: https://github.com/Zimbra/zm-mailbox/blob/develop/store/docs/soap-admin.txt

<CreateVolumeRequest>
<volume type="..." name="..." rootpath="..."
compressBlobs="..." compressionThreshold="..."/>
</CreateVolumeRequest>
<CreateVolumeResponse>
<volume id="{id}"/>
</CreateVolumeResponse>
Notes:
id: ID of volume
type: type of volume;
1 = primary message volume
2 = secondary message volume
10 = index volume
name: name or description of volume
rootPath: absolute path to root of volume, e.g. /opt/zimbra/store
compressBlobs: boolean value that specifies whether blobs in this
volume are compressed
compressionThreshold: long value that specifies the maximum uncompressed
file size, in bytes, of blobs that will not be compressed
(in other words blobs larger than this threshold are compressed)
isCurrent: 1 if the volume is current, 0 if not
-----------------------------
<GetVolumeRequest id="{id}"/>
<GetVolumeResponse>
<volume id="{id}" type="..." name="..." rootpath="..."
compressBlobs="..." compressionThreshold="..."
isCurrent="0|1"/>
</GetVolumeResponse>
-----------------------------
<GetAllVolumesRequest/>
<GetAllVolumesResponse>
<volume .../>+ <!-- each volume element is same as in GetVolumeResponse -->
</GetAllVolumeResponse>
-----------------------------
<ModifyVolumeRequest id="{id}">
<volume [type="..."] [name="..."] [rootpath="..."]
[compressBlobs="..."] [compressionThreshold="..."]/>
</ModifyVolumeRequest>
<ModifyVolumeResponse/>

https://github.com/Zimbra/zm-mailbox/blob/develop/store/src/db/hsqldb/db.sql has Zimbra tables, you can inspect columns in table “Volume”:

compress_blobs BOOLEAN NOT NULL,
compression_threshold BIGINT NOT NULL,

https://github.com/Zimbra/zm-mailbox/blob/develop/store/src/java/com/zimbra/cs/store/BlobBuilder.java : “useCompression” and “getCompressionThreshold” is being used in method “append”.

For Zimbra Object Store extension (you can see my post here), the class we used to extend https://github.com/Zimbra/zm-mailbox/blob/develop/store/src/java/com/zimbra/cs/store/external/ExternalStoreManager.java has “storeIncoming” method. Here, its value is being set to “false”.

public Blob storeIncoming(InputStream data)
throws IOException, ServiceException {
return storeIncoming(data, false);
}
@Override
public Blob storeIncoming(InputStream data, boolean storeAsIs) throws IOException,
ServiceException {
BlobBuilder builder = getBlobBuilder();
// if the blob is already compressed, *don't* calculate a digest/size from what we write
builder.disableCompression(storeAsIs).disableDigest(storeAsIs);
return builder.init().append(data).finish();
}

The default store manager of Zimbra https://github.com/Zimbra/zm-mailbox/blob/develop/store/src/java/com/zimbra/cs/store/file/FileBlobStore.java also has “storeIncoming(InputStream in, boolean storeAsIs)” method.

Happy Coding!