Archive

Archive for October, 2016

Improvements in mail deduplication

October 4th, 2016 Comments off

The no_duplicate plugin tracks exact duplicates, precisely incoming mail files having the same SHA1 fingerprint as a previously imported mail file.

Up to now, such duplicates could be discarded by simply declaring in manitou-mdx configuration file:


[name@example.com]
incoming_preprocess_plugins = no_duplicate

But when a manitou-mail database is used to only sync new messages from an IMAP server, with hierarchical tags reflecting folders, a message move across IMAP folders is interpreted as a duplicate coming in.

It’s fine and actually desirable not to import the message again, but ideally we’d want to see it in its new folder.

The no_duplicate plugin can now do that by acting both as a incoming_preprocess_plugin and as a incoming_postprocess_plugin.
The first step recognizes the duplicate, and optionally, updates the tags of the message instance already in the database.
The second step associates the SHA1 fingerprint of a newly imported message to its unique ID, which is necessary for the optional tags update to work, if a duplicate of this message comes in the future with different tags.

The declaration taking advantage of this new feature looks like:

[name@example.com]
incoming_preprocess_plugins = no_duplicate({update_tags=>1})
incoming_postprocess_plugins = no_duplicate

For more information on manitou-mdx plugins, see the documentation.

Categories: Database, New features Tags: