Archive

Archive for the ‘New features’ Category

Version 1.6.0 released

March 20th, 2017 No comments

Manitou-Mail 1.6.0 is released and available to download.

This version adds operators in the searchbar (on message senders, recipients, status, dates, attachments, tags), a statistics panel with charts and exportable results, and the creation of users and groups within the interface.

The users management features also include access rights checked at the database level, and the possibility of restricting certain accounts to certain identities, using policies with PostgreSQL’s Row Level Security feature.

Pre-compiled binaries for Windows and macOs come ready-to-use with Qt-5.5 libraries (including WebKit), and binary packages for Linux Debian 8 and Ubuntu 14.04 and 16.04 are also available through the APT repository (see the download page).

 

If upgrading from a previous version, make sure to run the server-side command:

manitou-mgr --upgrade-schema
Categories: New features Tags:

Improvements in mail deduplication

October 4th, 2016 Comments off

The no_duplicate plugin tracks exact duplicates, precisely incoming mail files having the same SHA1 fingerprint as a previously imported mail file.

Up to now, such duplicates could be discarded by simply declaring in manitou-mdx configuration file:


[name@example.com]
incoming_preprocess_plugins = no_duplicate

But when a manitou-mail database is used to only sync new messages from an IMAP server, with hierarchical tags reflecting folders, a message move across IMAP folders is interpreted as a duplicate coming in.

It’s fine and actually desirable not to import the message again, but ideally we’d want to see it in its new folder.

The no_duplicate plugin can now do that by acting both as a incoming_preprocess_plugin and as a incoming_postprocess_plugin.
The first step recognizes the duplicate, and optionally, updates the tags of the message instance already in the database.
The second step associates the SHA1 fingerprint of a newly imported message to its unique ID, which is necessary for the optional tags update to work, if a duplicate of this message comes in the future with different tags.

The declaration taking advantage of this new feature looks like:

[name@example.com]
incoming_preprocess_plugins = no_duplicate({update_tags=>1})
incoming_postprocess_plugins = no_duplicate

For more information on manitou-mdx plugins, see the documentation.

Categories: Database, New features Tags:

Users management in the interface

September 21st, 2016 Comments off

Starting with version 1.6, Manitou-Mail will allow the creation of users and groups from within the user interface, as shown in the screenshot below:

ui-users-1

Although PostgreSQL has merged users and groups into “roles” years ago (since version 8.1), we intentionally stick to the old terminology, because it’s easier to describe the model. Users are accounts for persons connecting to the database. Groups are entities having a set of permissions, and to which users are assigned. Users can belong to multiple groups.

Under the hood, users correspond to PostgreSQL roles having the LOGIN attribute, so they can log in (assuming proper permissions), whereas groups are roles that don’t have this attribute. Permissions are associated to groups through SQL GRANT commands.

Manitou-Mail chooses to associate permissions to groups, instead of individual users, given its focus on team work and shared mail corpuses. For a set of permissions that apply to a single person, a group will have to be created with only that user as member.

The set of permissions currently handled is shown in the snapshot below. More fined-tuned permissions will probably be added in the future depending on users needs.

 

ui-users-2

Version 1.6 can be built from the git repository; pre-compiled binaries and packages will be made available soon.

Categories: New features, User Interface Tags:

Operators in the search bar

July 9th, 2016 Comments off

Until now, the search bar in the user interface did not support query
terms to search on metadata.
I’m glad to say that commits 2ddddaae and a1cbe72a add support for filtering
by date and message status right from the search bar, introducing
five operators:

  • “date:” must be followed by an iso-8601 date (format YYYY-MM-DD),
    or by a specific month (format YYYY-MM), or just a year (YYYY).
    It selects the messages from respectively that day,or month, or year.

  • “before:” has the same format but selects messages dated
    from this day/month/year or an earlier date.

  • “after:” is of course the opposite, selecting messages past
    the date that follows.

  • “is:” must be followed by a status among read,replied,forward,archived,sent.
    Criteria can be combined by using the option several times, as statuses are cumulative, not mutually exclusive,

  • “isnot:” is of course the opposite of “is”. It accepts the same arguments
    and filters out the messages that have the corresponding status bit.
    “is:” and “isnot:” can also be combined, for instance: “is:archived isnot:sent”.

A few more search bar operators are likely to be added to that list, as it’s a pretty handy and fast way to express basic queries.

Categories: New features, User Interface Tags:

Improved Debian packages for manitou-mdx 1.4.0

October 15th, 2015 Comments off

An APT repository is now available for manitou-mail, with the latest manitou-mdx packages for amd64 and i386 architectures.
Create /etc/apt/sources.list.d/manitou.list with these contents:

# for Debian 7 or 8
deb http://manitou-mail.org/apt/ jessie main
# for Ubuntu 14.04 and other current versions
deb http://manitou-mail.org/apt/ trusty main

(the distribution codename is obtained from `lsb_release -c` )

then run:

apt-get update
apt-get install manitou-mdx

The first time, the installer now asks if it should install the database.

Screenshot from 2015-10-10 23:45:38

When choosing Yes, it creates a new database as indicated (assuming a default PostgreSQL cluster is up and running), and auto-starts (configured in /etc/default/manitou-mdx)

The default source of mail defined in /etc/manitou-mdx.conf is also the Maildir directory for manitou as a user (/var/lib/manitou/Maildir). Then if the system is configured to use Maildir, messages directed to the local address manitou@localhost will end up directly into the database.

Hopefully, these changes will make it easier to start with manitou-mdx compared to the previous versions of the package.

On Debian, the default maildrop destination is still mbox files (/var/mail/$USER), and delivery happens through procmail. maildir is always a better choice. With procmail, this is configurable by putting in /etc/procmailrc:

DEFAULT=$HOME/Maildir/

If not using procmail, for example if postfix is responsible for the maildrop, set in /etc/postfix/main.cf

home_mailbox = Maildir/

See https://wiki.debian.org/MaildirConfiguration for more details.

Categories: Administration, New features Tags:

Version 1.3.0 released

October 4th, 2012 Comments off

Version 1.3.0 is now available for download, with major improvements to the full text search and the ability to import mailbox archives from within the user interface. Here is what’s new in more detail:

  • Accent-insensitive search, word exclusion, better word parser, smaller inverted index.
  • User interface to submit and control server-side import of mailbox files.
  • Combine headers and body in the same view for better usability.
  • Add email syntax checks in composer.
  • Bugfixes: set forwarded status on forwarded messages.
  • Help system: use a simple integrated browser instead of the external Qt assistant program.
  • Integrated indexing of html parts.
  • Text extractors on attachments replace indexer plugins.
  • Add partial and full reindexing capabilities to manitou-mgr.

The user interface is available as an installer for MS-Windows and a disk image for Mac OS X. For Linux, precompiled packages are available for the latest Debian and Ubuntu distributions, for both x86 and amd64 architectures.
Users of other operating systems are encouraged to compile from source and open issues at github if problems arise.

To upgrade from previous versions, it’s strongly suggested to replace attachments indexers by words extractors (see the previous blog post: Upgrading attachments indexers). Once done, it’s also suggested to rebuild entirely the inverted index. See Upgrading manitou-mdx in the documentation.

Categories: New features Tags:

Upgrading attachments indexers

September 26th, 2012 Comments off

Version 1.3.0 is ready to be released with some major improvements in the full-text search and full-text indexer. A couple of packaging issues are still being worked on, but the code won’t change significantly from the 1.3.0 tags on the master branches at github, for both manitou-mail-mdx and manitou-mail-ui.

The list of changes from the previous version is currently visible in the NEWS file in the sources.

It should be noted that 1.3.0 will require some adjustments in how attachments in various formats are indexed with the help of user-supplied scripts.

Up to now, indexer plugins were used to index the contents of attached files in PDF or DOC, or even HTML formats.

Starting from 1.3.0, this is no longer desirable: they should be integrated with a new method called “words extractors” and declared in manitou-mdx main configuration file. Example:

index_words_extractors = application/pdf: /opt/scripts/pdf2text \
	application/msword: /opt/scripts/word2text

The user-supplied scripts should extract words from the contents in custom format passed to their standard input, and output these words encoded in utf-8 to the standard output.

For installs that didn’t index attachments with plugins, it doesn’t matter. Upgrading to 1.3.0 will just cause starting to index HTML contents, which it does now internally by default, so no manual action is required.

On the other hand, for installs that used indexer plugins, a preliminary step to upgrade to 1.3.0 would be to convert these to word extractor scripts. It’s nothing particularly difficult. As an example, here is a ready-to-use script that extracts words from MS-Word files with antiword.

#!/bin/sh

t=$(tempfile --suffix=.doc) || exit 1
trap "rm -f -- '$t'" EXIT
cat >>$t
antiword -i1 "$t" || exit 1

rm -f -- "$t"
trap - EXIT
exit 0

This is a preliminary step because it will be recommended to rebuild the inverted word index when upgrading to 1.3.0 , and doing this involves reindexing attachments as well.
Up to version 1.2, that was not possible with indexer plugins. That’s one of the reasons why plugins get deprecated as a way to index attachments contents. In addition, the lack of integration with the words vectors cache was a performance drag, and 1.3 solves that as well with its words extractors method.

Categories: Development, New features Tags:

Attachment uploader reloaded

February 24th, 2012 Comments off

The attachment uploader is a plugin that solves the problem of attachments that are too big to be transferred by mail. It requires no specific action from the person who composes the mail: on the client side, the user attaches the files as usual. This is important because the users are often not aware that sending huge files by e-mail is troublesome.

The upload happens on the server side: before passing outgoing messages to the delivery service, the plugin checks for attachments bigger than the configured size, and when found, transfers them to a web server, under a random-generated directory name. Inside the message, the contents are replaced by the URL pointing to them.

Until recently, the attachments had to be sent by FTP to the web server. What’s new is that it can now be SSH instead, provided that the Net::SFTP::Foreign Perl module is installed on the server.

The new version of the plugin comes with manitou-mdx 1.2.0 and is named attach_uploader_ssh.
(source code on github)

To activate the plugin and connect it to a sender’s identity, it needs to be declared in manitou-mdx configuration file.
Example:

[mymail@domain.tld]
outgoing_plugins = attach_uploader_ssh({host=>"www.myserver.tld", login=>
"sshuser", base_url=>"http://attached.myserver.tld", path=>"/var/www/attached.myserver.tld", maxsize=>2000000})

There is a “password” field that could be used but in this example it is assumed that an SSH key lets the script connect without a password. This is just one of the multiple security options and choices that open up when switching from FTP to SSH.

And here is how a picture larger that 2Mb would appear in the message (as a text/plain part)

The attached file is available at:
http://attached.myserver.tld/JXlafBybAKGrwE5yyOIbKA/PICT34.JPG
Categories: New features, Usage Tags:

Version 1.2.0 released

December 31st, 2011 Comments off

Version 1.2.0 is now available for download.
What’s new?

  • More functions are available in the filter expressions: age(), date(), date_utc(), now(), now_utc(), identity(), rawsize().
  • New equality and inequality operators for numeric comparisons.
  • A mdx plugin to modify headers of outgoing mail with dynamic contents.
  • A new incarnation of the attach_uploader plugin that supports ssh for file transfers.
  • Migration to new Qt assistant and compatibility fixes for Qt-4.7 and 4.8.
  • Drag and drop for attached files in the mail composer.
  • A new dynamic column for recipients in the messages list.
  • Many improvements in the filter editor: filters can now be reordered and tested from within the interface, and the user interface for actions has been redesigned.
  • New filter actions: Set identity, set header, remove header.
Categories: New features Tags:

Version 1.1.0

August 8th, 2011 Comments off

Manitou-Mail version 1.1.0 is now available. What’s new? Mostly, the mail merge capabilities for sending mailings to potentially large lists of users. Mailing templates and run-time information are stored in the database, and while the actual sending is done server-side, it can be monitored from the user interface.
Mailings can be html or text only or both, and may use placeholders with contents imported from CSV files.

Additionally, the UI now uses the new full text search function for faster results.

This time, there’s a deb package for manitou-mdx that is expected to work with all reasonably recent debian/ubuntu releases, and another deb package for the user interface, which is created for ubuntu 10.
See the download page.

Categories: New features Tags: