Routing outgoing mail in manitou-mdx

daniel — Sun, 18 Oct 2009 21:25:14 +0000

The default command invoked as the delivery agent for manitou-mdx is `sendmail -f $FROM$ -t` where $FROM$ is replaced by the sender’s email address, which matches whats is called the sender’s identity in Manitou-Mail. On a typical Unix system, this command generally corresponds to the Mail Submission Agent that is installed and responsible for routing the outgoing messages. The sendmail name doesn’t necessarily imply that the MSA is the sendmail SMTP server itself, it can be postfix, or exim, or esmtp, or other programs that have adopted the same name and command line arguments for historical reasons and for the sake of interoperability.
Anyway in manitou-mdx, if this default command is not suitable, the administrator can replace it either globally, or per sender’s identity. A sender’s identity is declared in manitou-mdx configuration file by simply declaring a mailbox with its email address. In the Manitou-Mail user interface, the sender’s identities are configured in the preferences and choosed in the composer.

A typical reason to use different delivery agents when using different identities is that messages may have to be routed to different SMTP servers with different authorization methods. For example, some servers will simply reject messages that have an unexpected From address.
Also there are other cases such as messages from a GMail address that should be routed to a Google SMTP server in order to be signed with a proper DomainKeys header field.
Indeed, while it’s not mandatory, some receivers may pre-sort as spam or reject messages from GMail addresses that are not signed as the Google SMTP servers do with a DomainKey signature (not trusing these messages being the point of DKIM). Let’s see how to route messages written in Manitou-Mail from a GMail address to Google SMTP servers.
I’ve used msmtp for a simple, easy to configure Mail Submission Agent. esmtp is also a candidate but its debian package makes it an alternative to postfix and I happen to want it to supplement postfix, not replace it. msmtp, on the other hand, is a supplementary package that doesn’t conflict with the default MSA.

The procedure to use msmtp is quite simple:
Create a $HOME/.certs directory if none already exists.
Create a $HOME/.msmtprc file (with perm 0600) containing:
# gmail account auth on host smtp.gmail.com port 587 user USERNAME@gmail.com password XXXXXX from USERNAME@gmail.com tls on tls_trust_file /home/daniel/.cert/ThawtePremiumServerCA.crt
Obviously USERNAME is to be replaced by your GMail login.
The cert file is to be extracted from the set of Thawte certificates available at: https://www.verisign.com/support/thawte-roots.zip, with this command:
unzip -p thawte-roots.zip 'Thawte SSLWeb Server Roots/thawte Premium Server CA/Thawte Premium Server CA.pem' > ~/.certs/ThawtePremiumServerCA.crt

And in manitou-mdx’s configuration file, we have something like:
[common] # various things [USERNAME@gmail.com] local_delivery_agent = msmtp -f $FROM$ -t

UPDATE:

the mentioned certificate is no longer accepted, now we should use Equifax_Secure_CA.crt. I located the file in the debian package named “ca-certificates”, so changing my .smtprc to:

tls_trust_file /usr/share/ca-certificates/mozilla/Equifax_Secure_CA.crt

Indexing HTML parts

daniel — Sat, 15 Aug 2009 13:12:44 +0000

While HTML integration is improving in Manitou-Mail, the current version (0.9.12) does not index the contents of HTML parts. This is generally not a problem because messages tend to carry a text version inside a multipart/alternative MIME construct, and that version gets indexed so that the message can still be retrieved by the words it contains. But still, some people send HTML-only messages, in which case we want to automatically extract the text from the HTML and pass it to the indexer.

It’s relatively easy to write a manitou-mdx Perl plugin that does just that, by using a CPAN module to do the HTML to text conversion: HTML::FormatText

Apart from the usual init and process functions that are described in the mdx plugins reference, we need to provide two functions: one that recursively descends the MIME tree to find the html parts, and another that extracts them to text and pass them to the indexer.

sub index_contents {
  my ($fh, $ctxt)=@_;
  my $html;
  my $text;
  {
    local $/;
    $html = $fh->getline();
  }
 
  if (defined $html) {
    my $tree = HTML::TreeBuilder->new;
    $tree->parse_content($html);
    my $formatter = HTML::FormatText->new(leftmargin=>0, rightmargin=>78);
    $text = $formatter->format($tree);
  }
  if (defined $text) {
    Manitou::Words::index_words($ctxt->{'dbh'}, $ctxt->{'mail_id'}, \$text);
  }
}
 
sub process_parts {
  my ($obj,$ctxt) = @_;;
  if ($obj->is_multipart) {
    foreach my $subobj ($obj->parts) {
      process_parts($subobj, $ctxt);    # recurse
    }
  }
  else {
    my $type=$obj->effective_type;
    if ($type eq "text/html") {
      my $io = $obj->bodyhandle->open("r");
      index_contents($io, $ctxt);
      $io->close;
    }
  }
}

The full source code and download link are available on the wiki

Manitou-Mail Blog » manitou-mdx

Routing outgoing mail in manitou-mdx

UPDATE:

Indexing HTML parts