Sympa mailing list manager

Sympa version

Tested with Sympa 3.4.4.3.

Overview

Sympa is a mailing list manager with a good web interface, good archiving, good handling of user accounts and preferences, and support for virtual hosts.

The instructions below give you a configuration in which all your mailing lists are on virtual hosts. It takes advantage of support for virtual hosts in Postfix in order to simplify the process of managing lists: configuration needs to be done only once per virtual host, rather than once per list.

This configuration includes spam and virus filtering, using Spamassassin and Clam AntiVirus.

Installation

Prerequisites

Required configuration:

• StandardApache
• StandardPostfix
• StandardSpamassassin
• StandardClam

Install Berkeley DB headers:

sudo apt-get install libdb3-dev

Install MHonArc:

sudo apt-get install mhonarc

Install Perl modules in StandardPerlModules.

Install perl-suid:

sudo apt-get install perl-suid

Create a sympa user and group:

sudo adduser --system --home /usr/local/sympa --group sympa

Make a directories for Sympa web archives and bounces:

sudo mkdir -p /var/mail/sympa/arc
sudo mkdir -p /var/mail/sympa/bounce
sudo chown -R sympa.sympa /var/mail/sympa

Sympa Distribution

Download the Sympa source code. Install /usr/local/configure-wrappers/configure-sympa (make it world-executable).

tar zxf sympa-x.y.z.tar.gz
cd sympa-x.y.z
/usr/local/configure-wrappers/configure-sympa
sudo make

Say yes to CGI::Fast, DBD::mysql and everything else it suggests, except for the LDAP and SSL-related things and the drivers for other databases.

sudo make install

It will ask you lots of questions. Just take the defaults; we're going to set those values later. Copy the Sympa icons into /var/www/icons/sympa (run this command in the source directory):

sudo mkdir -p /var/www/icons/sympa
sudo cp wwsympa/icons/* /var/www/icons/sympa

Fix some mistranslations in Sympa's templates:

sudo perl -pi -e 's/Welcome in/Welcome to/g' /usr/local/sympa/bin/etc/templates/*.tpl
sudo perl -pi -e "s/you never had a password from that server/you've never had a password from this server/g" /usr/local/sympa/bin/etc/wws_templates/*.tpl

Install /etc/init.d/sympa, make it executable, and make start and stop links to it:

sudo chmod a+x /etc/init.d/sympa
sudo update-rc.d sympa defaults

Logging

sudo touch /var/log/sympa

Add this line to /etc/syslog.conf:

local1.*  /var/log/sympa

Restart the logging daemon:

sudo /etc/init.d/syslogd restart

Templates for new lists

Sympa comes with a set of templates for creating various types of lists. For each type of list, there's a directory in /usr/local/sympa/bin/etc/create_list_templates. The defaults aren't very good, so replace them with better ones, which we can keep in /usr/local/sympa/etc/create_list_templates. Download create_list_templates.tar.gz, then:

cd /usr/local/sympa/etc
sudo tar zxvf /path/to/create_list_templates.tar.gz
sudo rm -rf /usr/local/sympa/bin/etc/create_list_templates

This provides three templates:

Discussion list

Open subscription (but confirmation is required). Posting is open to subscribers; non-subscribers must confirm when posting. Archives are public. Only the list owner may review the list of subscribers. The subject line of all postings begins with the list name in brackets. The digest version is sent out twice a week. By default, if a message is posted in both plain text and HTML, subscribers will get the plain text version.

Moderated discussion list

Anyone can post without confirmation, but all postings must be approved by a moderator; otherwise the same as a discussion list.

Restricted working group

Similar to a discussion list, but the owner must also approve subscriptions. Postings from non-subscribers are moderated. The digest version is sent out once a week.

Newsletter

Similar to a discussion list, but only moderators may post (and they must confirm their postings). There is no digest version.

Spam protection for email addresses in archives

Add the following to /usr/local/sympa/conf/sympa.conf:

# Spam protection
spam_protection javascript
web_archive_spam_protection cookie

The cookie option is the only one that works correctly for the web_archive_spam_protection variable. The other options, at and javascript, substitute the list address for the sender's address (see bug report). Unfortunately, the cookie option blocks search engines.

Listmaster alias

Add the following alias to /etc/aliases:

listmaster: root

Then run this command:

sudo newaliases

MySQL Database

Make a MySQL database for Sympa (replace some_password with a password for the sympa MySQL user):

mysql --user root --password < sympa-x.y.z/src/etc/script/create_db.mysql
mysql --user root --password
mysql> grant all on sympa.* to sympa@localhost identified by 'some_password';
mysql> grant all on sympa.* to sympa@"%" identified by 'some_password';

Main configuration file

Edit the following variables in /usr/local/sympa/conf/sympa.conf:

domain

myserver.example.org (use your hostname)

listmaster

listmaster@example.org (use your domain name)

db_type

mysql

db_name

sympa

db_host

localhost

db_user

sympa

db_password

(the database password you chose)

wwsympa_url

http://myserver.example.org/wws (use your hostname)

antivirus_path

/usr/local/clamav/bin/clamscan

antivirus_args

--stdout

Note that this links Clam AntiVirus into Sympa.

Spam Filtering

Install /usr/local/sympa/etc/scenari/include.send.header. This traps any messages that Spamassassin has flagged as spam, and replies to the sender, asking for confirmation in order to send the message.

Sympa Web Interface

Edit the following variables in /usr/local/sympa/conf/wwsympa.conf:

arc_path

/var/mail/sympa/arc

bounce_path

/var/mail/sympa/bounce

use_fast_cgi

1

title

(pick a title, e.g. Example.org Mailing Lists)

In /usr/local/apache/conf/httpd.conf, uncomment this section:

<IfModule mod_fastcgi.c>
    FastCgiServer /usr/local/sympa/bin/wwsympa.fcgi -processes 3 -idle-timeout 120

    <Location /wws>
        SetHandler fastcgi-script
    </Location>
      
    ScriptAlias /wws /usr/local/sympa/bin/wwsympa.fcgi
</IfModule>

Start Sympa:

sudo /etc/init.d/sympa start

Restart Apache:

sudo /etc/init.d/apache restart

You should now find a Sympa web interface at http://myserver.example.org/wws. You won't be able to use it, though, unless you add a lot of aliases to /etc/aliases, including one for each list. Instead of doing this, it's easier to use virtual hosts, as described below.

Making a Virtual Host

Our example is lists.example.info, but you can use any hostname, and you can make as many virtual hosts as you want. The example web interface will be located at http://lists.example.info. In the configuration files described below, make sure to change any occurrences of lists.example.info to the virtual hostname you want to create.

Please note that even though a single Sympa instance can support multiple virtual hosts, it cannot support two lists with the same name on two different virtual hosts (see the Sympa FAQ).

Configure DNS

Configure DNS for example.info, to include A and MX records for lists.example.info. In our example, both point to the same IP address as myserver.example.org.

Create an Apache Virtual Host

Create an Apache virtual host, as described in StandardApache. Add the following to the VirtualHost section of /usr/local/apache/conf/vhost/lists.example.info:

  <Location /wws>
      SetHandler fastcgi-script
  </Location>
      
  ScriptAlias /wws /usr/local/sympa/bin/wwsympa.fcgi

The result should look like this:

<VirtualHost 10.0.0.243:80>
  ServerAdmin webmaster@example.org
  DocumentRoot /var/www/lists.example.info/htdocs
  ServerName lists.example.info
  ErrorLog /var/www/lists.example.info/logs/error.log

  CustomLog /var/www/lists.example.info/logs/access.log common

  # Prevent 'Cross-Site Tracing' attacks
  # (http://www.apacheweek.com/issues/03-01-24#news)
  <IfModule mod_rewrite.c>
    RewriteEngine on
    RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK)
    RewriteRule .* - [F]
  </IfModule>

  <IfModule mod_alias.c>
    RedirectMatch ^/$ http://lists.example.info/wws
  </IfModule>

  <Location /wws>
      SetHandler fastcgi-script
  </Location>
      
  ScriptAlias /wws /usr/local/sympa/bin/wwsympa.fcgi
</VirtualHost>

Create a Sympa Virtual Robot

Create the directory where the Sympa virtual robot configuration will be stored:

sudo mkdir /usr/local/sympa/etc/lists.example.info

In that directory, create a file called robot.conf (you'll want to change at least http_host, wwsympa_url, listmaster and title for your virtual host):

NOTE: if you want multiply listmasters (listmaster in robot.conf), separate them by ","

## Config for lists.example.info Virtual Robot in Sympa

## This hostname will be compared with 'SERVER_NAME' ENV var in wwsympa.fcgi
## To deduce which Virtual Robot is being queried
http_host  lists.example.info

# URL of wwsympa interface
wwsympa_url http://lists.example.info/wws

## Listmaster's email address.  Note that this doesn't use the virtual hostname;
## this means that there is one listmaster for all the virtual hosts, and that the
## listmaster only needs one Sympa password.  If you want each virtual host to
## have a different listmaster, use the email address of the person who should
## be the listmaster for this virtual host - separate multiple emails byt ",".
listmaster listmaster@example.org

## Title for the ML service
## It is displayed as a title in the web interface
title Example Mailing List Service

# Anyone can submit a list creation request, but it must be approved by
# the listmaster.
create_list  public_listmaster

## The default action when entering the web interface
## Can be 'lists', 'home' or 'which'
default_home  home

## You can define custom colors for the Virtual Robot's
## Web interface
bg_color #ffffff
dark_color #006600
light_color #ddffdd
selected_color #006699
shaded_color #ccccff

Configure Postfix and Mail Aliases

Add the following to /etc/postfix/virtual:

# Virtual alias table for lists.example.info (see /etc/aliases)
/^lists\.example\.info$/                 DOMAIN
/^postmaster@lists\.example\.info$/      postmaster
/^listmaster@lists\.example\.info$/      example-listmaster
/^mailer-daemon@lists\.example\.info$/   mailer-daemon
/^abuse@lists\.example\.info$/           abuse
/^(.*)-owner@lists\.example\.info$/      example-lists-bounce+${1}
/^(.*)@lists\.example\.info$/            example-lists+${1}

The idea is that Postfix will take any email whose To address matches the regular expression in the first column, and send it to the corresponding local user in the second column, or to a corresponding aliase defined in /etc/aliases. The first line is required just to tell Postfix to handle the virtual domain. Then we have a few lines of essential aliases to local users. The last two lines require some explanation.

We want to define only one alias for all the lists in the virutal domain, rather than an alias for each list; that way, we only have to modify /etc/aliases when we add a new domain. We can do this by using the 'recipient address extension' supported by Postfix. This is an optional string following a + sign in the the local part of the address. In the example above, the ${1} will result in a local address extension containing the list name. If someone sends an email to list foo@lists.example.info, Postfix will first look for the alias example-lists+foo in /etc/aliases. We don't have to define such an alias; we only need to define the alias example-lists. When Postfix doesn't find an alias matching the complete local address including the extension, it will look for an alias for the local address without the extension, i.e. example-lists. We'll define this alias, and use it to support all the lists for the lists.example.info domain. In the definition of the alias, we'll use the value of the extension (which Postfix will still provide), as shown below, to tell Sympa which list the message was addressed to.

In the right-hand column of the table, you can use whatever name you want instead of example, but it must be different for each virtual host.

If your virtual hostname is CNAME alias, you'll need to include the above stanza twice: once using the canonical hostname in the left-hand column, and once using the CNAME alias. (The entries in the right-hand column can stay the same.) Otherwise, Postfix will get email containing the canonical name, and refuse to accept it, thinking it's for another host.

Add the following to /etc/aliases:

# Aliases for sympa lists at lists.example.info virtual domain
example-lists:         "| /usr/local/sympa/bin/queue ${EXTENSION}@lists.example.info"
example-lists-bounce:  "| /usr/local/sympa/bin/bouncequeue ${EXTENSION}@lists.example.info"
example-listmaster: listmaster@example.org

These aliases must match the ones used in /etc/postfix/virtual. They use the EXTENSION environment variable, whose value is set by Postfix (see man 9 local for details); it contains the recipient address extension, which in our case is the list name.

Tell Postfix to reload its config files:

sudo newaliases
sudo postmap /etc/postfix/virtual
sudo postfix reload

Create Categories

Sympa puts every list in a category, but the default categories aren't generally useful. You'll want to override them for your virtual host. Using the defaults in /usr/local/sympa/bin/etc/topics.conf as a model, create /usr/local/sympa/etc/lists.example.info/topics.conf. If the format of this file isn't clear, see the Sympa documentation.

Restart Sympa and Apache

sudo /etc/init.d/sympa restart
sudo /etc/init.d/apache restart

Create the Listmaster Account

Now you can visit http://lists.example.info/wws. Have Sympa send you the password for user listmaster@example.org. You can then log in and experiment.

Customizing

our requirements and reasons for it

Socialtools runs custom version of templates and wwsympa.fcgi. At the moment, we give groups their own virtual host to run lists, hence we end up with very few lists per virtual host (for arguments that challenge this approach, see domain ecology). That makes list categorization imposed by sympa contra productive - it forces people to choose category when categories do not exist, it also adds unnecesary click extra to reach lists. What we needed instead was list of all lists on the home page of each virtual host - that was the most important requirement. In addition to that, adding hosted by bit at the bottom of the page, having quick tips and puting login/passwd detail in the left column on the home page were layout ideas we took from http://lists.riseup.net.

how we did it

wwsympa.fcgi (in /usr/local/sympa/bin/)

In order to have access to the data structure that holds lists info (perl hash called which) we had to change do_home subroutine that gets called each time lists home page is visited. All we added to it is a single line, call to do_lists subroutine which gives us access to lists data structure for home page templates. Here's do_home now:

## WWSympa Home-Page
sub do_home {
    &wwslog('info', 'do_home');
    # all variables are set in export_topics
    &do_lists();

    return 1;
}

templates

We edited global templates that live in /usr/local/sympa/bin/etc/wws_templates. Two changed templates: home.us.tpl, main.us.tpl, and badded one loginbanner2.us.tpl are attached at the bottom of this page.

issues

We only did it for us language templates. If user switches to other language, list web pages will revert to default sympa ones.

Upgrading

Do these steps as described above:

1. Compile and install the Sympa source distribution.
2. Fix the mistranslations in the templates.
3. Delete /usr/local/sympa/bin/etc/create_list_templates.
4. Install /etc/init.d/sympa.

Upgrading to Sympa 5.0.1

Sympa can upgrade an existing installation by installing over it; this worked well in our case (from 3.4.4.3.), except I had to do two things manually, to fix error messages when starting the new version:

sudo mkdir -p /var/lock/subsys/sympa

And edit /usr/local/sympa/conf/sympa.conf, changing the name of the parameter msgcat to localedir.