FacebookTwitterFlickrYoutuberss

Thunderbird Sync

From Zentyal Linux Small Business Server
Revision as of 12:19, 19 August 2014 by Admin (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Title: Author(s): BBKing
Date: 02 Oct 2011
Version(s): 2.0, 2.2
Zentyal profiles: Zarafa



Contents

Intro

The reason for creating this howto is to share my experiences with others (many of the topics covered here come up very often in the forum) and to document the changes I made to finally have a working sync groupware I've been dreaming of the last 5 years...;)

A quick overview over the supported programs:

**Program / Plattform** **Mail** **Contacts** **Calendar** **Notes** **ToDos/Tasks**
Outlook OK OK OK OK OK
Thunderbird 3.1 OK Levelbossmike: Read only!**
Z-Sync: R/W*
SabreDAV: not yet tested
OK - not yet tested
Thunderbird 10.1 OK Levelbossmike: Read only!***
Z-Sync: R/W!****
SabreDAV: not yet tested
- (needs more tests) - not yet tested
Thunderbird 14 OK Levelbossmike: SoGo not compatible with TB14
Z-Sync: Not compatible with TB14
SabreDAV: not yet tested
- (needs more tests) - OK
Android 2.1 OK OK OK - -
Android 4.0 OK OK OK - -
iOS 4-5.1 OK OK OK - OK

Test currently not possible on my system

  • needs SoGO connector, needs patch for correct UTF-8 support
  • needs SoGO connector, same patch breaks UTF-8 support
  • creating contant in Z-Sync:OK, creating contant in Webaccess: after a sync with Z-Sync the contact is gone! Caution, possible data loss!


Enabling Zarafa is pretty straight forward. People, who used to access their mails through Roundcube, might be confused when it comes to enabling Zarafa. I recommend to disable the Webmail interface in "Module Status" and assign IMAP + IMAPS services to Zarafa under the "Groupware" menue.:


Mail Configuration



Zarafa Gateway Configuration


The reason for this is as follows: to my understanding DoveCot serves you your mails over IMAP/IMAPS when you access them through RoundCube. The path to the mailboxes is /var/vmail/<domain>/<username> Zarafa stores the emails in MySQL, and because of a config dependency RoundCube relies on either IMAP or IMAPS and I haven't managed to configure it to access Zarafa.

Important: I just have found out, that the domain you specifiy under Groupware/Virtual domain must be the first email address entry on the User properties page! Otherwise the mails will be still delivered to dovecot and not through Zarafa! So e.g. your virtual domain in Zarafa is snakeoil.com, your user MUST have its first email-address like myuser@snakeoil.com !


Virtual Domains


Groupware


Mail Account Settings

Moving mails to Zarafa

So having activated IMAP+IMAPS for Zarafa, you can copy your old mails from your old server over to Zarafa (I recommend not to switch off the existing server until the new one is running perfectly!). Copying mails is covered well in Documentation/Community/HowTo/MigrateMailToZarafa , just don't forget that your old mailserver needs the user@domain.xx username format, while Zarafa is happy with the username only. If you have problems with imapsync, you can try http://www.yippiemove.com/, they offer a mail migration service - it's not free and I haven't tried the service yet.


Migrating BlackBerry data over Outlook, moving Outlook .pst files to Zarafa

I had to backup my contacts from my BlackBerry to outlook and then export it from there to Zarafa. To do so, install the Zarafa client and the Zarafa Migration Tool on Windows, which used to reside in http://download.zarafa.com/community/beta/6.40/6.40.13beta2-30778/windows/ You can also move your existing Outlook content (mails, contacts, calendars, notes) to Zarafa with the help of the migration tool. The only issues I discovered were, that from a german Outlook the calendar entries were imported to a folder called "Kalender" instead of "Calendar". Same for the contacts here. By using drag&drop you can pretty easily move the entries to the right folder.


To allow connection from the migration tool and Outlook, you have to change in the file /usr/share/ebox/stubs/zarafa/server.cfg.mas the line

    server_bind		= <% $server_bind %>

to

    server_bind		= 0.0.0.0 


Save the file and install the Zarafa-licensed service by

    $ sudo apt-get install zarafa-licensed

to enable access with Outlook and the migration tool.Afterwards do a

    $ sudo /etc/init.d/ebox zarafa restart

and optionally a

    $ sudo /etc/init.d/zarafa-licensed start

In the "Firewall" menue, you have to open up the port for Zarafa access, the standard port is 236, which we will change later on. Create a new service "Zarafa" under the menue "Services" and edit it. Enter these values: Protocol: TCP/UDP Protocol Source port: ANY Destionation port: 236


Services Groupware


and save them. Now head over to the "Firewall" + "Packet filter" menue and under "Filtering rules from external networks to Zentyal" choose "Add new" and enter these values: Decision: Accept Source: ANY (or pre-define an object to grant access only to limited persons) Service: Zarafa

Packet Filter


Save the changes and ebox will restart the firewall module. Now you should be able to connect to Zarafa (you can check this by using nmap to discover open ports). If you get the message "Cannot contact license server" in the migration tool or Outlook, you have to restart thezarafa-licensed service!


The migration is covered in the Zarafa docs in http://www.zarafa.com/wiki/index.php/Zarafa_pst_migration,the only problem I discovered was the difference with a german Outlook and an english Zarafa profile. The contacts were imported to "Kontakte" so I had to manually drag&drop them to the "Contacts" folder. I haven't had too much appointments so I haven't taken care of the calendar yet - probably same problem here, too.

When the migration has worked you are all set to connect with Outlook to Zarafa - just create a new "Zarafa 6" account in Outlook (make sure it's the topmost account in the list if you have other accounts there - otherwise you won't see the content of your folders!!).

Outlook sync is finished with that.

Thunderbird + Lightning access

Coming to Thunderbird, I'll start with the calendar, that was the easiest way for me. I went straight with SSL, so just change in /usr/share/ebox/stubs/zarafa/ical.cfg.mas the port to 8444 (or whatever unused port number you like) insert this port under "Services" in the "Zarafa" entry we created above, save the changes and restart Zarafa+zarafa-ical if you changes the port number.

Install the Lightning extention with the help of your favourite package manager and open the calendar part in Thunderbird. Create a new network calendar, use CalDav as protocol and enter this link: https://<your_ip_here>:<your_ical_ssl_port_here>/caldav/<your_username_here>/

In my case it's: https://192.168.10.88:8444/caldav/bbking/

If everything went well, you shouldn't see the error symbol in the left frame in Lightning and optionally you should be able to enter a new appointment in the calendar, which is supposed to show up in Outlook and the Webaccess GUI of Zarafa under http://<your_ip_here>/webaccess . Lately I experienced high IO loads (600-900k/s) when a Lightning client connected. MySQL was causing the high IO-Load - no solution on this issue yet.

ToDos / Tasks

Recently, I discovered, that you also can sync the Tasks with Thunderbird. Create another calendar, like the one above, but call this time, enter the following: https://<your_ip_here>:<your_ical_ssl_port_here>/caldav/<your_username_here>/Tasks

In my case it's: https://192.168.10.88:8444/caldav/bbking/Tasks

(Pay attention to start Tasks with a capital!) After this step, you should be able to see your tasks in Thunderbird as well.


Turning over to IMAP/IMAPS access: As we allowed IMAP+IMAPS services from Zarafa, it is possible to connect to Zarafa with Thunderbird. At first I had some trouble, because I changed the port number of the Zarafa-server and the Gateway could not connect. When I entered my username/password combo I kept getting "Login to server 192.168.10.88 failed" and can't log in. So pay attention to the line

    server_socket   =       http://localhost:236/zarafa

and change the port number accordingly.

Sending mails with Thunderbird:
Configure Thunderbird as follows:

Server Settigns

SMTP Server


Pay attention that for the IMAP account the username is only the username without the domain, while the smtp setup requires the username with the domain!!!

Z-Push

Setting up z-push access with Android and iPhone4 was pretty straight forward, and it seems to be working. Enter the username, password and only the hostname for the exchange server (NOT http://hostname/Microsoft-Server-ActiveSync !!)

Here is a step-by-step screenshot-diashow for android 2.3 on a Samsung ACE:


Z-Push1

Z-Push2

Z-Push3

Z-Push4

Z-Push5

Z-Push6

Z-Push7

Z-Push8

Z-Push9

Z-Push10

Z-Sync

I ordered a licence for Z-Sync for 19 €

Before doing anything, create a BACKUP of your contacts!! Or use a test-account alternatively!

BUG: creating contant A in Z-Sync: A visible in Webaccess. creating contant B in Webaccess: after a sync with Z-Sync contact B is gone! Caution, possible data loss!

Changed to the directory with the z-sync download (you only get it if you pay for a licence!)

    $ unzip z-syncXYZ.zip <br>
    $ sudo mkdir /usr/share/z-sync <br>
    $ sudo cp -r z-syncXYZ/* /usr/share/z-sync/ <br>
    $ sudo chown -R www-data:www-data /usr/share/z-sync/logs <br>
    $ sudo chown -R www-data:www-data /usr/share/z-sync/syncdata <br>
    $ sudo chmod -R 775 /usr/share/z-sync/logs <br>
    $ sudo chmod -R 775 /usr/share/z-sync/syncdata <br>
    $ sudo /etc/init.d/apache2 reload

Added the following line /etc/apache2/sites-available/zarafa-webaccess:

    Alias /z-sync /usr/share/z-sync

Installed the .xpi file in Thunderbird. Unfortunatelly, on my system with Thunderbird 3.1, after configuring the add-on, it did not work and exited with an error alert: "Server not found at: "

On Thunderbird 10.1 it worked, although you don't get a response when it syncs - definetly a place for improvements. On the first run it somehow dropped ALL of my contacts - luckily I had a backup.

It seems that it works as it should. Contact pictures won't get synced, though.

CardDAV Support

OK, first of all, this gives you only read-only support at the moment, but it's more than nothing.

You'll need the SoGo AddOn for Thunderbird, download and install it from here: http://www.sogo.nu/fr/downloads/frontends.html . Pay attention which version you need! I'm still using Thunderbird 3.1, so please let me know if this is accurate for newer versions as well!
You'll need the LevelBossMike Zarafa Plugin from https://community.zarafa.com/pg/plugins/project/397/developer/rvjr/contact-access-via-webdav-and-carddav
Additional info can be found here: http://www.zarafa.com/wiki/index.php/CardDAV_Interface and I am quoting from that page parts of this howto:


    $ sudo apt-get install php-pear

since the next two commands from the wiki give me an error, I use the next two commands below:

    $ sudo pear install channel://pear.php.net/File_IMC-0.4.3
    $ sudo pear install channel://pear.php.net/HTTP_WebDAV_Server-1.0.0RC7

Change to your download locations with LevelBossMike,

    $ unzip LevelbossMike-Zarafa_Carddav-a140752.zip
    $ cd LevelbossMike-Zarafa_Carddav-a140752
    $ sudo cp zarafa_carddav.php /usr/share/zarafa-webaccess/community_carddav.php
    $ sudo vi /etc/apache2/sites-available/zarafa-webaccess

and add this line to the config:

    Alias /carddav /usr/share/zarafa-webaccess/community_carddav.php

afterwards reload the config:

    $ sudo /etc/init.d/apache2 reload

Now, check your contacts under (login with your Zarafa-users' credentials)

http://z1.snakeoil.com/carddav/

When I opened the page all Umlauts and accented characters where messed up and I started debugging. I pretty soon had the web listing right, but the contacts in Thunderbird where still showing strange letters, so I went on with debugging. Analysing the code showed me, that the plugin always converts to UTF-8 and doesn't check the string for the encoding format and converts it to UTF-8 anyway.

So I made a patch for this and it works for me in Thunderbird. You can skip the patch and try getting the contancts in Thunderbird first to see if they are messed up or not.

    $ cd LevelbossMike-Zarafa_Carddav-a140752/
    $ wget -O utf8.patch http://pastebin.com/raw.php?i=Q2J7tDwt
    $ patch -u zarafa_carddav.php utf8.patch
    $ sudo cp zarafa_carddav.php /usr/share/zarafa-webaccess/community_carddav.php

Here are a few pictures how to configure Thunderbird (sorry for the german labels)


Install the SoGo AddOn:

CardDAV

CardDAV1


Open the AddressBook in Thunderbird:

CardDAV2


CardDAV3

Right click on the new addressbook and choose "synchronize"

It takes a while until the first contacts start to appear, be patient!


CardDAV4

Installing officially signed SSL-certificates

I went to http://www.startssl.com and got my own, officially signed certificates for my Zentyal-Box. Here is a great howto (http://forum.zentyal.org/index.php/topic,616.msg2340.html#msg2340), regarding the installation of the signed certificates.

I will use Javier's howto here and append my remarks to it:


At first, you have to understand, that there are 2 different Apache instances running. One is doing the normal webpages, the other one is serving the web admin pages. These instances have 2 different config files.

Securing webpages (Zarafa's webaccess, Roundcube, etc.): 1) put the certificate in the file /etc/apache2/ssl/ssl.cert 2) put the certficate key file in the file /etc/apache2/ssl/ssl.key 3) create the pem file concatening the two previous files, like this:

   $ cat /etc/apache2/ssl/ssl.cert /etc/apache2/ssl/ssl.key > /etc/apache2/ssl/ssl.pem
   $ chmod 0400 /var/lib/ebox/conf/ssl.pem/ebox.pem

here I went over to the /usr/share/<ebox/zentyal>/stubs/webserver and added the following line to default-ssl.mas :

    $ SSLCertificateChainFile "/etc/apache2/ssl/sub.class1.server.ca.pem"

the sub.class1.server.ca.pem is issued by StartSSL, this will tell the browser who is the CA.

Do a

    $ sudo /etc/init.d/zentyal webserver restart

and your official certificates should be working.

Securing the admin interface: 1) put the certificate in the file /var/lib/ebox|zentyal/conf/ssl/ssl.cert 2) put the certficate key file in the file /var/lib/ebox|zentyal/conf/ssl/ssl.key 3) create the pem file concatening the two previous files, like this:

   $ cat  /var/lib/ebox|zentyal/conf/ssl/ssl.cert /var/lib/ebox|zentyal/conf/ssl/ssl.key > /var/lib/ebox|zentyal/conf/ssl/ssl.pem<br>
   $ chmod 0400 /var/lib/ebox/conf/ssl.pem/ebox.pem

TODO: verify inserting official CA

    $ sudo /etc/init.d/zentyal apache restart

will restart you admin interface, which will have the official certificates as well.

Securing IMAP with an official certificate:

After my initial tests still did not satisfy Thunderbird, I've found a great tutorial on http://www.howtoforge.com/securing-your-ispconfig-3-installation-with-a-free-class1-ssl-certificate-from-startssl

1) if you have a StarSSL certificate, get their CA files and give them speaking names:

    $ cd /home/youruser/<br>
    $ wget https://www.startssl.com/certs/ca.pem <br>
    $ wget https://www.startssl.com/certs/sub.class1.server.ca.pem <br>
    $ mv ca.pem startssl.ca.crt<br>
    $ mv sub.class1.server.ca.pem startssl.sub.class1.server.ca.crt <br>

2) At this point you have different options. Either you go with the official supported way and use the hooks for Zentyal, see http://trac.zentyal.org/wiki/Documentation/Community/HowTo/CustomizeConfigFiles

or, if you want to do a quick hack, you can do it this way (keep in mind, after an update, these changes might be gone!!!):

I refer to www.snakeoil.com.key and www.snakeoil.com.crt as the files you got from StartSSL as your certificate and your passwordless private key. I also suppose, you keep them in your home directory.

    $ cd /home/<youruser>
    $ cat startssl.sub.class1.server.ca.crt startssl.ca.crt > startssl.chain.class1.server.crt
    $ cat www.snakeoil.com.{key,crt} startssl.chain.class1.server.crt > www.snakeoil.com.pem


Depending where you keep your official certificate and the key, please change the locations accordingly.

    $ cd /etc/postfix/sasl
    $ ln -s /home/youruser/www.snakeoil.com.crt smtpd.cert 
    $ ln -s /home/youruser/www.snakeoil.com.key smtpd.key 
    $ ln -s /home/youruser/www.snakeoil.com.pem postfix.pem 
    $ ln -s /home/youruser/startssl.chain.class1.server.crt startssl.chain.class1.server.crt 
    $ cd /etc/dovecot/ssl/ 
    $ ln -s /home/youruser/www.snakeoil.com.pem dovecot.pem 
    $ cd /etc/zarafa/ssl/ 
    $ ln -s /home/youruser/www.snakeoil.com.pem ssl.pem 
    $ ln -s /home/youruser/www.snakeoil.com.crt ssl.cert 
    $ ln -s /home/youruser/www.snakeoil.com.key ssl.key

Not sure why you have to put the certificate in Zarafa's config directory as well, but it cost me about 3 hours of search to find out why my server still served my old, outdated certificate....

You have to pay attention to the references to the ssl files in /usr/share/zentyal/stubs/zarafa/gateway.cfg.mas: Change the refrences to whereever you keep your SSL-files, in my case it would be:

    ssl_private_key_file    =     /etc/zarafa/ssl/ssl.key <br>
    ssl_certificate_file    =     /etc/zarafa/ssl/ssl.cert <br>


4) restart the mail system and Zarafa with:

    $ sudo /etc/init.d/zentyal mail restart
    $ sudo /etc/init.d/zentyal zarafa restart

Check /var/log/zarafa/gateway.log for errors. If you get something like this:

Wed Sep 10 00:59:41 2013: SSL CTX certificate file error: error:02001002:system library:fopen:No such file or directory 
Wed Sep 10 00:59:41 2013: Error loading SSL context, POP3S and IMAPS will be disabled 

then you probably have a reference error to some of the 2 files mentioned under 3)

TODO: check whether the certificates are in place after a reboot. Explain the use of hooks.

Multiple domains in Zarafa: Rewriting outgoing email addresses

Imagine, you have Zarafa enabled for your company, snakeoil.com . You also own a domain for your family, lets call it myfamily.com. You would like to offer all the benefits of a groupware system to your family members as well - there is only one problem: in the Zarafa community version you can only use one domain that can fully exploit the advantages of Zarafa. In fact, when your family members use Thunderbird, it's not a problem - but as soon as you use webaccess and Z-Push, your family members will be sending mails with username@snakeoil.com, but will receive all emails sent to username@myfamily.com.

In webaccess you can define different outgoing addresses, but you have to pay attention from which you will be sending emails (and the one with snakeoil.com will be the default!). In the new webapp you cannot do that - so it's time to find out if we can rewrite the outgoing addresses automatically.

Yes, we can (big thanks to stif http://forum.zentyal.org/index.php/topic,6401.msg37032.html#msg37032), but you have to keep some things in mind. Firstly, make a copy of your current main.cf.mas

    $ cd /etc/zentyal
    $ sudo mkdir stubs
    $ sudo mkdir mail
    $ sudo cp /usr/share/zentyal/stubs/mail/main.cf.mas /etc/zentyal/stubs/mail
    $ sudo vi /etc/zentyal/stubs/mail/main.cf.mas

in main.cf.mas add the following line:

    smtp_generic_maps = hash:/etc/postfix/generic

in main.cf.mas change the following lines from

    dovecot_destination_recipient_limit = 1
    virtual_transport = dovecot

to

    virtual_transport = zarafa<br>
    zarafa_destination_recipient_limit = 1


Save the file and create a new file under /etc/postfix/generic and open it for editing:

    mum@snakeoil.com <use tabs here> mum@myfamily.com
    dad@snakeoil.com <use tabs here> dad@myfamily.com
    bro@snakeoil.com <use tabs here> bro@myfamily.com

you can add as many entries and as many different domains as you want. When done, save the file and create generic.db to make your file readable to postfix and restart postfix itself to make the changes valid.

    $ sudo postmap /etc/postfix/generic
    $ sudo /etc/init.d/zentyal mail restart


From now on, emails sent by your family members should be delivered with the email address username@myfamily.com instead of username@snakeoil.com.

Hint: Don't try this config, when the 2nd domain (=in this case myfamily.com) is not hosted on the Zentyal server (and the emails for myfamily.com are retrieved through the retrieval service in the User Corner). It will cause Zarafa send all emails to the server where myfamily.com is hosted.


REMEMBER: when you change anything in the postfix setup over the web interface, it will NOT take any effect until you don't merge the new file from /usr/share/zentyal/stubs/mail/main.cf.mas with /etc/zentyal/stubs/mail/main.cf.mas !!!!
/etc/zentyal/stubs/mail/main.cf.mas overwrites anything from other sources, this could be a problem when upgrading!!!

Personal tools
Namespaces

Variants
Actions

Zentyal Wiki

Zentyal Doc
Navigation
Toolbox