Push email, contacts and calendar with Zimbra 6 Opensource & Z-Push – ActiveSync
As a few of you may have realised, im a great fan of Zimbra however one real short coming is the lack of ability to sync contacts and for real time push email – instead having hack around with Funambol and/or rely on IMAP/POP for email. In the Opensource edition anyway.
For me, i just could not justify the expense of the Network Edition license for ActiveSync and back-up – so i set about finding opensource equivalents to allow me to get the functionality i required. Now whilst this may not scale well for larger deployments, for me it does what i need! If Zimbra offered a more sensible licensing model, that would make Network Edition plausible for 5 users (rather than the 25 user minimum) then i would go down this route…..as it stands they dont, so one of the beauties of opensource software is being able to find alternatives for free!
Another solution for email is to use Zarafa, however its not quite an all in one solution like Zimbra and i have to confess that the webmail is too much like ‘Outlook’ for me! However, it does have one thing going for it and thats the inclusion of ‘Z-Push’. Z-Push is a PHP based opensource ActiveSync solution – ie a way to sync systems other than Exchange with Windows Mobile, Apple iPhone, Android etc. In this case, we’re going to use Z-Push with a specific Zimbra backend plugin to enable push email and syncing of calendar and contacts to an iPhone – without stumping for the Zimbra Network Edition license cost.
You can read more about Z-Push here:
http://z-push.sourceforge.net/soswp/
The above combined with the below, and an Apache server, is all we need:
http://sourceforge.net/projects/zimbrabackend/
So first we need to download Z-Push to our web root, im doing this via an SSH session – utilising version 1.3 RC2 of Z-Push:
billabong:/home/sync/public_html# wget http://download.berlios.de/z-push/z-push-1.3RC2.tar.gz
–2010-02-17 15:08:33– http://download.berlios.de/z-push/z-push-1.3RC2.tar.gz
Resolving download.berlios.de… 195.37.77.141
Connecting to download.berlios.de|195.37.77.141|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 106534 (104K) [application/x-download]
Saving to: `z-push-1.3RC2.tar.gz’100%[======================================>] 106,534 –.-K/s in 0.1s
2010-02-17 15:08:34 (813 KB/s) – `z-push-1.3RC2.tar.gz’ saved [106534/106534]
billabong:/home/sync/public_html#
Now we unzip/untar the package
billabong:/home/sync/public_html# tar zxvf z-push-1.3RC2.tar.gz
z-push/
z-push/state/
z-push/include/
z-push/include/z_RFC822.php
z-push/include/mimeDecode.php
z-push/include/utils.php
z-push/include/z_ical.php
z-push/include/z_tnef.php
z-push/backend/
z-push/backend/diffbackend.php
z-push/backend/vcarddir.php
z-push/backend/ics.php
z-push/backend/maildir.php
z-push/backend/imap.php
z-push/backend/backend.php
z-push/wbxml.php
z-push/memimporter.php
z-push/LICENSE
z-push/zpushdefs.php
z-push/zpushdtd.php
z-push/proto.php
z-push/config.php
z-push/statemachine.php
z-push/index.php
z-push/streamer.php
z-push/INSTALL
z-push/version.php
z-push/streamimporter.php
z-push/compat.php
z-push/request.php
z-push/debug.php
billabong:/home/sync/public_html#
Next we set permissions 777 for the ‘state’ directory, so that the web server can write files – note you could restrict this further if you so desire, ie so that just the user that Apache runs as can write to it:
billabong:/home/sync/public_html# chmod 777 z-push/state
billabong:/home/sync/public_html#
Now we need to edit the Apache configuration , to add an alias:
Alias /Microsoft-Server-ActiveSync /home/site/public_html/z-push/index.php
This will depend on the version of Apache that youre using, for me its Apache2 – the configuration file for my virtual host is stored at /etc/apache2/sites-enabled/
Add the above line, customising as you need, to the correct virtual host configuration file for which you wish to enable Z-Push – if you do not have access to your Apache config then this will not work! Because i want to use this over HTTPS, i need to make sure i add it to the correct virtual host, ie with the SSLEngine on and running on port 443:
<VirtualHost xxx.xxx.xxx.xxx:443>
SuexecUserGroup “#1003″ “#1004″
ServerName sync.fubar.com
Alias /Microsoft-Server-ActiveSync /home/sync/public_html/z-push/index.php
DocumentRoot /home/sync/public_html….
SSLEngine on
….
</VirtualHost>
We now have the basics of Z-Push set up, however we dont yet have it connecting to Zimbra’s SOAP interface to retrieve email, contacts and calendar entries. This is where the Z-Push Zimbra Backend comes in…
Next we go back to the directory to where we downloaded and extracted Z-Push to, inside this directory is another called ‘backend’:
billabong:/etc/apache2/sites-enabled# cd /home/sync/public_html/z-push
billabong:/home/sync/public_html/z-push# ls
backend include memimporter.php statemachine.php wbxml.php
compat.php index.php proto.php streamer.php zpushdefs.php
config.php INSTALL request.php streamimporter.php zpushdtd.php
debug.php LICENSE state version.php
billabong:/home/sync/public_html/z-push# cd backend/
billabong:/home/sync/public_html/z-push/backend# ls
backend.php diffbackend.php ics.php imap.php maildir.php vcarddir.php
These are the different options we have to interface with Z-Push, as you can see there is currently no Zimbra. Lets fix that – note at time of writing revision 18 didnt work for me, revision 16 however does – checking out from subversion/svn you will get the latest by default, however from SourceForge you can also just download specific versions. Clicking on the revision number, ie 16, from the following URL http://sourceforge.net/projects/zimbrabackend/develop, allows you to download just that version. Then click on ‘zimbra.php’, there will then be a ‘download’ link, right click and copy and then use wget with the URL copied to your clip board. We then rename the downloaded file to be just ‘zimbra.php’, note this MUST exist in the backend directory inside Z-Push:
billabong:/home/sync/public_html/z-push/backend# wget “http://zimbrabackend.svn.sourceforge.net/viewvc/zimbrabackend/zimbra.php?revision=16&pathrev=16″
–2010-02-17 15:33:15– http://zimbrabackend.svn.sourceforge.net/viewvc/zimbrabackend/zimbra.php?revision=16&pathrev=16
Resolving zimbrabackend.svn.sourceforge.net… 216.34.181.65
Connecting to zimbrabackend.svn.sourceforge.net|216.34.181.65|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: unspecified [text/plain]
Saving to: `zimbra.php?revision=16&pathrev=16′[ <=> ] 110,039 267K/s in 0.4s
2010-02-17 15:33:16 (267 KB/s) – `zimbra.php?revision=16&pathrev=16′ saved [110039]
billabong:/home/sync/public_html/z-push/backend# mv zimbra.php\?revision\=16\&pathrev\=16 zimbra.php
billabong:/home/sync/public_html/z-push/backend# ls
backend.php ics.php maildir.php zimbra.php
diffbackend.php imap.php vcarddir.php
billabong:/home/sync/public_html/z-push/backend#
So now we have the required files, we just need to configure Z-Push with the Zimbra backend – edit the config.php file in the z-push root directory and add the following lines:
define(‘ZIMBRA_URL’, ‘https://URL-TO-ZIMBRA’);
define(‘ZIMBRA_SYNC_CONTACT_PICTURES’, true);
define(‘ZIMBRA_VIRTUAL_CONTACTS’,true);
define(‘ZIMBRA_VIRTUAL_APPOINTMENTS’,true);
define(‘ZIMBRA_VIRTUAL_TASKS’,true);
define(‘ZIMBRA_IGNORE_EMAILED_CONTACTS’,true);
define(‘ZIMBRA_HTML’,true);
Note im using HTTPS to connect to Zimbra, you may need to modify this to HTTP if required. Then in the same config.php file, edit the variable to make Z-Push use the Zimbra backend:
$BACKEND_PROVIDER = “BackendZimbra”;
That should be pretty much it…fire up your iPhone, or which ever device you wish to use! Note, you dont need to enter a ‘domain’ in the mobile configuration, just leave that blank.
A couple of gotcha’s i came across:
- Folder names cannot have .’s in, this will break the sync for further email folders and/or contacts/calendar.
- One or two folders for some reason did not display on the mobile device, creating a new folder copying the mail over and then renaming the folder worked fine for me
- When adding calendar/contacts entries from the phone, these are synced to Zimbra and then back down again – creating a duplication. This was/should be fixed in Zimbra backend revision 18, however i had some problems with that release!
Hi,
thanks for this nice guide, I did a migration from Zimbra 5 to 6 today and took this opportunity to also install the zimbrabackend + z-push integration.
Everything went well… until i tried to load my Mails.. First he asked me to confirm the self signed certificate and then i’m getting a connection error.
Could you give some more informations on how you configured the account on the iphone?
Thanks for this article.
I have been using Funambol for Contacts, IMAP email and iCal for Calendar. Unfortunately, Funambol wants to sync ALL contacts, including all of the emailed contacts. Really clutters up the iPhone. Not pretty.
So I was just starting to work on this and realized that it uses Apache. Does this mean that you are using this on another box? If not, how did you set it up to work on the same box as Zimbra?
Hey there, yes I chose to run this on another Apache box. However it is possible to run Apache on the same server…. Stay tuned for another article ;)
iMx
@SamG
http://www.slashdevslashnull.com/2010/02/how-to-bind-zimbra-to-a-specific-ip-run-apache-on-the-same-server/
How to bind Zimbra to a specific IP, allowing you to run Apache as well on the same server with multiple IPs
@Bernhard Rode
Have you specified the right protocol inside z-push and apache? ie http or https?
If you try to load the alias in a web browser what do you get? Ie https://sync.domain.com/Microsoft-Server-ActiveSync? Sign in with your Zimbra user login and you should see the following if successful:
GET not supported
This is the z-push location and can only be accessed by Microsoft ActiveSync-capable devices.
If not, id say you have an alias problem….and you’ll get the error youre getting.
iMx
So i followed this excellent guide and when firing up my iphone, i see nothing different. When i surf to http://domain.com/Microsoft-Server-ActiveSync using IE 8, i get the page cannot be displayed. When surfing to the same address on my iphone using safari, i get http error 500
problem accessing /service/extension/zimbrasync. reason:
system failure: extension http handler not found at /zimbrasync
powered by jetty://
That last line tells me that i am at least making it to the zimbra server but not sure what is wrong. Anyone else seeing this or offer a fix?
Here is what is in apache log:
[Mon Mar 01 21:44:23 2010] [error] [client 192.168.123.3] PHP Parse error: syntax error, unexpected ‘:’ in /srv/www/htdocs/sites/z-push/config.php on line 89
and here is line 89 of z-push conf:
define(‘ZIMBRA_URL’, ‘http://ip.ad.dr.ess:8080’);
Our zimbra runs on port 8080. I took out the :8080 and tried to access the above link with IE 8 and got the same page can not be displayed. Can you specify ports in the z-push conf?
Many thanks for this article!
I tried version 18 and had to config:
define(‘ZIMBRA_URL’, ‘ZIMBRA-HOSTNAME’);
to get a connection, so I don’t give any information about the protocol (but https is recommended by my Zimbra instalation!)
Everthing works fine (email, contacts and calendar) but two problems:
- when I send emails from my Iphone2G there is no body transmitted. They contain sender, receiver and subject, but no body :-(
- contact sync has problems with “äöüß” :-)
Any hints?
Hello.
I realize that z-push is continuously developing but.
I am just wondering how much functionality we can expect from z-push and zimbra when it comes to mail, calendar and contact.
For example. email sync. seems to happen whenever the software feels like syncing. Sometimes I get my mail immediately, sometimes I open my email app and then i get it and sometimes it just won’t sync down to my phone.
First time my contacts sync’d with my phone but there after all new contacts created on the desktop do no sync to the phone. Same with the Calendar. @v19 … is this just the current state of z-push for zimbra?
I set this up and it looks like it’s partially working for email. The problem is that the emails come through with all of the html tags showing. They aren’t rendered by the email program at all. When I was doing this through POP, all of the emails came through and looked fine. There are other problems getting emails through POP but they did look fine.
Any thoughts on why the HTML tags are showing up?
I was able to get an attachment through and that worked great.
Centos 5.4, Zimbra 6.04 in a VM, Droid (Android 2.1)
Thanks
I figured out the issue with the html tags. I changed:
define(‘ZIMBRA_HTML’,true);
to
define(‘ZIMBRA_HTML’,false);
This kept the tags from showing up in the emails. My email and calendar are working great now. Thanks for the article.
Apache already runs on the Zimbra server, on port 7780. This is to support aspell.
Hi there
I’m trying to install z-push on a Zimbra server. First of all this isn’t my server, I do not really know what they installed.
Zimbra 6 is installed on a Debian server. The whole Zimbra files are located at /opt/zimbra. The apache server is also installed in this location, I think it is kind of implemented in Zimbra. I do not really know where I can config the alias now. I found two different httpd.conf files one located at /opt/zimbra/conf/httpd.conf the other located at /opt/zimbra/httpd-2.2.14/conf/httpd.conf. I also didn’t found any VirtualHost configured.
Can anybody help?
Hi there,
You should ideally be using a separate instance of Apache, its easier when updating Zimbra etc.
iMx
05/03/10 22:15:25 [1631] END Setup { 73 Folders Loaded }
05/03/10 22:15:25 [1631] POST cmd FolderSync denied: Retry after sending a PROVISION command
I still cannot see the folder list, and my Android based device is still complaining about not being able to find folders etc…
In the debug.txt file, I do see a lot of folder names scrolling by, hence the ’73 Folders Loaded’ message.
I do have a cople of spaces in folder names I use, could that be it? or any other character not being recognized?
Thanks for the info!
Perhaps try setting up a fresh account, ie with nothing in it, and send some test mails? :) It does try to load everything in sequence, if something breaks the sync, it kills it for pretty much everything… But id go back to basics, if that works fine, then try removing some of the funkiness…spaces etc, you might find some new bugs!
Am running with the latest release on my iPhone without too many issues…
Hello,
thanks for the tut :-) I’d like to set up a vm for syncing with my milestone. Which Distro in which version do you use? Debian lenny?
Thanks.
Personally i use KiddieOS, ie Ubuntu – it gives a nicer upgrade path between releases than Debian and just generally simplifies things for me. Running KiddieOS 8.04, Hardy.
If i had my way, id use Opensolaris – but alas thats not supported!
thanks :-)
Great how to, I may rewrite it and dress it up a bit and throw it on my blog. But will give you full credit.
Many thanks for this article! finally it works.
This is my scenario:
* SamsungOmnia, WM6.1 professional,
Sync (mail, contacts, calendar) OTA (WI-FI & GPRS/UMTS) OK
* debian lenny + Apache2 + PHP 5
z-push 1.3 + zimbraBackend 42
* Ubuntu Linux 8.04.4
zimbra 6.07 FOSS
Thansk for the Article its comming onto test right now :-)
I Agree with you with the licensecost – no ife to say more:
I got 2 Therories about Zimbras license policy maybe both are true:
First: They just grazy and life on another Planet. Zimbra is in fact a Beta Products with compared to Exchange 2010 (and microsoft microcosm) limited in function.
But it is even for a service Provider much more expensive – you get the hole microsoft collab suite for 8 dollar including 50 gig space – or email and activesync for 3 dollar) while zimbra cost you at least 3.5 a month BUT only 25 user blocks util you get 100 – you get run often on 7 or 8 dollar tco per seat if youre unlucky with 29 seats or so. But this cost for ms are for enusers. Serviceproviders get muc better conditions… and 5 user blocks
So Theorie 1, they are just nuts and think this will work anyway and look at the small dollar (maybe knowing they cant compete so try to press as much from existing customers….
Theorie 2 – (and i beginning belive that more and more)
Everyone was scared Microsoft buying Yahoo and zimbra and shut it down – so they let buy zimbra by their old friendly vmware (even if they compete on other sectors they where good friends for a time anyway) and just keep it at a near useless or at least to expensive to use and not functional complete (maybe they get fundings for stalling from ms)
if that isnt the case – honestly i dont know which idiot is in charge of the project – nearly everything they do in the big .. is a obiviouse mistake – or a reproduce of existing technology and ways…
dont get me wrong im using zimbra too because of lack of useable alternatives .. and im refusing network too
but honestly i think this project is gonna die if not someone else is gettingto into charge or it get sold to a company who really wanna compete on that market…
there would be sooo much place for inovation – some of that wont be that expensive or hard – it would be so easy to make a competeable pricestructure – but instead of that they invest power into an apliance (hmm if you ask me the very least you need for this application but ok shall be they are vmware)
But still massive lacks of basics (very basics)…
No really im sasd about that situation .. maybe the license let it fork someday – if that happen ill gonna be part of active development
in the meantime thanks for your support for the opensource edition and your timesaving howto