SyncML - Push Email, Contacts, and Calendar Data

SyncML (Synchronization Markup Language)

I wanted to pass on some information about how simple SyncML can be if used with a good SyncML client. I've used Funambol with Thunderbird and found it a breeze to setup and use. For those that haven't heard of Funambol or SyncML, here's a tutorial on how to setup with Thunderbird.

About Funambol

...Funambol (pronounced Fu-NOM-bol) is derived from the Latin words funis (rope) and ambulare (walking), meaning a tightrope walker. Just as a tightrope walker must be strong, brave, disciplined and nimble, Funambol continuously balances the needs of our open source community and the market...

Here's a little background on SyncML from the Wiki link above,

SyncML (Synchronization Markup Language) is the former name (currently referred to as: Open Mobile Alliance Data Synchronization and Device Management) for a platform-independent information synchronization standard. Existing synchronization solutions have mostly been somewhat vendor-, application- or operating system specific. The purpose of SyncML is to change this by offering an open standard as a replacement. Several major companies such as Motorola, Nokia, Sony Ericsson, LG, IBM and Siemens AG already support SyncML in their products, although LG do not support it in all their phone models, preferring to use their own proprietary syncing protocols such as LG Sync SPP. Philippe Kahn was instrumental in the founding vision for synchronization with Starfish Software, later acquired by Motorola. The founding vision as expressed by Kahn was: "Global synchronization and integration of wireless and wireline devices".

SyncML is most commonly thought of as a method to synchronize contact and calendar information (personal information manager) between some type of handheld device and a computer (personal, or network-based service), such as between a mobile phone and a personal computer. The new version of the specification includes support for push email, providing a standard protocol alternative to proprietary solutions like BlackBerry.

The Wiki link above also includes a useful chart showing other supported SyncML clients and servers, please take a look to see if SyncML has a solution for you.

Since this tutorial is for Thunderbird contacts and calendar events you will need to install a calendar extension for Thunderbird, my recommendation is Lightning. Lightning is an add-on that is located on the Thunderbird Add-ons page.

Note: This tutorial was created when Funambol .5 was only compatible with Lightning .8. Make sure that you review compatibility requirements on both the Thunderbird Add-ons page and the open source page for Funambol.

After setting up Lightning proceed to download the Funambol plug-in, install, and configure.

1. Download Funambol for ThunderBird. Save the file. The file type is .xpi used for add-ons in Thunderbird. You can save the file to the Thunderbird extensions directory or you can save to a different location and browse for the file location when you add Funambol to Thunderbird.

2. Add Funambol to Thunderbird. If you saved the .xpi file in the Thunderbird extensions folder and restart Thunderbird it should see the new plug-in and ask to install, or you can select Tools - Add-ons - Install and browse to the location where you saved Funambol. Then restart Thunderbird.

3. Configure Funambol to connect to your SyncML server. After Thunderbird has restarted select Tools - Funambol Plug-in... and then select the Options button. This will bring up your Account configuration tab. Enter your SyncML server URL and the username and password for your SyncML account, usually your email address and password.

If your SyncML server requires unique Remote folder names for Contacts and/or Calender events you can specify the Remote name in the Details... button of the Synchronize tab.

4. Synchronize your contacts and calendar events. That completes the configuration of Funambol. Typically your Thunderbird contacts and calendar events will sync with your mail servers Webmail contacts and calendar. Login to Webmail and you should see if the sync was a success.

If in the event the Sync does not work you can view the logs in Funambol from the Log tab - View Log... button.

How to Redirect SMTP Traffic

On occasion you may need the ability to redirect SMTP traffic either because an Internet host is down and your mail server is not able to send to the MX record or your client computers need a quick way to send to a new IP without changing the configuration on all email clients. Here's a few examples of what can be done to redirect SMTP traffic.


Mail Server filters or redirects - Condition filters or domain redirects

1. Condition filters can be very flexible in how a message is redirected. The condition of a filter can be sender, recipient, IP address, header, or even message body content. If the condition is met then action will be taken to redirect the SMTP traffic. A redirect could be to a different hostname or IP or to a different email address.
   
2. Other types of redirects for mail servers are backup domains. Your mail server likely has options to redirect mail if it's to a particular domain, then perform all the normal scans, such as, Antivirus, Antispam, other filters, and redirect to the backup domains IP address or hostname. This type of redirect is an easy way to redirect an entire incoming domain, which works perfectly for a gateway mail server.
   

DNS Server or Hosts file - Resolve domain or MX record to different IP.

1. If you administer your DNS servers you have a lot of control over how client computers resolve SMTP hostnames. Email clients do not resolve recipient domains to an MX record before sending mail, unlike a mail server they send directly to an A record hostname or an IP address. Configuring an email client to use a hostname will give you the ability to change the IP address of the hostname and essentially redirect SMTP at the email client level. For example, let's say your email clients send to mail.your_domain.com and this resolves internally to 192.168.1.8 (your production mail server). If you put up a new mail server on IP 192.168.1.9 you can redirect mail.your_domain.com to 192.168.1.9 using your DNS server giving you global control over all email clients using that hostname. Of course, there's always the problem with DNS cache that prevents this from taking affect immediately, in this case have your clients open a DOS prompt and type ipconfig /flushdns.
   
2. The hosts file is a good way to resolve hostnames on the client computers or the mail server without changing DNS server entries. The hosts file will be used before DNS lookup providing a client side resolver/redirector. This gives you control over individual computers. The hosts file on a windows computer is located in the following directory,
   
   C:\WINDOWS\system32\drivers\etc\
   
   Open the hosts file with any text editor and see the included examples. Please note that this file does not have a file extension - when saving the file make sure there is no extension. As in the above example we can change a few client computers using the hosts file and not affect the entire network. Here's an example hosts file entry,
   
   192.168.1.9 mail.your_domain.com
   
   This can also be done on the mail server computer, however, you need add the MX record and new IP to the hosts file because the mail server will lookup the MX record for delivery instead the the A record. There are lots of different possibilities using the hosts, just keep in mind that your computer will lookup the hostname in the hosts file before looking it up in the DNS server.

Firewall Router - Kerio WinRoute Firewall port mappings.

Firewall routers are another way to redirect SMTP traffic. Most have the ability to map ports to an internal or external IP. Kerio WinRoute firewall is a good example of a firewall router that will perform this task. What it can do is accept a request for port 25 (default SMTP port) and either translate the port to the requested hostname or map to a predefined hostname or IP. The policies for port mapping can be detailed enough where they only apply to certain IP's or the entire network.

Proxy - Stunnel.

If you are interested in a proxy I'd recommend using Stunnel. It not only provides SMTP redirection it can do it securely. Enjoy!

How to read POP3 logs

POP3 Session Explained. When an email client opens a connection to receive email it connects to a predefined hostname or IP address and connects to a listening mail server on port 110 (default port for POP3). The mail server greets the connecting client with domain and possibly mail server type and version. The POP3 mail session continues with single responses from the mail server and client. For example, after the mail server responds with the server greeting message the client is required to respond with the next allowed command for POP3. Once the client and server have negotiated authentication the POP3 client will request how many messages are on the server and retrieve and/or delete messages from the server.

Here's a simplified version of server/client POP3 conversation:

S: = Server

C: = POP3 Client

S: +OK test.com Mail Server
C: USER test
S: +OK
C: PASS ****
S: +OK 2 messages waiting
C: LIST
S: +OK 2 messages (34567890) octets
C: RETR 1
S: +OK
C: DELE 1
S: +OK
C: RETR 2
S: +OK
C: DELE 2
S: +OK
C: QUIT
S: +OK test.com

How POP3 Client Controls Session. As you can see the client controls what happens during the POP3 session. The POP3 client can retrieve any message in any order (although most will retrieve in order) and the POP3 client is in control of telling the POP3 server to delete messages. Only after the POP3 client has requested to delete messages from the server and after the QUIT command will the server actually delete the messages from the server.

Detailed POP3 Session. Typically your mail server will log the simplified version of a POP3 session, however, there can be a lot more going on in the background than what's logged. For example, the client and server may negotiate SSL certificate or different authentication types. Here's a more detailed POP3 log.

Server = »

Client = «

192.168.1.1 = the IP of the connecting POP3 client.

192.168.1.1 » +OK test.com Mail Server v1 POP3
192.168.1.1 « AUTH
192.168.1.1 » +OK List of supported mechanisms follows
192.168.1.1 » CRAM-MD5
192.168.1.1 » PLAIN
192.168.1.1 » LOGIN
192.168.1.1 » DIGEST-MD5
192.168.1.1 « CAPA
192.168.1.1 » +OK Capability list follows
192.168.1.1 » TOP
192.168.1.1 » USER
192.168.1.1 » APOP
192.168.1.1 » EXPIRE NEVER
192.168.1.1 » UIDL
192.168.1.1 » SASL CRAM-MD5 PLAIN LOGIN DIGEST-MD5
192.168.1.1 » STLS
192.168.1.1 « USER test
192.168.1.1 » +OK
192.168.1.1 « PASS *******
192.168.1.1 » +OK 7 messages (2379188) octets
192.168.1.1 « STAT
192.168.1.1 » +OK 7 2379188
192.168.1.1 « LIST
192.168.1.1 » +OK 7 messages (2379188) octets
192.168.1.1 « UIDL
192.168.1.1 » +OK 7 messages (2379188) octets
192.168.1.1 « RETR 1
192.168.1.1 » +OK 594172 octets
192.168.1.1 « DELE 1
192.168.1.1 » +OK Message deleted
192.168.1.1 « RETR 2
192.168.1.1 » +OK 594355 octets
192.168.1.1 « DELE 2
192.168.1.1 » +OK Message deleted
192.168.1.1 « RETR 3
192.168.1.1 » +OK 594404 octets
192.168.1.1 « DELE 3
192.168.1.1 » +OK Message deleted
192.168.1.1 « RETR 4
192.168.1.1 » +OK 595109 octets
192.168.1.1 « DELE 4
192.168.1.1 » +OK Message deleted
192.168.1.1 « RETR 5
192.168.1.1 » +OK 387 octets
192.168.1.1 « DELE 5
192.168.1.1 » +OK Message deleted
192.168.1.1 « RETR 6
192.168.1.1 » +OK 383 octets
192.168.1.1 « DELE 6
192.168.1.1 » +OK Message deleted
192.168.1.1 « RETR 7
192.168.1.1 » +OK 378 octets
192.168.1.1 « DELE 7
192.168.1.1 » +OK Message deleted
192.168.1.1 « QUIT
192.168.1.1 » +OK test.com closing connection
192.168.1.1 *** test@test.com 7 2379188 00:00:29 OK
192.168.1.1 Disconnected

Understanding How POP3 Works. Although in my experience POP3 issues are not that numerous, understanding how the POP3 protocol works and using the logs to isolate POP3 message collection problems can save you lots of time. Some of the more common problems are desktop virus scanners on the mail server computer, and more likely, on the POP3 client computer. Virus scanners will scan the message as it arrives and if not configured properly can give errors that may appear to be a mail server problem, for example, some desktop virus scanners use POP3 proxy to act as a middle man for the email client and the mail server, misconfiguration of proxy use can result in POP3 errors.

POP3 References:

POP3 Protocol RFC
http://www.faqs.org/rfcs/rfc1939.html

POP3 Extensions RFC
http://tools.ietf.org/html/rfc2449