Constant crashes/db corruption with clientson OS X Server accounts

[p0ddie]

Version: 2008
Operating System: Mac OS X 10.5 (Leopard)
Processor: Intel
Email Client: Exchange

Hi there,

let me post some details about the setup so we know what this is about.

Clients: multiple iMac 24" Core 2 Duo, all with Leopard 10.5.7
Server: Xserve with 10.5.6 server, dual 1 TB drives (RAID1) as storage
Server software: Kerio Mailserver 6.7

The clients have Open Directory server accounts (meaning every user can move to any iMac he wants, log in with his credentials and has his desktop and Office 2008 Microsoft User Data files).

The people working in this office all use Entourage 2008 (12.1.9 at present - I updated them every time, hoping the issues will be finally resolved) set up as an Exchange account with Kerio Mail Server. KMS is a fine product and works flawlessly under much heavier load with Apple Mail, Thunderbird and MS Outlook (2003 and 2007 in use in other places). This is not a Kerio issue is what I want to express

There are a number of email accounts, but the "main" email account (sort of the info@domain) is in use by about 4-8 people simultaneously. They coordinate their appointments and correspondence as a company with this one email account. I can't comment on the question if this is a good idea or not, as it's how the office boss likes it...

Anyway, since there is no "online mode" in Entourage as there is in Outlook (so mails are kept only on the server, not on the client), every user account of course has a quite large Entourage database (about 3 - 10 gigs, depending on how well I can tell the people to regularly move old mails to an archive folder).

This seems to be a major issue. The databases constantly crash and have to be remade (rebuilding, of course, will not cure this problem...) which means I delete the database, create a fresh one and let it repopulate (since there is no online mode...) with the mails, adresses and calendar items from the server.

I get bugged every week because someone's database died or behaves funny (mails only showing as blank lines, emails saved to drafts won't re-open or can't be sent, all Office 2008 apps become unresponsive and have to be force-quitted - losing all unsved changes etc etc). This is VERY annoying because they keep telling me they a) are used something more stable from Outlook (which Entourage, by a long shot, is NOT but they don't understand) and b) they are not interested in an alternative to Entourage (without having seen one yet).

What is the nature of these problems? Why can't Entourage just work?

Some more infos: When the Entourage database resided on a local harddrive, the same problems kept reappearing. I tried mobile accounts (the 10+GB database is synced to the server every login and logoff), no cure.

For the account with several users, I have a theory: While Entourage of person A writes 3 mails and deletes 5, Entourage of person B deletes another 5 mails and writes 5 more. Then, when Entourage of person C updates the database, it sees missing mails and mails it has not written (because these are actions from persons A and B) and thus goes into a fetal position thinking the database is corrupt... while it's just some changes on the server.

The other theory is (but then again, this was also the case with local databases) that the databases residing on the server hard drives (server accounts...) give too much of a latency for the database daemon to work correctly -> stating the DB is corrupt or doing silly things.

I won't cry about the use or uselessnes of the monolithic huge Entourage database, the lack of "online mode" or something, I just want a solution so these people can use Entourage and get off my back.

Sorry for not keeping this brief, but I hope I didn't miss any vital information this way.

Pretty please, help me sort this out! Thank you so much!

 

[p0ddie]

One tiny thing I forgot to mention: Clients all have at least 2 GB of RAM (most have 4 GB), server has a whopping 32GB of RAM and is a quadcore 2 GHz Xeon. The server is not very busy, it only serves MS Office / pdf files and is the mail server for a max of 500 mails/day.

Clients are connected to the server via Gigabit on a Linksys/Cisco switch.

 

[Diane Ross]

One tiny thing I forgot to mention: Clients all have at least 2 GB of RAM
(most have 4 GB), server has a whopping 32GB of RAM and is a quadcore 2 GHz
Xeon. The server is not very busy, it only serves MS Office / pdf files and is
the mail server for a max of 500 mails/day.

Let's look at the 32GB. Your computer needs a minimum of 10% free for
virtual memory. So when you look at the 32 GB, how much of this is actually
used up by the 10%?

 

[Diane Ross]

There are a number of email accounts, but the "main" email account (sort of
the info@domain) is in use by about 4-8 people simultaneously. They coordinate
their appointments and correspondence as a company with this one email
account. I can't comment on the question if this is a good idea or not, as
it's how the office boss likes it...

I don't claim to be an expert with Exchange mail, but in general having the
database on a server and not on the computer is not a good idea. Then add
multiple users for that one account and IMHO you are going to have problems.

I'll let the Exchange experts comment on sharing one account with up to 8
users.

Anyway, since there is no "online mode" in Entourage as there is in Outlook
(so mails are kept only on the server, not on the client), every user account
of course has a quite large Entourage database (about 3 - 10 gigs, depending
on how well I can tell the people to regularly move old mails to an archive
folder).

I can help with this. There is a beta version of the script (AutoArchive
Exchange Account to "On My Computer") for Leopard now. All reports seem to
indicate it's working with no problems.

AutoArchive Exchange Account to "On My Computer"

Move all e-mail older than 60 days from your Exchange account folders to
archive folders within the "On My Computer" folder tree. (can be modified)

<http://www.entourage.mvps.org/exchange/autoarchive.html>

There is 'wasted space' in Entourage each time a message is deleted.
However, this is not all bad, since the wasted space is used up with new
data as it is added. This way, over a period of time (assuming that you are
deleting stuff as you go) the database will probably reach an optimum size,
after which the stuff you delete is roughly equal to the stuff you add, and
no further growth takes place.

If you want to compare the size of the database to the actual size, you can
use File --> Export --> select the default action export as Entourage
archive (.rge) file all items. Compare the size of the .rge file to the
database. The difference is wasted space.

Recommended reading:

Optimize Entourage to better work with Exchange
<http://tinyurl.com/56ea7k>

 

[p0ddie]

Let's look at the 32GB. Your computer needs a minimum of 10% free for
virtual memory. So when you look at the 32 GB, how much of this is actually
used up by the 10%?

Hi,

I don't quite get the question, sorry... the system volume is another drive with enough free spae to accomodate for virtual memory, however, the mirrored 1 TB drives is solely for data storage.

 

[p0ddie]

Diane,

thank you for your thorough replies.

I don't claim to be an expert with Exchange mail, but in general having the
database on a server and not on the computer is not a good idea.

If you relate to the server accounts: then Entourage with its need for the database is incompatible with server accounts (be it OS X Open Directory or Windows Active Directory). I don't think the Mac BU of MS intentionally crippled the software *this* much. (by the way, this works flawlessly with Outlook on Windows server accounts, be it with or without offline caching, so it can't be a conceptional issue).

If you talk about leaving mails on the server and not downloading them from the server:

I am sorry, but this is how a majority of email setups look. That's exactly what the online mode in Outlook is for. If you have all your email data on the Exchange server, why have a copy in your Outlook file if you work with server accounts? It would mean you have the mail files in your profile and your mail server, wasting space. Plus, if you swap workstations or just use a different cubicle because you feel extra special today, it would mean you wouldn't have access to all of your archived mails since they are local only - or your workstation would sync for half an hour (depending on the size of the local mail database), cloging the network and putting strain on the mail server.

Also, keeping all mail on the server is - at least since the IMAP standard was developed in '86 - the way to go with email if you have the storage capacity. If your client breaks, then all your mail is gone? Not wit mails on the server. That's why there is such a huge aftermarket for Exchange archiving software (they run out of space but they want their data to be readily available and stored on the server).

Then add
multiple users for that one account and IMHO you are going to have problems.

At least with IMAP accounts, this has never been a problem, but I do agree, it is a rather clumsy way of dealing with a centralized contact solution.

And to play the cynic: regarding the sarbanes oxley act and many other laws regulating the archiving of email, using pop3 to delete your mails from the server or archiving them to local folders without a very decent backup (which a central mailserver tends to have more likely than a local client machine) is grossly negligent.

I'll let the Exchange experts comment on sharing one account with up to 8
users.

I would be very interested in their opinion, especially in terms of Entourage dealing with it.

I can help with this. There is a beta version of the script (AutoArchive
Exchange Account to "On My Computer") for Leopard now. All reports seem to
indicate it's working with no problems.

Sorry, but if my "local" behemoth of an Entourage data file dies every other week, the *last* thing I want is pulling mail off the server and into local clients.

The mails are being archived into the public folder structure of the mail server under a dedicated archive folder (with proper permissions) so Entourage won't have to deal with 2 year old mails in 50GB databases.

Move all e-mail older than 60 days from your Exchange account folders to
archive folders within the "On My Computer" folder tree. (can be modified)

Tried this with a rule in Entourage (why would I need an AppleScript to do that btw, it works flawlessly with rules...) to push mails older than x days into the public folders/archive folder, but - of course - Entourage then caches the entire public folders into its database, too :-/

If you want to compare the size of the database to the actual size, you can
use File --> Export --> select the default action export as Entourage
archive (.rge) file all items. Compare the size of the .rge file to the
database. The difference is wasted space.

an easier way for me is to just check out what the mail server tells me about used space and number of o

 

[p0ddie]

whoops, got cut off there for some reason.

an easier way for me is to just check out what the mail server tells me about used space and number of files in the account. I have seen 3 gigs of mail in the account vs. a 18 gig Entourage DB.

Another thing, just as a side note, they don't just use mail, they also use the calendar and contacts of their own account and the delegated contacts/calendar of their boss (9k contacts and more added every day) in the accounts.

I know this is the stupidest thing to do, having 9k contacts subscribed from another user vs. putting them into the public folders (these are the general contacts after all) but this is how they want to work and I stopped trying to convince them otherwise, it just won't work. The boss thinks he need all 9k contacts with him on his iPhone (he is apparently inapt to use the global adress lookup feature of the iPhone & mail server).