Ent 2004 - Exchange / Large Mail DB Issues

[mikey79]

All,

Just wanted to put a message out there regarding some issues I've been
having since

implementing Ent 2004 in our Exchange environment.


Now, the disclaimer for this message is that I do understand that a lot
of what I am

proposing or asking about here is quite "far out" and may be quite
unachievable. I

definitely am aware of that already, and would prefer that people don't
bother with the

"there's no way that's possible" type responses.

Maybe the only answer to this is that it is incorporated in a proper
update from MS? (we

can only pray!). But I am hoping that in the mean time there is someone
out there that can

help with any possible workarounds?


Now, first I have to say that we initially setup all our Macs with Ent
X connecting into

Exchange. With a few of our Mac clients having used Ent X for quite a
while for POP/SMTP

(before we got Exchange) and having a relatively stable / happy time
using it I thought it

would be the same connecting through to Exchange... *coughs*

But, I'm sure most of you are aware of the problems that Ent X has
connecting through to

Exchange - all the crashing, DB rebuilds required, etc etc... Not to
mention lack of functionality as compared to PC Outlook or even Mac
Outlook 2001 / OS 9... *coughs louder*


Anyways - when Ent 2004 came out + I was able to get my hands on a new
Exchange server

media kit (to get the Ent 2004 client installer for free! ) - I
eagerly went about

installing this because ANYTHING would be better for us than having to
put up with the

incompetence of Ent X.

OK - so Ent 2004 was installed and pretty much everyone has been happy
ever since. Except

ONE GUY! Who happens to be a DIRECTOR of the company I work for and
as such needs to be kept happy


Now, this guy has a very large mail DB. Around 15,000 messages - with
the Sent Messages

folder alone containing around 9,000 messages. On top of the amount of
messages, he has a

large folder structure for all his client and internal mail to be
stored into.


All the other guys have smaller, much more managable DBs. But this one
guy has this large

DB, and basically the one thing that was good about Ent X was the fact
that it used IMAP

for Exchange connectivity. So, with using IMAP as compared to WebDAV in
Ent 2004 it didn't

need to constantly sync itself with the server and bring every new
message it found

straight away.

It worked more on an "on demand" basis. You click on a folder, and THEN
it would download

messages as required.

So, basically with the large mail DB it didn't have much of a problem.
It left each folder

alone (other than Inbox / Outbox, etc) and didn't worry about sync'n
until a "user action"

required it to.


Now, with all the great features of Ent 2004, the stability of the mail
DB, of it's

Exchange connectivity - the ONE great failing of it is it's use of
WebDAV which constantly

syncs EVERY folder (including all Public Folders) and brings down each
message the minute

it sees it there.


Now, in some ways this a good feature - if you are using a small, more
managable DB then

you will have no need to wait for server message downloads again - and
also Public Folder

viewing is nice and snappy, etc.


The BIG problem here with our friend with the large DB, is the fact
that his DB is so huge

that the cycle that Ent 2004 goes through with sync'n messages simply
takes FOREVER!

Up to 20 minutes he claims - altho I think that may be a bit
exagerrated. But, nevertheless

it takes a good chunk of time - and the whole issue is that the "Inbox"
folder is stuck within that sync and is only carried out ONCE within a
full cycle.

What this results in is a very long delay in received email actually
being shown / downloaded into the "Inbox". From this guy's POV this is
simply unacceptable - and quite understandably so.


Now, obviously there is a case here to say that this DB should simply
need to be cut down

with older messages archived, etc - which would of course solve the
problem - but this guy needs to carry out searches across these
messages often. So, they need to be very easily accesible. And also
need to be "protected", ie. part of our back up process.

Any of Entourage's archival options ("Export" menu feature or drag-drop
to Finder) are

nowhere near sufficient for the amount of messages, and the large
folder structure we are talking about here.


There are many, many workarounds I have thought of, but each brings
it's own pros + cons to

the table, and NOTHING is as good as keeping all the messages within
the single Exchange DB as it is - something that is accessible through
all clients (PC - Outlook, Mac - Ent 2004, Web - OWA) and is also kept
within the Exchange server for data protection / backup.


So, what I was thinking was - the ONLY way around this easily is simply
to make the Ent

2004 > Exchange sync process able to do any of the following:


1. Have the ability to flag EVERY folder within a mail DB as either "on
demand sync" or "always sync". So, basically all folders that aren't
needed to be constantly sync'd can be pushed to "on demand", and start
acting like Ent X IMAP / Exchange folders used to. This will largely
decrease the amount of time a sync takes and solve the issue.

Is there any reason why the switch to WebDAV in Ent 2004 would change
the possibility of this occuring as it used to in Ent X?


2. If the above isn't possible, then make an "Inbox" folder sync occur
at much

higher priorities, so that no matter how long the full sync takes,
messages are still being

received at regular intervals.

Why can't the "Inbox" folder just be constantly sync'n at the same time
as "main sync" process takes place?

Bandwidth issues? Surely not?!



Now, both the above seem to be quite low-level solutions that would
most probably need to become an MS Update to the actual code running
all these functions within the program.

The question is - how easy would these things be to implement for MS?
How could pressure be put on them to implement this in the next
Entourage update?


There are definitely no settings that can get this working within the
Entourage Prefs.


I understand there are MANY things out there on the wishlist for Ent
2004 - but this is a HUGE issue in my books. And as such it is a HUGE
issue with a rather simple fix just waiting to be implemented by MS...
(please MS!)


Are we expected to just have mail DBs with small folder structures and
HUNDREDS of messages rather than THOUSANDS? When at the same time an
Ent X / Exchange could handle this setup with no problem?

Or are we just expected to wait for this sync cycle to complete and
believe "20 minutes isn't that long a time, so it's OK?"

Or are we expected to just change back to using IMAP/SMTP protocols for
Exchange functionality and lose a lot of the other good things that
come with the Ent 2004 / WebDAV setup?


The organisation I work for is rather small + as such only ONE guy is
experiencing this - with the other mail DBs being much smaller. I would
expect this to be much more of a problem at a larger organisation.

It's quite strange I have heard nothing about this here?! Is anyone
else experiencing this? Or are they just putting up with it?


Is there, ANY way of getting the above noted options working by other
means? AppleScript? Some kind of changing of "internal software" prefs
that could somehow get this working how I wanted?


Any help would be MUCH appreciated!
TIA + Merry Christmas!
Michael

 

[Corentin Cras-Méneur]

would prefer that people don't bother with the "there's no way that's
possible" type responses.

Yeah, then you are taking a vast chance that no one will answer...
I'll still try to answer some of your comments :-\

So, what I was thinking was - the ONLY way around this easily is simply
to make the Ent

2004 > Exchange sync process able to do any of the following:


1. Have the ability to flag EVERY folder within a mail DB as either "on
demand sync" or "always sync". So, basically all folders that aren't
needed to be constantly sync'd can be pushed to "on demand", and start
acting like Ent X IMAP / Exchange folders used to. This will largely
decrease the amount of time a sync takes and solve the issue.

<don't hit me>
There's no way that's possible
</don't hit me>

No seriously. This is not supported.

Is there any reason why the switch to WebDAV in Ent 2004 would change
the possibility of this occuring as it used to in Ent X?

Yes. In X Entourage used IMAP for Exchange whereas as you are well
aware, Entourage 2004 uses WebDAV. You can still configure the account
as IMAP though, but then you lose Address Book and Calendar synching.
You can still query the GAL by setting up a Directory Service against
the LDAP.

2. If the above isn't possible, then make an "Inbox" folder sync occur
at much

Again, I wish I could tell you you can do this..... but you can't.

Now, both the above seem to be quite low-level solutions that would
most probably need to become an MS Update to the actual code running
all these functions within the program.

Yep, and maybe on the server side as well.

The question is - how easy would these things be to implement for MS?
How could pressure be put on them to implement this in the next
Entourage update?

You can ask for it, but that looks like a lot of coding both on
Exchange and Entourage to me.

There are definitely no settings that can get this working within the
Entourage Prefs.
Nope.


[...]
Or are we expected to just change back to using IMAP/SMTP protocols for
Exchange functionality and lose a lot of the other good things that
come with the Ent 2004 / WebDAV setup?

Well you asked for workarounds and that's definitively the only one that
comes to my mind.

The organisation I work for is rather small + as such only ONE guy is
experiencing this - with the other mail DBs being much smaller. I would
expect this to be much more of a problem at a larger organisation.

Not a problem here (but we have a 40MB limit on the accounts).

[...]

TIA + Merry Christmas!
Michael

well since I'm a little late I'd wish you a happy new year instead :_)

Corentin

 

[mikey79]

Hi Corentin,

Thanks for replying...

Well, I guess you have now put all my little glimmers of hope to rest
with regards to having a single "pure" Ent 2004 / Exchange account
setup without the need for some work arounds being put in place to
rectify the situation...

I knew that what I was putting forth was quite ambitious, but I wanted
to put it out there before I conceeded defeat...


I guess all I can do now is forward my request to MS and hope that they
actually pay some attention to it?!

Any suggestions with regard to the best way to get their attention on
this? Or simply use the "Send Feedback on Entourage" function from the
"Help" menu in Ent 2004?


I can only hope they do something about this - it is quite a big issue
- though only affecting people will very large DBs - but still,
anything that brings email delivery down to such a slow pace should
surely need to be looked at...


As for what I will do now - what I have decided is to do the following:

- Create a separate email account for user on Exchange specifically for
archival purposes

- Access this account via Ent 2004 / IMAP, and move the bulk of all
archived messages across from the main Ent 2004 / Exchange based
account to this one

- Continue accessing this account via IMAP as needed, all the while
still having the Exchange account being accessed for standard sending /
receiving mail, and all the other Exchange functionality required


So, this way we get the best of both worlds - ability to continue to
access Exchange functionality - synching Calendar / Contacts, Public
Folders, etc - and then also removing the large amount of data that is
slowing down the whole client / server synch process, and have this
available through IMAP "on demand"...


Until MS do something about this issue, I think that this is the best
balance achievable... No loss in functionality, but a major gain in
performance...

A bit annoying to have to have two separate accounts to access for the
user, but I'm sure he'll agree it's a lot better than having to wait
20+ minutes for an email to arrive in his Inbox!
Once again, thanks for your help!
Michael

 

[Corentin Cras-Méneur]

Hi Michael,
[...]

Any suggestions with regard to the best way to get their attention on
this? Or simply use the "Send Feedback on Entourage" function from the
"Help" menu in Ent 2004?

You can sure use this page, and hopefully, they'll also read this
newsgroup (but I can't promise anything).


[...]

As for what I will do now - what I have decided is to do the following:

- Create a separate email account for user on Exchange specifically for
archival purposes

- Access this account via Ent 2004 / IMAP, and move the bulk of all
archived messages across from the main Ent 2004 / Exchange based
account to this one

- Continue accessing this account via IMAP as needed, all the while
still having the Exchange account being accessed for standard sending /
receiving mail, and all the other Exchange functionality required

Yeah, that should sure do the trick (though I agree that it has to be
tedious for you :-\ ).

So, this way we get the best of both worlds - ability to continue to
access Exchange functionality - synching Calendar / Contacts, Public
Folders, etc - and then also removing the large amount of data that is
slowing down the whole client / server synch process, and have this
available through IMAP "on demand"...


Until MS do something about this issue, I think that this is the best
balance achievable... No loss in functionality, but a major gain in
performance...

A bit annoying to have to have two separate accounts to access for the
user, but I'm sure he'll agree it's a lot better than having to wait
20+ minutes for an email to arrive in his Inbox!
Once again, thanks for your help!

I can only imagine. I'm glad you found a way to regain accessibility to
the functions you needed though. I hope the future update they mentioned
in a recent announcement will do something about this.


Corentin