any way to fix a database that was open during backup?

[arlomedia]

Version: 2008
Operating System: Mac OS X 10.5 (Leopard)
Processor: Intel
Email Client: pop

Hello,

I handle the IT for a 10-person, Mac-based company. Everyone uses Entourage and everyone backs up their computer daily using Synchronize Pro X. This week one of my users lost some email, so I went to restore the Office 2008 Identity, and that's when I learned about the problem of Entourage databases becoming corrupted if they are open while they are backed up. Apparently this is well known, but it was news to me.

Okay, so now that I understand why my backups are corrupted, is there anything I can do about it? I downloaded and ran EntourAid, which restored most of the received mail, but none of the sent mail. In our company, the mail we send is at least as important as the mail we receive, so I'm not out of the woods yet.

It seems that if a shareware author with a version 1.0.1 utility can restore received mail, Microsoft should have a solution for restoring all mail -- like maybe the Database Utility should know how to fix it -- or am I just dreaming here? In any case, this is a shocking discovery that basically invalidates our entire backup process. Once I get past my immediate problem of restoring this lost mail, I have a lot more work to do figuring out a new process that is as convenient as our current backup process (to ensure participation) but that actually serves a purpose by creating usable backups.

Thanks in advance for any suggestions.

-Arlo

 

[Corentin Cras-Méneur]

Hello,
Hi,

I handle the IT for a 10-person, Mac-based company. Everyone uses
Entourage and everyone backs up their computer daily using Synchronize
Pro X. This week one of my users lost some email, so I went to restore
the Office 2008 Identity, and that's when I learned about the problem of
Entourage databases becoming corrupted if they are open while they are
backed up. Apparently this is well known, but it was news to me.

yeah, it sort of makes sense to me though. It's the nature of all big,
monolythic databases. It's bad karma to do anything with them in the
Finder (or through backups) if they are opened.

Okay, so now that I understand why my backups are corrupted, is there
anything I can do about it? I downloaded and ran EntourAid, which
restored most of the received mail, but none of the sent mail. In our
company, the mail we send is at least as important as the mail we
receive, so I'm not out of the woods yet.

I am really afraid there is nothing you can do at this point. The
database format cannot be manipulated and I don't think you can recover
anything from it.

It seems that if a shareware author with a version 1.0.1 utility can
restore received mail, Microsoft should have a solution for restoring
all mail -- like maybe the Database Utility should know how to fix it --
or am I just dreaming here?

The database utility can fix a corrupted database where a little thing
went wrong. It cannot fix a database that was being modified as it was
backed-up. The corruption lies much deeper than a simple messed-up
message or directory.

In any case, this is a shocking discovery
that basically invalidates our entire backup process. Once I get past my
immediate problem of restoring this lost mail, I have a lot more work to
do figuring out a new process that is as convenient as our current
backup process (to ensure participation) but that actually serves a
purpose by creating usable backups.

We use Exchange servers where I work and they backup the accounts from
the servers themselves (I would do the same for IMAP).
It doesn't do a thing for e-mails "on my computer", which would require
independent backups.
Personnally, I archive my e-mails in EagleFiler (but there are many
other options out there).
I had written an article on EagleFiler for the Entourage Help Page a
while ago, but I'm not sure whether Diane ever published it.


Corentin

 

[Diane Ross]

Thanks in advance for any suggestions.

Can you open the Identity? If yes, then try exporting as MBOX files.

Export folders as MBOX files using a script
<http://www.entourage.mvps.org/database/export_mbox.html>

The script will fail on a folder with corruption. At this point, as long as
Entourage is open, you can...

1) try deleting the messages in this folder and see if the script will
resume (I believe it will start over)
2) drag individual messages to the desktop
3) try manually dragging that folder to the desktop
4) skip that folder and try dragging the remaining folders

If you tried exporting as Entourage archive (.rge) file, most likely it will
fail. The archive file doesn't handle corruption. Tip: running this is a
good way to detect corruption and at the same time create a backup of your
data.

Do you have any backups just prior to the problem? If you have a really
large drive with the OS install and Office, you can try rebuilding there.
It's going to take a 500GB or larger drive or larger.

--
Diane
Entourage Help Page <http://www.entourage.mvps.org/>
Entourage Help Blog <http://blog.entourage.mvps.org/>
YouTalk <http://tinyurl.com/bzcrjy> <-- Entourage mailing list
Twitter: follow <http://twitter.com/entouragehelp>

 

[Diane Ross]

I had written an article on EagleFiler for the Entourage Help Page a
while ago, but I'm not sure whether Diane ever published it.

I haven't published yet. It's on the top of my list.

--
Diane
Entourage Help Page <http://www.entourage.mvps.org/>
Entourage Help Blog <http://blog.entourage.mvps.org/>
YouTalk <http://tinyurl.com/bzcrjy> <-- Entourage mailing list
Twitter: follow <http://twitter.com/entouragehelp>

 

[Kerry]

Thanks in advance for any suggestions.

-Arlo

You've unfortunately run into what too many of us have. Corentin is
right though that if the backup database is corrupted that's that and
rebuild is unlikely to resoive the problem. Synchronize Pro I believe
does not do incremental backups is that right? I don't use that and
have found that even if Entourage is open my backups will work. I
sometimes have to go back in time to find one where damage hasn't
happened as a result of say sync services. In your case, you are
trying to recover e-mail so you probably can't go back too far anyway.
Due to the debate about backing up a databaseb while open I have
automator close all my open Windows except for Quickeys before my
database systems start backing up at 3 in the morning using Quickeys.
After they are completed Quickeys restarts the computer when the final
database system has run and then puts the computer to sleep and I'm
ready to go the next day. My suggestion would be to use IMAP so that
your mail is always on an IMAP server and since you're multi-user
(don't know how you manage backup) look at Retrospect which does
incremental backups and is designed for multi-user environments (check
out V8 beta but don't use this but rather the current 6.1 as there are
bugs that they are working on before they go Gold). Data Backup also
provides incremental backups and does this in a rotating fashion (very
nice). Finally, you might want to see how users your are using
Entourage and if they aren't using its advanced functionality you
might want to consider using something else such as just the Apple
apps and Mail (not as feature rich but extremely efficient disk usage
and backup and no significant problems/bugs) or if they are doing
something a little more sophisticated look at Contactizer Pro and/or
Daylite. Scan this Newsgroup as you'll find all the information you
need here to develop a going forward strategy.

 

[arlomedia]

Can you open the Identity? If yes, then try exporting as MBOX files.

I can't open the identity. Entourage says it is corrupted and offers to run the Database Utility; I can open that and do a complete rebuild, but after the rebuild Entourage still says it is corrupted and I end up back where I started.

I've done more testing and learned what changes I need to make to get usable backups in the future. But for someone who doesn't know about this issue and finds themselves in my position, I'm surprised there's no recourse. Since this is a known problem, I'd love to see Database Utility upgraded so that it could handle it. That seems feasible for Microsoft to do, since a shareware developer was able to do most of this work on his own. I've already posted a feature request.

 

[Corentin Cras-Méneur]

I haven't published yet. It's on the top of my list.

Do you always read *every single post* in the groups Diane?? ;-)

Corentin

 

[Diane Ross]

Do you always read *every single post* in the groups Diane?? ;-)

No, just ones that look interesting.

--
Diane Ross, Microsoft Mac MVP
Entourage Help Page <http://www.entourage.mvps.org/>
Entourage Help Blog <http://blog.entourage.mvps.org/>
YouTalk <http://tinyurl.com/bzcrjy> <-- Entourage mailing list
Twitter: follow <http://twitter.com/entouragehelp>

 

[Kerry]

I can't open the identity. Entourage says it is corrupted and offers to run the Database Utility; I can open that and do a complete rebuild, but after the rebuild Entourage still says it is corrupted and I end up back whereI started.

I've done more testing and learned what changes I need to make to get usable backups in the future. But for someone who doesn't know about this issue and finds themselves in my position, I'm surprised there's no recourse. Since this is a known problem, I'd love to see Database Utility upgraded so that it could handle it. That seems feasible for Microsoft to do, since a shareware developer was able to do most of this work on his own. I've already posted a feature request.<<

-----

This is unfortunate but I can assure you that you're not the first one
to end up in this mess with Entourage although I know this won't make
you fell any better. What'd I'd do, if you have the time, is call
Microsoft and ask them to resolve this for you. they won't be able to
as this is a known issue and is unresolvable. Indicate the impact that
this has had on your organization (time, costs etc etc). Ask that this
is submitted to the powers that (tons of this has gone across from
support but to no avail in over a year of Entourage being on the
market). You will not pay for the case if you've used up your free
incidents and if not this incident can be re-activated as it is
unresolved. At least you can express your views to somone but it is
discouraging considering this and so much is problematic in Entourage.

In terms of a method Microsoft could deal with this in Entourage is
just to make Entourage work reliably. Secondly, as things do go wrong
in databases but in the case of well designed, industrial strength
databases not so much, they need to get rebuild to work reliably. In
your case though, due to the nature of the damage a utility probably
could not fix this kind of damage. So backups become critical. The one
thing Microsoft could easily do though is add a function that backs up
Entourage automatically say daily when the system is idle or you close
down Entourage etc etc. I have a database that does this and backs up
its information in background when my system is idle and I have it set
for 14 days worth of backups. However, in the years I have used this
the database only failed once but in that one time these backups were
a life saver. There aren't really any lifeboats on ship Entourage.