"Data Loss" due to Jet Engine Cache?

[al]

Hi All,

my problem is that data is stored into databases by background modules, but
at some point in time the data may be inaccessible because it is stored in
some kind of buffer or cache. Data can be read until this point, but after
this point simply no data is stored in the table (no error reported to the
clients). Once I shut down the Storage Module, the data is somehow flushed
into the database. There are multiple similar databases to which data is
stored by the Storage Module, but only one or two may be affected. The
problem seems to occur at random.

I suspect that the data is written into the buffer of the Jet Engine.
According to http://office.microsoft.com/en-us/access/HP010321611033.aspx the
MaxBufferSize may be calculated to 500MB! Workaround would be to reduce the
buffer, but I want to know what causes the Jet Engine to switch to the buffer
and not go back to normal operation.

Sidenote: I noticed in one case that after a couple of days, ~16000 Records
were flushed to the database without me shutting down the Storage Module. So
maybe the MaxBuffer limit was reached and a flush initiated. After this
flush, data was stored into the buffer again, new data didn't appear in the
DB until I shut down the Storage Module (which initiated another flush).

Background information:
I have a system that consists of several VC++ 6 modules running as a Windows
Service and accessing data from an Access Database using DAO. The Storage
Module opens the database upon startup, stores data every few seconds into
one table and closes it only upon shutdown. The Logging Module opens a
different table on demand (e.g. every hour) and stores values to it. No data
is deleted The databases are bound to a running production process, for each
process a new database is created. The databases are opened by client modules
via UNC from the network (however UNC is only used for reading. Writing is
done locally, no UNC). Transactions are not used for writing. While the
caching error occurred, I noticed that the Logging Module received an error
while writing to the table "Table could not update. Currently locked by user
xyz". However nobody else except the Logging Module itself is writing to this
table.

Also note that there is an AntiVirues running on the system, I currently
don't know whether this has on-access scanning capabilities or whether the
MDBs are ignored by the scanner.

Anybody ever had this problem? How can I determine the reason for Jet
switching to the cache as storage? How can I prevent this? Any ideas for
further debugging this issue (tools etc.)?

Is this even the correct group?

thanks
alex

 

[Stefan Hoffmann]

hi Al,

Once I shut down the Storage Module, the data is somehow flushed
into the database. There are multiple similar databases to which data is
stored by the Storage Module, but only one or two may be affected. The
problem seems to occur at random.

Shutdown, Storage Module?

I have a system that consists of several VC++ 6 modules running as a Windows
Service and accessing data from an Access Database using DAO. The Storage
Module opens the database upon startup, stores data every few seconds into
one table and closes it only upon shutdown.

I suspect the service shutdown is what are you referring to? This may be
the problem, if you don't have implemented it correctly.

While you're in a service I would try setting FlushTransaction Timeout
to Zero.

Transactions are not used for writing.

And use tranactions.

Also note that there is an AntiVirues running on the system, I currently
don't know whether this has on-access scanning capabilities or whether the
MDBs are ignored by the scanner.

Some AV can cause troubles, so turning it of for some tests or excluding
*.mdb/*.mde files makes sense.

I have seen some failing services, which where all poorly implemented.
But, I'm not a VC++ guy, so you should discuss the design of a service
in an more appropriate newsgroup.

The most important part for these services to fail was the poor
threading behavior.


mfG
--> stefan <--