D
David Rasmussen [MS]
I see there have been some interesting threads on embedded files, and
apologize for my absence in responding to them. Rather than replying on what
became a very deep thread, I thought I'd start a new one with a clear
explanation of how things work.
1. As of B2TR "inserted files" (aka embedded files) are actually stored
within the .one section files. That is, when you insert a file on a page in
OneNote, it gets stored in the relevant section file in your notebook
folders. You no longer will see "thicket" folders (xx_onfiles) alongside the
seciton file, they are not used anymore.
2. We made this change for a number of reasons, but mainly because lots of
other replication systems underneath us (e.g. Windows Offline Files caching,
Folder Share, Groove, other sync tools you might use) could do some pretty
nasty things to us when they are moving/copying/renaming the section files
separately from the embedded files in the folder. That might even result in
loss of those files for the user. That would be bad, we take data integrity
very seriously so we decided to be more robust when co-existing with these
tools. Second, users could get themselves in a mess (e.g. move the .one
section files but not the thicket folder with it etc.). That would also be
bad, could result in data loss etc.
3. There is an exception to the above change. When we are syncing to a
notebook on SharePoint we still leave the embedded files in separate
"thicket" folders alongside the section file. This is for performance reasons
because we have to send whole files on SharePoint when making updates, we
can't just selectively update some bits within the file. So big files are
expensive. Also, the sync/replication systems mentioned above (Windows
Offline Files, Folder Share, Groove etc.) don't generally operate on
SharePoint document libraries, so that problem doesn't exist there.
4. So the definitive repository of your embedded files is in the .one
section files along with your notebooks. Just like all notebook data, we do
replicate them down to our cache for a whole bunch of reasons discussed in
earlier threads (performance, offline availability, etc.). In our cache, for
performance reasons we store them as separate files still (the way we store
things in our cache vs user files is completely separable and we have
different performance / architecture considerations for each). This in no way
means that this is the only or definitive place where these files are stored
(i.e. it's just a cached copy for our operational needs).
5. Our cache is a "garbage collected" store. In simple terms, this means
that when you delete something or close a notebook, it won't disappear from
there immediately. When we've accumulated a certain % of no longer in use
stuff, and the machine is not busy we come along later and clean things up,
remove things that are no longer referenced etc. You can always force this
operation at anytime by choosing the "Optimize All Files Now" button in the
Tools->Options->Save tab. You will note when you do this that it can take
some time, that's why we don't do these operations immediately everytime you
delete/close something. The garbage collected store results in much better
performance for the user.
6. I STRONGLY recommend you ignore the cache folder in the system "Local
Settings" directory and certainly don't go messing with it and messing around
with files in there. That's probably not going to lead to a happy result for
you. If your notebooks are fully in sync and you're a super power user, and
you want to rebuild the cache, yes that can be done (basically delete ALL the
files there including the .onecache file itself, I wouldn't mess with them
individually). This may have been helpful in earlier beta builds when we
still had sync issues. At B2TR and RTM though I really can't think of a good
reason at all to recommend you do this.
7. One very unfortunate limitation of what we could with embedded files in
this version is that our search will not produce hits within the embedded
files. I won't go into all the technical details and issues dealing with
indexing engines etc., but I will say this pains us greatly, it's somethign
we really wanted to do, we thought long and hard about it, but we just
couldn't do that work in this release. It's a lot of work to do right and we
focused most of our search energy on just getting fast indexed search for
OneNote content right.
8. Audio and video recordings made with OneNote are treated just like other
embedded files. All the same issues, challenges etc. apply so we handle them
in the same way.
apologize for my absence in responding to them. Rather than replying on what
became a very deep thread, I thought I'd start a new one with a clear
explanation of how things work.
1. As of B2TR "inserted files" (aka embedded files) are actually stored
within the .one section files. That is, when you insert a file on a page in
OneNote, it gets stored in the relevant section file in your notebook
folders. You no longer will see "thicket" folders (xx_onfiles) alongside the
seciton file, they are not used anymore.
2. We made this change for a number of reasons, but mainly because lots of
other replication systems underneath us (e.g. Windows Offline Files caching,
Folder Share, Groove, other sync tools you might use) could do some pretty
nasty things to us when they are moving/copying/renaming the section files
separately from the embedded files in the folder. That might even result in
loss of those files for the user. That would be bad, we take data integrity
very seriously so we decided to be more robust when co-existing with these
tools. Second, users could get themselves in a mess (e.g. move the .one
section files but not the thicket folder with it etc.). That would also be
bad, could result in data loss etc.
3. There is an exception to the above change. When we are syncing to a
notebook on SharePoint we still leave the embedded files in separate
"thicket" folders alongside the section file. This is for performance reasons
because we have to send whole files on SharePoint when making updates, we
can't just selectively update some bits within the file. So big files are
expensive. Also, the sync/replication systems mentioned above (Windows
Offline Files, Folder Share, Groove etc.) don't generally operate on
SharePoint document libraries, so that problem doesn't exist there.
4. So the definitive repository of your embedded files is in the .one
section files along with your notebooks. Just like all notebook data, we do
replicate them down to our cache for a whole bunch of reasons discussed in
earlier threads (performance, offline availability, etc.). In our cache, for
performance reasons we store them as separate files still (the way we store
things in our cache vs user files is completely separable and we have
different performance / architecture considerations for each). This in no way
means that this is the only or definitive place where these files are stored
(i.e. it's just a cached copy for our operational needs).
5. Our cache is a "garbage collected" store. In simple terms, this means
that when you delete something or close a notebook, it won't disappear from
there immediately. When we've accumulated a certain % of no longer in use
stuff, and the machine is not busy we come along later and clean things up,
remove things that are no longer referenced etc. You can always force this
operation at anytime by choosing the "Optimize All Files Now" button in the
Tools->Options->Save tab. You will note when you do this that it can take
some time, that's why we don't do these operations immediately everytime you
delete/close something. The garbage collected store results in much better
performance for the user.
6. I STRONGLY recommend you ignore the cache folder in the system "Local
Settings" directory and certainly don't go messing with it and messing around
with files in there. That's probably not going to lead to a happy result for
you. If your notebooks are fully in sync and you're a super power user, and
you want to rebuild the cache, yes that can be done (basically delete ALL the
files there including the .onecache file itself, I wouldn't mess with them
individually). This may have been helpful in earlier beta builds when we
still had sync issues. At B2TR and RTM though I really can't think of a good
reason at all to recommend you do this.
7. One very unfortunate limitation of what we could with embedded files in
this version is that our search will not produce hits within the embedded
files. I won't go into all the technical details and issues dealing with
indexing engines etc., but I will say this pains us greatly, it's somethign
we really wanted to do, we thought long and hard about it, but we just
couldn't do that work in this release. It's a lot of work to do right and we
focused most of our search energy on just getting fast indexed search for
OneNote content right.
8. Audio and video recordings made with OneNote are treated just like other
embedded files. All the same issues, challenges etc. apply so we handle them
in the same way.