Can I control the text recognition of my ink?

[AdamK]

Is there anyway I can tell OneNote how to interpret my handwritten ink
without converting it to text?

I know that I can do a text search on my ink. In fact, I think this is
one of the most powerful features of OneNote. However, it doesn't
always recognize my handwriting correctly (which is completely
understandable since my handwriting is horrible!). I would like to
tell OneNote how to interpret my ink without actually visually
converting it to text within the note, so that I can preserve the
spatial arrangement of my ink while still getting the benefits of
accurate text search.

It seems like this is at least theoretically possible since it must
have an 'internal representation' of the text interpretation of each
handwritten word on the page in order for the search to work as it
currently does. I just want to be able to edit this 'internal
representation' so it is more accurate. Perhaps it could even use this
info to interpret more accurately in the future.

It would be really cool if there was a mode where the text
interpretation of each handwritten word would hover below it on the
page, so I could quickly verify the interpretation accuracy for the
whole page and edit any misinterpretations.

 

[Rainald Taesler]

Is there anyway I can tell OneNote how to interpret my handwritten ink
without converting it to text?

To make a long story short: NO.

I know that I can do a text search on my ink. In fact, I think this is
one of the most powerful features of OneNote. However, it doesn't
always recognize my handwriting correctly (which is completely
understandable since my handwriting is horrible!).

The same in my side.

I would like to
tell OneNote how to interpret my ink without actually visually
converting it to text within the note, so that I can preserve the
spatial arrangement of my ink while still getting the benefits of
accurate text search.

Unfortunately that is not possible.

It seems like this is at least theoretically possible since it must
have an 'internal representation' of the text interpretation of each
handwritten word on the page in order for the search to work as it
currently does. I just want to be able to edit this 'internal
representation' so it is more accurate. Perhaps it could even use this
info to interpret more accurately in the future.

It works different from your conceptual model.
There is no way to edit anything in so far.

It would be really cool if there was a mode where the text
interpretation of each handwritten word would hover below it on the
page, so I could quickly verify the interpretation accuracy for the
whole page and edit any misinterpretations.

I can not say anything in the technical details, especially not in how
far the advanced recognition features introduced with Vista are used by
ON. And I can not even say how these features of customizing the
recognition may be used as these features are only available in the
English version of Vista.
Bur as this an "in general" topic if TabletPC usage, I suggest that you
might in the TabletPC newsgroup. Although there has not been much
traffic there lately, it might be that one of the knowledgeable people
like Josh Einstein might come by.

Rainald

 

[AdamK]

To make a long story short: NO.

I was afraid that would be the answer.

It works different from your conceptual model.
There is no way to edit anything in so far.

I just poked around a bit and found the OneNote 2007 XML Schema, the
OneNote 2007 COM API, and OMSpy. Every inked word on a OneNote page is
represented in XML by a <one:InkWord> tag, which includes a
'recognizedText' field. It looks to me like the value in this
'recognizedText' field is exactly what OneNote uses during a text
search. So it seems like it might be possible to create a PowerToy/Add-
in thingy that would allow a person to edit these fields? This would
be dependent on a number of factors, one of which is how "static"
these fields are. That is - is this field left alone by OneNote once
it is computed, or is it getting repeatedly rewritten each time I
write on the page or every few minutes or something? Editing the field
wouldn't do much good if it got overwritten by OneNote on a regular
basis.

I can not say anything in the technical details, especially not in how
far the advanced recognition features introduced with Vista are used by
ON. And I can not even say how these features of customizing the
recognition may be used as these features are only available in the
English version of Vista.
Bur as this an "in general" topic if TabletPC usage, I suggest that you
might in the TabletPC newsgroup. Although there has not been much
traffic there lately, it might be that one of the knowledgeable people
like Josh Einstein might come by.

Thanks for the suggestion, I didn't know about that group. Although, I
think this is turning out to be a OneNote specific question, because
the potential answer seems to lie in developing a tool for OneNote
that provides access to information in the OneNote XML format, which
seems to be specfic to OneNote as opposed to the handling of ink on
the TabletPC more generally.

Any OneNote developer types out there who can speak to whether editing

Rainald

Thanks for your help!

 

[Ilya Koulchin]

I just poked around a bit and found the OneNote 2007 XML Schema, the
OneNote 2007 COM API, and OMSpy. Every inked word on a OneNote page is
represented in XML by a <one:InkWord> tag, which includes a
'recognizedText' field. It looks to me like the value in this
'recognizedText' field is exactly what OneNote uses during a text
search. So it seems like it might be possible to create a PowerToy/Add-
in thingy that would allow a person to edit these fields? This would
be dependent on a number of factors, one of which is how "static"
these fields are. That is - is this field left alone by OneNote once
it is computed, or is it getting repeatedly rewritten each time I
write on the page or every few minutes or something? Editing the field
wouldn't do much good if it got overwritten by OneNote on a regular
basis.

The field will generally be left alone after it is computed. The one
exception would be if you were to add more ink to the page, such that
OneNote decides that some part of the new ink is part of the old word,
or some part of the old word is part of the new word/drawing that got
added. If that were the case, then OneNote would re-parse the ink and
might get different recognition results.

 

[Erik Sojka]

It can already do this, sort of.

OneNote stores multiple versions of the converted text in its index if the
handwriting isn't clear. You can see these entries if you right-click on
an Inked word. The multiple possible converted choices are displayed in
the context menu. So while there is no way to choose a correct one without
converting it permanently to text, it does take into account the
possibility of a wrong conversion.

 

[AdamK]

I tried, and failed, to use the OneNote COM API to change how already
generated ink is recognized. So, unless anyone has any ideas, I'm
going to have to agree with Rainald that it is currently impossible to
do this in OneNote. (Rainald - I should have listened to your original
reply, but I figured it was worth a try anyway!) I think it would be a
really nice feature to add to OneNote in the future - it would provide
a nice compromise between being at the complete mercy of the built-in
recognition, or needing to convert all ink to text.

Here's roughly what I did:
In OneNote I created a new page with a text title and a single ink
word on the page.
I used the code from John Guin's TableSum PowerToy as a starting point
(http://blogs.msdn.com/johnguin/archive/2007/12/10/table-sum-powertoy-
for-onenote.aspx).
I called GetPageContent, with pageInfoToExport =
piBinaryDataSelection, to get the full XML for the page, including the
ink.
I made two changes to the XML:
1. I changed the CDATA for the <T> element in the title.
2. I changed the value of the 'recognizedText' attribute of the
<InkWord> element.
Then I called UpdatePageContent and returned the modified XML.
No exception was thrown, and the title changed to my new value, so I'm
confident the UpdatePageContent call succeeded.
But when I tried searching for the new recognizedText string in
OneNote, it found nothing. And when I called GetPageContent again, and
reretrieved the XML, my change to the title <T> element was intact,
but my change to the recognizedText attribute of the <InkWord> element
was missing, and the attribute was back to its original value.

I then thought that perhaps changing only the recognizedText attribute
of the element wasn't enough for OneNote to notice that it needed to
actually update that element on the page. So I tried again, but this
time, instead of changing the recognizedText attribute of the existing
<InkWord> element, I created a copy of the <InkWord> (and its parent
<OE> stripped of its objectID) and set the recognizedText attribute of
this new element. The result was that the inked word on the page was
successfully duplicated, but on inspection of the XML, its
recognizedText attribute was the same value as the other <InkWord>,
not the value I had set it to.

So, my best guess at what is happening is that when an <InkWord>
element is passed to OneNote via UpdatePageContent (either an already
existing InkWord for modification, or a new InkWord for addition),
OneNote is ignoring the value of the recognizedText attribute, and is
instead calling the recognizer to compute a new value. If this is the
case, I don't see any way to programatically set the value of this
attribute using the COM API, and I don't know of any other way to try
to set it. Of course, I would be happy to be wrong about this!

 

[Rainald Taesler]

I tried, and failed, to use the OneNote COM API to change how already
generated ink is recognized.

Thanks for reporting back.

So, unless anyone has any ideas, I'm
going to have to agree with Rainald that it is currently impossible to
do this in OneNote. (Rainald - I should have listened to your original
reply, but I figured it was worth a try anyway!)

No, IMO it was for sure worthwhile to experiment a bit.
What I said was *assumptions*, not even "educated" ones (as said I'm
knowledgeable at all if it comes to the internals of ON).

I think it would be a
really nice feature to add to OneNote in the future - it would provide
a nice compromise between being at the complete mercy of the built-in
recognition, or needing to convert all ink to text.

You may post this as a suggestion for the next version in "Connect", the
place for submitting suggestions and bug reports to the developers team:
http://connect.microsoft.com/onenote/

When done, pls drop a note with the URL in here so that we can jump in
and vote for your suggestion.

Rainald

 

[AdamK]

You may post this as a suggestion for the next version in "Connect", the
place for submitting suggestions and bug reports to the developers team:http://connect.microsoft.com/onenote/

Ok, maybe its just too early on a Monday for me, but I can't figure
out how to submit a suggestion from the Connect page. I go to the
page, and make sure I am logged in, and then I click the "Feedback"
link on the left side of the page. From the next page I click the
large orange "submit feedback" link, which goes to a form for
submitting feedback, but this form is already set to "Feedback Type:
Bug" and "Feedback Form: Submit a bug". I see no option to specify
that I want to make a suggestion as opposed to submit a bug. I must be
missing something obvious...

 

[Rainald Taesler]

Ok, maybe its just too early on a Monday for me, but I can't figure
out how to submit a suggestion from the Connect page.

Let's try to sort it out ;-)
Pls excuse me should the wording not be 100% correct. Connect speaks
German with me and I have to translate what it says ...
For better understanding pls see my screenshots
ftp://ftp.hs-heilbronn.de/vdb/onenote/on_connect1.png

I go to the page, and make sure I am logged in, and then
I click the "Feedback" link on the left side of the page.
From the next page I click the large orange "submit feedback"
link, which goes to a form for submitting feedback,

When I do this, I get a page which in the middle has the headline
"Select feedback form"
https://connect.microsoft.com/onenote/feedback/CreateFeedback.aspx
Below that there are two sub-headlines for choices, each with a link
below:

- Forms for bugs:
Link "Submit a bug"

- Forms for suggestions:
Link "Suggestion"

Just click on the latter.

but this form is already set to "Feedback Type:
Bug" and "Feedback Form: Submit a bug".

I do not get a form prior to clicking on one of the links.

I see no option to specify
that I want to make a suggestion as opposed to submit a bug. I must be
missing something obvious...

It's riddle.
You may try to "refresh" your browser.

I normally take different route:
When I start with https://connect.microsoft.com/onenote/ I get the
"Welcome ..." screen, the "MS Office OneNote Start Page" (I assume that
you get this one too).
In the middle there are 3 links with bold font and a rather big
font-size:
1. Submit Feedback
2. Search feedback
3. Download ...

Clicking on "Submit Feedback" opens the afore described page
"Select feedback form".

Then I do what's said above.

Never ever had the problems you are reporting.

Rainald

 

[AdamK]

I figured out my problem...

In order to submit suggestions about OneNote 2007, once you have fully
signed up for a Windows Live ID, you still need to do the following:
1. Go to https://connect.microsoft.com/directory/
2. Find the listing for "Microsoft Office OneNote"...
A. Click "Office" under "Categories" on the left hand side.
B. Select "Connection Name (A-Z)" in the "Sort By" drop down in
the main list.
C. "Microsoft Office OneNote" should be towards the bottom of the
page.
3. Click the "Apply Now" link next to "OneNote 2007" in the listing
('applying' is an instant process).
Now Rainald's instructions for making suggestions will work:
ftp://ftp.hs-heilbronn.de/vdb/onenote/on_connect1.png

If you don't "Apply Now", and you just go straight to the Microsoft
Office OneNote Connect website, you will be able to submit bugs, but
not make suggestions. You will see this:
http://mypage.iu.edu/~akrawitz/onenote2007/on_connect_no_apply.png

Probably the strangest thing about all of this is that, not only is
there no way to 'apply' from the Microsoft Office OneNote Connect
webpage, there isn't even any indication on this page that one might
need to apply, or that anything is missing from the page because you
haven't applied. I guess I'll go see if there is a 'Microsoft Connect'
Microsoft Connect website, so I can submit a suggestion about how to
improve the process of submitting suggestions.

 

[AdamK]

You may post this as a suggestion for the next version in "Connect", the
place for submitting suggestions and bug reports to the developers team:http://connect.microsoft.com/onenote/

When done, pls drop a note with the URL in here so that we can jump in
and vote for your suggestion.

Ok, here is my suggestion that users be able to edit the recognized
text for handwriting in OneNote:
https://connect.microsoft.com/onenote/feedback/ViewFeedback.aspx?FeedbackID=428549

Comment, rate, etc... as you see fit.

 

[Rainald Taesler]

Ok, here is my suggestion that users be able to edit the recognized
text for handwriting in OneNote:
https://connect.microsoft.com/onenote/feedback/ViewFeedback.aspx?FeedbackID=428549

Comment, rate, etc... as you see fit.

Thanks for posting the suggestion.
I voted and commented.

Rainald

 

[Rainald Taesler]

I figured out my problem...

Great!!

Thanks fro your comprehensive recipe.
I will keep it for further reference in the future. As I rather often
suggest to post suggestions I'm thinking adding your roadmap as a
footnote to such postings.

[...]

Probably the strangest thing about all of this is that, not only is
there no way to 'apply' from the Microsoft Office OneNote Connect
webpage, there isn't even any indication on this page that one might
need to apply, or that anything is missing from the page because you
haven't applied.

I'm sorry that I did not have this mind. I've been visiting the
Connect-site for ON for more than 2 years and in the meantime I simply
have forgotten that there is an application procedure. ;-) ;-)

I guess I'll go see if there is a 'Microsoft Connect'
Microsoft Connect website, so I can submit a suggestion about how to
improve the process of submitting suggestions.

I'm with you!
Things could really be easier and explained better.

Thanks again
Rainald
P.S. Could you see my screen-shot montage in your browser? Mine refuses
to show it. {grrrhhh}

 

[Knightnet]

Ahh, welcome to the strange world of Microsoft!

I have to say that I fully agree with your comments about the Connect
system, it is arcane to say the least. You end up going round in
circles trying to do something that should be really easy. Anyone
would think that Microsoft really didn't want you to
participate ...... ????

 

[AdamK]

Ahh, welcome to the strange world of Microsoft!

I have to say that I fully agree with your comments about the Connect
system, it is arcane to say the least. You end up going round in
circles trying to do something that should be really easy. Anyone
would think that Microsoft really didn't want you to
participate ...... ????

This thought certainly crossed my mind repeatedly as I searched in
vain for the 'Suggestion' form.

For the record, there actually is a Connect site for Connect - its
called the Microsoft Connect Improvement Program:
https://connect.microsoft.com/Connect
Of course, you need to go to the Connection Directory and find it
there and 'Apply now' before you can participate:
https://connect.microsoft.com/directory
And unlike the OneNote Connect, it took a few hours for me to get
'approved' to be able to participate.

At least they are trying?

 

[Rainald Taesler]

Ahh, welcome to the strange world of Microsoft!

I have to say that I fully agree with your comments about the Connect
system, it is arcane to say the least. You end up going round in
circles trying to do something that should be really easy. Anyone
would think that Microsoft really didn't want you to
participate ...... ????

LOL
In fact they want participation!
But obviously the designers of the web-pages were not fully aware of the
proper workflow :-( :-(

Rainald

 

[Rainald Taesler]

For the record, there actually is a Connect site for Connect - its
called the Microsoft Connect Improvement Program:
https://connect.microsoft.com/Connect
Of course, you need to go to the Connection Directory and find it
there and 'Apply now' before you can participate:
https://connect.microsoft.com/directory

Thanks for reporting!

And unlike the OneNote Connect, it took a few hours for me to get
'approved' to be able to participate.

Yeah. For some sections one gets an immediate access.
For others one has to wait.
Seems as if for the latter some manual/intellectual check is
implemented.

Rainald