Check the formatting of the Page Number character style to make sure that it
doesn't include the underlining; updating the style definition would be the
quickest way to remove the underlines if it does. Otherwise, you'll just
have to View | Header and Footer, select the PAGE field, and Ctrl+U to
remove the underlining (repeating as needed if you have multiple, unlinked
sections).