What is the 'on my computer' folder?
It's the folders saved on your computer, as opposed to the ones on your
Exchange or IMAP server.
You don't. Even if you never use any of the other folders you use the
Outbox, and it may be advisable to use the other folders for local
archiving.
If your Exchange/IMAP account is your Default Account, those folders will
appear above On My Computer. Collapse the triangle and all you're dealing
with is one line in your Folder List.
Can I also remove the news server or the mail views items?
You can remove the News Server at Tools > Accounts > News, but you cannot
remove Mail Views.