It all really depends on how much information you need to store and how much
effort you want to put into it. Some folks create an Excel spreadsheet for
keeping track of that sort of thing, others use Access. Still others might
add a custom field either to the Tasks or Journal items in Outlook and use
that. In my case, for example, I use Outlook's Journal feature. I use
Category for the company/client, and I added a Paid field to indicate
whether a given item has been paid.
For invoices, when you copy from the Journal and paste into Excel, it goes
across as a table, and can be mathematically manipuplated there.