This couldn't happen if of course the e-mails are no longer on the e-mail server. You might want to check that
you are not saving copies on the e-mail server. It shouldn't happen in any case but check that first.
What version of Outlook do you have? What sort of mail account(s)? When
you get duplicates, do you get *all* the mail you've left on the server
over again, or just some of it?