Heh! Trap for new switchers!
Mac OS X keeps file management away from all applications. You always
delete and move files from the Finder. It keeps the place tidier that
way.
Unless, of course, you want to unleash your inner geek, and utter
unix-flavoured rm commands from terminal. ;-)