I've noticed the same problem. If I check my mail from another device & delete it from the server, it stops re-pushing. But deleting it from the Droid sometimes does nothing 4-5 times in a row, I'll get it re-sent a minute later. Not sure what's causing it, it seems to happen randomly to me.
Yes, I keep getting the same emails sent back to me, I think the issue is that the droid server when refreshed, it keeps rechecking your original email server whether it be yahoo, gmail or hotmail and then when it finds that there are emails that hasn't yet been opened from the original server then it keeps sending them back to you, even though u might have opened it from ur droid device but the fact that it's not read from the original server this keeps it as unread, meaning that you'll have to delete ur email from a PC as well as your droid device if u don't want to encounter this problem