What software are you using on your PC?
How have you set up the connection - POP, IMAP?
It doesn't sound like you are using the gmail app and accessing gmail via a web browser on your pc. I don't think you'd be having problems then. It sounds like you've set up some sort of sync between your email app and a client like Outlook. It's been a while since I did that, but I believe if you set it up as a POP account it will automatically remove it from the server. You would need to set it up using IMAP and then you would have the options as to either delete it from the server or leave it there.
More info is at Setting up & troubleshooting email accounts