Things running in the background can drag the phone down yes, but it depends on what those are. Twitter clients/rss feed readers/email clients/Facebook clients all generally run in the background to give updates, but don't slow the phone down. Personally, I have over 150 apps on my phone (according to the most recent backup I just did with Titanium and that does include system apps) and have no problems with slowdowns.
The biggest culprits tend to be a task killer of some sort, an antivirus of some sort and/or a bady written app.
To the OP, if you're experiencing slowdowns on the phone, there are a couple of apps that will let you know which app is killing the CPU, though I'd say look at your battery usage first, since usually something taking up battery will also kill the CPU. Once you find out the culprit for the slowdown, better advice can be given for handling it.