Yeah you have to be rooted to remove default apps. Default apps are in the /system partition. Whereas downloaded apps are in the /data partition. Data apps can be uninstalled no problem but, you have to have root access and permissions to remove /system apk's
If you're rooted, and don't care to pay a couple dollars, download Root Explorer and go to system, and then go to app. Click the button at the top to mount /system as r/w. Then long press the app you want to delete, and simply click Delete!
If you don't wanna get Root Explorer, go to a terminal emulator or adb shell on your computer and type:
su
mount -o rw,remount -t yaffs2 /dev/block/mtdblock4 /system
rm /system/app/nameofapp.apk
mount -o ro,remount -t yaffs2 /dev/block/mtdblock4 /system
sync
reboot
And that will remove the /system apps