True Voice over IP is not built into Google Voice. Google relays calls via the analog / pots phone system. With Gizmo5 in google's pocket, the "real" functionality of GV and VOIP is comming. Until then, here is a work around. I have this working and call quality is pretty good for not using cell minutes.
1: Get a Google Voice account. I recommend the fully functional version, as in a NEW number assigned to you by google.
2: Go to sipgate.com. Sign up for a sipgate one account (free).
If you have a Gizmo5 account, great, we'll use that to.
3: Sign up for an account at sipsorcery.com. (you must use a windows PC with Silverlight installed)
4: Download and install Sipdroid to your droid. The market also has it. Do not sign up for PBX's.com.
I would recommend signing into Google Voice, Sipgate and Sipsorcery in one window and under different tabs as you will be going back and forth between them.
Then open: Unlimited Free Calling with Google Voice in a NEW window. READ THE ENTIRE PAGE. pay special attention to linking gizmo 5 and sipgate accounts to sipsorcery. Pay special attention to crating sipbindings.
PAY VERY SPECIAL ATTENTION to the dial plan, as this is where the link is made between Google Voice and your droid.
Add your credentials to sipdroid and you can now make calls via the internet and bypass Verizon all together.
(edit: Mods feel free to move this if I placed it in wrong forum)