There was a time when I removed the plug and music did blare through the speakers.
I think that was when I had an incoming call (which paused the music), then I removed the headphone plug to use the phone like a normal person (you can use the headphone to listen during a call though). After the call ended, it resumed the music through the speaker. That's how I recall it anyway.
Otherwise it's a nice feature, even the ipod mini had it (but not my nano 3g).