Yeah, search is your friend. The Droid either doesn't have a CPU temp monitor, or it doesn't work right, I haven't figured out which. But the battery has a thermal runaway point of 140f, which is lower then the CPU (about 170f). You don't want it to get near 140f.
The battery - the battery isn't rated for anything much over 45C. When it comes to overclocking, doing CPU intensive stuff is always dangerous. There is an app called SetCPU that can help with that though (automatically clock down the CPU when the temperature gets too high).
I'm overclocked to 1.2Ghz and I've never gotten to the point where SetCPU had to downclock the phone. Why? Cuz my fingers (holding the phone) told me when it got too hot - then I manually downclocked the phone to 550Mhz or so (when doing games, GPS'ing, and browsing for long periods of time when not plugged in).