I've been testing the new Google Search for playing music and here's what I've figured out so far (and some of these have already been posted in this thread). Look at this as a big recap:
Google Search can search for music through a few different apps, such as Slacker, Pandora, LastFm, etc. You have to pre-install these on your phone. It does NOT search from your music files stored on your phone.
When you do your first music search, you get to select from a list of installed apps to search for the music (Slacker, Pandora, LastFM, etc.). If you click on the "Make this Default" box, then it will always use that source for whenever you do a music search. If you don't check that box, then it will ask you each time.
For example, you say "Listen to Led Zeppelin". You specify that you want it to search "Slacker" and check the "make this default" box. From now on, everytime you search for music using Google Search, it will automatically open Slacker.
If later you change your mind don't want Slacker to be your default, you have to go to Settings-->Applications-->Manage Applications-->Slacker and then find the "Launch by Default" button and click "Don't launch by default". Then when you use Google Search for another song, you will get the menu to pick an app.
I wish this could also search from my own music stored on the phone. Maybe this will be added sometime later. For now, you can use a 3rd party app that's capable of searching for music stored on your phone, such as "Choice Dialer" (which costs $4).
mSpot is an app that can stream music from your home computer to your phone. When it streams, it also stores it on your phone. Google Search is currently able to search your music files through mSpot. I don't like mSpot as I transfer all my music files to my phone anyways. But that's just me, some people like this.