a novel way to set gains?

One issue with using only a DMM to set the gains, is that the impedance of the subwoofer will vary with frequency and will vary completely different from the impedance that the DMM presents to the amp while testing.  When you connect the DMM to the amplifier, it has a purely resistive load on the amplifer and you will end up getting a different output to the subwoofer.  The DMM has a very, very high resistance which presents a different load on the amplifier.  So you can set the output voltage of your amplifier using the DMM, but the subwoofer will get a completely different output from the amplifier.

The best way is to use your ears.  If your subs start to make that "different sound" you're in the bad zone.  If you subwoofer is a low power handling one, then you'll notice it very early while turning up your gain.

*impedance load*  sorry sorry haemphyst

What I want to do is get a 60 Hz sine wave mp3, play it through the headunit (Pioneer DEH-P680MP from 2005 w/ 6 volt pre-amps) with it at full volume/ as loud as I will use it

It's important, as well, to find the clipping level of the deck.  You could very well find that it may be below the highest volume level you intend to allow it to go.  Here again is where listening to the clipping sound in the pure sine wave tone is important.  Once this deck level is found, you make a note of that level and use it to find gain level of the amp.  And because, as megaman said, impedance varies by frequency, you should have a few tones available (all recorded at 0db) in the target driver's range.

A voltage reading using a meter cannot be accurate in finding the highest clean deck output.

As DYohn said, it is not enough in itself to use the DMM to find proper gain level.   I understand that you are concerned that by setting gain by ear you may not be able to properly match amp output to deck output without giving the sub too much power.  But as you are actually listening to the output (wear protection) when using the hearing method,  you are also able to watch as well as listen to the sub.  If there is too much discrepency (mismatch) between amp and sub you will likely not find a suitable gain level...but you would notice violent behavior from the sub, if that is the case, while gain is at lowest level.  In a case such as yours, use a gentler music source first before switching to the sine wave tones. 

That is to say, test with a music source while amp gain is at lowest setting just to see if you should proceed with trying to match the gear you want to use.

The tones are hard on a driver, so keep the tests short in duration and allow some cooling time if this becomes a difficult setup.

Build the box so that it performs well in the worst case scenario and, in return, it will reward you at all times.