door panel on 92 Saturn SC

I just got a componet speaker set, kinda crappy, but the speakers that were in there were rotted out, anyway, with a little time, i got the rear ones in, (from a different set), but now im trying to get the door ones in (actually i want to put the crossover in the door, otherwise i would have to rip out the entire dash to get to wires i need, i know it sounds a lil farfetched, but A) theres no place to mount the all-inclusive crossovers, except the door and B) i cant find any of the wires unless its behind the center console, which i also do not want to take apart. Anyway, i cant get the door panel off, all the little christmas tree thingys are out, and the instructions say to pull the panel streight up, but i cant, all the screws are out, and like the door handle mount wont budge, thats whats still connected, if anyone has any ideas please post.

thanks