midrange enclosure is too small?

I'm having trouble finding enough space to enclose my 4" midrange, and would like a little advice. However, since this is my first post, I think it's only polite that I should introduce myself first, but feel free to skip to the juicy bits below. Several years ago I was heavily in to my in car audio, even buying a particular car because of the speaker mounting possibilities, but I became disillusioned (and poorer!) after several smash 'n' grabs, which saw me lose thousands of pounds worth of kit. Now older and wiser, I've found my old components out, and have decided to take a tentative step back into my old hobby. My 'best' sounding system ended up being the simplest, with components up front, a single 12 running sealed, and just the one amp (ZX450). I'm after going back to that simple kind of set up, and although I've already learned much from this forum, I could do with a little help please. And so to my question(s): I have the 3-way JL component set (XR 653-CS), but because of a lack of kickwell space, I need to mount everything in the doors. To make sure the drivers don't interfere with each, I'm trying to make a separate encosure within the door for the 4" midrange. Trouble is, JL recommend at least 2.3Liters (0.08 cuft) but I can only find room for half this. In your collective experience, would such a small enclosure really affect a mid so much, especially since it'll be going nowhere near any bass frequencies? Would stuffing the box help? Or since the door cavity won't be completely sealed, and probably more like 'free air' would the affect of the midbass speaker on the mid be low enough for me to get away with simply running them in the same space? Sorry for all the questions, and lack of detail, but I thought I might as well get them all out at once :) Thanks in advance.

Thanks for replying. I'm well used to stripping a car to a bare shell inside, to do a thorough deadening job. I didn't bother when I first started out, but later found it made such a difference as to be almost essential, and money/time well spent. I'll try the mids straight into the door cavity then, and see how we go. :)