it can be done.
i used my bivy as a stuff sac, folded it in to a nice bag, and used bungee cords to strap it on the outside between my valice and air mattress.
i put my PT kit and my 2nd pair of boots in it. the stuff is packed tight, but i can carry it, and get to all the stuff without too much...