I think honestly that the MENTAL aspect is the biggest factor for the WHOLE military, not just the infantry. If you're mentally weak, it doesn't matter how physically tough you are, you're gonna give up.
and to be honest from what i've seen and heard, the majority of the infantry are not huge...