Bacteria and Immunity

If low doses of infectious bacteria, such as those on unwashed hands, are enough to cause infection even in healthy people, and bacteria can survive on surfaces for a long time, why do we not all get ill all the time?