The Fleeting Grace of the Habitable Zone