The crucial point here is the difference from what happens with falling bodies: a ball may fall, but it can also come back up, by rebounding, for instance. Heat cannot. This is the only basic law of physics that distinguishes the past from the future. None of the others do so.
