The deterministic equations of classical physics (or the operation of U in quantum physics, for that matter) have no preference for evolving in the future direction. They can be used equally well to evolve into the past. The future determines the past in just the same way that the past determines the future. We can specify some state of a system in some arbitrary way in the future and then use this state to compute what it would have had to be like in the past.
