our observed universe did start out in a low-entropy state, and entropy has been going up ever since. That’s the ultimate origin of the direction of time. Just as there is an arrow of space nearby because we live close to the Earth, there is an arrow of time because we live close (relatively speaking) to the Big Bang.
