Never Flinch (Holly Gibney, #4)

by Stephen King

The dog, he thinks. Someone will wonder why it’s out with its leash trailing. Or it will come back. I should have let it go.

I don’t understand this sentence. He did let the dog go, right? So why’s he saying he should have let the dog go?