As the cosmos continued to cool, the photons that had been born in the visible part of the spectrum lost energy to the expanding universe and eventually slid down the spectrum, morphing into infrared photons. Although the visible light photons had become weaker and weaker, they never stopped being photons.
