The chief barrier to getting to the moon and back again was the energy budget. Putting a pound of payload into earth orbit takes a lot of energy coming off the launch pad, which means a lot of propellant; taking a pound all the way to the moon requires that much more; taking a pound all the way to the moon and back again requires the most of all.
