A final note is on caffeine. Most people find it’s fine to have caffeine while nursing, and there is certainly no literature suggesting risks to the baby. However, some babies are quite sensitive to caffeine and get very fussy and irritable. If you find this is the case, you may have to avoid it.