There are a variety of gripe waters out there, so make sure you read the ingredients on the labels first. Some gripe waters contain sugar, alcohol and sodium bicarbonate, which are not good things to be giving a baby. Let’s take a look at what ingredients should be in your baby’s gripe water.