Thanks for the helpful advice!! I just want to add that if you get your hot water from a hot water storage tank, you also need to insulate it. Insulation with an R-value of at least 24 is preferable. This could reduce heat loss by up to 45% and save you around 10% in water heating costs.
The good news: you can find pre-cut water tank insulation jackets from around $20.