It is your wedding and this is a personal choice of the bride and groom.
Some people feel as though children at a wedding, means the adults cannot enjoy themselves.
As a mother of two children, it is a novelty to have a night out. So I would welcome this decision if I was invited to an adults only wedding. I personally would not be offended at all.
If you decide not to have children at your wedding, you must let your guests know their children are not invited, in a polite way. Perhaps, write adults only on the invitation. You or write the names of the guests on the. RSVP cards, so there can be no confusion.
Ultimately, if your budget does not allow more guests or you feel as though the vibe will not be right with children, it is your day and you have the right to decide what you want.
Good luck!