How to get more families in Manor Lords

Trying to get new families in Manor Lords is a bit of a puzzling process. Usually in a game like this, you just let nature take its course and watch as your new town expands on its own. Manor Lords' population mechanics are a bit trickier, however; perhaps because the number of available workers and soldiers is the game's main hurdle to overcome. 

If it was too easy to gain population, you'd immediately be able to take advantage of every resource and steamroll the map with a tide of militia.