First, they have to come at a pace that is politically digestible or there will be a significant backlash from the population, which would be counterproductive. As a society, we need to come to a consensus about what this level would be. For now, we are probably comfortable with about 15,000 to 25,000 migrants a year. Below this range, it is mathematically impossible to prevent the population from shrinking at current fertility levels.
