Worlds that aren’t watched have a tendency to blink into nothingness and be forgotten, filling the belly of some cosmic terror, creating yet another hole in the pastry. When a world gets eaten and a hole opens up, it’s easier for the baklava-eaters to shove their nasty little hands in and pry more pieces loose.
