Just as on a Special Forces team, everyone on a Scrum team has to know what everyone else is doing. All the work being done, the challenges faced, the progress made, has to be transparent to everyone else. And if the team gets too big, the ability of everyone to communicate clearly with everyone else, all the time, gets muddled.