Whether you accumulate diamonds, people, things, money, knowledge, it is all the same: it is just different ways of constructing your ego. What materials you use to construct varies from person to person, but you are always using external inputs to construct. That is not the purpose of yoga.