Your immediate subordinate managers are responsible for their relationships with their directs. The way you maintain your relationship with your skips (and even levels below that, if it applies to you) is by keeping a strong relationship with your directs and relying on them to maintain relationships with theirs. Any other model for this just doesn't scale.