I think Grandmother Calloway created enough friction with their daughters that he felt like any more from him would be fatal. So he was quiet and kind and nonconfrontational, and his true wishes could only be surfaced after he was gone. It’s kind of sad he lived his life without truly expressing himself and with fearing what would happen if he did.
