The Product Owner has to be reliable, consistent, and available. Without access to him, the team won’t know what to do, or what order to do it in. They rely on the Product Owner for “the vision” and, also, market intelligence on what is important. If the Product Owner isn’t available to the team, the whole process can fall apart.