One can express this concern at a more metaphysical level: God is unique; therefore, his speech is unique. Why should we accept that which isn’t the speech of God to have equal authority to that which is the speech of God? If you want to put something else into the “infallible rule” category alongside the very words of God, you will need a good reason.
