To summarize, we discussed several types of noise. System noise is undesirable variability in the judgments of the same case by multiple individuals. We have identified its two major components, which can be separated when the same individuals evaluate multiple cases: Level noise is variability in the average level of judgments by different judges. Pattern noise is variability in judges’ responses to particular cases. In the present study, the amounts of level noise and pattern noise were approximately equal. However, the component that we identified as pattern noise certainly contains some
...more
