Reliability And Reliability

Reliability Reliability is the extent to which scores obtained from similar or parallel instruments, by different observers or raters, or at different times yield the same or similar results (Streiner, 2003c). Importantly, reliability applies to the scores obtained from an instrument rather than the instrument itself. This is one of the most commonly made measurement mistakes in psychology. One way to establish reliability is to create alternate forms of an instrument. To create alternate forms, we take one instrument and compile another similar instrument that measures the same construct. If they yield the same or similar results, then the instruments are said to be equivalent or parallel forms.

There are several types of reliability.

The degree of agreement between the scores of raters is the interrater reliability of that instrument. Instruments of this nature include semistructured interviews, observational coding systems, behavior checklists, or performance tests.

The most common type of reliability is the internal consistency of an instrument, often indicated by coefficient alpha or Cronbach’s alpha. Internal consistency refers to the similarity or consistency of scores for the items or elements within an instrument. This is an alternative to creating alternate forms. In essence, the items within an instrument are split in half to create two halves, an alternate form within the instrument itself. To have high internal consistency, it is best to have homogeneous items, items that are likely to be highly correlated due to their similarity.

It is important to note that high reliability of scores does not guarantee that those scores are a valid representation of the construct they are intended to measure. Reliability does not guarantee validity; however, it does determine how valid scores obtained from an instrument can be. The upper limit of the validity coefficient can be determined by taking the square root of the reliability

For example, if we administer a measure of depression to a sample of participants all diagnosed with Major Depressive Disorder, the reliability of those scores does not apply if we administer this instrument to the population at large. For the reliability coefficient to be relevant to a certain population, the population needs to be similar to the sample that was used to assess the reliability initially.

How reliable scores should be depends largely on what the instrument is being used for and to which population it will be administered. Reliability is typically lower for research purposes as compared to clinical use. Researchers can afford a ballpark estimate of reliability because of the abstract nature of their work while scores from assessments used in a clinical setting have a direct effect on the life of an individual. Reliability can also be too high, such as in when items on an instrument are overly redundant or are too