Research Concepts
Reliability and Validity
A measurement is valid if it measures what it is supposed to measure.
Reliability usually refers to one or more of three kinds of phenomena.
- Inter-rater reliability. This is the degree to which different persons taking the measurements will obtain similar results.
- Test-retest reliability. This is the degree to which the outcomes are stable over time.
- Internal consistency reliability. This is the degree to which the various items on the instrument are in agreement. This can also be applied to different instruments (for example if different instruments are used in pre- and post-testing) and in this case is called parallel reliability.
All measures have an effective range, that is can only discern differences up to a certain granularity. For example, a thermometer used for making candy one's kitchen would not be sufficiently accurate to use in refining petroleum fractions. Researchers need to be sure that their measures are sufficiently sensitive that they can detect the differences under study.
Validity usually refers to one or more of three different kinds of phenomena:
- Concurrent validity. This basically checks to see if your measures agree with a measurement already known to be valid.
- Content validity. This is the extent to which your measure is able to include all of the effect that you are attempting to account for.
- Predictive validity. In order for a measurement to be useful in predicting future events, it must have predictive validity.
A study is internally valid if it is able to show a causal relationship between the dependent and independent variables. A study is externally valid if it is generalizable to the entire population.
