Skip to main content

Survey Research

Designing Scales

When a survey question seeks to measure an abstract or complex concept—such as an attitude, opinion, or behavior—a single question is often insufficient. Instead, researchers develop scales, which are composite measures composed of multiple individual questions, known as items. Scales allow for more nuanced and reliable measurement of a construct by capturing different facets of it and minimizing the measurement error associated with any single item. The design of these scales is a critical step in creating a valid survey instrument

Common Scale Types

While numerous scales exist, a few types are foundational in survey research due to their versatility and well-understood properties

The Likert Scale

Perhaps the most widely recognized scale, a Likert scale measures the extent to which a respondent agrees or disagrees with a series of statements. Each statement is designed to tap into the underlying construct of interest. Respondents are typically given five, seven, or nine ordered response options, such as “Strongly Disagree,” “Disagree,” “Neither Agree nor Disagree,” “Agree,” and “Strongly Agree.” A key feature of a true Likert scale is that the final score for a respondent is typically calculated by summing or averaging their responses across all items, creating a composite score that represents their overall position on the construct ### The Semantic Differential Scale {-}

This scale is designed to measure the connotative meaning of an object, event, or concept. Rather than asking for agreement, it presents a respondent with a pair of bipolar adjectives at opposite ends of a continuum. The respondent then marks a point along the continuum that reflects their perception. For example, a new product might be rated on a series of scales such as “Innovative . . . . . . . Traditional,” “High Quality . . . . . . . Low Quality,” and “Necessary . . . . . . . Unnecessary.” This technique is particularly powerful for capturing feelings and attitudes in branding, marketing, and psychological research

The Guttman Scale

Also known as a cumulative scale, the Guttman scale is composed of a series of items that are hierarchical in nature. The items are ordered so that a respondent who agrees with a particular item is also expected to agree with all previous, “weaker” items. For example, in measuring attitudes toward recycling, a Guttman scale might include items like: “I am willing to place a can in a public recycling bin,” “I am willing to separate my household trash for recycling,” and “I am willing to pay a small fee to support local recycling efforts.” Agreement with the final, most “difficult” item implies agreement with the first two. A successful Guttman scale indicates that the construct being measured is unidimensional, and it provides a clear, ordinal rank of respondents

Reliability and Validity Considerations

Creating a scale goes far beyond simply choosing a format. The ultimate goal is to produce data that is both reliable and valid. These two concepts are the cornerstones of measurement quality

Reliability

Refers to the consistency and stability of a measurement. A reliable scale will produce similar results under consistent conditions. If a person takes the same survey twice in a short period, a reliable scale should yield roughly the same score, assuming their true attitude hasn’t changed

  • Internal Consistency: This is a key form of reliability for scales. It assesses whether the different items that make up the scale are measuring the same underlying construct. If a scale is designed to measure “job satisfaction,” then all the items should be correlated with one another. A high score on an item about “satisfaction with pay” should correspond with a high score on an item about “satisfaction with work-life balance.” Statistical measures like Cronbach’s alpha are often used to assess internal consistency
  • Test-Retest Reliability: This is assessed by administering the scale to the same group of people at two different points in time. High correlation between the scores from both administrations indicates good test-retest reliability

Validity

Refers to the accuracy of a measurement—that is, whether the scale is actually measuring what it is intended to measure. A scale can be highly reliable (consistent) but not valid (accurate). For example, a scale that consistently measures a person’s height five inches too short is reliable but not valid. There are several forms of validity to consider:

  • Face Validity: At a minimum, does the scale appear to be measuring what it’s supposed to measure? This is a subjective, “at a glance” assessment by experts or respondents. While not a strong form of evidence, a lack of face validity can undermine respondent confidence
  • Content Validity: This assesses whether the scale’s items cover the full range of the construct’s meaning. For example, a scale measuring “Depression” that only asks about sadness but ignores other dimensions like anhedonia (loss of pleasure) or changes in sleep patterns would have poor content validity
  • Criterion Validity: This evaluates how well the scale’s score correlates with an external, established criterion. This criterion could be a behavior, a diagnosis, or the score from another well-established “gold standard” survey. For instance, a new scale measuring political engagement should correlate highly with voting records
  • Construct Validity: This is the most comprehensive form of validity. It assesses whether the scale truly measures the theoretical construct it purports to measure. It involves examining how the scale relates to a variety of other measures, in ways that are predicted by theory. For instance, a valid measure of “self-esteem” should be positively correlated with measures of “confidence” and negatively correlated with measures of “anxiety.”

In summary, designing effective scales is a deliberate and methodical process. The choice of format—be it Likert, Semantic Differential, or another type—provides the structure, but the careful crafting of items and rigorous evaluation of the scale’s reliability and validity are what ultimately ensure that the data collected is meaningful, trustworthy, and scientifically sound