Learn in your language
Skip to content
A Level · Cambridge (CIE) · 9990

Evaluate validity, reliability and replicability

Judge whether a measure captures its intended behaviour, whether it is consistent and whether another researcher could repeat the procedure.

Section Paper 2 · Research Methods

Cambridge International AS & A Level Psychology 9990 · 2028–2030 · Version 1 · MC-VALIDITY / MC-RELIABILITY

Original StudyForA teaching mapped to the Cambridge 2028–2030 syllabus. These lessons are independent study support, not Cambridge material, official mark schemes or awarded grades. No lesson reproduces a research article or Cambridge question.

Before you begin

  • This pathway supports the 2028–2030 syllabus only. AS candidates take Papers 1 and 2; A Level candidates also study two of the four specialist options and take Papers 3 and 4. The separate 2027 syllabus is not mapped here.

Work on paper. Try each question before revealing help, and explain why your method works.

Quick readiness check

Two observers agree on 95 of 100 coded events. Which reliability type is being assessed?

What you will learn

  • Explain validity and ecological validity
  • Evaluate subjectivity, objectivity, demand characteristics and generalisability
  • Distinguish reliability types and replicability
  • Apply evaluation to new research scenarios

Validity asks whether the evidence supports the intended interpretation

A measure has stronger validity when it captures the intended concept rather than a different one. Ecological validity concerns how far a task or setting reflects the real-life behaviour of interest. Subjective measures depend on judgment or report; objective rules can reduce some judgment, but a consistently scored measure can still measure the wrong thing.

Demand characteristics occur when participants infer the study purpose and change their behaviour. Generalisability concerns applying findings beyond the studied people, setting or time. A narrow sample or artificial task may limit generalisation, but one limitation does not automatically make all evidence useless.

Reliability is consistency; replicability is whether the procedure can be repeated

Inter-rater or inter-observer reliability concerns agreement between different people recording or scoring the same behaviour. Test-retest reliability concerns whether a measure gives consistent results when repeated under comparable conditions. Low reliability can add noise and weaken comparisons; high reliability alone does not prove validity.

A replicable procedure is described clearly enough for another researcher to repeat it. Standardised instructions, materials, timing, sampling and scoring support replication. A replication may produce different findings because of sampling variation or changes in context; the procedure should report what was held constant and what changed.

Sources for this lesson

Pause and explain

A clear procedure can be repeated, but the repeated study finds a different result. What can be concluded?

Put the idea to work

Worked example

Two observers count helping behaviour using the same video but disagree about what counts as a ‘help’. Which quality is most directly in question?

Show the worked solution
  1. The same behaviour is being coded by different observers.
  2. The issue is agreement between observers, which is inter-observer reliability.
  3. The definition may also need operational clarification; agreement alone would not prove the measure captures all helping behaviour.

Answer Inter-observer reliability is weak; clearer operational categories may improve agreement, though validity still needs separate evaluation.

Try a topic test

Choose Easy, Normal or Difficult. Save your answers and open worked review after finishing.

Test this topic →

From guided practice to a new situation

Write your answer and reasoning first. Use a hint only if you are stuck. The model solution is for self-checking; your written work is not automatically marked.

1 · Guided

A ‘confidence’ measure records how many times a participant taps a desk. Give one validity concern and one way to improve the measure.

Give me a hint

Ask whether tapping directly captures the intended construct.

Compare my reasoning
  1. Desk taps may reflect restlessness, habit or the room rather than confidence.
  2. The operational measure may lack construct validity.
  3. Use a justified self-report or a validated behaviour measure and compare more than one indicator if appropriate.

Desk-tap frequency may not validly measure confidence; justify a direct measure and check whether it corresponds to the intended construct.

Look for these in your work

  • Mismatch identified
  • Validity term applied
  • Specific improvement proposed
2 · Independent

A test gives similar scores each time but measures reaction to bright light rather than the intended fear of crowds. Evaluate reliability and validity.

Give me a hint

Separate consistency from whether the right idea is measured.

Compare my reasoning
  1. Similar scores suggest test-retest consistency under comparable conditions.
  2. The measure appears to capture light sensitivity, not fear of crowds.
  3. It may be reliable but invalid for the stated construct.

The measure appears reliable but lacks validity for fear of crowds.

Look for these in your work

  • Reliability evaluated
  • Construct mismatch explained
  • The two qualities kept distinct
3 · Transfer

A researcher wants another school to repeat an observation of classroom interruptions. Name information needed to make the procedure replicable and one generalisability limit.

Give me a hint

Another observer needs the operational rules and sample context.

Compare my reasoning
  1. Specify the interruption categories, observation interval, observer position, lesson type, sampling schedule and scoring rule.
  2. Train observers and report how disagreements are handled.
  3. One school or subject may not represent classrooms elsewhere, limiting generalisability.

Provide a standardised category system, setting and sampling protocol, and observer training; findings from one school may not generalise to different schools or lessons.

Look for these in your work

  • Replicable procedure detailed
  • Scoring consistency addressed
  • Generalisability limit tied to sample

Common mistakes

  • Treating a reliable measure as necessarily valid.
  • Using ecological validity as a synonym for all forms of validity.
  • Saying a study is not replicable just because a later result differs.
  • Claiming that one unusual participant always makes a study invalid.
Recall without your notes

What can you explain now?

Can a measure be reliable but invalid?

Compare with the explanation

Yes. It can produce consistent scores while measuring something other than the intended concept.

Consistency is reliability; correspondence to the intended construct is a validity question.

After trying it yourself, choose your next review. This is your self-assessment.

Your review choice appears on Today. Sign in to sync it across devices.

Make it stick

How do validity, reliability, replicability and generalisability answer different evaluation questions?

If you needed the solution, close it and solve the problem again from a blank page. A correct answer is stronger when you can explain the reason for each step.

Keep learning Psychology

Course scope and assessment

Seven original lessons map all 15 AS research-method and methodological-concept sections to explained Guided, Independent and Transfer practice. All 12 compulsory AS core studies have original study-specific teaching and staged practice; five additional A Level methodology lessons cover research design and validity. All 15 Clinical, all 15 Consumer (2.1.1–2.5.3) and all 15 Health (3.1.1–3.5.3) Psychology sections link to original teaching and staged practice. All 15 Organisational Psychology sections 4.1–4.5 now have original teaching and explained staged practice; the six new 4.4–4.5 lessons bring the planning map to 92 of 92 indexed sections. Consumer lessons cover physical and psychological environments; decision models, heuristics and mistakes; packaging, selling and buying; advertising, self-monitoring, product placement, brand recognition and slogans. Evaluation boundaries are explicit: studies use particular products, samples, tasks and settings; gaze, recall, self-report, EEG indices, ratings, choices and purchases are different outcomes. Health evidence is bounded: Laba et al. measured hypothetical preferences; Riekert and Drotar raise a non-response concern; Chung and Naya's cap tracked openings, not swallowing; Chaney et al. was a short caregiver-reported pilot; Yokley and Glenwick examined immunisation attendance in one public-health setting. MacLachlan et al. report one patient, not a controlled treatment effect; Brudvik et al. compared separate child, parent and physician ratings in one emergency department. Chandola et al. found a cohort association; Wang et al. measured responses during a monitored arithmetic task; Evans and Wener observed commuters; Bridge et al. found mixed self-rated outcomes in radiotherapy. Janis and Feshbach measured short-term self-reports in one school; Lewin et al. tested a rehabilitation package, not information alone. Tapper et al. and Fox et al. studied bounded school and worksite programmes; incentive effects do not establish transfer or rule out reporting effects. Weinstein measured comparative expectations, while Seligman’s framework is conceptual and Shoshani and Steinmetz used a school-level comparison. These findings do not establish universal health effects or diagnose individuals. Need-theory examples use a child measurement study, not a workplace test; SMART is a practical checklist, not a motivational theory. Landry et al. compared reward framing with student volunteers in a lab; this does not establish universal effects in workplaces. Cuadrado et al. (2008) measured undergraduate evaluations of presented leaders, not workplace performance or hiring; this does not establish a universal gender effect. The historical 1995 Munchausen criteria are not a current diagnostic checklist. Cambridge’s 2028–2030 Reference List treats Oruč et al. (1997) as an example of genetic explanation although its participants had bipolar I; gambling disorder is listed in Cambridge’s Clinical topic but classified under addictive behaviours in ICD-11. Candidates choose two options, but all four remain listed so schools can select a route. The 92-section count is a planning index, not a Cambridge score. Remaining work is deeper coverage review, separate-year syllabus reconciliation, timed-paper practice and human subject-expert review. The separate 2027 and 2024–2026 syllabuses are not reconciled into this route. Practice is not official grading, a timed-paper simulation or human subject-expert review.

StudyForA · Learn with understanding

Original teaching content and links to official sources. Check your exam year and qualification before using a resource.