solvedProblem hub
The headline finding flips when the sample is weighted
Created Nov 6, 2026, 8:30 AM · Updated Nov 10, 2026, 5:10 PM
The messy version
Unweighted, the effect is a clean 6.2 points in favour. Applying the demographic weights the funder asked for, it goes to -1.4 and the confidence interval crosses zero. Same respondents, same question wording. Where it came apart: under-25 respondents: 11% of the sample, 24% of the population weight applied to that cell: 2.2 and that cell is the only one where the effect runs the other way So the whole result rests on about forty people who are being counted twice over. That is not a finding, it is a sampling problem wearing a finding's clothes. Need to say plainly in the write-up which estimate is reported and why, rather than quietly picking the flattering one.
When you're ready
Take it for a run.
Start a thinking session and come back with one concrete next move.
History
Thinking sessions
On Nov 8, 2026, 7:45 AM, you took this problem for a Manual session. 20 minutes.
Earlier note: Recomputed with the funder's weights and without. Decided to report both, with the caveat stated up front.
On Nov 6, 2026, 12:00 PM, you took this problem for a Manual session. 28 minutes.
Earlier note: Checked which cell the weights were doing the work in. It was the under-25s, on their own.