The research behind the method

NormSignal doesn't propose a new personality theory. It applies findings that personnel selection, decision science and game theory have each tested for decades to the final-stage hiring decision. This page lists those findings and their sources.

Structure beats intuition

The most consistent finding in selection research: asking every candidate the same questions and rating answers against predefined standards clearly beats free-form conversation. In current meta-analyses the structured interview leads every selection method examined. [1, 6, 7, 8, 16, 19]

Kahneman (1955)Nobel 2002As a 21-year-old lieutenant he rebuilt the Israeli army's interview: rate six traits one by one, leave the overall judgement to the end. It stayed in use for decades.
Conway, Jako & Goodman (1995)The more structured the interview, the more closely different interviewers agree on the same candidate.
Campion, Palmer & Campion (1997)Defined the fifteen components of interview structure: same questions, job-based content, per-answer rating, anchored scales, more than one interviewer.
Schmidt & Hunter (1998)Pulled 85 years of selection research into one analysis; structured interviews clearly outperform unstructured ones.
Levashina et al. (2014)Reviewed three decades of research on the structured interview; structure raises validity, reliability and fairness together.
Sackett et al. (2022)Recomputed the validity of selection methods: structured interviews top the list at .42, ahead of job knowledge and cognitive ability tests.

Dilemmas without a right answer

Seeing what people choose in realistic work situations, rather than asking what kind of person they are, says more about work behaviour. Making them choose between two acceptable options reduces faking good. [11, 13, 14, 17]

Kristof-Brown et al. (2005)Person–group and person–supervisor fit relate consistently to satisfaction, commitment and intention to leave.
McDaniel et al. (2007)Answers to realistic work dilemmas show a meaningful, replicable relationship with job performance.
Lievens, Peeters & Schollaert (2008)Reviewed evidence that situational judgement methods are seen as fair by candidates and show smaller group differences than traditional tests.
Cao & Drasgow (2019)Formats that force a choice between two acceptable options reduce faking good in high-stakes settings.

Norms are mutual expectations

In game theory, behaviour in a team is shaped not only by individual preferences but by what people expect of each other. Those expectations are often wrong and invisible; once visible, they can be discussed and changed. [2, 4, 5, 9, 10, 12, 15]

Schelling (1960)Nobel 2005Focal points: people coordinate without talking because they decide by predicting what others will do.
Lewis (1969)Conventions are patterns everyone keeps because they expect the others to keep them too.
Prentice & Miller (1993)Pluralistic ignorance: group members systematically misjudge what the others think, and adjust their behaviour to the misjudgement.
Ostrom (2000)Nobel 2009Groups build cooperation on mutual expectations more than written rules; cooperation strengthens when those expectations are visible.
Prelec (2004)Showed a game-theoretic way to collect honest subjective data even where no answer can be checked.
Bicchieri (2006)A social norm is not what a person prefers but what they expect of others, and what they believe others expect of them.
Krupka & Weber (2013)Built a coordination-game method to measure norms: people answer 'what would others find appropriate' with striking consistency.

Reducing noise

Two interviewers reaching different conclusions about the same candidate is the least discussed problem in decision quality. Standards defined in concrete behaviour and independent ratings narrow that gap. [3, 18]

Smith & Kendall (1963)Behaviourally anchored scales: defining a strong and a weak answer in concrete behaviour beforehand cuts arbitrariness between raters.
Kahneman, Sibony & Sunstein (2021)Nobel 2002The 'noise' between experts looking at the same facts can be cut with structure and independent assessments.

How we apply these principles

To keep results reliable, the session content, the scoring key and how the report is produced aren't published. We share them one-to-one with customers and independent experts under NDA.

Request a demo

Bibliography

[1]Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux. (Account of the 1955 Israel Defense Forces interview.)
[2]Schelling, T. C. (1960). The Strategy of Conflict. Harvard University Press.
[3]Smith, P. C., & Kendall, L. M. (1963). Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 47(2), 149–155. DOI
[4]Lewis, D. (1969). Convention: A Philosophical Study. Harvard University Press.
[5]Prentice, D. A., & Miller, D. T. (1993). Pluralistic ignorance and alcohol use on campus: Some consequences of misperceiving the social norm. Journal of Personality and Social Psychology, 64(2), 243–256. DOI
[6]Conway, J. M., Jako, R. A., & Goodman, D. F. (1995). A meta-analysis of interrater and internal consistency reliability of selection interviews. Journal of Applied Psychology, 80(5), 565–579. DOI
[7]Campion, M. A., Palmer, D. K., & Campion, J. E. (1997). A review of structure in the selection interview. Personnel Psychology, 50(3), 655–702. DOI
[8]Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274. DOI
[9]Ostrom, E. (2000). Collective action and the evolution of social norms. Journal of Economic Perspectives, 14(3), 137–158. DOI
[10]Prelec, D. (2004). A Bayesian truth serum for subjective data. Science, 306(5695), 462–466. DOI
[11]Kristof-Brown, A. L., Zimmerman, R. D., & Johnson, E. C. (2005). Consequences of individuals' fit at work: A meta-analysis of person–job, person–organization, person–group, and person–supervisor fit. Personnel Psychology, 58(2), 281–342. DOI
[12]Bicchieri, C. (2006). The Grammar of Society: The Nature and Dynamics of Social Norms. Cambridge University Press. DOI
[13]McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb, W. L. (2007). Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology, 60(1), 63–91. DOI
[14]Lievens, F., Peeters, H., & Schollaert, E. (2008). Situational judgment tests: A review of recent research. Personnel Review, 37(4), 426–441. DOI
[15]Krupka, E. L., & Weber, R. A. (2013). Identifying social norms using coordination games: Why does dictator game sharing vary? Journal of the European Economic Association, 11(3), 495–524. DOI
[16]Levashina, J., Hartwell, C. J., Morgeson, F. P., & Campion, M. A. (2014). The structured employment interview: Narrative and quantitative review of the research literature. Personnel Psychology, 67(1), 241–293. DOI
[17]Cao, M., & Drasgow, F. (2019). Does forcing reduce faking? A meta-analytic review of forced-choice personality measures in high-stakes situations. Journal of Applied Psychology, 104(11), 1347–1368. DOI
[18]Kahneman, D., Sibony, O., & Sunstein, C. R. (2021). Noise: A Flaw in Human Judgment. Little, Brown Spark.
[19]Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068. DOI

These sources define the principles the method rests on; they are not validation results for NormSignal itself. Our own validation study runs alongside pilot customers and will be published here when complete.

Bibliography

[1]Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux. (Account of the 1955 Israel Defense Forces interview.)

[2]Schelling, T. C. (1960). The Strategy of Conflict. Harvard University Press.

[3]Smith, P. C., & Kendall, L. M. (1963). Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 47(2), 149–155. DOI

[4]Lewis, D. (1969). Convention: A Philosophical Study. Harvard University Press.

[5]Prentice, D. A., & Miller, D. T. (1993). Pluralistic ignorance and alcohol use on campus: Some consequences of misperceiving the social norm. Journal of Personality and Social Psychology, 64(2), 243–256. DOI

[6]Conway, J. M., Jako, R. A., & Goodman, D. F. (1995). A meta-analysis of interrater and internal consistency reliability of selection interviews. Journal of Applied Psychology, 80(5), 565–579. DOI

[7]Campion, M. A., Palmer, D. K., & Campion, J. E. (1997). A review of structure in the selection interview. Personnel Psychology, 50(3), 655–702. DOI

[8]Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274. DOI

[9]Ostrom, E. (2000). Collective action and the evolution of social norms. Journal of Economic Perspectives, 14(3), 137–158. DOI

[10]Prelec, D. (2004). A Bayesian truth serum for subjective data. Science, 306(5695), 462–466. DOI

[11]Kristof-Brown, A. L., Zimmerman, R. D., & Johnson, E. C. (2005). Consequences of individuals' fit at work: A meta-analysis of person–job, person–organization, person–group, and person–supervisor fit. Personnel Psychology, 58(2), 281–342. DOI

[12]Bicchieri, C. (2006). The Grammar of Society: The Nature and Dynamics of Social Norms. Cambridge University Press. DOI

[13]McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb, W. L. (2007). Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology, 60(1), 63–91. DOI

[14]Lievens, F., Peeters, H., & Schollaert, E. (2008). Situational judgment tests: A review of recent research. Personnel Review, 37(4), 426–441. DOI

[15]Krupka, E. L., & Weber, R. A. (2013). Identifying social norms using coordination games: Why does dictator game sharing vary? Journal of the European Economic Association, 11(3), 495–524. DOI

[16]Levashina, J., Hartwell, C. J., Morgeson, F. P., & Campion, M. A. (2014). The structured employment interview: Narrative and quantitative review of the research literature. Personnel Psychology, 67(1), 241–293. DOI

[17]Cao, M., & Drasgow, F. (2019). Does forcing reduce faking? A meta-analytic review of forced-choice personality measures in high-stakes situations. Journal of Applied Psychology, 104(11), 1347–1368. DOI

[18]Kahneman, D., Sibony, O., & Sunstein, C. R. (2021). Noise: A Flaw in Human Judgment. Little, Brown Spark.

[19]Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068. DOI