Behavioral interviews, oriented

An unstructured interview is a conversation that ends in an impression. A structured one is a measurement. Both feel like interviews; only one produces comparable data.

The behavioral variety rests on a specific premise, stated plainly in the U.S. Office of Personnel Management's guidance: the best predictor of future behavior on the job is past behavior under similar circumstances. That is why the questions ask what you actually did rather than what you would do.

The standardization is not bureaucratic decoration. If two candidates are asked different questions, their scores are not comparable, and any difference between them may be an artifact of what they happened to be asked. Consistency is the thing that makes the comparison mean anything.

Concretely: OPM's own guide builds one from the competency Interpersonal Skills. Every candidate hears the same question — describe a situation in which you dealt with people who were difficult, hostile or distressed; who was involved, what you did, what resulted — and every answer lands on the same five-level scale, from Awareness to Expert. Change one candidate's question and their score sits on a different instrument.

Rule: if the questions change between candidates, you no longer have a measurement — you have a set of impressions with numbers attached.

Reliability is consistency. If two trained interviewers watch the same answer and rate it very differently, the scale is not doing its job, and whichever interviewer you happened to draw becomes part of your score. Agreement between raters is the version of reliability that structured interviews care about most.

Validity asks the harder question: does this actually predict performance on the job? A method can be highly reliable and invalid at the same time — everyone agrees on the measurement, and the measurement is of something irrelevant. That combination is dangerous precisely because the agreement feels like evidence.

The trap worth naming is the one that feels most like expertise. An interviewer who is confident they can read candidates is describing a subjective impression, not a validated instrument, and confidence is not one of the properties above. Structure exists because unaided judgment performs worse than practitioners believe.

Fairness sits alongside both: if a method screens out a protected group at a substantially different rate, that pattern needs examining regardless of how well it predicts.

Concretely: OPM's assessment guidance gives the reliability case as two agencies using similar writing tests for the same job: if an applicant scored low at one and high at the other, the hiring decision would turn on which test was drawn rather than on writing skill. Its summary: reliability places a limit on validity, and an assessment that says nothing about job performance "is of no value".

Rule: ask of any interview question, "would another trained rater score this answer the same, and does the answer relate to the job?" Failing either makes it decoration.

The first step in building a structured interview is not writing questions. It is a job analysis: examining what the job actually involves, which competencies the work requires, and how the two connect. Subject matter experts — people who know the work — confirm that list rather than a hiring manager guessing at it.

A competency is an observable, measurable pattern of behavior needed to do the job. That word "observable" is load-bearing. A trait like conscientious is a description of a person; a competency has to be something a candidate can demonstrate and a rater can see evidence of. Competencies that cannot be observed cannot be scored, whatever their intuitive appeal.

This is also where the strongest defense of an interview comes from. If a question traces back to a competency, and that competency traces back to a documented task, the question is job-related by construction. A question with no such lineage is one nobody can justify when asked why it was scored.

The common mistake is treating the analysis as paperwork completed once. When the job changes, the competencies change, and questions written for the old job quietly stop measuring anything.

Concretely: OPM's sample situational item shows the lineage in one row. Job task: performs investigative work to obtain information, gather evidence or verify facts. Competency: Integrity/Honesty. Question: a suspect in a money-laundering inquiry offers to get you a car well below market value — what do you do? Remove the task and the question is a character test nobody can defend as job-related.

Rule: for any interview question, ask which competency it targets and which job task requires that competency. A question that cannot answer both is not job-related.

Naming a competency is only half the specification. "Problem solving" describes a new analyst and a division director alike, and the same rating scale cannot serve both. Proficiency levels resolve this by defining what the competency looks like at each altitude, anchored in concrete behavior rather than adjectives.

What separates one level from the next is usually scope: the size of the problem, the number of people and functions involved, the time horizon, and how much ambiguity had to be resolved without instruction. A candidate describing a decision affecting their own work and one describing a decision affecting several teams are demonstrating the same competency at different levels.

The discrimination that matters in practice is between adjacent levels, and it is genuinely hard. The gap between the top and bottom of a scale is obvious to anyone; the gap between level three and level four is where raters actually disagree, which is why level descriptions carry behavioral examples rather than relying on words like "advanced".

The trap is rewarding articulacy. A candidate who describes a small piece of work vividly can outscore one who describes a much larger piece flatly, and the scale is supposed to prevent exactly that.

Concretely: on OPM's leadership scale, Level 1 (Awareness) applies a competency "in the simplest situations" under "close and extensive guidance"; Level 5 (Expert) handles "exceptionally difficult" ones and "serves as a key resource and advises others". For Interpersonal Skills, a Level 3 illustration is restoring a working relationship between two angry co-workers; a Level 5 one is defusing an emotionally charged meeting with external stakeholders. Same competency; the scope moved.

Senior executive selection changes the framework: a published set of executive core qualifications, not a per-job analysis, scored through narrative accounts of leadership experience — and scope discriminates there too; a well-told small initiative loses to a plainly told large one. That set changed on the page's own date, 1 October 2025. The five now published are Commitment to the Rule of Law and the Principles of the American Founding, Driving Efficiency, Merit and Competence, Leading People, and Achieving Results, each with named competencies beneath it. A narrative written to a qualification that is no longer on the list, however strong, maps to nothing a rater can score — so read the current list on OPM's Executive Core Qualifications page first.

Rule: rate the scope of what was done, not the polish of the telling; for a leadership narrative, lead with the challenge and its scale, against the list as it stands today.

The federal guidance lays out interview construction as a sequence: analyze the job, choose the format and write the questions, build the rating scales, create probes, pilot-test the questions, write the interviewer's guide, and document the process. The order is the argument — scales and probes are designed before anyone is interviewed, not improvised afterwards.

Good questions share concrete properties. They tie to the job's responsibilities, are open-ended, are clear and concise, avoid jargon, and are written at a reading level the candidates can be expected to have. A question that fails on jargon is measuring vocabulary; one that fails on clarity is measuring interpretation.

Each question should target a single competency. Bundle two into one question and the rating becomes ambiguous — a middling score could mean solid on one and weak on the other, and nobody can tell which.

Probes are prepared in advance for the same reason. When an answer is missing the detail the scale needs, the interviewer asks a planned follow-up rather than an improvised one that might lead the candidate.

Pilot-testing catches what review misses: questions that read cleanly and turn out to be unanswerable.

Concretely: OPM's guide pairs its Interpersonal Skills question with probes written in advance and sorted by what they retrieve — situation ("what factors led up to it?"), action ("what is the first thing you did?"), outcome ("is there anything you would have done differently?") — and tells interviewers to use very similar probes for every candidate. A follow-up invented on the spot for one candidate is a different question, and the comparison is gone.

Rule: one question, one competency, and every follow-up written before the first interview.

The halo effect is letting a rating on one competency drive the rating on another — scoring someone highly on problem solving because they communicated well, irrespective of what they actually demonstrated on problem solving. Central tendency is rating everything at the middle of the scale, the guidance's example being all threes on a five-point scale, which produces data with no discrimination in it. Leniency is rating all candidates high regardless of performance; strictness is the mirror. And interpersonal bias is letting factors unrelated to performance affect the judgment.

Note the shape they share. Each replaces a specific judgment about a specific competency with a general impression of the person, and each yields ratings that look complete while carrying almost no information.

The remedies are procedural rather than attitudinal, because deciding to be unbiased does not work. Rate each competency as its question is answered rather than at the end; take notes on what was said rather than how it felt; and where several interviewers are involved, rate independently before discussing, so the first opinion voiced does not become everyone's.

Concretely: OPM's panel procedure has each panelist review their notes and rate alone the moment the candidate leaves, with notes that show the reasoning behind every rating, before anyone compares. Its list of interviewing mistakes says why: judgments formed "within the first few minutes", unfavorable information weighted more than favorable, and each candidate rated against the one interviewed just before.

Rule: score the answer, one competency at a time, while it is still in front of you.

Treating the interview as the assessment is the most common error in hiring design. It is one method, and the others have different economics.

Situational judgment tests present realistic scenarios and ask what the candidate would do. That is the mirror image of the behavioral question: future-oriented rather than past-oriented, resting on the premise that intentions are closely tied to behavior. They scale cheaply because they need no interviewer.

Work samples ask the candidate to perform a piece of the actual job. Their strength is directness — you are observing the work rather than a report of the work — and their cost is that they are expensive to build and burdensome for candidates, which limits how early in a process they can sit.

Evaluations of training and experience score what a candidate has already done, from their own account. They are cheap and they filter volume, and they are the most inflatable of the three, because the candidate writes the input.

Methods differ in how easy they are to fake, and the ones that scale best are usually the easiest.

Three further methods look like they measure the person rather than the performance, and each has a narrow legitimate use and a much wider popular one. Personality measures must be job-relevant to be defensible — measuring a trait because it is interesting, rather than because the job analysis showed the work requires it, produces a score nobody can justify — and being self-reported, they invite candidates to present themselves in the direction they believe is wanted. Emotional intelligence measures are less settled: different instruments define the construct differently, some as an ability to be tested and some as a mix of traits, and the two are not interchangeable despite the shared name. Reference checks have the opposite problem — near-universal and almost never structured, so an unstructured call reproduces every rating error above, with the candidate having chosen the referee.

Concretely: OPM's situational twin of the Interpersonal Skills question is a scenario — a very angry client says your office sent her an overdue check five days ago and nobody will help; how would you handle it? — answerable by someone who has never faced one. A work sample for an administrative assistant instead has the applicant transcribe a memo or file a stack of paperwork. And a task-based experience rating scores higher the more tasks an applicant ticks; OPM's counter to inflation is telling applicants the answers will be verified, then verifying.

Rule: choose a method for the competency and the stage, not out of habit; expect fakability to rise as cost falls; and remember none of these measures the person in general — each measures one construct, and using it outside that construct is where the claim stops being supportable.

Because the question asks about past behavior, the rater is looking for evidence of behavior — a specific episode, not a general policy. "I always make sure the team is aligned" describes a disposition and gives a rater nothing to place on a scale.

A usable answer carries three things. Enough context that the difficulty is visible, since an action only demonstrates a competency relative to what it was up against. The action itself, in the first person: what you decided and did, which is the part most often lost to "we". And the outcome, because the same action can be competent or reckless depending on how it turned out and what you learned.

Choosing which episode to tell matters more than polishing the telling. The strongest example is the one where the competency being asked about was genuinely the hard part, at the largest scope you can honestly claim.

Two shapes read badly to a rater working from a scale. The hypothetical answer, which describes what you would do when asked what you did — it cannot be evidence of past behavior. And the collective answer, where the team is the subject throughout and no individual contribution is scoreable.

Concretely: two answers to the question above. "I always make sure people feel heard" gives the rater nothing; so does "we calmed the situation as a team". What scores is an episode: a site manager refused to cooperate after a missed delivery; I asked what single issue was driving it and rescheduled the inspection around that; the delivery landed and the relationship held. That is the shape OPM's probes reconstruct — situation, action, outcome.

Rule: one episode, first person, with the difficulty and the outcome both visible.

Eligibility comes first and is categorical: whether you may be considered at all, which depends on the hiring path rather than on how good you are. Qualification standards come next and are specific — required experience is defined at a level of detail that includes what counts as specialized experience for a given grade, and where education may substitute for experience.

The practical consequence is unintuitive. The application is being screened against a written standard, not read for impressiveness, so an application that omits detail because it seems obvious can be scored as not meeting a requirement that the candidate actually meets.

Announcements are worth reading as specifications rather than advertisements. The competencies named there are, in a well-built process, the same competencies the interview will score — which makes the announcement the most reliable available prediction of what you will be asked.

Concretely: OPM's assessment-strategy guidance describes the multiple-hurdle design — the least expensive assessments first to pare down the pool, the labor-intensive ones later, when few remain. The written screen comes first: an experience evaluation — OPM's crediting plan or rating schedule — scores what the applicant reports against benchmarks derived from the job analysis. Leave out relevant experience because it seemed obvious, and you are scored against a benchmark you did not answer.

Rule: read the announcement as the scoring key, and write the application against the standard rather than against your sense of what is impressive.

The U.S. Equal Employment Opportunity Commission publishes what employers may not do across the hiring process. Discrimination on the basis of race, color, religion, sex — including transgender status, sexual orientation and pregnancy — national origin, age from forty, disability, or genetic information is prohibited in advertising, recruitment, application and hiring alike.

Two mechanisms are worth separating. One is direct: refusing to consider someone because of a protected characteristic. The other is indirect, and it catches practices nobody designed to discriminate. A job advertisement seeking "recent college graduates" may discourage applicants over forty; relying on word-of-mouth recruitment through a workforce that is already homogeneous can produce hiring that mirrors it. Neither requires an intention to exclude.

The disability rules turn on timing, and the line is unusually crisp. An employer may not ask disability-related questions or require a medical examination until after making a conditional job offer — not even if it intends to look at the answers only at the later stage. Before that point it may ask whether an applicant can perform specific job functions, with or without reasonable accommodation, and may ask applicants to describe or demonstrate how they would perform them, provided every applicant for that job is asked the same thing.

Rule: consistency is the defense. A question asked of one candidate and not another is the shape most of these problems take.