Screening Workflow¶

screen() runs multiple IER indices, applies flagging rules, and returns a structured result.

Configure with a single IndexOptions object:

from ier import IndexOptions, screen

result = screen(data, options=IndexOptions(scale_min=1, scale_max=5))
print(result["indices_used"])
print(result["flag_counts"])
print(result["valid_index_counts"])
print(result["consensus_flags"])
print(result["errors"])

Result keys¶

Key	Meaning
`scores`	Per-index score arrays
`flags`	Per-index boolean flags
`thresholds`	Actual per-index cutoffs (`None` for presence flagging)
`threshold_sources`	`fixed`, `percentile`, or `presence` provenance per cutoff
`percentiles`	Requested tail percentile per sample-relative cutoff; otherwise `None`
`flag_counts`	Total flags per respondent
`valid_index_counts`	Available index scores per respondent
`consensus_eligible`	Whether each respondent meets the optional completeness rule
`consensus_flags`	Respondents meeting the configured multi-index agreement threshold
`min_flags`	Number of per-index flags required for consensus (default: 2)
`min_valid_indices`	Required available scores for consensus, or `None`
`indices_used`	Successfully computed indices
`errors`	Soft failures (missing config, invalid data for an index)
`summary`	Mean/std/min/max, valid/unavailable counts, flagged count, and valid-score flag rate per index
`n_respondents` / `n_indices`	Size metadata

Respondent tables¶

screen_table() turns a result into an ordered mapping of respondent-aligned columns, matching the CLI CSV schema without its label column. Arrays are returned by reference, so the table itself copies nothing and should be treated as read-only; DataFrame constructors may still make their own copies:

import pandas as pd
from ier import IndexOptions, screen, screen_table

df = pd.read_csv("responses.csv", index_col="participant_id")
result = screen(df, options=IndexOptions(scale_min=1, scale_max=5))
table = pd.DataFrame(screen_table(result), index=df.index)

For polars, or any consumer without a row index, add identifiers as a leading respondent column. They must be strings, one per respondent, and cannot end with a NUL character, which NumPy string arrays would silently drop:

import polars as pl

frame = pl.DataFrame(screen_table(result, respondent_ids=df.index.astype(str).tolist()))

Use include_scores=False or include_flags=False to keep only one kind of per-index column. composite_table() provides the same layout for composite_summary() and composite_scores_summary() results: composite_score, valid_index_count, then one {name}_score column per successfully scored component.

Index agreement¶

Consensus flags are most informative when indices capture different behavior. index_agreement(result, kind) compares every pair of indices used:

Kind	Cell value
`overlap`	Respondents flagged by both indices; the diagonal is each index's flagged count
`jaccard` (default)	Shared flags divided by respondents flagged by either index; `NaN` when neither flags anyone
`spearman`	Rank correlation of suspiciousness over respondents with finite scores on both indices, using average ranks for ties

Spearman scores are oriented by each index's flag direction before ranking: indices that flag low scores, such as irv, psychsyn, and person_total, are negated. A positive correlation therefore always means both indices rank the same respondents as more suspicious, even when one flags low scores and the other high scores. Unregistered score names keep their raw orientation. Spearman pairs with fewer than three shared scores, or with constant ranks, are NaN. Presence-flagged indices such as onset are excluded from Spearman matrices because their scores are event positions rather than severities. Co-flag matrices for 1,000,000 respondents and 11 indices take about 0.1 s.

from ier import index_agreement, plot_index_agreement

names, overlap = index_agreement(result, kind="overlap")
fig = plot_index_agreement(result, kind="jaccard")

plot_distributions(result) marks each index's applied cutoff and shades its flagged tail, so percentile choices can be checked visually alongside plot_flagged_heatmap(result) and plot_flag_counts(result).

Defaults¶

Default indices are NumPy-only and require no extra item metadata:

irv, longstring, longstring_pattern, mahad, psychsyn, person_total, markov, u3_poly, midpoint, acquiescence, guttman

Mahalanobis distances and all direct flagging methods ("chi2", "iqr", and "zscore") are available in the NumPy-only base install.

Config-gated indices¶

Pass configuration to include indices that need survey metadata:

from ier import IndexOptions, screen

result = screen(
    data,
    indices=["evenodd", "mad", "semantic_syn", "infrequency"],
    options=IndexOptions(
        evenodd_factors=[5, 5],
        mad_positive_items=[0, 1, 2],
        mad_negative_items=[3, 4, 5],
        mad_scale_min=1,
        mad_scale_max=5,
        semantic_item_pairs=[(0, 1), (2, 3)],
        infrequency_item_indices=[9],
        infrequency_expected_responses=[5],
        infrequency_missing="fail",
    ),
)

Missing required config is recorded in result["errors"] instead of aborting the whole screening run. composite() uses the same soft-fail policy.

Balanced acquiescence¶

The default acquiescence score uses the normalized response mean. Balanced surveys can isolate yea-saying from item content by pairing positively and negatively worded items:

result = screen(
    data,
    indices=["acquiescence"],
    options=IndexOptions(
        scale_min=1,
        scale_max=5,
        acquiescence_positive_items=[0, 2],
        acquiescence_negative_items=[1, 3],
    ),
)

ier screen responses.csv --indices acquiescence --scale-min 1 --scale-max 5 \
  --acquiescence-positive-items 0,2 --acquiescence-negative-items 1,3

Both lists use 0-based positions in the scored matrix and pair in order. They must be nonempty and equal in length; invalid pairs become a normal soft failure, or a structured command error with --strict, rather than dropping trailing items silently.

Supply raw agreement responses for both item polarities. Negative items are not reverse-scored when measuring acquiescence (see Hinz et al., 2007). On a 1–5 scale, agreeing with all items scores 1, disagreeing with all items scores 0, and agreeing with one polarity while disagreeing with the other scores 0.5.

Reverse-keyed items¶

Consistency and person-fit indices need reverse-worded items recoded, while sequence and response-style indices such as longstring, irv, and acquiescence need the responses as presented. Name the reverse-worded columns once and keep one screening run:

from ier import IndexOptions, screen

result = screen(
    data,
    indices=["evenodd", "individual_reliability", "guttman", "longstring", "irv"],
    options=IndexOptions(
        scale_min=1,
        scale_max=5,
        evenodd_factors=[6, 6],
        evenodd_method="halves",
        reliability_factors=[6, 6],
        reverse_keyed_items=[1, 4, 7, 10],
    ),
)

ier screen responses.csv --indices evenodd guttman longstring irv \
  --scale-min 1 --scale-max 5 --evenodd-factors 6,6 --evenodd-method halves \
  --reverse-keyed-items 1,4,7,10

The keyed indices evenodd, individual_reliability, guttman, lz, gpoly, u3poly_fit, and ht read one recoded copy made with reverse_score(); every other index reads data unchanged, so consensus flags combine both views. ier indices lists each index's input in its keyed_responses column, and index_catalog() reports it as uses_keyed_responses. Positions are 0-based in the scored matrix, and explicit scale bounds keep an unobserved endpoint from shifting the recoding. Invalid items or responses outside the bounds become soft failures of the keyed indices only, or a contextual error with strict=True. See Reverse-keyed items for the full table.

Consensus completeness¶

An unavailable component score is not a flag. When incomplete item data or a soft-failed index could leave a consensus decision supported by too few signals, set a minimum availability requirement:

result = screen(
    data,
    indices=["irv", "longstring", "infrequency"],
    options=options,
    min_flags=2,
    min_valid_indices=3,
)

reviewable = result["consensus_eligible"]

Rows below the minimum remain unflagged even if their available signals meet min_flags. The result always reports valid_index_counts and consensus_eligible; with the default min_valid_indices=None, every row is eligible and prior consensus behavior is preserved. Both counts accumulate one index vector at a time rather than constructing a respondent-by-index matrix.

ier screen data.csv --indices irv longstring infrequency \
  --infrequency-item-indices 3 --infrequency-expected-responses 5 \
  --min-flags 2 --min-valid-indices 3 --format json

Attention-check missing responses¶

infrequency exposes four explicit missing-response policies through IndexOptions.infrequency_missing, the standalone missing= argument, and CLI --infrequency-missing:

Policy	Behavior
`pass`	Treat missing checks as correct; legacy default
`fail`	Treat missing checks as failures
`omit`	Exclude missing checks from proportions; no observed checks yields `NaN`
`propagate`	Return `NaN` when any configured check is missing

Set infrequency_proportion=True (or --infrequency-proportion) to score the failure share instead of the count. infrequency_flag(..., proportion=True) supports the same policy with an inclusive cutoff between zero and one.

Bogus items and self-reported diligence items often accept several answers. Replace the expected responses with one inclusive range per item through IndexOptions.infrequency_acceptable_ranges=[(1, 2), (5, float("inf"))] or --infrequency-acceptable-ranges '1:2,5:'; responses outside a range fail.

For production batches that require every requested index to succeed, enable strict mode. The first failed index raises a contextual ValueError:

result = screen(data, indices=["irv", "mad"], options=options, strict=True)

Parallel index scoring¶

Each selected index reads the same validated matrix independently. Larger multi-index runs can opt into standard-library worker threads:

result = screen(data, workers=4)

ier screen responses.npy --workers 4 --format npz --output screening.npz

The default is workers=1. Parallel results, soft failures, and strict failures retain selection order. Additional workers can improve throughput when NumPy kernels release the interpreter lock, but their temporary workspaces may overlap and raise peak memory. Benchmark representative matrix sizes and worker counts.

Reusing computed scores¶

Threshold and consensus sensitivity checks do not need to recalculate expensive indices. Pass the score mapping from one screening run to screen_scores():

from ier import screen, screen_scores

initial = screen(data, indices=["irv", "longstring", "mahad"])

lenient = screen_scores(
    initial["scores"], errors=initial["errors"], percentile=90, min_flags=1
)
strict = screen_scores(
    initial["scores"],
    errors=initial["errors"],
    percentiles={"irv": 99, "longstring": 99, "mahad": 99},
    min_flags=2,
)

The reusable path accepts registered index names, preserves their mapping order, and returns the same structured result contract as screen(). All vectors must be one-dimensional, non-empty, equally sized, and contain only finite values or NaN. Compatible float64 arrays are retained by reference and never mutated. Pass errors=initial["errors"] to retain the first run's soft failures in the returned result. Failed indices remain selected when validating cutoff overrides and min_valid_indices, but contribute no scores, flags, or availability. For example, if three indices were selected and one failed, a three-index minimum still makes every respondent ineligible when the scores are reused. Without errors, only the supplied score names are selected and the error mapping is empty. Failure names must be registered, absent from the score mapping, and paired with nonblank string messages.

On the bundled 10,000-respondent, 80-item benchmark, evaluating five tail percentiles from retained scores takes 5.1 ms and 1.6 MiB peak temporary allocation instead of 175.7 ms and 21.5 MiB for five full runs, a 34.2x speedup without another dependency.

Persist the same raw score mapping directly or through CLI NPZ output and reload it later:

from ier import load_score_archive, save_score_archive, screen_scores

save_score_archive(
    "screening-scores.npz", result["scores"], errors=result["errors"],
    n_respondents=result["n_respondents"],
)
saved = load_score_archive("screening-scores.npz")
revised = screen_scores(
    saved["scores"], errors=saved["errors"], n_respondents=saved["n_respondents"],
    percentile=99, min_flags=3,
)

The public writer can also store aligned respondent IDs. The loader returns those identifiers and any recorded soft failures.

The explicit n_respondents count is optional when score vectors are available, and is checked against their lengths. If every selected index failed, the empty score mapping and retained errors can still be saved and replayed by supplying the count. Respondents retain zero flags and zero available scores; a positive min_valid_indices makes them ineligible for consensus. Empty mappings without failure provenance are rejected.

Missing responses¶

Missing-response rate is available as an opt-in registry index:

options = IndexOptions(missing_item_indices=[0, 1, 4, 5])
result = screen(
    data,
    indices=["missing_rate"],
    options=options,
    thresholds={"missing_rate": 0.2},
)

It is not a default because planned skip logic can create legitimate omissions. Use missing_item_indices when the same item subset is required for every respondent. The option applies through screen() and all composite helpers; the direct equivalent is missing_rate(data, item_indices=[...]).

For respondent-specific branching, supply a Boolean matrix matching the response matrix. True cells identify expected responses and false cells are excluded from both the missing count and denominator:

import numpy as np

applicable = np.array(
    [
        [True, True, False, False],
        [True, True, True, True],
    ]
)
options = IndexOptions(missing_applicable_mask=applicable)
result = screen(data, indices=["missing_rate"], options=options, min_flags=1)

Rows without any applicable selected items receive NaN and are not flagged. missing_rate_flag() accepts the same applicable_mask argument for direct flagging.

The CLI equivalent is --missing-applicable-mask PATH. Use a headerless numeric 0/1 matrix, optionally gzip-, bzip2-, or xz-compressed, or an uncompressed Boolean .npy matrix that is memory-mapped read-only. A mask must match the response matrix's rows and selected item columns in order, excluding IDs and metadata. It can be combined with --missing-item-indices to restrict the required subset. Invalid values or dimensions fail before scoring and preserve existing output. Use - for mask standard input when responses come from a file; both inputs cannot consume standard input together. Response header, delimiter, preamble, and missing-token options apply only to the response file.

Flagging¶

Most indices use percentile thresholds (percentile=95 by default).
High-direction indices flag above the percentile; low-direction indices flag below 100 - percentile.
Sample cutoffs use linear interpolation among available scores, retaining finite cutoffs even for extreme score units. Retained score arrays keep their original order; missing scores remain unflagged.
Override individual tail settings with percentiles={"irv": 90, "longstring": 99}. Values use the same directional convention as the global setting, and unspecified indices retain the global percentile.
Pass fixed cutoffs with thresholds={"irv": 0.25, "longstring": 8}. Fixed thresholds are inclusive: high-direction scores at or above the cutoff and low-direction scores at or below the cutoff are flagged. Other indices keep using the configured percentile.
An index cannot have both fixed and percentile overrides. Presence-mode indices accept neither. Results retain every cutoff's source and requested percentile.
onset uses presence flagging: any detected changepoint is flagged.
consensus_flags marks respondents flagged by at least min_flags indices. Use screen(..., min_flags=1) for single-index workflows.

Response times (out of band)¶

response_time* helpers take timing matrices (seconds or other duration units), not Likert item responses. They are intentionally not registered in screen() / composite() because mixing domains would silently mis-score respondents. Call them directly or use the dedicated CLI command:

from ier import (
    load_response_time_archive,
    response_time,
    response_time_flag,
    response_time_score_flags,
    save_response_time_archive,
)

median_rt = response_time(times, metric="median")
flags = response_time_flag(times, cutoff_percentile=5)
stricter_flags = response_time_score_flags(median_rt, cutoff_percentile=1)

saved = load_response_time_archive("timing.npz")
revised_flags = response_time_score_flags(
    saved["scores"],
    threshold=1.0,
    direction=saved["flag_direction"],
)
save_response_time_archive(
    "revised-timing.npz",
    saved["scores"],
    revised_flags,
    threshold=1.0,
    metric=saved["metric"],
    flag_direction=saved["flag_direction"],
    respondent_ids=saved["respondent_ids"],
)

ier response-time timings.csv --metric median --percentile 5
ier response-time timings.csv --metric consistency --threshold 0.05 --format csv
ier response-time timings.csv --metric mixture --components 2 --random-seed 42
ier response-time timings.csv --metric median --format npz --output timing.npz
ier response-time timings.csv --metric effort --effort-max-threshold 10 --format csv

Direct timing metrics and consistency scores use low-tail flagging. Mixture probabilities use high-tail flagging. Fixed thresholds include equality; derived percentile cutoffs exclude ties, matching the other public flagging workflows. Response time effort (--metric effort) also uses the low tail but follows response_time_effort_flag(): it flags RTE strictly below a fixed 0.90 unless --threshold or --percentile supplies another cutoff. Retained direct scores use direction="low" by default; pass direction="high" for mixture probabilities. This sensitivity path never recomputes row summaries or refits the mixture. The NPZ loader also validates that archived flags agree with their stored threshold and suspicious-tail direction before reuse. The matching writer performs the same checks before creating a CLI-compatible archive. Mixture fitting excludes respondents whose median time is missing, infinite, or non-positive. Its posterior normalization remains stable when ordinary Gaussian density calculations underflow for an extreme valid observation.

CLI¶

Blank fields in comma-, tab-, or semicolon-delimited input are loaded as missing values (NaN) and follow each index's documented missing-data behavior.

ier screen data.csv --scale-min 1 --scale-max 5 --indices irv longstring
ier screen data.csv --min-flags 3
ier screen data.csv --min-flags 2 --min-valid-indices 3
ier screen data.csv --threshold irv=0.25 --threshold longstring=8
ier screen data.csv --index-percentile irv=90 --index-percentile longstring=99
ier screen data.csv --indices irv mad --strict
ier screen data.csv --format json --output screen.json
ier screen data.csv --format csv --evenodd-factors 5,5 --indices evenodd irv
ier screen data.csv --indices missing_rate --missing-item-indices 0,1,4
ier screen data.csv --indices missing_rate --missing-applicable-mask applicable.npy
ier screen data.csv --indices infrequency \
  --infrequency-item-indices 3,7 \
  --infrequency-expected-responses 5,1 --infrequency-missing fail
ier screen data.csv --id-column participant_id --item-columns q1,q2,q3,q4
ier screen survey-export.csv --skip-rows 2 --id-column participant_id
ier response-time timings.csv --metric median --threshold 1.0
ier screen data.csv.gz --format json --output screening.json.gz
ier screen data.npy --indices irv longstring --format json
ier screen data.npy --indices irv longstring --format npz --output screening.npz
cat data.csv | ier screen - --indices irv longstring --format json
ier --version

--item-columns accepts comma-separated header names and may be repeated. It lets screen and composite commands ignore unselected metadata columns while preserving the requested item order. Repeatable --item-column NAME accepts one exact header name, including commas; mix both forms to retain command-line selection order. Any item-index options and applicability-mask columns refer to that selected order. For survey exports with metadata columns, repeatable --item-pattern 'Q*' selects matching headers in header order, and repeatable --exclude-column NAME leaves named columns unscored after any selection (or, alone, scores every column except the ID column and the excluded names).

Header detection defaults to --header auto. Use --header present when the first row contains ambiguous names such as numeric item codes, or --header absent to ensure a malformed first data row is reported instead of interpreted as a header. Named column selection cannot be combined with --header absent.

Use --skip-rows N when an export begins with report titles or metadata. The loader discards exactly N physical lines before delimiter sampling and header detection, including blank lines, across plain, compressed, and standard-input streams. The option does not apply to .npy matrices.

Repeat --missing-value TOKEN to treat exact, whitespace-trimmed survey-export markers such as NA, ., or -99 as missing numeric cells. Configured markers participate in automatic header detection, so a headerless first row containing a missing marker is retained as data. Identifier and unselected metadata columns are not transformed.

The loader removes one leading UTF-8 byte-order mark from delimited input before delimiter and header detection. The normalization occurs once for plain, compressed, and standard-input streams; markers elsewhere remain invalid numeric content.

Uncompressed .npy input is memory-mapped read-only and must contain one non-empty, two-dimensional, real numeric array. It has no headers, so --id-column, --item-column, --item-columns, --missing-value, and --delimiter do not apply. Compressed .npy input is not supported because it cannot be memory-mapped.

Use - as the data path to read a forward-only standard-input stream, and use --output - to select standard output explicitly. Input and output paths ending in .gz, .bz2, or .xz are compressed or decompressed transparently with no optional package. CSV rows and JSON respondent arrays are emitted in bounded chunks across all destinations.

JSON output is standards-compliant: unavailable or non-finite scores and summary statistics are encoded as null. CSV output represents non-finite scores as empty cells so numeric columns remain compatible with spreadsheet and statistics tools. NPZ output preserves those values and typed flags exactly; its versioned layout is documented in CLI output formats.