2026-10-03

belief formation

belief formation concerns how information becomes confidence in a proposition.

the central question is not merely whether a belief is true, but what mechanism determines whether it survives. a person can hold a true belief through a poor process, or a false belief through a process that strongly resists correction.

primary failures

  • social substitution: beliefs are selected or maintained for affiliation, loyalty, identity, sympathy, antagonism, status, or belonging rather than truth.

    • the proposition may look factual while functioning socially.
    • disagreement then threatens a social or moral position rather than merely a proposition.
    • argument often fails because the stated reasons are not the actual support structure of the belief.
  • belief-first reasoning: confidence is established before investigation; evidence is then selected, interpreted, or generated to defend the commitment.

    • the person is more interested in preserving the belief than testing it.
    • contrary evidence becomes something to defeat rather than information to learn from.
  • repetition mistaken for evidence: repeated exposure produces familiarity and confidence without corresponding independent support.

    • 100 feeds repeating compatible claims can produce overwhelming subjective certainty while adding little evidential value.
    • social concurrence can be mistaken for independent confirmation even when everyone inhabits the same information environment.
  • asymmetric scrutiny: favorable evidence receives weak scrutiny while unfavorable evidence receives strong scrutiny.

    • weak supportive evidence is accepted.
    • strong contrary evidence is attacked for imperfections.
    • source, wording, motive, or minor uncertainty becomes important primarily when the conclusion is unwelcome.
  • missing-information blindness: people commonly feel that they understand an issue even when decisive information is absent.

    • a coherent account feels complete because the missing variables are not represented mentally.
    • unknown evidence cannot generate the subjective feeling of contradiction.
  • intuition outside calibration: gut feeling is trusted in abstract, distant, statistical, technical, or unfamiliar domains where ordinary intuition has little basis for accuracy.

    • comprehensibility is mistaken for plausibility.
    • strangeness is mistaken for improbability.
    • intuitive moral or causal stories substitute for domain knowledge.
  • commitment coupling: beliefs become attached to identity, worldview, morality, reputation, social position, or prior advocacy.

    • revising the proposition then carries costs unrelated to truth.
    • stronger coupling makes evidence less able to move confidence.
  • emotional weighting: fear, pity, outrage, disgust, admiration, or hope determines attention or evidential weight.

    • emotion can determine which facts enter the foreground before explicit reasoning begins.

    • moral concern about a person or group can silently become a factual judgment about events involving them.

truth vs other functions

factual belief formation has one governing criterion: whether the available evidence discriminates in favor of the proposition.

other human functions are legitimate in their own domains but become epistemically destructive when they control factual belief:

  • affiliation: which answer belongs to my side?
  • identity: which answer fits who i am?
  • morality: which answer expresses the correct sympathy or condemnation?
  • status: which answer marks me as informed, enlightened, independent, respectable, etc.?
  • affect: which answer feels reassuring, satisfying, frightening, or meaningful?
  • entertainment: which account is dramatic, elegant, surprising, or memorable?

the failure is not the presence of these motives. it is functional substitution: they become selection criteria for factual propositions.

evidence handling

  • selective exposure: seek information likely to confirm existing beliefs.
  • selective attention: notice congenial evidence while contrary evidence remains backgrounded.
  • cherry-picking: select a favorable subset while ignoring the relevant comparison set.
  • source veto: reject a claim because of who presents it rather than defects in the evidence.
  • authority substitution: accept a claim because of the speaker rather than the support for the claim.
  • consensus substitution: treat agreement as sufficient evidence without examining why agreement exists.
  • familiarity effect: repeated claims acquire credibility merely because they are easier to process.
  • vividness effect: memorable anecdotes outweigh broader but less salient evidence.
  • premise laundering: unsupported assumptions enter reasoning unnoticed because they are familiar, implied, socially accepted, or embedded in a larger argument.
  • pseudo-independence: dependent repetitions are counted as separate confirmations.
  • causal support vs epistemic support: an environment can strongly cause a belief without strongly supporting its truth.

common inference failures

  • criticism of a -> support for b: weakening one explanation does not establish an alternative.
  • anomaly -> collapse: an unresolved observation does not invalidate the surrounding body of knowledge.
  • local -> global: failure of one study, person, institution, or claim is generalized beyond the dependency carrying the failure.
  • provenance -> truth value: facts about origin, motives, incentives, character, or affiliation substitute for examination of the proposition.
  • correlation -> causation: association is treated as establishing a causal relation.
  • possibility -> actuality: showing that something could have happened is treated as evidence that it did.
  • conceivability -> plausibility: ability to imagine a coherent account is mistaken for evidence that it is likely.
  • absence of explanation -> preferred explanation: ignorance about a mechanism is treated as support for a favored alternative.
  • explanatory fit -> truth: ability to accommodate known facts is treated as sufficient without comparing alternatives.
  • one-sided explanation: evidence for a proposition is considered without asking what the competing proposition predicts.
  • certainty from consistency: internal coherence is mistaken for correspondence with reality.
  • category substitution: a moral, political, social, or emotional judgment is silently substituted for the factual question being asked.

attention failures

reasoning can be distorted before evidence is explicitly evaluated.

  • salience selection: some facts dominate awareness while equally relevant facts remain psychologically absent.
  • moral foregrounding: sympathy or condemnation determines which part of an event receives attention.
  • narrative compression: complex events are reduced to roles such as victim, oppressor, hero, traitor, establishment, or outsider.
  • person substitution: attention shifts from what happened to whether one likes, admires, identifies with, or pities the actors.
  • question substitution: "is p true?" becomes "does p support my side?", "does p express the right values?", or "does p preserve my worldview?"

resistance to correction

  • asymmetric updating: favorable evidence strongly raises confidence while unfavorable evidence weakly lowers it.
  • self-sealing belief: contrary evidence is reinterpreted as further evidence for the belief.
  • moving criteria: evidential requirements change when contrary evidence satisfies the previous standard.
  • ad hoc rescue: auxiliary explanations are introduced mainly to prevent revision.
  • unfalsifiable commitment: no conceivable observation is permitted to materially reduce confidence.
  • rationalization: reasons are constructed after commitment and presented as though they produced it.
  • certainty without reconstruction: confidence greatly exceeds the strength of any argument or evidence the person can recover and defend.
  • sunk commitment: previous time, effort, reputation, or emotional investment increases resistance to revision.
  • public commitment: changing belief becomes costly because previous advocacy must be repudiated.

correction principles

  • truth relevance: confidence should change for reasons bearing on whether the proposition is true.
  • independence: repeated or correlated information should not be counted as independent confirmation.
  • symmetry: comparable evidence should receive comparable scrutiny regardless of which conclusion it favors.
  • proportionality: confidence should track evidential strength.
  • adversarial testing: actively seek observations that could make the belief weaker.
  • comparative evaluation: alternatives must explain the relevant evidence, not merely attack competitors.
  • scope: conclusions should not extend beyond what their evidence supports.
  • revisability: factual beliefs must admit possible evidence that would lower confidence.
  • uncertainty: missing information should lower confidence rather than disappear from the model.
  • separation: affiliation, morality, identity, sympathy, status, repetition, and emotional satisfaction are not themselves evidence.
  • reconstruction: be able to state what evidence supports the belief, what evidence opposes it, and what would change the conclusion.
  • opposing competence: understand serious competing positions well enough to represent their strongest actual reasons rather than a convenient caricature.

diagnostic signs

  • confidence is much stronger than the recoverable evidence.
  • contrary evidence changes the subject toward motives, identity, morality, or source affiliation.
  • standards of proof differ depending on which side benefits.
  • many examples are available but few independent evidential chains can be identified.
  • the person can defend the belief but cannot describe what would falsify or weaken it.
  • the strongest opposing argument cannot be stated accurately.
  • emotional certainty increases while informational content remains unchanged.
  • the belief survives repeated failure of its stated justifications.
  • changing the belief would threaten belonging, identity, status, or moral self-conception.

central distinction

  • bias: truth remains the governing objective, but processing is distorted.
  • epistemic substitution: another function becomes the effective governing objective.

epistemic substitution is the deeper failure. once the belief is primarily serving affiliation, identity, morality, affect, or status, ordinary factual rebuttal may address only the visible justification rather than the mechanism maintaining the belief.

central question

  • weak: what does this person believe?
  • stronger: why does this belief survive?
  • decisive: what evidence could make this person less confident?

a belief-forming process is epistemically sound to the extent that its beliefs survive because of evidence capable of both supporting and defeating them.