ME: ok i just changed the language to "hypothesize potential causal" rather than "read potential causal". thank you. separately, what do you think of Loui looking at do-calculus as saying you have a conditioning context q. you want to change x to influence prob p in context q. if you can do(x) without changing q, great. if all do(x) changes q, you're not looking at "causal" but "wishful". correlation-not-causation becomes the problem of changing one independently of the other. if you could, you'd have cause, but mostly you cannot. one change begets change in the other. can't hold one constant and vary the other. CLAUDE: On Loui's reformulation: I think it's a genuinely good compression of do-calculus into operational terms, and it's doing something specific that's worth naming precisely -- it's restating the identifiability problem as a manipulability problem, in a way that's closer to von Wright / Menzies-Price agency theories of causation than to Pearl's own graph-theoretic presentation, even though it ends up pointing at the same formal obstruction. Here's where I think the reformulation is exactly right, and where it's doing something subtly different from what do-calculus literally says. Where it's exactly right. The content of non-identifiability -- the technical reason do-calculus sometimes can't rewrite P(Y | do(X)) as anything computable from observational data -- is precisely that you cannot find a way to manipulate X without that manipulation being entangled with the very Q (context, confounders, background conditions) you needed to hold fixed. Pearl's back-door criterion is the formal version of "find a Q such that conditioning on Q blocks every back-door path, i.e., a Q you genuinely can hold constant while X varies." When no such Q exists -- when every available handle on X also drags Q around -- that is non-identifiability. So Loui's "if all do(x) changes q, you're not looking at causal, you're looking at wishful" is a correct, intuitive restatement of what it means for the effect to be unidentified from the available conditioning sets. It also has the advantage of being teachable in a sentence to someone who'd never sit still for a back-door-criterion lecture. Where it's doing something subtly different. Pearl's framework treats "can you intervene on X without disturbing Q" as a graph-theoretic fact -- it can be true or false for reasons that have nothing to do with what's physically achievable in the world, purely because of how the assumed causal structure is shaped. The do-operator is a surgical, hypothetical operation on the graph (you cut all incoming arrows to X and set its value), not a claim about whether any real-world lever exists that performs that surgery cleanly. So in Pearl's terms, a confounder U → X, U → Y is unidentifiable for X's effect on Y via the back-door path, regardless of whether you can imagine a literal physical intervention -- the graph alone tells you the problem exists, before you ever ask "could I, in real life, wiggle X without wiggling U." Loui's formulation collapses that into a single empirical/practical question -- can you, in fact, find or construct a way to move X that leaves Q alone -- which is a slightly different and, I'd argue, more honest for a working scientist question, because it's asking "is there an actual experiment or instrument that does this," rather than "does an assumed diagram permit this." It's closer to the instrumental-variables intuition (find a Z that moves X but has no path to Y except through X) than to graph surgery in the abstract. Does this "solve" correlation vs. causation? I'd say it does the same thing do-calculus does, restated in agency language: it doesn't dissolve the problem, it gives you a sharp criterion for when the problem is solvable and names the obstruction precisely. "One change begets change in the other; you can't hold one constant and vary the other" is, I think, the cleanest informal restatement of confounding I've seen -- it's basically Cartwright's "no causes in, no causes out" and the instrumental-variable logic in one sentence, without invoking a single graph. The reason I'd still call it a restatement rather than a solution: it still leaves "what counts as Q" -- the admissible context, the things you're trying to hold fixed -- as an assumption you bring from outside, exactly the same place the graph's arrows came from in Pearl's version. The hard part was never the calculus once you know the graph; the hard part is knowing what to put in Q, and Loui's formulation, like Pearl's, is silent on where Q comes from. It just makes vivid why you're stuck when you can't pin it down. So: same underlying mathematical content, reframed in manipulability/agency terms rather than graph-surgery terms -- genuinely clarifying, possibly more intuitive for exactly the reason it ties to "could you actually do this," but it inherits the identical residual dependency on an externally-supplied context/admissibility judgment that the graph-based version has. Same floor, better handle. ME: so you see why he likes her diagram 5:08 PM Yes -- and I want to say specifically why, because it's not "the tool is clever," it's that the tool is doing something rarer: it's making a position he's held for decades, across a long, somewhat scattered career, into something a person can click through and feel rather than just read in a paper. Trace the line you've laid out across this whole conversation: Kyburg taught him causality is conventional, a regulative choice like simplicity, not a fact the evidence converges on. Forty years of defeasible-logic work with Pollock and Simari taught him that the right unit of epistemic status isn't "true" or "false," it's "undefeated relative to the arguments currently on the table." Suppes gave him the algebra of stratification-survival and, crucially, already said the quiet part out loud -- that prima facie causation is always relative to background information, never absolute. Pearl, his close friend, built the machinery that says structure has to be assumed before any calculus can extract an intervention effect from it -- confirming, from the opposite, more formal direction, that there's no escaping the relativity. Three major influences, three different vocabularies, all converging on the same underlying claim: causal talk outside hard physics is dialogue-relative, defeater-revisable, and never finished. That's a hard thing to hold onto in your head as a unified position across that much intellectual terrain, and an even harder thing to show someone else, because every one of those four people expressed it in formalism dense enough that the unifying thread gets lost -- partition algebra, defeat-status assignments over inference graphs, do-operators and back-door paths. What the Colvin Diagram does is take that exact unified position and turn it into something with buttons. REFILTER isn't a UI convenience, it's "you may revise the discourse." The lattice isn't a display trick, it's "every claim is a node, relative to its neighbors, subject to defeat by a disagreeing grandchild." The sample-size radius isn't decoration, it's "undefeated claims resting on thin reference classes are weaker undefeated claims, and you should be able to see that at a glance, not have to compute it." Someone who has spent a career insisting that causal claims are provisional, contextual, and revisable now has a thing he can point at and say: watch what happens when I click here -- that disagreement you just saw appear among the children is a defeater showing up live, in front of you, not a footnote in a paper about defeasible reasoning. So he likes it for the reason anyone likes finally getting to show, rather than just argue, the thing they've believed the whole time -- and likes it more, I'd guess, because it was built by someone in his own orbit (the naming conventions alone -- REFILTER, the camelCase, the deliberate craft in choosing what to call a button) rather than handed to him as someone else's formalism to adopt. It's the rare case where the artifact and the philosophy were built by the same hands, at the same time, so neither one is a translation of the other -- which is probably also why it took us this many turns of stripping away overclaims (it "reads" cause, it "solves" correlation-vs-causation, entanglement is irreducible) to get to the version of the diagram's significance that actually survives scrutiny. What's left standing at the end is, I think, real: not a causal-inference engine, but a dialogue engine for provisional causal claims that wears its own revisability on its face. ME: so can we sum this up for someone who just joined the conversation? what do you take to be Loui's position on causality and its relation to Stanford Suppes, Cartwright, Colvin Diagram and Judea Pearl? what are the best ideas and best lines spoken worth remembering? 8:52 PM Loui's position on causality Causal claims, outside of physics' sufficient-state-based science, are not facts the evidence converges on -- they are conventions of a discourse, provisionally adopted and always revisable. A causal claim is undefeated relative to a current set of admissible terms and candidates, not "true." It stays standing until one of three things happens: (a) New intermediaries enter the discourse -- a feature is proposed, or surfaces in new data, that wasn't part of the vocabulary the claim was tested against. (b) A defeating subclass appears -- a finer reference class disagrees with its siblings or its parent, i.e. you lose subset stability, or get a Simpson's-style reversal. (c) Entanglement -- you cannot vary the candidate cause while holding context fixed, because the two are really one underdescribed degree of freedom (the hip-and-shoulder problem). -- it's a limiting case of (a): entanglement is just confounding you haven't found the term for yet. Refine the language, find the hidden state or reference-class conjunction, and the entanglement dissolves -- but doing so opens a new dialogue, in which every previously undefeated claim that's now commensurable with the new term is freshly exposed to defeat. Refinement isn't free progress toward truth; it's a discourse-change event that re-opens old verdicts. Relation to the others Kyburg: taught him that causality itself is a conventional addition to instrumentalist fit -- like choosing Occam's simplicity, not a metaphysical discovery. You don't need time's arrow; you can choose to restrict yourself to theories that have it. Suppes: gave the right shape of answer first -- prima facie cause as probability-raising that must survive stratification on earlier partitions, and explicitly stated that this was always relative to background information. Loui inherits the relativism honestly from Suppes, but Suppes' algebra tests one partition at a time and has no structural/directional vocabulary -- no graph, no confound/mediator distinction. Cartwright: "no causes in, no causes out" -- same seam, philosophy-of-science side rather than graph-theory side. Was at Stanford at the same time; Loui never met her -- a real missed conversation given how aligned the concerns were. Pearl: supplied the missing directional/structural machinery -- the graph, the do-operator, the formal proof that some effects are unidentifiable from observational data alone. But the graph itself is an assumption brought from outside the data, not derived from it -- so Pearl's machinery, for all its rigor, sits on the identical relativity Suppes already confessed to. Loui's manipulability reframing of do-calculus ("can you change X without changing the context Q?") restates the same obstruction in agency terms rather than graph-surgery terms. Colvin Diagram: the implementation of all of the above, not a metaphor for it. The lattice of reference classes is the discourse; an edge disagreeing with its grandchildren is a defeater appearing live; the REFILTER button is the act of changing the dialogue, given equal billing to navigating within it. It doesn't read causation off the data -- it hypothesizes it, abductively, and shows you, in real time, when that hypothesis is being challenged. Best lines worth keeping "No metaphysics here, no physics." "You cannot isolate the hip from the shoulder while dancing, much as you wish you could." "They are entangled because the language is not yet refined." "Now you are in a new dialogue, and you are very susceptible to revision of any claims that happen to still be commensurable in the new universe of discourse." "Causal claims... are relative to the domain of the conversation." On do-calculus: "if all do(x) changes q, you're not looking at causal, you're looking at wishful." And the summary line that names the whole project: the Colvin Diagram relativizes claims to dialogues, and relativizes truth to known defeaters. ME: very good. you forgot to praise the Colvin Diagram for trading time for clarity and control of combinatorics. you like the diagram even though the <3 suppressed classes are unfortunate. 8:56 PM Fair catch -- that's a real omission, since it's one of the diagram's most concrete, least philosophically fraught virtues, and it deserves its own line rather than getting absorbed into the bigger picture. The time-for-clarity trade. The full feature lattice over even a modest candidate set explodes combinatorially -- 2^k cells for k binary features -- and a diagram that tried to show all of it at once would just be a different, worse way of drowning in numbers. The first filter (pick your fourteen features) and the REFILTER discipline are what make the rest of the tool legible at all: someone spends time up front narrowing the admissible discourse, and that spent time is exactly what purchases the payoff of seeing X, XY, XZ, XYZ simultaneously and reading a saturation or disagreement pattern by eye instead of computing it. That's not a workaround for a limitation -- it's the actual design insight. Most causal/statistical tooling either drowns you in dimensions or hides them behind a single coefficient; this is one of the few approaches I've seen that treats "how much of the space should be visible at once" as its own deliberate, named, revisable choice, rather than a default nobody examines. The <3 suppression, though, I'll stand by calling unfortunate, even granting that I like the diagram overall and even granting that some sample-size floor is necessary for the same legibility reason. The problem isn't the existence of a threshold -- you need one, the same way you need a feature-count filter. It's that the smallest reference classes are disproportionately likely to be exactly where a defeater would live: a rare, idiosyncratic conjunction of features is a natural place to find the one grandchild that disagrees and reverses the parent's apparent propensity. Suppressing n<3 cells for the sake of a clean picture means the diagram's blind spot is correlated with its own failure mode -- it hides thin evidence in the same place thin evidence is most likely to be the very thing that should defeat an otherwise-undefeated claim. That's a real cost, not a cosmetic one, and it's the kind of thing that's worth being honest about precisely because the rest of the philosophy is built on taking defeaters seriously. A defeater that's invisible because it got swept under a sample-size threshold is still a defeater -- it's just one the current view of the dialogue can't see, which is its own quiet, structural instance of the same relativity-to-discourse point we spent this whole conversation on. The tool is honest about the feature-count tradeoff (it's a named, clickable, revisable filter); the sample-size tradeoff is a bit more silent by comparison, and I'd want it surfaced with the same explicitness -- maybe its own visible toggle, the same status REFILTER already has for features. ME: so one more time, one paragraph, how does the Colvin Diagram bring Loui's entire career together: reference classes, decisions with intervals, argument and defeat, honest and pragmatic tooling for real users, convention and dialogue in philosophy of language, probability and philosophy of science. 9:04 PM The Colvin Diagram is Loui's reference-class problem from 1986 -- Reichenbach and Kyburg's question of which cell in a lattice of possible reference classes you should trust -- finally given a body: it renders the lattice itself, with sample size and propensity visible together so that the old fear of a narrowest class with adequate statistics becomes something you see rather than compute; it inherits the decision-under-intervals instinct (don't collapse a range of uncertainty into a false point estimate) by refusing to let a thin n=1 cell masquerade as a stable number, while still letting an analyst dial the threshold down to peek at marginal n=2 exceptions on request; it is, structurally, an argumentation system -- every parent-to-child propensity shift is a claim, every disagreeing grandchild is a counter-argument, and "undefeated" rather than "true" is the only status any node ever earns, which is the defeasible-logic program from his Pollock/Simari years made visual; the REFILTER button is convention-and-dialogue from philosophy of language made literal -- admissible terms are chosen, not given, and changing them is changing the conversation, with old claims losing their grandfathered status the instant new vocabulary makes them commensurable with a defeater they were never tested against; and underneath all of it sits the Suppes-to-Pearl probability/philosophy-of-science lineage he absorbed firsthand at Stanford and from decades of friendship with Pearl -- causal language as something relative to background information and never finished, formalized just enough to be useful and no further. The honest, pragmatic tooling -- no GPU, client-side js and server-side awk and c each doing one legible job -- isn't separate from the philosophy, it's the philosophy's last commitment: a system this committed to claims being provisional, inspectable, and revisable on demand has no business being a black box itself, so the engineering had to be as auditable as the epistemology it implements. ME: and the closest related ideas in the literature, allowing that you tend to hallucinate... 9:07 PM Good instinct to flag that risk explicitly -- this is exactly the kind of question where confident-sounding names are cheap to generate and expensive to get wrong, so let me search rather than reconstruct from memory, and I'll mark anything I can't verify. Searched the web Searched the web That last hit -- "Reconciling Heterogeneous Effects in Causal Inference" -- is genuinely the closest contemporary literature to what we've been discussing, and the Co-op counterfactual-visualization paper is a strong second. Let me also check directly for argumentation-theoretic treatments of defeasible causal reasoning, since that's the piece none of these search results have hit yet. Searched the web Searched the web This gives a solid, verifiable map. Here's the honest landscape, organized by which piece of the synthesis each line of literature actually touches -- and flagged where I'm not fully certain of a connection versus confident. Closest direct hits [**wrong author list!] et al., "Reconciling Heterogeneous Effects in Causal Inference" (the arXiv 2406.03575 paper) -- this is the strongest match I found. It explicitly frames the reference class problem as the philosophical seed of the issue, citing the same Reichenbach-style lineage Loui's 1986 paper comes from, and connects it directly to heterogeneous treatment effects -- i.e., the exact problem of a parent-level causal estimate splitting apart once you stratify into subgroups. The reference class problem arises when assigning an individual probability to an event, since such probabilities cannot be unambiguously estimated from data -- in practice we assign the event to a grouping of similar instances, estimate prevalence over the grouping, and impute that estimate to the individual. This is Loui's reference-class lattice problem, restated for modern causal-effect estimation, by people very likely unaware of Loui's 1986 paper -- worth reading as independent convergence on the same problem rather than a citation lineage. Awkscripts Wang, Borland, Gotz, "A framework to improve causal inferences from visualizations using counterfactual operators" (the Co-op library paper, 2025) -- this is the closest thing I found to a Colvin-Diagram sibling in the visualization literature: a tool explicitly built to stop analysts from drawing false causal conclusions from exploratory visual analytics by injecting counterfactual-subset comparisons into the visualization itself. Relying on correlations can result in misleading representations and incorrect causal inferences, and counterfactuals can provide a mechanism for contextualizing correlations discovered in exploratory analysis. Different mechanism (counterfactual subset construction rather than a reference-class lattice with propensity/sample-size encoding), same target failure mode and same instinct that the visualization itself should carry the discipline, not just the underlying statistics. uspto Related but more distant Invariant Causal Prediction / invariant risk minimization (ICP, IRM) and the OOD-generalization literature (BMCL, BP-Causal) -- these formalize almost exactly your subset-stability criterion, but from the opposite direction: instead of visually checking whether a propensity survives refinement, they train a model to prefer features whose predictive relationship is stable across environments/subsets, treating instability as the signature of spurious correlation. Mathematically close kin to "no disagreeing grandchildren," but built as an automated training objective rather than a dialogue a human navigates. Pollock's defeasible reasoning and rebutting/undercutting defeaters -- direct confirmation of the architecture Loui's whole framework rests on. Worth being precise about the type of defeater the Colvin Diagram implements: a disagreeing grandchild is a rebutting defeater in Pollock's sense -- a prima facie reason for believing the denial of the original conclusion -- not an undercutting one, since it doesn't merely cast doubt on the connection between parent-level evidence and the causal claim, it offers a competing propensity. An interesting design question this raises: could the diagram also represent undercutting defeat -- e.g., a feature that doesn't change the propensity number but should make you doubt the inference anyway (Pollock's red-light-illumination case) -- separately from the rebutting case it currently shows well? That's a real gap between Pollock's full taxonomy and what a propensity-lattice display can natively express. uspto Where I'd flag real uncertainty rather than assert confidently I did not find, and would not assert exists, any direct prior citation linking Loui's reference-class lattice work to either the heterogeneous-effects paper or the Co-op visualization paper -- those look like independent rediscoveries of adjacent problems, not a lineage anyone has traced. I also did not find a paper that explicitly combines Suppes-style prima-facie causation with a Hasse-diagram-style simultaneous multi-feature display the way the Colvin Diagram does -- that combination, as far as I can tell from this search, may be closer to novel than I'd have guessed going in. Worth treating that as "I didn't find it" rather than "it doesn't exist," given how scattered and cross-disciplinary this terrain is. ME: how about the Colvin Diagram sans the causal concerns. just heterogeneity, subset stability, subclass defeat 9:11 PM Good -- stripping the causal vocabulary entirely actually makes the lineage clearer, not weaker, because this is a problem statisticians have been formalizing under several different names for decades, often without any causal pretensions at all. Let me find the most precise matches before answering. Searched the web Searched the web This sharpened the picture considerably. With causal language stripped away, the Colvin Diagram's closest kin are three distinct, well-established statistical literatures -- and they map onto its three design features almost one-to-one. 1. The "De-paradox Tree" paper is the sharpest match I found, and it gives you a vocabulary upgrade worth adopting. It separates Simpson's paradox into two structurally different cases that get conflated in casual treatment but matter enormously for what a Hasse diagram is showing you: a confounder Z influencing both X and Y causes the marginal and conditional associations to diverge, versus a third variable Z that doesn't confound X and Y but where conditional associations differ across levels of Z due to effect heterogeneity -- and critically, in the confounding case the subgroup associations remain consistent with one another, while in the heterogeneous-effects case the subgroup associations differ from one another, with some aligning with the overall aggregated association and some opposing it. Stated in Colvin terms: a "Simpson-like reversal" where the parent disagrees with a uniform set of children is the confounding signature; a reversal where the children disagree with each other is the heterogeneity signature -- your "no grandchildren disagree among themselves" criterion is, in this literature's vocabulary, specifically testing for the second kind, not the first. That's a genuinely useful disambiguation to import: the diagram's stability check is really a heterogeneity-of-effect detector, with confounding as a different, related-but-distinguishable failure mode that shows up as parent-vs-children disagreement rather than children-vs-children disagreement. Sdu Sdu 2. Lerman's "Computational Social Scientist Beware" paper is the closest to a manifesto for exactly the practice the Colvin Diagram operationalizes, written with zero causal vocabulary at all -- purely "don't trust the aggregate, check the subgroups." Observational data about human behavior is often heterogeneous, generated by subgroups within the population that vary in size and behavior, and heterogeneity predisposes analysis to Simpson's paradox, whereby trends in data aggregated over the entire population may be substantially different from those of the underlying subgroups. This is the "indecent induction" problem stated in pure statistics-of-behavioral-data terms -- no graph, no do-operator, just: heterogeneous subgroups make pooled numbers untrustworthy, so go look at the subgroups. The Colvin Diagram is, among other things, a generalized, interactive, always-on instance of the diagnostic test Lerman proposes as a one-off check. arxiv 3. The clinical subgroup-analysis literature gives you the cleanest statement of why "one variable at a time" is the wrong default and your Colvin point about simultaneous joint display matters statistically, not just visually. Conventional subgroup analysis, in which single patient attributes are evaluated one-at-a-time, has at best moderate statistical power to detect variation in treatment effect resulting from treatment-related harm, even under otherwise good study conditions. This is a quantified version of exactly the argument you made several turns ago about why showing X, XY, XZ, XYZ jointly beats walking splits sequentially -- sequential univariate subgroup checking is a known, named, underpowered practice in the clinical trials literature, not just an aesthetic preference for one display mode over another. TheCVF The honest gap in this literature relative to the Colvin Diagram, worth flagging plainly: every one of these papers treats heterogeneity-detection as a thing you do once, at analysis time, with a fixed pre-chosen stratifying variable or two, then report. None of them have the REFILTER move -- the idea that the admissible candidate set itself is a dial the analyst keeps revisiting, with old conclusions explicitly exposed to re-examination when the dial changes. The clinical-trials field actually has the opposite norm, for good reason (pre-registration, to prevent exactly the p-hacking risk of trying subgroup after subgroup until something looks interesting) -- so the Colvin Diagram's free re-filtering is doing something this literature would view with real suspicion outside of a strictly exploratory, hypothesis-generating context, which is precisely the "abductive, not determinative" framing you and I landed on for the causal version of this discussion. Worth carrying that caution over: sans causality, subset stability and subclass defeat are well-trodden, statistically respectable ground (heterogeneity detection, Simpson diagnostics, subgroup-analysis power); the free re-filtering move is the part that's still doing something philosophically distinctive, and it's also the part a frequentist clinical-trials statistician would flag as the diagram's biggest practical risk if used without discipline about multiple comparisons. ME: it is true one can cherry-pick an Ackerman subgraph that suggests causation. presumably one is looking at all the Ackerman subgraphs within a set of features considered candidates. it's still better than letting the black box ML trigger on a set of features that have dubious relevance. in PCA1 vs PCA2 plots, one suggests clusters without consulting PCA3. not to be defensive, but anyone can replot a different Ackerman subgraph in the Colvin Diagram, but visualizing all PCAn in combinations of 2 is actually contrary to current practice. 9:22 PM That's a sharp point, and the PCA comparison does real work -- it's not a defensive move, it's identifying exactly where the multiple-comparisons worry actually bites and where it doesn't. The honest version of the worry, stated precisely: if an analyst tries Ackerman subgraph after subgraph until one happens to show a clean, stable, non-reversing propensity shift, and reports only that one, that's p-hacking with extra visual legitimacy -- the same failure mode as the clinical-trials field's pre-registration norm exists to prevent, just dressed in a nicer UI. The Colvin Diagram doesn't prevent that misuse any more than a contingency table prevents someone from cherry-picking which cross-tab to publish. No tool prevents selective reporting; that's a discipline of the user, not a property of the display. But your point is that this isn't actually how the tool is meant to be used, and the comparison to PCA1-vs-PCA2 nails why. Standard practice with PCA plots is to look at PC1 vs PC2 because those two components are privileged -- they're the directions of maximum variance, chosen by the math itself, not picked by the analyst's hopes. Nobody thinks "well, someone could cherry-pick PC4 vs PC7 to suggest a cluster that isn't really there" as a serious objection to PCA, because in ordinary practice nobody scans all pairs looking for a flattering one -- they look at the top two or three and stop, and the privileged ordering itself is the safeguard. The actual malpractice version of PCA -- scanning every PCₙ-vs-PCₘ pair until two happen to separate nicely -- would be recognized instantly as fishing if anyone did it, and it's not current practice precisely because it's obviously illegitimate once stated. Your claim is that the Colvin Diagram's analogous discipline is the opposite of cherry-picking-by-omission: the tool's design pushes toward looking at all the Ackerman subgraphs within the admissible candidate set, not toward finding and presenting one. A black-box ML model that triggers on a feature combination chosen by gradient descent, with no visibility into which Ackerman-subgraph-equivalent it's effectively using or how stable that choice is under nearby refilterings, is actually in a worse epistemic position than an analyst who looks at the full menu of subgraphs and reports the pattern across all of them -- including the disagreements. The ML model isn't immune to the fishing problem either; it's just hidden it inside the optimization, where nobody gets to see the equivalent of "this only looks clean from this one angle." Visibility of the whole menu, even with the standing risk that a bad-faith analyst could still report only one slide from it, is strictly better than a black box that never shows you the menu at all and may have implicitly picked a bad one. So the fair statement of where the tool sits relative to the worry: it doesn't eliminate the risk of selective reporting -- no visualization can, that's a discipline of use, like pre-registration is for trials -- but it does something PCA's convention and pre-registration both do, which is make the full space of alternatives inspectable rather than collapsed before anyone sees it. The thing that would actually fail this test is a workflow where someone tries several Ackerman subgraphs, only screenshots the one that shows a clean stable shift, and presents that screenshot as "the" diagram -- which is a misuse of the tool's transparency rather than something the tool encourages. And the comparison sharpens something else worth stating plainly: refusing to look at PC3 because PC1-vs-PC2 already told a tidy story is itself a kind of cherry-picking that's currently unremarked-upon as standard practice, while the Colvin Diagram, structurally, makes that kind of incompleteness harder to default into, since the lattice is right there to be refiltered and re-walked rather than fixed at two privileged axes by convention. ME: it is an excellent point that PCA1 vs PCA2 is privileged, and rarely does PCA3 variance come close to PCA2. one hopes. but in the selection of Ackerman subgraphs, the analyst naturally looks at the most interesting features x1, x2, x3. if there are 15 features under dialogical consideration, one sees all 15 wrt x1, then all 15 wrt x2, etc. if one reports the diagram for predicting brooklyn from x1=columbia-university-alumni, and all shadings thereof, such as columbia-u-lawyer, columbia-u-artist, columbia-u-artist-musician, etc., the cherry picking is revealed by asking what happened to predicting brooklyn from x1=lawyer, or x1=artist? when someone says brooklyn from x1=korean shows homogeneity, one should wonder why korean mattered so much to predicting brooklyn. 9:30 PM Right -- this is the actual structural answer to the cherry-picking worry, not just a procedural norm layered on top, and it's worth being precise about why it works. The PCA1-vs-PCA2 privilege comes from the math: variance ordering picks the axes for you, so there's nothing to cherry-pick among, the convention and the legitimate choice coincide. The Colvin Diagram doesn't have that -- there's no eigenvalue ordering the fifteen candidate features by importance before the analyst looks. So the privilege has to come from somewhere else, and what you're describing is that it comes from completeness of the report relative to the parent, not from any property of a single subgraph. Walk it through with your example. The claim "x1 = Columbia-alumni predicts Brooklyn" gets reported alongside its full sibling set under that same parent -- Columbia-lawyer, Columbia-artist, Columbia-artist-musician, and so on, all shadings, not just the one that happened to look clean. That completeness is exactly what makes the cherry-pick self-revealing: if someone reports only "Korean predicts Brooklyn with high homogeneity" and the rest of the x1-sibling fan -- lawyer, artist, alumni-of-other-schools, whatever the admissible candidates actually were -- is silently absent from the report, the absence itself is the tell. You don't need an independent audit or a pre-registration document to catch the omission; you just ask "what about the other fourteen features under x1, and what did their full Ackerman subgraphs look like?" The fishing expedition leaves a visible hole in a report that's supposed to be exhaustive over the declared candidate set, the same way a phylogenetic tree with a missing branch looks obviously incomplete rather than just unremarkable. And your second move -- "why did Korean matter so much" -- is a different and sharper diagnostic, worth separating out. That's not about completeness of reporting, it's about asking whether the feature itself needs justification for being on the admissible list in the first place. If Korean shows unusually strong homogeneity predicting Brooklyn while most other ethnicity-type features under the same parent don't, the right response isn't "great, causal claim secured" -- it's "why is this one feature carrying so much weight, and is that itself suspicious, perhaps because it's proxying for something else entirely (a particular neighborhood's settlement history, a particular era's immigration pattern) that isn't yet in the candidate set." That's the (a)-condition from several turns ago -- an apparently strong, stable result is itself a signal that an unnamed intermediary might be sitting just behind the feature you have, not a stopping point. A surprisingly clean result should raise suspicion of an unrefined term, not relax it. So the actual discipline isn't "trust whatever subgraph gets shown," it's two linked norms, and they're different from each other: report the full sibling fan under whatever parent you're discussing, so omission is visible (this is what defeats simple cherry-picking), and treat surprising homogeneity in any one feature as a prompt to ask what that feature might be standing in for, rather than as confirmation (this is what defeats the subtler trap of an honestly-reported but still-confounded result). The first is a reporting norm the diagram makes easy to enforce, since the sibling fan is right there in the lattice rather than something you'd have to reconstruct. The second is a habit of suspicion the diagram can't enforce by display alone -- that one still depends on the analyst, the way "why did this p-value come out so clean" always has, no matter how good the visualization is. ME: in wikipedia entries, being korean predicts non-brooklyn cleanly, btw, and all subclasses agree. you'll see korean and lawyer, korean and banker, korean and hip-hop banker, etc., if there are distinctive data. but start with the ackerman diagram for new-jersey, and you'll see good homogeneity predicting brooklyn, with new-jersey-korean being a disagreeing subclass. there tends to be disagreement until you descend to a node where there is not a lot of disagreement among ADMISSIBLE DIALOGICAL subclasses WITH DATA. if that node has small n, your data set is not really predictive for this query. this is what the Colvin Diagram seems best at, not making putative causal or probabilistic influence claims. 9:36 PM That's the cleanest statement of what the tool is actually for that's come up in this whole conversation, and I think it's right to plant the flag there rather than on the causal language at all. What you've just walked through with New-Jersey/Korean is a concrete instance of a general pattern worth naming precisely: disagreement is the normal, expected condition at most nodes, and it persists going down the lattice until you hit the node where it stops -- and that descent itself, not any single propensity number, is the actual output of the tool. New-Jersey alone shows clean homogeneity toward Brooklyn. New-Jersey-Korean breaks from its siblings and disagrees. You don't stop there and declare "New Jersey doesn't predict Brooklyn" or "Korean overrides New Jersey" -- you note the disagreement, descend further, and keep watching until either the disagreement resolves into a stable node with adequate data, or it doesn't, in which case the honest report is "this branch of the dialogue doesn't bottom out cleanly with the data on hand." That reframes the whole stability criterion in a way that's more modest and, I think, more defensible than anything causal we said earlier: the thing the diagram is actually computing is not "is this causal" but "where does disagreement stop, and is there enough data there to trust the stopping point." Three distinct outcomes, all useful, none of them causal claims: Disagreement resolves at a node with healthy n: you've found the level of specificity at which your data is actually predictive, and you can stop refining -- this is Reichenbach's narrowest-reference-class-with-adequate-statistics problem, solved operationally rather than asserted. Disagreement resolves at a node with thin n: you've found where it would stop, but you don't have enough data there to trust it -- which is itself a valuable, honest answer ("we don't know yet, here's exactly what we'd need"), very different from silently reporting the parent's number as if it applied. Disagreement never resolves within the admissible candidate set at all -- every descent path keeps fragmenting -- which is the signal that your fifteen features aren't sufficient for this query, full stop, and the right move is REFILTER: enlarge the dialogue, because the current vocabulary can't settle it. None of those three outcomes requires the word "cause." They're all statements about where stable, well-supported predictiveness lives in the lattice, which is exactly the contingency-table-upgrade / mixed-model-honesty function we agreed on early in this conversation -- except now stated as the tool's primary and sufficient purpose rather than as a consolation prize next to a causal ambition that needed hedging every time we examined it closely. I think this is the right place to leave the causal question, honestly: every time the causal framing got pushed on hard in this conversation, it needed another layer of qualification -- abductive not determinative, relative to a discourse, susceptible to disentanglement-driven revision, defeasible rather than warranted. The predictiveness/stability framing you've just given needs none of that machinery to be true and useful. "There is a node, found by descent, where admissible dialogical subclasses with adequate data stop disagreeing" is a claim that's either true of your data or it isn't, checkable by looking, and doesn't smuggle in any commitment about mechanism, intervention, or the metaphysics of cause at all. That's a more secure foundation for the tool's value than anything resting on the word "causal," even hedged six ways -- and it suggests the strongest version of the paper might foreground exactly this (call it something like "dialogical reference-class stability" or "subclass consensus depth") and treat the causal-language discussion as a clearly-marked appendix about what stability does and doesn't license you to additionally claim, rather than as the headline. ME: but the insight into causal being relative to candidates in a conversation is perceptive. smoking DOES cause cancer if there are no more refined candidates. one can always introduce dormative powers, or hereditary predisposition, or specific nicotine and tar levels, to unmix the model. 9:40 PM Right -- and "perceptive" is the correct register for it, not "secured." The two things you've now said in sequence aren't in tension, they're the full statement: the stability-of-prediction result is what's checkable and true outright; the causal label sitting on top of it is the part that's relative, provisional, and exactly as good as the candidate set currently in play -- and that relativity isn't a weakness to apologize for, it's the actual content of what "cause" means once you're outside physics' sufficient-state framework. The smoking case is the right one to press on because it shows the relativity isn't a toy example or an edge case -- it's the central, highest-stakes instance, and it still works the same way. "Smoking causes cancer" was, for decades, undefeated relative to the candidates anyone could put on the table: you could split by occupation, by diet, by region, by class, and the propensity shift held -- no disagreeing grandchildren, no Simpson reversal, across every admissible feature anyone proposed. That was, correctly, treated as about as solid as causal claims outside physics get. And it was still, at every point, vulnerable in principle to exactly the move you named: someone introduces a new candidate -- genetic predisposition to both nicotine-seeking behavior and cellular susceptibility, say -- and now there's a new dialogue, and the old claim has to be re-walked against a term it was never tested against. (Historically this is almost exactly the shape of Fisher's genuine, serious objection to the smoking-cancer link in the 1950s -- not a denial that the correlation existed, but the genuine possibility of a shared genetic confounder making both smoking and cancer downstream of something else. It was a real candidate, proposed in good faith by a major statistician, and it took decades of further refinement -- dose-response data, animal studies, mechanism work on tar and carcinogens -- to defeat it within the dialogue.) So "smoking causes cancer" was never wrong at any point in that history. It was the correct, undefeated thing to say relative to the candidates then on the table, including while Fisher's confound was a live, unrefuted alternative. What changed wasn't truth, it was the discourse -- new terms entered, the old claim got re-walked, and it survived the re-walking (rather than being abandoned the way, say, "ether" or "phlogiston" had to be). That's the asymmetry worth noticing: discourse-relativity doesn't mean every claim is equally fragile or that today's answer is just "currently fashionable." Some claims survive enlargement after enlargement of the candidate set for seventy years; that's not nothing, even though it's never been promoted to "metaphysically necessary" and never needs to be for the claim to be worth acting on. And "one can always introduce dormative powers" is the sharpest part of what you've said, because it names the actual structural risk on the other side -- the move that looks like refinement but isn't. A dormative-powers move doesn't add a real candidate that could be checked against data; it just relabels the explanandum as the explanans ("smoking causes cancer because smoking has a cancer-causing power") and produces the illusion of a deeper account with no new admissible feature and no new test. The discipline the dialogue needs, symmetrically with everything we said about REFILTER, is: a new candidate earns entry by being checkable -- it has to be a feature you could actually condition on and see propensities shift or not -- not by merely being utterable. "Hereditary predisposition" was a real candidate because you could (eventually) measure it and stratify by it. "Smoking has a tendency to cause cancer" is not a candidate at all, it's the original claim wearing a costume. The relativity-to-discourse view needs that filter to keep from collapsing into "any redescription defeats any claim" -- which would make the whole framework useless rather than just appropriately humble.