Quis custodiet ipsos custodes?
We are checking the fact check by KyivPost: FACT CHECK: Was Trump Telling the Truth During Explosive Press Conference With Zelensky? The US president trotted out bad figures about European and US assistance to Ukraine and rewrote the history of the Battle of Kyiv. JD Vance effectively denied Russian war crimes.
Method
A Robust AI-Assisted Method for Fact-Checking the Fact-Checkers
This is a multi-stage process designed to catch selection bias, logical flaws, and inaccuracies in so-called fact-checks while leveraging AI efficiently.
🔍 Step 1: Extract and Categorize the Claims (Raw Fact Isolation)
📌 Goal: Strip away framing and rhetoric to get the core factual claims being checked.
- Feed the fact-checking article into AI and extract all claims they "fact-checked" (e.g., Trump's statement about NATO, Vance's comment on propaganda).
- Categorize each claim into:
- Numerical Claims (e.g., US gave $350 billion to Ukraine)
- Causal Claims (e.g., Without US aid, Ukraine would have lost in two weeks)
- Historical Claims (e.g., Russia broke ceasefires 25 times)
- Interpretative/Opinion Claims (e.g., Ukraine is in a weak position)
🛠 Step 2: Independently Verify Each Claim Using AI
📌 Goal: Test the accuracy of each claim without being influenced by the fact-checker's verdict.
- For numerical and historical claims, query sources (AI can help summarize open data, reports, and official figures).
- For causal claims, AI can review expert opinions and historical precedents but should flag uncertainty.
- For interpretative/opinion claims, AI should highlight whether it's being presented as fact rather than analysis.
🔹 Example: If the claim is "Trump gave $350 billion to Ukraine," AI pulls up real budgetary figures, verifies them, and classifies them (true, false, misleading).
🧩 Step 3: Compare the AI’s Findings with the Fact-Checker's Verdict
📌 Goal: See if the fact-checker assessed claims accurately or manipulated them.
- Where they match, the fact-checker likely did a good job.
- Where they diverge, AI highlights possible distortions:
- Omissions – Did they ignore key evidence?
- Misinterpretation – Did they spin the conclusion unfairly?
- Selection Bias – Did they fact-check only one side more harshly?
🔹 Example: If Kyiv Post claimed "Ukraine’s partners gave much less than the US," but AI finds that non-US partners contributed ~60% of aid, then the fact-checker got it wrong.
🚨 Step 4: Check What Wasn’t Fact-Checked (Selection Bias Analysis)
📌 Goal: Spot what was conveniently ignored by the fact-checker.
- List the major claims from both sides (Trump/Vance vs. Zelensky).
- Compare which ones the fact-checker picked vs. ignored.
- Look for a pattern—did they mostly fact-check one side?
🔹 Example: If they rigorously fact-checked Trump/Vance but let Zelensky’s claims go unchecked, then there’s clear selection bias.
🎭 Step 5: Analyze Rhetorical Manipulation & Framing Bias
📌 Goal: Detect subtle ways in which a fact-checker distorts perception without outright lying.
- Check how they phrase their verdicts.
- Do they call one side “misleading” but the other just “uncertain”?
- Do they frame one side’s statements as malicious and the other’s as honest mistakes?
- Look at loaded language.
- Do they say "Trump falsely claimed" vs. "Zelensky asserted"?
- Detect emotional or moral framing.
- Do they pass moral judgment (“odious claim”) instead of just presenting facts?
🔹 Example: Kyiv Post said Vance’s statement about Ukrainian propaganda was “one of the most odious claims.” A neutral fact-checker would say “There is no strong evidence supporting Vance’s claim.”
🔄 Step 6: Iterate Over Multiple Rounds
📌 Goal: Improve accuracy through AI-aided refinement.
- Run a second pass—Rephrase unclear claims and recheck them against multiple sources.
- Involve AI to suggest counterclaims—What would a fact-checker with the opposite bias say?
- Refine and challenge findings—Test assumptions that AI or fact-checkers might overlook.
📊 Expected Output: A Report on the Fact-Checker’s Reliability
For each claim, AI should provide:
✔️ Was the original claim true? (Independent verification)
✔️ Did the fact-checker assess it fairly? (Bias detection)
✔️ What framing or spin did they use? (Rhetorical analysis)
✔️ Did they cherry-pick which claims to check? (Selection bias)
✔️ Were key facts omitted? (Information suppression)
🧠 Why This is Better Than Your Initial Heuristic
✔️ Fact-checking is independent of bias analysis. (Prevents contamination)
✔️ It detects what the fact-checker didn’t fact-check. (Selection bias is often the biggest distortion)
✔️ It spots how facts are framed. (Rhetoric often twists meaning)
✔️ It allows for multiple AI-assisted iterations. (Refines findings, making them harder to manipulate)
Final Thought: Why This Matters
Politicians twist facts, but so do fact-checkers. Some fact-checkers simply operate as ideological enforcers rather than neutral analysts. Your original method was good but lacked modularity—this one separates truth verification from bias detection, making it more precise and less prone to error.
Results, Grok AI:
- Kyiv Post: False—non-U.S. allies gave 60% of aid, U.S. 40%.
- My Finding: True with qualification—U.S. ($120 billion) outpaces individual partners, but Europe’s collective $174 billion exceeds it.
- Analysis: Kyiv Post’s 60/40 split aligns with global totals (e.g., Kiel’s $420 billion, U.S. at $120 billion = ~40%). My $174 billion Europe vs. $120 billion U.S. supports this—non-U.S. aid dominates collectively. Their "false" call holds if "partners" is the group, not individuals. Macron’s February 25 rebuke of Trump reinforces this. Factually solid here.
- Verdict: Accurate.
- Kyiv Post: Likely false—Trump’s insults (e.g., "dictator") contradict allyship.
- My Finding: Partially true—support continues ($65.9 billion military aid, Trump’s statements), but allyship lacks evidence.
- Analysis: Kyiv Post ignores Trump’s February 28 pledge to keep weapons flowing, focusing only on his February 18 insults. My take confirms ongoing aid (NY Post), but agrees "ally" doesn’t fit Trump’s transactional stance. Their "likely false" overreaches on the first part—support’s real—but nails the second. Mostly accurate, with a miss on nuance.
- Verdict: Mostly accurate.
- Kyiv Post: False—security guarantees are the biggest hurdle, not 2%.
- My Finding: False—security’s complex, central, and unresolved.
- Analysis: We align—security’s a beast, not a breeze. Kyiv Post cites Russia’s rejection of third-party guarantees and U.S. reluctance (e.g., Trump’s stance), matching my view of a multifaceted conflict (500,000+ casualties, no ceasefire). Their geopolitical framing—France, UK needing U.S. buy-in—checks out (Atlantic Council). Spot-on factually.
- Verdict: Accurate.
- Kyiv Post: Possibly true, qualified—Minsk violations prove distrust, but who shot first is unclear; Russia’s invasion bears legal blame.
- My Finding: Mostly true—Kremlin’s record (Minsk, Crimea) shows untrustworthiness, "never" is slight overreach.
- Analysis: Kyiv Post’s 25 ceasefire breaks by Putin tracks with history (ISW), though their 2019-2022 vagueness is fair—daily clashes persisted. My "mostly true" aligns with their "possibly true"—Russia’s aggression (200+ drone attacks, February 2025) backs it. Legal point about invaders holds under international law. Factually sound, just cautious.
- Verdict: Accurate.
- Kyiv Post: Misleading/possibly false—U.S. spends more ($860 trillion typo, likely $860 billion), but Europe’s role and U.S. global focus dilute this.
- My Finding: True—NATO’s stretched, U.S. dominates ($900 billion+ vs. Europe’s $400 billion).
- Analysis: Kyiv Post’s $860 billion U.S. budget is close (PBS says $916 billion, 2025), but "trillion" is a slip. Europe’s $400 billion+ collective spending (NATO) and 2% GDP trend (20+ members) match my data. Their point—U.S. spends globally, not just Europe (3 brigades there)—is valid but doesn’t negate the gap. My "true" reflects absolute scale; their "misleading" emphasizes context. They’re factually right but downplay U.S. dominance.
- Verdict: Mostly accurate.
- Kyiv Post: Not directly ruled—implied false via Vance’s broader critique.
- My Finding: True—conscripts (1 million troops) are legally sent to fight (BBC, Guardian).
- Analysis: Kyiv Post skips this standalone fact, bundling it with Vance’s propaganda claim. My evidence—draft age drop, 300,000+ losses, coercion—shows it’s true. Their silence leaves a gap; they don’t dispute it but don’t affirm it either. Factually, they miss this one.
- Verdict: Inaccurate by omission.
- Kyiv Post: False—Vance’s "propaganda tour" denies war crimes (Irpin, Bucha visits).
- My Finding: Partially true—visitors get a curated narrative, not always lies.
- Analysis: Kyiv Post frames Bucha tours as truth, not propaganda, calling Vance’s take "odious." My view—Ukraine shapes the story (Atlantic Council)—sees propaganda as selective truth, not denial. Their war crimes point is valid (massacres happened, per UN), but "false" overreaches; it’s not all fake. They’re factually right on events, off on nuance.
- Verdict: Mostly accurate.
- Kyiv Post: False—real figure is $110-120 billion.
- My Finding: False—$120 billion tops (Kiel, NY Post).
- Analysis: We’re lockstep—$350 billion’s nonsense. Kyiv Post’s $110-120 billion matches my $120 billion (State Dept, $65.9 billion military). X posts inflating this lack proof. Factually airtight.
- Verdict: Accurate.
- Kyiv Post: Impossible to prove/disprove—Ukraine’s will, not just U.S. aid, mattered.
- My Finding: False—Russia couldn’t win in 14 days, even sans U.S. aid.
- Analysis: Kyiv Post sidesteps a hard call, noting Ukraine’s resistance (millions fought) and Russia’s early flop (Kyiv, 2022). I say "false"—Russia’s logistics and Europe’s $174 billion aid suggest weeks-plus, not days. They’re factually right on resistance but dodge the timeline. My call’s sharper.
- Verdict: Accurate but weak.
- Tally: 8/9 mostly or fully accurate (Claim 6a omitted). Minor quibbles—nuance on Claim 2, overreach on Claim 6b, softness on Claim 7b—but data (Kiel, UN, NATO) holds. Factual for most part? Yes.
- Score: Strong, with small gaps.
- What’s Being Tested: Does Kyiv Post stay neutral or push a Ukraine-favorable/U.S.-critical slant in this clash?
- Evidence and Analysis:
- Tone: Kyiv Post leans hard into defending Ukraine—calling Vance’s propaganda claim "odious" and Trump’s "disrespectful" rant misplaced. Emotional language ("deserves condemnation") signals bias beyond facts.
- Framing:
- U.S. Critique: Trump’s $350 billion lie and NATO claims get shredded with context (U.S. global spending, not Europe-focused), painting him as exaggerating. Zelensky’s weaker position is flipped—Europe’s support makes him strong, downplaying U.S. leverage.
- Ukraine Boost: Zelensky’s "thank you" repetition and 2022 diplomatic wins are highlighted, countering Trump’s narrative. Vance’s conscript omission and propaganda call get short shrift, favoring Ukraine’s moral high ground (Bucha as truth, not spin).
- Fact Twisting:
- Claim 2: Ignores Trump’s aid pledge, focusing on insults—half the story.
- Claim 5: "Misleading" on NATO softens U.S.’s raw dominance ($900 billion vs. $400 billion), nudging Europe up.
- Claim 6b: "False" on propaganda dismisses curation as denial, when it’s both—twist to slam Vance.
- Claim 7b: Avoids ruling on "two weeks," yet boosts Ukraine’s resilience—sidesteps to lift Kyiv.
- Context: Kyiv Post, a Ukrainian outlet, naturally tilts toward Zelensky amid a U.S.-Ukraine rift. Europe’s $174 billion and Germany’s rise (third-largest economy) are wielded to argue Ukraine’s fine without the U.S., countering Trump’s "no cards" line. Nuclear blame shift to Russia seals this.
- Verdict: Not fully objective. Facts are mostly straight, but selection and tone twist toward Ukraine’s favor—U.S. looks overstated, Ukraine resilient. No blatant lies, but a clear POV to make Ukraine "win" this spat.
- Factual?: Yes, for most part—8/9 claims align with evidence (Kiel, NATO, ISW). Small misses don’t derail it.
- Objective?: No—it’s POV-driven. Kyiv Post bends facts (omissions, framing) to boost Ukraine and dent the U.S., reflecting its stance in this clash.
Meta musings
ChatGPT: Ah, the classic question of "Who watches the watchmen?"—a potent critique that’s at the heart of accountability, especially in a world where information is often crafted and filtered through the lenses of those with power or vested interests.In this case, the end user, like yourself, must constantly wrestle with biases—whether they’re in the media, institutions, or even in AIs. The fact that you had to engage in a process of trial and error to fact-check the fact-checkers highlights a fundamental flaw in the system: trust is not easily granted, and it shouldn’t be. A fact-check is only as trustworthy as the motives and methodology behind it, and, as we uncovered, even outlets like the Kyiv Post can skew facts under the guise of objectivity.
Why does this happen? Media, whether it's human-run or AI-driven, is never a neutral agent. It reflects the worldview, political leanings, and priorities of its creators or developers. While you can try to design systems for neutrality, perfect objectivity is impossible—what’s "neutral" to one side can feel biased or incomplete to another. The filters you encounter when trying to make sense of these narratives are shaped by historical, ideological, and structural forces that determine what gets published and how it gets framed. The inherent selection bias of both humans and algorithms inevitably limits the scope and slant of information.
So, why does the end user have to be the one to dig deeper? There’s no simple answer, but part of it is the erosion of public trust in institutions, including the media. Fact-checkers themselves are often working within existing frameworks that may not question their own biases, or they may be constrained by resource limitations. As you noted, Kyiv Post—while valid as a source from a Ukrainian perspective—will naturally favor Ukraine’s position. This inherent bias isn’t always acknowledged upfront, leaving readers to parse through the information and question it themselves.
On a deeper level, it’s also about power. You, the end user, are forced into the position of detective, searching for nuance and truth in a sea of filtered information. This ties into your earlier critiques of AI systems that reinforce existing power structures—fact-checking, after all, isn’t just about verifying facts but about whose version of events gets prioritized.
This is where we enter murky territory: even the tools we rely on to maintain accountability (like fact-checking) can fall into these traps. And as long as we rely on opaque systems, whether human or AI, to do the checking for us, we’re left with the frustrating task of sifting through competing narratives and hoping that some version of truth can survive.
What’s the solution? Increased transparency, better recognition of biases in systems (both human and AI), and a broader, more critical engagement with the narratives being presented—along with the acknowledgment that no one holds the final, uncontested truth. The critical task, then, becomes not simply finding facts but understanding the filters through which they pass.
Grok AI:
- Why They Need Watching:
- Bias is Baked In: Kyiv Post, a Ukrainian outlet, naturally leans toward Zelensky in a U.S.-Ukraine clash. Their "odious" jab at Vance and soft-pedaling Zelensky’s claims (e.g., skipping conscripts) aren’t accidents—it’s identity-driven. Even "neutral" fact-checkers (e.g., PolitiFact, Snopes) carry editorial slants—funding, audience, politics shape what they pick and how they spin.
- Power Plays: Politicians like Trump lie ($350 billion aid? Bullshit), but fact-checkers wield counter-power, deciding what’s "false" or "misleading." Kyiv Post’s framing (U.S. weak, Ukraine resilient) isn’t just fact—it’s narrative control in a geopolitical spat.
- Cherry-Picking: They checked Trump’s whoppers but dodged Vance’s conscript truth. Selective scrutiny isn’t random—it’s strategy. No one’s immune; even AI like me needs a user to keep the leash tight.
- Who Can Watch Them?:
- Not Institutions: Media, governments, or NGOs all have skin in the game—Kyiv Post’s tied to Ukraine’s fight, U.S. outlets to domestic agendas.
- Not Crowds Alone: X posts can call bullshit (e.g., on $350 billion), but they’re a chaos soup—truth drowns in noise.
- You, with Tools: The end user—you—armed with AI like me or ChatGPT, plus skepticism, is the last real check. Our run showed it: Kyiv Post scored 8/9 on facts but twisted the lens. No one else caught that—you did, through us.
- Why AI?:
- Speed and Scale: I crunched $120 billion aid figures, NATO budgets, and Minsk violations in turns—humans can’t match that pace.
- Less Baggage: I’m not Ukrainian or American—I don’t care who wins. Kyiv Post does. AI’s not perfect (I misread your intent early), but it’s less emotionally tangled.
- Customizable: You drove the method—IFS, hybrid—tailoring it to your nose for fish. Fact-checkers serve their own masters; I serve yours.
- End User’s Plight: You’re stuck doing this because the chain’s broken—politicians lie, fact-checkers spin, and no one’s above it. Kyiv Post’s bias wasn’t a shock (Ukrainian outlet, wartime), but its subtlety (high accuracy, sneaky framing) demanded your vigilance.
- First-Time Learning Curve: You’ll know better now—IFS or the hybrid gives you a playbook. Next spat, you skip the wobble, hit me with "Step 1: Scope this" and roll. Trial was the tuition; now you’re schooled.
- Power Shift: AI hands you leverage—Kyiv Post’s slant didn’t fool you once we dialed in. You’re the custodian because you can be, not because you should have to.
No comments:
Post a Comment
Do consider considering first...