The data shows a C+. The data shows a C. Two of the most heavily capitalized AI companies in the world, Anthropic and OpenAI, have received near-failing grades on an AI safety index. The headline is a governance signal. The subtext is a data problem. As someone who has spent the last seven years building on-chain analytics models, I see this not as a technology report, but as a ledger entry with missing source documents.
We are being asked to evaluate a company's trustworthiness without seeing the underlying transaction log. That is a dangerous prompt.
Context: The Governance vs. Capability Gap
AI safety indexes are not benchmark suites. They do not test reasoning, code generation, or multi-step logic. They measure governance. Public commitments, internal red-teaming protocols, external audit accessibility, transparency disclosures, and stated alignment processes. It is a scorecard of administrative intent, not a performance test of the model itself.
The report places Anthropic at C+ and OpenAI at C. Both fall within the 'failing' band. This signals that even the industry's leaders have not turned safety governance into a standardized, auditable practice. The ledger never lies, only the narrative hides. The narrative here is that safety is a priority. The ledger shows it is a slide deck.
From my own experience with the 2018 ICO audits, I learned that a team's promise is a liability until verified on-chain. We audited 47 smart contracts and found critical vulnerabilities in 12, a 25% failure rate. The lesson was that intent without execution is just a bug in the governance layer.
The current AI landscape mirrors that. The intent to be safe is publicly declared, but the execution is fragmented across subjective rubrics and non-standardized evaluations. The data suggests the industry is still in its 2018 phase, building hype on promises rather than proof.
Core Analysis: The Evidence Chain Is Broken
My main concern is not the grade itself, but the failure to provide a chain of custody for the evidence. The original report does not disclose the scoring methodology. There is no breakdown of metrics, no definition of the rubric, no mention of whether the score includes red-team results, jailbreak rates, or actual safety incidents. It is a verdict without a discovery.
In my work at Dune Analytics, I would never publish a dashboard that just showed a total with no explanation of the SQL query behind it. That would be a flagged as a red flag for data integrity. The same standard applies here.
When I built the GARCH volatility model for NFT floor prices in 2021, the methodology was as critical as the output. I had to show the transaction volume, the sample window, and the whale wallet concentration before anyone accepted the conclusion. The AI safety index presents a conclusion without a methodology, leaving it vulnerable to misinterpretation.
The report also mentions a deepening relationship with the military. This is a critical variable, but it is treated as a footnote rather than a data point. What is the exact nature of this relationship? Is it a dual-use infrastructure contract, or is it a co-development of autonomous targeting systems? The absence of detail turns an ethical concern into a speculative narrative. Tracing the ghost liquidity back to its source requires that we look at the wallets, not just the price.
The report also conflates 'safety commitment' with 'safety outcome'. A company can sign a pledge for safety and still have a model that is easily jailbroken. The correlation between governance paperwork and actual model robustness is unproven. I have seen this in DeFi audits, where a project with a thorough audit report still gets exploited because the audit did not cover the specific attack vector. The same logic applies here.
Contrarian: The Business of Being Safe
Here is the counter-intuitive angle. The grades are low, but the market does not care. The capital markets continue to price AI companies on capability, user growth, and ecosystem lock-in. Safety scores have not yet entered the pricing model. This creates a blind spot for investors.
We are looking at a classic mismatch between fundamental risk and market valuation. In 2022, when I analyzed the stablecoin depeg, the data showed that 30% of risky positions were undercollateralized. The market was still pricing in stability until the moment of the crash. The C+ and C grades are the same. They are a warning signal that the market is currently ignoring.
A second contrarian point is the reaction to the military relationship. The public outcry is predictable, but the data suggests that this is not a new phenomenon. AI companies have been working with defense departments for years. The real question is not whether they do it, but whether the contracts are transparent. A public partnership with a defense agency, documented and disclosed, is a governance feature, not a bug. The lack of disclosure is the actual risk.
Implications: The Next Signal
For AI companies, the challenge is to treat the safety index as a data quality issue. The next report will be published in a quarter. The response to the C grades should be to publish the methodology, the weighting, and the raw evidence. The industry needs an open-source safety audit template, not a new marketing report.
Based on my experience in the crypto space, the first step is to standardize the audit checklist. I reduced my own audit review time by 40% in 2018 by standardizing the process. The AI industry needs to do the same for safety, creating a clear, verifiable, and replicable framework for evaluating risk.
I am not interested in the narrative of who is safer. I want to see the data that proves it. The next signal to watch is whether the scoring institution publishes its rubric. If they do not, the score is simply a narrative tool, not a data point.
If the data is not verifiable, it is just a narrative. The question is: are we going to audit the score, or just accept the grade?