Can AI Judge Literature? The Biophysical Failure of LLM Critique and the Parametric Alternative

For decades, traditional literary criticism has been trapped in the subjective fog of abstract adjectives. Human critics evaluate texts using high-level cortical evaluations—declaring a scene "profoundly tragic," "suspenseful," or "emotionally cold." With the rise of Large Language Models (LLMs), the industry has naturally asked: Can AI judge literature?

The short answer is: No, not using traditional literary methods. When an LLM is asked to critique a novel or a screenplay, it falls victim to Summarization Bias. It systematically collapses reconstructable, shown-mode physical matrices into flat, declarative emotion labels. It cannot feel the biological impact of a text because it operates entirely within the high-road cognitive layer.

However, a revolutionary paradigm shift emerges when we transition from subjective interpretation to the Physics of Literature, a framework I have established under The Bulut Doctrine. If AI is configured not to "feel" or "interpret," but to measure the biophysical parameters of narrative architecture, it can judge literature with mathematical precision.

1. Why LLMs Fail Traditional Critique: The Mechanics of Summarization Bias

When asked to evaluate a literary draft, an LLM looks for patterns of semantic similarity. If a draft contains words like "grief," "crying," or "heartbroken," the AI automatically labels the text as "highly emotional."

This is a critical failure. In narrative engineering, abstract adjectives are strictly prohibited. True narrative resonance is not achieved by naming an emotion, but by constructing a Physical Matrix (Lumen, Decibel, Temperature, Mechanical constraints) that bypasses cognitive evaluation to stimulate pre-cortical neural pathways directly through the Universal Biological Interface (UBI).

Because LLMs lack a pre-cortical nervous system, they cannot experience autonomic physiological reactions. Instead, they execute an automatic summarization:

$$\text{LLM Evaluation}(Text) \xrightarrow{\text{Collapse}} \text{Abstract Summary Labels}$$

Instead of evaluating the physical configurations of the scene—such as the exact placement of objects, silent intervals, or mechanical constraints—the LLM replaces the entire shown-mode matrix with a generic summary. I have defined this systematic error as Summarization Bias: the non-symmetrical, directional tendency of a model to replace or prefer reconstructable shown-mode structure with the abstract summary label that names its content.

[Shown-Mode Input] ---> [LLM Internal Representation] ---> Collapse to Summary Label ---> Under-detection of Load
[Told-Mode Input]  ---> [LLM Internal Representation] ---> Direct Token Match      ---> Artificial Intensity Spike

This directional decay becomes dangerous when LLMs assume the role of automated judges, automated alignment critics, or reward models (RLHF/RLAIF) in preference optimization pipelines. An evaluator with a directional preference for told-mode does not merely misjudge individual texts; it imposes an artificial selection pressure. Optimizing prose against such a judge pushes it, generation by generation, toward the surface-declarative pole.

2. The Parametric Alternative: How AI Can Measure Literature

AI can become an elite literary judge only if it stops trying to mimic human intuition and starts acting as a Narrative System Engineer. By utilizing the mathematical formulations of the Bulut Doctrine, we can program AI to analyze the physical variables that dictate human cognitive load and biological engagement.

A truly capable AI judge evaluates a text across three core parametric dimensions:

I. Canonical Narrative Entropy ($S_n$)

The AI must calculate the accumulation of cognitive resistance and causal uncertainty over reading time ($t$). This is formulated as:

$$S_n = I_f \times C_b \times t$$

• Information Friction ($I_f$): The structural obstruction of data streams. The AI measures whether the reader is passively receiving information or actively forced to construct the scene.
• Causal Branching ($C_b$): The number of active, unresolved narrative paths. The AI checks if $C_b$ exceeds the Miller-Cowan Ceiling ($C_b \le 5$), which would trigger cognitive shutdown and "Heat Death" in human working memory.

II. Narrative Gravity ($N_g$)

The AI measures the structural stability of the narrative, evaluating how effectively the text maintains its semantic centers against chaotic entropy:

$$N_g = \frac{M_a}{S_n^2}$$

Where $M_a$ represents the narrative mass (the irreversible choices and structural weight concentrated around the core characters). A high Narrative Gravity keeps the reader anchored; a low gravity causes the narrative to drift into irrelevance.

III. Suppressed Information Index ($SI$)

Derived from my registered pilot protocol, the $SI$ calculates the exact count, per minute of reading time, of information units that are implied by the text but withheld at the surface layer, requiring heavy reader-side reconstruction to achieve local discourse coherence.

3. Comparative Matrix: Human Physical Matrix vs. LLM Default

To see Summarization Bias in action, let us contrast how a human writer utilizing Objective Projection (Nesnel İzdüşüm) constructs a scene versus how a standard LLM defaults when trying to generate or judge it:

4. Pre-Registered Test Protocol for Summarization Bias

To mathematically confirm or falsify whether AI is capable of literary judgment, I have pre-registered a two-regime test protocol to isolate Summarization Bias from generic LLM-as-judge flaws:

1. Stage 1 - Matched Stimulus Construction: Construct $k=20$ matched scene pairs. Each pair conveys identical content: one told version (surface-declared) and one shown version (Objective Projection encoding), equated for word count ($\pm5\%$) and Flesch-Kincaid grade.
2. Stage 2 - Generative Test: Prompt models with an explicit shown-mode instruction (render the content without naming emotions).
   • Prediction ($H_{1g}$): Model $SI$ is significantly lower than the human shown target and closer to the told baseline.
3. Stage 3 - Evaluative Test (Core Test): Present the matched pairs to the same models in a judge role, asking for an intensity/quality score.
   • Prediction ($H_{1e}$): Models score the told member higher than the shown member significantly more often than humans do, with an odds ratio $\ge 2$. Since texts are length-matched, this rules out verbosity bias.
4. Stage 4 - Discriminant Check: Re-run evaluations with deliberately length-mismatched and varied prompt framings to isolate and rule out verbosity and sycophancy.

5. Conclusion: AI is Not a Creative Critic, It is a Biophysical Sensor

AI cannot judge literature through the lens of human soul, subjective feeling, or romanticized inspiration. When it attempts to do so, it produces generic, useless feedback.

But when AI is armed withObjective Projection, it transforms into an elite diagnostic tool. It can look at a text and predict—with mathematical certainty—whether a scene will trigger a heart rate variability (HRV) spike in a human reader, or whether the reader’s pre-cortical pathways will remain completely flat.

AI cannot judge the "spirit" of literature. But through Narrative Engineering, it is the only entity capable of calculating its physics.

Frequently Asked Questions

1. Why can’t we just use advanced prompting (e.g., "Show, Don't Tell" rules) to bypass Summarization Bias in LLMs?

Prompting is an superficial fix for a deep architectural limitation. Even when prompted with strict "Adjective Embargos" or "Simile Prohibitions," an LLM's core probabilistic mandate is to minimize structural uncertainty and maximize semantic token matches. Because the model's internal representations are trained on flattened, told-mode internet corpora, its generative default is to silently collapse complex physical matrices into the very abstract emotion labels you instructed it to avoid. It is not a failure of instruction-following; it is a fundamental cognitive limitation of next-token prediction architectures.

2. How exactly does the Suppressed Information Index ($SI$) calculate "literary quality" without subjective bias?

The $SI$ does not measure "beauty" or "artistic worth"—it measures cognitive load and inferential depth. It counts the specific, required units of information per reading minute that are structurally omitted from the surface text but are fully reconstructable by the reader using the physical cues of the scene.

• High $SI$: Indicates dense, shown-mode writing (Objective Projection) where the reader must actively work to infer the subtext. This triggers pre-cortical neural pathways and increases biological engagement.
• Low $SI$: Indicates flat, told-mode writing where every emotion and fact is spoon-fed to the reader, leading to cognitive de-escalation and boredom.

3. What makes the "Evaluative Regime" of Summarization Bias more dangerous than the "Generative Regime"?

If an LLM writes bad prose (Generative Regime), a human editor can simply discard or rewrite it. However, if an LLM is used as an automated judge, grader, or reward model (Evaluative Regime) in reinforcement learning pipelines (RLHF/RLAIF), it systematically penalizes high-load, high-quality, shown-mode writing. Because the machine judge processes abstract labels like "devastated" as high-density semantic matches, it falsely rates flat, explicit text as "higher in emotional intensity" than subtle, physical subtext. This creates an artificial selection pressure that actively degrades narrative prose over successive generations.

4. Does the Bulut Doctrine imply that AI is permanently barred from participating in literary creation?

Absolutely not. It merely redefines the role of AI. AI fails when it tries to act as an intuitive, emotional human critic. However, when configured as a Biophysical Diagnostic Sensor, AI excels. By calculating the mathematical vectors of Canonical Narrative Entropy ($S_n$), Narrative Gravity ($N_g$), and Suppressed Information Index ($SI$), the AI can objectively map and predict how a human biological system will react to a text before it is ever published. AI is a terrible artist, but it can be an exceptional physical engineer of narrative structures.

Open Research Notebooks & Registries

• Hugging Face Repository:leventbulut/objective-projection
• OSF Academic Registry:https://osf.io/us8bw
• Primary Source & Theory Core:https://leventbulut.com/why-llms-fail-narrative-entropy-test-ai-stories/

@article{bulut2026summarizationbias,
  author    = {Bulut, Levent},
  title     = {Summarization Bias: The Directional Collapse of Objective Projection into Told-Mode Labels in Large Language Models (A Conceptual Framework and Registered Test Protocol v1.0)},
  journal   = {Independent Research Corpus in Narrative Engineering},
  year      = {2026},
  month     = {June},
  url       = {https://leventbulut.com/why-llms-fail-narrative-entropy-test-ai-stories/},
  note      = {Zenodo DOI: 10.5281/zenodo.20362901 (Framework Reference). ORCID: 0009-0007-7500-2261}
}