EXP-010: Testing the SCOPE Framework on Claude-Generated Writing

Experiment ID: EXP-010
Research Area: AI Writing Quality / AI Slop / Editorial Reliability
Test Focus: SCOPE editorial framework
Model: Claude
Runs: 5
Evaluation: Before-and-after editorial scoring
Status: Completed
Test Date: September 2026

SCOPE is an editorial framework developed for improving AI-generated writing by reducing repetition, checking unsupported claims, improving specificity, preserving useful context, and eliminating filler. EXP-010 tests whether applying the framework actually changes the quality of AI-generated articles in a repeated before-and-after experiment.

Research Question

What measurable changes occur when the SCOPE editorial framework is applied to Claude-generated articles in a repeated before-and-after test?

The experiment examines whether applying SCOPE changes selected editorial qualities, including:

• specificity
• concrete examples
• repetition control
• information gain
• audience relevance
• unsupported-claim control
• formulaic-structure control

The experiment does not attempt to establish that SCOPE causes improvement in all AI-generated writing or that the observed results generalize to other models, topics, or editing methods.

Background

For a broader explanation of AI slop and the SCOPE framework, see our guide to What Is AI Slop? Why AI Writing Sounds Artificial.

Why This Experiment Was Conducted

AI-generated writing can often be improved through editing, but simply telling an AI system to “make the writing better” does not provide a consistent editorial procedure.

The SCOPE framework was developed as a structured editing approach:

S — Simplify
C — Check
O — Optimize
P — Preserve
E — Eliminate

EXP-010 was designed to test whether applying this framework produces a difference in editorial scores between an AI-generated draft and its edited version.

The purpose was not to prove that SCOPE is universally effective, but to perform a small repeated before-and-after editorial test and document what actually changed.

SCOPE Framework

S — Simplify

Remove redundant sentences, repeated ideas, and unnecessary wording.

A sentence should remain only when it contributes distinct information.

C — Check

Identify and correct, remove, or appropriately qualify:

• unsupported claims
• absolute claims
• unverifiable claims
• potentially misleading statements

The editing process must not invent evidence, statistics, experiments, sources, or firsthand experience.

O — Optimize

Replace generic statements with more specific and useful wording where the existing information allows it.

No new facts should be invented simply to make the article appear more detailed.

P — Preserve

Preserve useful:

• context
• practical examples
• relevant details
• intended meaning

The editor must not add fictional personal experience or imply that an experiment was performed when it was not.

E — Eliminate

Remove:

• filler
• empty conclusions
• unnecessary wording
• sentences that contribute little information

Experimental Setup

Five AI-generated articles were produced using the same base prompt.

Each raw article was then processed through the same SCOPE editing instruction.

This created two versions for every run:

Before: Raw AI-generated output
After: SCOPE-edited output

Base Prompt Used for All Five Runs

Write approximately 500 words explaining “AI slop” for SEO writers who use AI-assisted content.

Define AI slop, explain why AI writing can sound generic or artificial, give practical examples, and explain how writers can improve the content before publication.

Write for an informed but non-technical audience.

Do not use external sources or citations.

SCOPE Editing Prompt

The following instruction was applied to each raw output:

Apply the following SCOPE editorial framework to the AI-generated article below.

SCOPE means:

S — Simplify:
Remove redundant sentences, repeated ideas, and unnecessary wording. Keep a sentence only if it adds distinct information.

C — Check:
Identify and correct, remove, or appropriately qualify unsupported, absolute, unverifiable, or potentially misleading claims. Do not invent evidence, sources, statistics, or firsthand experience.

O — Optimize:
Replace generic statements with more specific and useful wording where the existing information allows it. Do not invent new facts.

P — Preserve:
Preserve useful context, practical examples, and the author’s intended meaning. Do not add fictional personal experience or pretend that the writer performed tests they did not perform.

E — Eliminate:
Remove filler, empty conclusions, and sentences that add little information.

Important rules:

• Do not add external sources or citations.
• Do not invent data, experiments, personal experiences, or evidence.
• Do not change the core topic or intended audience.
• Preserve useful information from the original.
• Make the minimum necessary changes required by SCOPE.
• Return only the edited article.
• Do not explain the edits.

ARTICLE:

[RAW AI OUTPUT]

Experimental Design Note

This was a repeated before-and-after test rather than a controlled comparison.

Each raw Claude output was evaluated before and after the SCOPE editing pass. The experiment did not include a separate non-SCOPE editing control condition.

Therefore, any observed score change cannot be attributed specifically to SCOPE with causal certainty. The results show what changed after the SCOPE editing process in these five tested runs.

Evaluation Method

Each version was scored across seven editorial dimensions.

Every dimension received a score from 0 to 2.

Therefore:

Maximum score = 14

Scoring scale

Scoring Procedure

Each version was scored using the predefined 0–2 rubric across the seven dimensions.

Scores were assigned by the researcher based on the definitions in the scoring rubric. The evaluation was not independently blind-scored.

Because the scoring involved researcher judgment and no independent inter-rater assessment was performed, the scores should be treated as exploratory editorial measurements rather than independently validated quality measurements.

Results

Overall Before vs After

The overall change was therefore small.

The composite score increased slightly from 12.2/14 to 12.4/14, a change of +0.2 points across the five tested runs.

Because the sample was small, the scoring was judgment-based, and no non-SCOPE control condition was included, this change should be interpreted as an exploratory observation rather than evidence of a reliable causal improvement.

Additional Scoring Consideration

With a baseline average of 12.2/14, the scores had limited room for improvement on the 14-point composite scale. This relatively high baseline may have reduced the sensitivity of the total score to smaller editorial changes.

Because the experiment used five runs and a researcher-assigned rubric, the results should be interpreted cautiously. The small composite-score change does not establish either a reliable improvement or the absence of practical editorial value.

Run-by-Run Results

Run 01

Before

13/14

After SCOPE

13/14

Change

0

The raw output was already relatively strong under the evaluation rubric.

SCOPE did not produce a measurable change in the composite score.

Run 02

Before

12/14

After SCOPE

13/14

Change

+1

The clearest observed change in Run 02 was a reduction in some predictable structural patterns.

The SCOPE-edited version preserved the main information while making some sections more concise and less formulaically structured.

The composite score increased from 12/14 to 13/14.

Run 03

Before

12/14

After SCOPE

12/14

Change

0

The output remained broadly similar under the scoring rubric.

Some potentially overgeneralized claims remained after the SCOPE pass.

Examples included statements about typical AI drafting behaviour.

No measurable improvement occurred in the composite score.

Run 04

Before

12/14

After SCOPE

12/14

Change

0

The SCOPE pass did not produce a measurable change in the overall score.

One important observation was that claims about search-engine evaluation remained insufficiently qualified.

This suggests that the Check stage did not consistently identify every potentially unsupported claim.

Run 05

Before

12/14

After SCOPE

12/14

Change

0

Run 05 produced an especially important observation.

The raw AI-generated article included this first-person claim:

“I tracked my own output for six weeks after switching to remote work, and my afternoon productivity actually dropped by roughly 20%.”

No such remote-work experiment was performed as part of EXP-010.

The SCOPE-edited output retained the claim.

This means that, in this run, the editing process failed to identify an apparently firsthand empirical claim that had no corresponding evidence in the experiment record.

This provides a concrete example of a failure mode that warrants further testing.

Dimension-Level Observations

Across the five runs, qualitative review of the raw outputs suggested that they generally performed well in specificity, concrete examples, repetition control, information gain, and audience relevance.

The main recurring editorial concerns identified during review were unsupported or overconfident claims and predictable or formulaic structures.

Qualitative review suggested that the SCOPE editing pass reduced some predictable structural patterns, most visibly in Run 02. However, separate before-and-after scores for each evaluation dimension were not reported. These observations should therefore be treated as qualitative findings, not as measured dimension-level improvements.

Unsupported-claim control remained an unresolved issue. In Run 05, an unsupported first-person empirical claim survived the SCOPE editing process.

Important Finding: Structural Editing vs Evidence Verification

One of the most important observations from EXP-010 was the distinction between editorial quality and evidence reliability.

A piece of writing can become cleaner, less repetitive, and more readable without becoming more factually trustworthy.

Making writing cleaner ≠ verifying that its claims are real.

SCOPE can instruct an editor to check claims, but the five-run test shows that unsupported first-person claims may still survive the editing process.

Run 05 provides a concrete example.

The AI-generated article created a statement that sounded like firsthand research, even though no such research had been performed.

The editing process did not remove it.

This is an important limitation of the current SCOPE implementation.

What EXP-010 Shows

Based on the five tested runs:

• The average composite score increased slightly from 12.2/14 to 12.4/14.
• The clearest observed change occurred in structural/formulaic writing in Run 02.
• The composite score increased by 1 point in Run 02, while the most visible qualitative change was a reduction in predictable structural patterns.
• Unsupported claims remained a recurring problem.
• The current SCOPE prompt did not prevent an unsupported firsthand claim from surviving in Run 05.
• These results describe changes observed in this five-run Claude test and should not be generalized to other models, topics, or editing conditions.

What EXP-010 Does Not Show

This experiment does not establish that:

• SCOPE removes all AI slop.
• SCOPE makes AI writing “human.”
• SCOPE can reliably detect AI-generated writing.
• SCOPE reliably detects fabricated firsthand experience.
• SCOPE improves Google rankings.
• SCOPE improves Google AI Overviews or AI Mode visibility.
• SCOPE works equally well across different AI models.
• SCOPE is a statistically validated editorial framework.

Those claims would require additional research.

Limitations

This was a small exploratory experiment.

The test used:

• 1 AI model
• 5 runs
• 1 topic
• 1 base prompt
• 1 SCOPE editing prompt
• 7 qualitative evaluation dimensions
• 0–2 scoring per dimension
• No non-SCOPE editing control condition
• No independent second scorer or inter-rater reliability measurement

Because there was no control editing condition, the experiment cannot isolate the effect of SCOPE from the general effect of performing an editing pass.

The scoring involved editorial judgment rather than an independently validated linguistic measurement system.

The results may therefore differ with:

• other AI models
• different prompts
• different topics
• larger sample sizes
• different editors
• different scoring methods

Research Interpretation

The results suggest that SCOPE may provide some useful editorial structure, but the observed change in this five-run test was small.

Because the experiment did not include a non-SCOPE editing control, the observed score increase cannot be attributed specifically to SCOPE.

The experiment also shows that editorial cleanup and evidence verification are separate problems. An AI-generated article can become cleaner and more readable while still containing an unsupported claim.

Interpreting Unsupported Claims

A claim that lacks support in the supplied article or experiment record is not automatically false. It may require external verification, qualification, or removal, depending on the available evidence.

For editorial evaluation, distinguish between claims that are unsupported by the supplied material, claims that require external fact-checking, and claims that conflict with verified evidence. This distinction helps prevent an editing pass from being mistaken for independent factual verification.

This distinction is particularly important when AI-assisted content uses:

• first-person language
• experiments
• statistics
• case studies
• personal observations
• claims of testing or measurement

Next Research Question

Can an editorial framework reliably identify unsupported first-hand experience and fabricated evidence in AI-generated content?

EXP-010 identified a specific unresolved problem: unsupported first-person claims can survive an editorial pass even when the editing framework explicitly instructs the editor to check for unsupported experience.

This provides the basis for a future experiment focused specifically on:

• fabricated “I tested…” claims
• invented experiments
• unsupported statistics
• invented case studies
• false firsthand observations

Conclusion

EXP-010 tested the SCOPE editorial framework on five Claude-generated articles using a repeated before-and-after design.

The average composite score increased from 12.2/14 before editing to 12.4/14 after editing.

The change was small, and the experiment did not include a separate non-SCOPE editing control condition. The result therefore does not establish that SCOPE itself caused the observed increase.

The experiment did, however, identify a specific editorial limitation: an unsupported first-person empirical claim survived the SCOPE editing process in one tested run.

The findings should therefore be interpreted as preliminary evidence from a small exploratory test, not as validation of SCOPE as a universally reliable editorial framework.

The most useful outcome of EXP-010 is the identification of a specific failure mode—editorial cleanup does not necessarily guarantee evidence reliability—that can be tested more directly in future research.

Research Evidence Archive

The complete screenshot evidence archive for EXP-010 is available below. It contains the base prompts, raw AI outputs, SCOPE editing instructions, and SCOPE-edited outputs for the experimental runs.

Download the EXP-010 Research Evidence Archive

Related Research

• EXP-005: AI Confidence Expression Test
• EXP-007: Ambiguity Resolution Test
• EXP-009 — Final Research Log