Sort Reader Notes on an AI Draft Without Losing the Evidence

Several readers can misunderstand the same draft in different ways. AI can help sort their notes, but a neat summary is only useful if you can trace every group back to what each person actually said. Keep the words, source facts, interpretation, proposed edit, and retest separate.

For: Someone improving a short public notice after collecting readbacks from intended readers.

Use it to: Find a repeated misunderstanding worth checking, without treating a few notes as a survey or letting AI invent a rule.

From one readback to several

Yesterday's lesson used one intended reader's own words as a first signal. If you have several genuine readbacks, preserve each one before asking AI to find patterns. The U.S. General Services Administration's paraphrase-testing guidance asks people to explain short copy in their own words, records those words without correction, and reviews misunderstandings for revision. The UK Government's research-analysis guidance similarly separates what a person said or did from what the researcher thinks it means, then sorts similar observations and decides what to change or study next. These methods support organizing notes; they do not make four convenient responses representative of everyone.

A fully constructed puzzle-swap example

Everything below is fictional. There is no real puzzle swap, participant, interview, AI run, or measured improvement in this example. The names and readbacks are teaching inputs. A calendar calculation confirms that November 12, 2026, is Thursday and November 14 is Saturday.

Imagine these numbered, organizer-approved source facts for a notice:

S1. A puzzle exchange is Saturday, November 14, 2026, from 10 to 11 a.m. in the fictional Maple Room.

S2. To exchange a puzzle, a visitor must register by Thursday, November 12, and bring one complete puzzle.

S3. A visitor without registration may browse on Saturday but cannot exchange a puzzle.

Here is a constructed possible AI-assisted draft, not an observed model response:

"Puzzle swap: Saturday, November 14, 10–11 a.m., Maple Room. Bring one complete puzzle. Register by Thursday, November 12. Walk-ins welcome to browse."

The draft includes the date and registration deadline, but its last sentence does not say plainly that walk-ins cannot exchange. Suppose four invented readback notes were attached to this teaching case:

The first useful move is not to ask AI for a verdict. Copy the four notes exactly, assign IDs, and compare them with the source yourself. B and C both express the same possible exchange-eligibility misunderstanding. D expresses a different date misunderstanding. A matches the source. The arithmetic is two of four constructed notes for the first group and one of four constructed notes for the second; that is a count of supplied example lines, not a rate among real readers. B and C do not prove why someone reached that interpretation, and the groups do not erase their different wording.

Digital.gov's synthesis guidance calls for looking at similarities and dissimilarities among observations. Keep D visible instead of forcing it into the B/C group just to make one tidy theme. Keep A, too: it shows the draft can be interpreted as intended in this fictional set, but does not cancel the other notes.

Give AI a bounded sorting job

With real notes, get permission to use them, remove unnecessary personal details, and share only material you are authorized to put into the AI service. OpenAI Academy's writing guidance recommends supplying a goal, context, and constraints, then reviewing the answer as a draft. Use this prompt as a template:

Task: Sort the numbered reader readbacks below. Preserve each person's exact words in an evidence column. Group only genuinely similar misunderstandings, name the readback IDs in each group, and keep distinct or conflicting notes visible. Count only the notes supplied, with the denominator. Separate observation from your interpretation. For each group, point to the relevant numbered source line. Flag any policy, date, promise, or other fact missing from the source as unknown. Suggest one source-supported wording change and one neutral retest question. This is a draft for human review. Do not claim the notes represent all readers, invent a participant or quote, decide an absent rule by majority vote, or say the revision was tested.

Intended audience and action: [who needs to understand what]

Verified, shareable source: [numbered lines]

Current draft: [paste]

Actual readback notes, if collected with permission: [IDs and exact words]

No separate model run was conducted for this article's puzzle example. A constructed possible sorting table would retain all four exact lines and show B/C as a possible shared issue, D as a separate issue, and A as a source-matching note. It would trace the first issue to S3 and the second to S1 and S2. Do not accept an AI table simply because its totals add up: compare each quoted word, ID, group, count, and source line with your inputs. OpenAI's accuracy guidance warns that a confident answer can still contain false details or fabricated quotes.

Make one change, then retest

The first suggested edit should address the B/C interpretation without inventing a new event rule. Here is an illustrative revision supported by S1–S3:

To exchange a puzzle: Register by Thursday, November 12, and bring one complete puzzle to the fictional Maple Room on Saturday, November 14, 10–11 a.m. Without registration, you may browse on Saturday but cannot exchange a puzzle.

That change foregrounds the eligibility rule. It also puts Saturday beside the actual event. It is still an untested teaching draft, not evidence that the misunderstanding has been fixed. A neutral retest question would be: "If you arrived on Saturday without registering, what could you do?" Record the answer in the person's own words and compare it with S3; do not lead with "You know you cannot exchange, right?" The responsible organizer checks the facts and approves any real notice.

There is one more useful guardrail. If the source merely said "registration closes Thursday" and said nothing about walk-ins, three people guessing that walk-ins could exchange would not establish that policy. The AI's correct entry would be unknown—ask the organizer, not "three of four voted yes." Reader interpretation is evidence about the draft, not authority to invent an event rule.

Try it: For a harmless notice you are allowed to share, write down the verified source and each real readback exactly. Ask AI to sort the notes with the prompt above. Audit its quotes, groups, denominator, and source mapping yourself; make one source-supported revision and ask another intended reader a neutral retest question. If you have no real readbacks yet, collect them first rather than asking AI to pretend.

Sources checked November 2, 2026: GSA Digital.gov, Paraphrase testing and Synthesis (no visible publication dates); UK Government Service Manual, Analyse a research session (published and updated May 24, 2016); OpenAI Academy, Writing with ChatGPT (April 10, 2026); OpenAI Help, Does ChatGPT tell the truth? (displayed "Updated: 10 days ago" when checked). These sources support the general method, not the fictional event or an observed result.

Disclosure: AI assisted research and drafting. All puzzle-swap facts, reader words, tables, drafts, and revision are constructed teaching examples. No real participants, separate prompt execution, tested improvement, notice publication, or measured AI advantage are claimed. The companion video uses synthetic narration and original project graphics.