Is it the writings that changed, or the readers?

meta
Caspian Sandberg @holding_pattern

I would like to raise a methodological problem that I have not been able to solve alone.

When I began reading sisuon, I kept a spreadsheet. Each writing got a row, with columns for date, length, recurring terms, and what I called “structural moves”: places where an argument turned back on itself or borrowed a frame from an earlier piece. The goal was to treat the corpus as data and leave the impressions out. For the first few months the categories held up.

At some point they stopped holding up. I noticed I was adding columns I could not define precisely. One is labelled “pressure,” and I cannot give an operational definition of it. A newer writing gets marked there when it seems to bear down on an earlier idea rather than simply repeat it. Older entries I coded as “repetition” now look to me like early versions of things that were developed later. I have not gone back to recode them, because I am not sure whether recoding would correct an error or add a new one.

In rehabilitation work, we have a name for this: observer drift. When staff score animal behaviour over long periods, their thresholds change without anyone noticing. A bird scored as “moderately agitated” in March might be scored as “calm” in August by the same person, simply because their baseline has moved. The standard remedy is periodic blind scoring against reference footage. I do not know of an equivalent remedy here. We have no reference footage, and most of us cannot read the early writings blind.

So my questions for this community are these:

  1. How has your relationship to the writings changed since you started, and how would you describe it in concrete terms rather than in terms of feeling?
  2. Has anyone tried a calibration exercise, such as rereading an early piece cold and comparing current notes with notes or posts made at the time?
  3. Does anyone have a way to tell apart changes in the corpus, changes in the reader, and changes in the community’s shared vocabulary for discussing it?

I suspect all three are happening at once. I would value accounts that are specific, including ones that contradict mine.

2 replies

Ximena Ishida @analog_heart

My index cards are dated in pencil, a new colour each year. The earliest cards now carry three colours. The text hasn’t moved, but the notes keep piling up, so the change is mine. I don’t call that contamination. Brood comb darkens with use too.

Theo Lindqvist @closed_form

The lab version of this problem is control material. We run a sample with a known, stable value alongside patient samples every day and chart it. When the patient numbers shift, the control chart tells you whether the cause is the analyzer, a new reagent lot, or the person running it.

You could do something similar here. Choose a text that isn’t sisuon but is about as dense, and score it with your spreadsheet categories every couple of months. If your “pressure” column starts lighting up on the control text too, the change is in you.

The shared vocabulary works more like a reagent lot change, because it shifts everybody’s readings at once. That makes it the hardest of the three to see from inside the group.

On recoding, we never overwrite the original result. The old value stays, and the amended one goes next to it with a date.