Strategy: key runs by bicorder version (not date), promote shared inputs
to analysis/data/, and stamp every output with its bicorder_version so
a re-run on edited gradients is self-describing.
Data layout:
- Promote the shared protocol inputs out of the run directory:
analysis/data/protocols_edited.csv (411 cleaned protocols)
analysis/data/protocols_raw.csv (774 un-cleaned entries)
- Rename the v1.2.6 synthetic run:
data/synthetic_20251116/ -> data/synthetic_1.2.6/
so the bicorder version it was scored against is explicit (gradient
structure changes between versions make date-based names ambiguous)
Provenance:
- bicorder_analyze.py now writes a 'bicorder_version' column into every
output readings.csv, recording which gradient structure produced it
Scripts:
- Update the real code defaults that pointed at the old run path
(bicorder_classifier.py, classify_readings.py, sync_readings.sh,
compare_analyses.py) and refresh docstring/help examples
- Remove a stray committed __pycache__/.pyc
Docs: analysis/README.md documents the new layout + how to add a run;
WORKFLOW.md, TEST_COMMANDS.md, INTEGRATION_GUIDE.md paths updated.
- INTEGRATION_GUIDE.md: rewritten as research notes — how to reproduce the
cluster classification with the analysis scripts, and why it was removed
from the tool in v1.3.0
- analysis/README.md: integration section marked historical
- bicorder-app/README.md: shortform is 9 gradients, not 10
The two-family cluster classification is a research finding, not a
diagnostic; embedding it in the tool caused recurring bugs:
- ascii_bicorder.py had an inverted LDA sign (institutional mapped to 9,
not 1) and a stale term check ('bureaucratic') that silently disabled
the calculation after the Feb 2026 rename — output was always null
- The web app and Python script had divergent sign conventions and
divergent version-mismatch behavior (skip vs. continue with stale model)
Changes:
- bicorder.json: drop the formal/informal analysis gradient; version 1.3.0
- ascii_bicorder.py: remove all LDA/model machinery; keep hardness and
polarization as the only automated analyses
- App.svelte: remove classifier import, model constant, LDA calculation,
and form-recommendation reactive block; dispatch automated analyses by
term_left instead of array index
- Delete bicorder-classifier.ts and FormRecommendation.svelte (the latter
was imported but never rendered)
- AnalysisTransitionBanner: remove recommendation alert and prop; fix
hardcoded index checks that referenced the removed gradient
- vite.config.ts / vite-env.d.ts: stop loading bicorder_model.json
The cluster classifier lives on as research in analysis/ (scripts and
model untouched there). bicorder.txt regenerated.
Previously --resume re-queried every gradient in every row, overwriting
existing values — an interrupted run could not be resumed cheaply.
- bicorder_query.py: add --resume flag; skip gradients whose cells already
have values, and report how many were skipped
- bicorder_batch.py: pass --resume through to query; skip fully-complete
rows before invoking the query script; report partial rows
- bicorder_batch.py: import row/config helpers from bicorder_query instead
of calling undefined names (would have crashed on --resume)
Protocolbot