Full pipeline per README command flow (steps 2-6): univariate averages and
distributions, multivariate analysis (2-cluster k-means: 209 relational vs
199 institutional; PCA/t-SNE/UMAP/factor/correlation/network/importance),
LDA separation and overlaid cluster plots, and LDA classifications (trained
on the 1.2.6 run via version-matched auto-selection). Cross-version summary
is on the console report: avg Euclidean distance 9.06 vs 1.2.6 (r=0.79).
Verified: bicorder-app has no classifier implementation and reads
bicorder_model.json nowhere (src, build config); ascii_bicorder.py doesn't
use it either. Corrected the stale 'read by bicorder-app at build time'
claims in README, WORKFLOW, and the export script docstring.
- README: new 'Command flow' section — dependency-ordered pipeline (setup →
batch generation → univariate → multivariate → LDA/cluster visualizations →
classify → cross-version compare → JSON/model export) with actual output
paths per step, using data/synthetic_1.4.0/ as the example
- README: document chart production — univariate_analysis --img publishes the
img/ summary charts; visualize_clusters added to the flow
- README: fix -b path in 'Adding a new run' (bicorder.json lives at repo root)
- WORKFLOW: script list and any-readings-CSV command block updated to match
411 protocols × 23 gradients scored against bicorder.json v1.4.0 with
gpt-oss:20b-cloud (same analyst standpoint as the 1.2.6 run); the
bicorder_version column makes the run self-describing. Analysis outputs to
follow in data/synthetic_1.4.0/analysis/.
- scripts/bicorder_common.py (new): single source of truth for historical
gradient renames (COLUMN_RENAMES), version detection (bicorder_version col
→ version col → data/<type>_<version>/ dir convention), and training-CSV
auto-selection (find_training_csv)
- classify_readings.py: auto-select training run by recorded bicorder version
(excludes the input itself to avoid circular training), canonicalize old
column names, version-mismatch warnings
- bicorder_classifier.py / export_model_for_js.py: share renames + dimension
loader; instructive error when clustering results are missing;
bicorder_version recorded in exported models
- compare_analyses.py: CLI (reference + comparison CSVs), rename
canonicalization so runs of different versions align on shared gradients,
Descriptor dedup (was silently skewing merges); legacy no-arg audit intact
- scripts/univariate_analysis.py (new): per-protocol/per-gradient averages,
distributions, summary stats; --img publishes the three README summary
charts to img/
- sync_readings.sh: defer classifier training to auto-matching; gitignore
analysis/.venv and __pycache__
Nathan renamed the Design gradient 'institutional' → 'formal' (v1.4.0).
The analysis scripts carry historical rename maps that route old data
(elite/institutional) into the current bicorder.json gradient names, and
those maps still stopped at 'institutional' — so old readings would have
silently misaligned against the v1.4.0 structure.
- bicorder_classifier.py / export_model_for_js.py: add
institutional_vs_vernacular → formal_vs_vernacular, keep elite→formal
- json_to_csv.py TERM_RENAMES: same two-step route
- convert_csv_to_json.py GRADIENT_MAPPINGS: elite_vs_vernacular now maps
to 'formal'
- bicorder_classifier.py demo ratings: use the current dimension name
- analysis/README.md: example run name 1.3.0 → 1.4.0
Strategy: key runs by bicorder version (not date), promote shared inputs
to analysis/data/, and stamp every output with its bicorder_version so
a re-run on edited gradients is self-describing.
Data layout:
- Promote the shared protocol inputs out of the run directory:
analysis/data/protocols_edited.csv (411 cleaned protocols)
analysis/data/protocols_raw.csv (774 un-cleaned entries)
- Rename the v1.2.6 synthetic run:
data/synthetic_20251116/ -> data/synthetic_1.2.6/
so the bicorder version it was scored against is explicit (gradient
structure changes between versions make date-based names ambiguous)
Provenance:
- bicorder_analyze.py now writes a 'bicorder_version' column into every
output readings.csv, recording which gradient structure produced it
Scripts:
- Update the real code defaults that pointed at the old run path
(bicorder_classifier.py, classify_readings.py, sync_readings.sh,
compare_analyses.py) and refresh docstring/help examples
- Remove a stray committed __pycache__/.pyc
Docs: analysis/README.md documents the new layout + how to add a run;
WORKFLOW.md, TEST_COMMANDS.md, INTEGRATION_GUIDE.md paths updated.
- INTEGRATION_GUIDE.md: rewritten as research notes — how to reproduce the
cluster classification with the analysis scripts, and why it was removed
from the tool in v1.3.0
- analysis/README.md: integration section marked historical
- bicorder-app/README.md: shortform is 9 gradients, not 10
The two-family cluster classification is a research finding, not a
diagnostic; embedding it in the tool caused recurring bugs:
- ascii_bicorder.py had an inverted LDA sign (institutional mapped to 9,
not 1) and a stale term check ('bureaucratic') that silently disabled
the calculation after the Feb 2026 rename — output was always null
- The web app and Python script had divergent sign conventions and
divergent version-mismatch behavior (skip vs. continue with stale model)
Changes:
- bicorder.json: drop the formal/informal analysis gradient; version 1.3.0
- ascii_bicorder.py: remove all LDA/model machinery; keep hardness and
polarization as the only automated analyses
- App.svelte: remove classifier import, model constant, LDA calculation,
and form-recommendation reactive block; dispatch automated analyses by
term_left instead of array index
- Delete bicorder-classifier.ts and FormRecommendation.svelte (the latter
was imported but never rendered)
- AnalysisTransitionBanner: remove recommendation alert and prop; fix
hardcoded index checks that referenced the removed gradient
- vite.config.ts / vite-env.d.ts: stop loading bicorder_model.json
The cluster classifier lives on as research in analysis/ (scripts and
model untouched there). bicorder.txt regenerated.
Previously --resume re-queried every gradient in every row, overwriting
existing values — an interrupted run could not be resumed cheaply.
- bicorder_query.py: add --resume flag; skip gradients whose cells already
have values, and report how many were skipped
- bicorder_batch.py: pass --resume through to query; skip fully-complete
rows before invoking the query script; report partial rows
- bicorder_batch.py: import row/config helpers from bicorder_query instead
of calling undefined names (would have crashed on --resume)
Introduce a simple, show-don't-tell entry screen that lowers the "what
counts as a protocol" barrier and frames the activity as a short journey:
- Lead line "Examine the protocols around you" over a rotating, cross-fading
list of deliberately diverse examples (traffic, code, kitchens, ritual,
governance, play)
- Prominent "Begin" CTA above a three-step path: Describe / Understand / Share
- "About the Bicorder" link opens the existing help modal
Shown on first arrival; returning users with a reading in progress skip
straight to the diagnostic (computed synchronously, no flash). Remove the
now-redundant inline description from the focused flow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The model (analysis/bicorder_model.json) maps a positive LDA score to
cluster 2 = Institutional/Bureaucratic = "formal", but ldaScoreToScale
added the score (5 + score*4/3), sending formal/institutional protocols
toward 9 (informal) and vice versa. bicorder.json defines this gradient
as 1 = formal, 9 = informal, so the score must be subtracted.
- Flip the sign: value = 5 - (ldaScore * 4/3); correct the doc comment to
state the model's actual sign convention
- Rename calculateBureaucratic -> calculateFormalInformal and update the
stale analysisOrder comment, matching bicorder.json's formal/informal terms
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the click-anywhere ASCII bar (whose hit-mapping was offset from
the displayed marker, so taps registered the wrong value) with a track of
nine discrete, exact tap targets that fill from the center outward to the
selection: [----==#--]. Direction and distance from center read at a glance.
- Exact per-position selection on touch and desktop; no more position math
- Keyboard support (arrows/Home/End), scoped so it doesn't trigger
the app-level screen navigation
- Center the bar and slim mobile padding so the full track fits narrow
viewports
- Apply the same control to the manual analysis gradient; render the
automated ones as a matching read-only, dimmed display
- Add a subtle gray "Value: #" indicator below the buttons on the
diagnostic gradients, matching the analysis screens
- Move the "auto-calculated" note under the ANALYSIS header instead of
after the value
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the intermediate readings/ subdirectory level — dataset naming
(synthetic_YYYYMMDD, manual_YYYYMMDD) already encodes what the data is.
Update all path references across scripts and docs accordingly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The web/ directory (bicorder-classifier.js, .d.ts, test-classifier.mjs)
was a prototype superseded by bicorder-app/src/bicorder-classifier.ts.
The only integration point between this analysis directory and the app is
bicorder_model.json, which Vite reads at build time.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move all scripts to scripts/, web assets to web/, analysis results
into self-contained data/readings/<type>_<YYYYMMDD>/ directories
- Add data/readings/manual_20260320/ with 32 JSON readings from
git.medlab.host/ntnsndr/protocol-bicorder-data
- Add scripts/json_to_csv.py to convert bicorder JSON files to CSV
- Add scripts/sync_readings.sh for one-command sync + re-analysis of
any dataset backed by a .sync_source config file
- Add scripts/classify_readings.py to apply the LDA classifier to all
readings and save per-reading cluster assignments
- Add --min-coverage flag to multivariate_analysis.py for sparse/shortform
datasets; also applies in lda_visualization.py
- Fix lda_visualization.py NaN handling and 0-d array annotation bug
- Update README.md and WORKFLOW.md to document datasets, sync workflow,
shortform handling, and new scripts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prevent horizontal overflow with overflow-x: hidden on body. Make the
gradient bar fill its container in focused mode (width: 100%;
min-width: 0) rather than holding a fixed minimum width.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Raise responsive layout breakpoints from 768px to 900px across all
components. This ensures narrow desktop browser windows (769–900px)
get the stacked, single-column layout rather than the three-column
grid that was too wide for those widths. The 768px breakpoint in
app.css is kept as-is since it only adjusts font sizes for iOS.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both export_model_for_js.py and bicorder_classifier.py now read
DIMENSIONS and KEY_DIMENSIONS directly from bicorder.json at runtime,
so the model stays in sync whenever gradient terms are renamed or
added. A COLUMN_RENAMES dict handles historical CSV column name
changes. The model now includes bicorder_version so the app's version
check works correctly.
Regenerated bicorder_model.json against bicorder.json v1.2.6 with
correct dimension names, 9 key dimensions from shortform flags, and
updated thresholds.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove user-select: none from body, keeping it only on interactive
elements (buttons, sliders, tooltips) where it prevents accidental
selection during interaction.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move the upload confirmation dialog above the action buttons so it's
immediately visible, and add overflow-safe CSS to prevent text from
spilling outside the dialog box on mobile.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Nathan Schneider
ntnsndr