Commit Graph

21 Commits

Author SHA1 Message Date
Nathan Schneider
60e83783ec Flatten data/readings/ → data/
Remove the intermediate readings/ subdirectory level — dataset naming
(synthetic_YYYYMMDD, manual_YYYYMMDD) already encodes what the data is.
Update all path references across scripts and docs accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 17:46:23 -06:00
Nathan Schneider
1a80219a25 Remove web/ prototype; update docs to reflect app integration
The web/ directory (bicorder-classifier.js, .d.ts, test-classifier.mjs)
was a prototype superseded by bicorder-app/src/bicorder-classifier.ts.
The only integration point between this analysis directory and the app is
bicorder_model.json, which Vite reads at build time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 17:39:25 -06:00
Nathan Schneider
897c30406b Reorganize directory, add manual dataset and sync tooling
- Move all scripts to scripts/, web assets to web/, analysis results
  into self-contained data/readings/<type>_<YYYYMMDD>/ directories
- Add data/readings/manual_20260320/ with 32 JSON readings from
  git.medlab.host/ntnsndr/protocol-bicorder-data
- Add scripts/json_to_csv.py to convert bicorder JSON files to CSV
- Add scripts/sync_readings.sh for one-command sync + re-analysis of
  any dataset backed by a .sync_source config file
- Add scripts/classify_readings.py to apply the LDA classifier to all
  readings and save per-reading cluster assignments
- Add --min-coverage flag to multivariate_analysis.py for sparse/shortform
  datasets; also applies in lda_visualization.py
- Fix lda_visualization.py NaN handling and 0-d array annotation bug
- Update README.md and WORKFLOW.md to document datasets, sync workflow,
  shortform handling, and new scripts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 17:35:13 -06:00
Nathan Schneider
f1ae9cac1f Derive classifier dimensions from bicorder.json automatically
Both export_model_for_js.py and bicorder_classifier.py now read
DIMENSIONS and KEY_DIMENSIONS directly from bicorder.json at runtime,
so the model stays in sync whenever gradient terms are renamed or
added. A COLUMN_RENAMES dict handles historical CSV column name
changes. The model now includes bicorder_version so the app's version
check works correctly.

Regenerated bicorder_model.json against bicorder.json v1.2.6 with
correct dimension names, 9 key dimensions from shortform flags, and
updated thresholds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 15:13:54 -06:00
Nathan Schneider
730075f757 Tweaks to analysis readme 2026-03-03 20:42:07 -07:00
Nathan Schneider
62a6b0ba2c analysis updates 2026-02-21 17:13:15 -07:00
Nathan Schneider
b004e9fb19 Analysis changes 2026-01-25 22:55:30 -07:00
Nathan Schneider
57b780fe95 Additional analysis in app and tweaked json descriptions 2026-01-25 22:48:54 -07:00
Nathan Schneider
d1f288cda8 Adjusted the classifier to reduce false flags 2026-01-14 22:44:32 -07:00
Nathan Schneider
6230e6da7b Added arrows to focused view for clarity 2026-01-14 22:35:27 -07:00
Nathan Schneider
3355ed9f02 Added synthetic_readings JSON files to analysis 2026-01-14 21:59:28 -07:00
Nathan Schneider
7829141756 Analysis updates and json tweaks 2026-01-14 21:42:23 -07:00
Nathan Schneider
1b508b911f Added classifer analysis to bicorder ascii and web app 2025-12-21 21:38:39 -07:00
Nathan Schneider
b541f85553 Additional analysis: examining clustering via LDA 2025-12-19 11:40:45 -07:00
Nathan Schneider
7be67c9eb5 Adjustments to analysis and AI disclaimer 2025-11-23 09:27:51 -05:00
Nathan Schneider
998c93b3f9 Legacy bicorder version added to analysis folder 2025-11-21 19:35:24 -07:00
Nathan Schneider
fa527bd1f1 Initial analysis 2025-11-21 19:34:33 -07:00
Nathan Schneider
04eee1360f Formatting and clarity improvements 2025-11-21 11:43:13 -07:00
Nathan Schneider
dcfd37fa4c Initial analysis complete 2025-11-16 23:47:10 -07:00
Nathan Schneider
815ed9d6f4 Set up analysis scripts 2025-10-30 10:56:21 -06:00
Nathan Schneider
d2da0425c6 Restructured data/ into analysis/ 2025-10-28 11:04:06 -06:00