Scanned sheet music to MusicXML

1. Introduction

1.1 Purpose

This document describes the design of a client-side web application that converts scanned sheet-music PDFs (pages that are images, such as IMSLP, CPDL or Internet Archive scans, photocopies and phone scans) into MusicXML 4.0 score-partwise. It is written for the people and agents who build, review and test the system, and it fixes the module boundaries, data shapes, algorithms and acceptance criteria they work to.

1.2 Scope

In scope:

• Image-only PDFs, plus PNG and JPEG page images. Born-digital vector PDFs are a secondary goal: they are rasterised and go through the same image path.
• Printed (engraved or typeset) Western staff notation. The primary repertoire is hymns and chorales in closed score (SATB on two staves with lyrics between them). Secondary: open-score SATB, a single melody with lyrics, voice and piano, and piano solo.
• Lyrics: OCR of the text rows, syllabification from the printed hyphens, extenders, several verses, and alignment to notes.
• Splitting several voices on one staff into separate <part>s when the staff is a vocal closed score, while keeping keyboard grand staves as one part with two staves.
• A node harness that runs the same core against a ground-truth corpus and reports accuracy.

Out of scope for v1: handwritten music, tablature, early (mensural or chant) notation, chord symbols and figured bass, dynamics and articulations beyond fermatas, editing the recognised score inside the app, and any server-side processing.

1.3 Definitions

2. Requirements

2.1 Functional requirements

2.2 Non-functional requirements

Accuracy targets. The v1 targets come from the research recommendations and are not yet measured:

For context, the best open system reports OMR-NED of 11.1–11.2 % on quartet scans and 17.0–17.8 % on Lieder scans for its 2M-parameter model (copisteria, *self-reported*), and Audiveris 5.11 reports 66.9 % and 51.8 % on the same sets [OMR §3.2]. No published system produces lyrics and voice-separated parts together [OMR §0].

Performance. Budget per 300 dpi page on a mid-range laptop, single-thread WASM [OMR §7.4] (estimates to be verified by the harness):

Heavy work runs off the main thread in a same-origin Web Worker, so the UI stays responsive. Peak memory for one 300 dpi A4 page (about 8.7 Mpx) should stay below 300 MB: gray, binary and staff-removed copies are Uint8Arrays, and the integral images are Float64Arrays that are freed after binarisation.

Size budget. Each supporting file ≤ 15 MB, the total ≤ 64 MB [WF]. Planned: phase 1 about 13–20 MB, phase 2 with ONNX about 30–40 MB [OMR §9.3]. The build fails if a limit is exceeded (§9).

Licensing. Everything shipped must be open source and compatible with public distribution: MIT, Apache-2.0, BSD or OFL. Nothing AGPL, GPL, LGPL or non-commercial is shipped (§11).

Accessibility. WCAG 2.2 AA for the page: keyboard operation of every control, visible focus, text contrast ≥ 4.5:1, an aria-live progress region, no information carried by colour alone (voice overlays also differ by marker shape and are listed in text), prefers-reduced-motion respected, and a usable layout at 400 px width with no horizontal page scroll (§8.5).

Privacy. The user's file never leaves the browser. There are no network requests after the page and its assets have loaded.

Portability. Current Chrome, Edge, Firefox and Safari. No SharedArrayBuffer and no crossOriginIsolated requirement.

3. Architecture overview

3.1 Pipeline

The core is a pure function from page bitmaps to a score model and MusicXML. It holds no DOM references. Environment-specific work (PDF rasterising, image decoding, OCR engines, ONNX sessions) is reached through an injected environment adapter.

flowchart TD
  IN["Input: PDF / PNG / JPEG bytes"] --> R["raster.js: rasterise, gray, shading normalise, Sauvola, deskew"]
  R --> S["staff.js: staff metrics s,t; staves; systems; staff removal; barlines; measures"]
  S --> Y["symbols.js: connected components, heads, stems, beams, flags, dots, rests, accidentals, clefs, key/time, ties"]
  S --> L1["lyrics.js (pass 1): text-row detection in inter-staff bands"]
  Y --> RC["recognize.js: chords, slots, pitch, voices per staff, rhythm repair"]
  L1 --> OCR["env.ocr: tesseract.js line OCR"]
  OCR --> L2["lyrics.js (pass 2): syllables, extenders, verses, alignment to reference voice"]
  RC --> P["parts.js: staff-group classification, split decision, naming, lyric copy"]
  L2 --> P
  P --> X["musicxml.js: MusicXML 4.0 partwise"]
  X --> OUT["convert() result: musicxml, score, overlays, warnings, stats"]
  ENV["env adapter: env-browser.js or env-node.js"] -.-> R
  ENV -.-> OCR
  ENV -.-> Y

Each arrow is a typed data structure defined in §5. Stages are run per page by index.js, except recognize (cross-system part continuity), parts (piece-level decisions) and musicxml, which run once over all pages.

3.2 Module layout

The layout is fixed by the build workflow, with one owner per file [WF]:

3.3 Environment adapters

interface Env {
  name: 'browser' | 'node'
  rasterize(bytes: Uint8Array, opts: {targetDpi?: number, maxPixels?: number})
      -> AsyncIterable<{pageIndex, bitmap: Bitmap, sourceDpi: number | null, isImageOnly: boolean}>
  decodeImage(bytes: Uint8Array, mime: string) -> Promise<Bitmap>
  createOcr(opts: {langs: string[]}) -> Promise<OcrEngine | null>
  createOnnxSession(modelName: string) -> Promise<OnnxSession | null>   // null when not shipped
  now() -> number                                                       // ms, for stats
  yield() -> Promise<void>                                              // lets the UI breathe
}
interface OcrEngine {
  recognizeLine(bitmap: Bitmap, opts: {whitelist?: string, psm?: 7})
      -> Promise<{text, conf, words: [{text, conf, bbox: [x0,y0,x1,y1]}]}>
  terminate() -> Promise<void>
}

The core never imports either adapter. index.js receives one as an argument, which makes the harness and the browser run byte-identical core code.

3.4 Threading

The browser runs the core in one same-origin module worker (assets/omr-worker.js, or a blob: worker built from the inlined bundle). pdf.js runs on the main thread (fake worker) and transfers page bitmaps to the OMR worker as ArrayBuffers. Tesseract runs in its own worker created by tesseract.js. ORT runs single-threaded inside the OMR worker. Progress events are posted back to the UI thread.

3.5 Error containment

Every stage is wrapped by index.js. A stage that throws on one page or one system yields an empty result for that unit plus a warning of severity error, and the pipeline continues. convert() returns partial MusicXML whenever at least one measure was recognised. Only input-level failures (unreadable file, encrypted PDF, zero pages) end the run with an error state.

4. Module design

Each module lists its responsibility, inputs and outputs, algorithms, failure modes and confidence. Thresholds are written in staff spaces s and line thickness t, and are starting values for tuning on the corpus.

4.1 raster.js

Responsibility. Turn input bytes into normalised page images: gray, binary, deskewed, at a scale where s is in a workable range.

Inputs/outputs. Uint8Array + MIME → Page[] (§5.1) holding gray, binary, skewDeg, scale (working pixels per source pixel) and the transform back to source coordinates.

Algorithms.

1. Rasterise through env.rasterize or env.decodeImage. Reject pages above 40 Mpx with a warning and downscale them to 40 Mpx.
2. Gray: luminance 0.299R + 0.587G + 0.114B, or the single channel when already gray.
3. Shading normalisation: estimate the background with a large-kernel blur (about 1/20 of the page width) or a morphological closing, and divide it out. Run line detection on both the original and the normalised copy and keep whichever yields more consistent staves, as copisteria does [OMR §2.2].
4. Binarise with Sauvola: T(x,y) = m·(1 + k·(σ/R − 1)), R = 128, k = 0.2–0.4, window about 2–4 s (31–61 px at 300 dpi), computed in O(1) per pixel from integral images of I and I² (Shafait, Keysers & Breuel 2008) [OMR §2.2]. Before s is known, use a 41 px window, then re-run with a window of 3 s once staff metrics exist. Compute Otsu as well and keep the image whose vertical run-length histogram gives the sharper t+s peak.
5. Deskew: maximise the variance of the horizontal projection profile over angles in ±5°, coarse 0.5° steps, then refine to 0.05° [OMR §2.3]. Rotate gray and binary with bilinear sampling. Local curvature is not dewarped here; staff.js models it with polylines.
6. Denoise: remove connected components of area below 0.02 s² that do not touch a staff line (salt-and-pepper), after s is known.
7. Scale normalisation: if s < 10 px, upscale the page 2× before recognition [OMR §9.5]; if s > 40 px, downscale towards 20 px. Rule-based steps do best around s = 16–24 px [OMR §2.1].

Failure modes. Very dark or very light scans (Sauvola leaves speckle or erases thin lines); colour scans with red rubrics; 2-up landscape pages; strong gutter curvature; pages with no music (title pages). Confidence. rasterConf = sharpness of the staff-line run-length peak, normalised to [0,1]; low values add a warning ("faint or noisy scan").

4.2 staff.js

Responsibility. Staff metrics, staves as line polylines, systems and staff groups, staff removal, barlines and measures.

Inputs/outputs. Page → StaffLayout (§5.2, §5.3): staves[], systems[], measures[], s, t, plus a staff-removed binary image noStaff.

Algorithms.

1. Run-length staff metrics. Over all columns, histogram vertical black runs (mode = t) and white runs (mode ≈ s − t). Following Cardoso & Rebelo (ICPR 2010), take the most common sum of a black run and the following white run as t + s, which is robust when one of the two histograms is noisy [OMR §2.4].
2. Projection-based staff detection per strip. Split the page into 8–16 vertical strips. In each strip, take the horizontal black projection, find peaks, and group peaks into runs of five lines spaced t + s ± 20 % [OMR §2.5]. Link five-line groups across adjacent strips by vertical overlap, which tolerates skew and curvature. Fit each line as a polyline through the strip centres.
3. Line tracking fallback. Where strips disagree (curved pages, broken lines), track each line column by column, following the black run nearest the predicted y; as a second fallback, use stable shortest paths (Cardoso et al., TPAMI 2009) [OMR §2.5].
4. Staff removal. Along each tracked line, delete black pixels whose vertical run through the line is at most about 1.75 t and is centred on it; keep longer runs because they belong to symbols (Dalitz et al., TPAMI 2008, "line track height") [OMR §2.6]. Keep the un-removed gray and binary images for notehead matching, because removal fragments hollow heads.
5. Barlines. In the staff-removed image, find vertical runs of width ≤ 0.4 s that span the staff from the top line to the bottom line (± 0.5 s) and have no notehead attached. Classify thin, thick, double, final and repeat (two dots in spaces 2 and 3 next to a thick-thin pair).
6. Systems and staff groups. Two vertically adjacent staves belong to one system when a brace, bracket or vertical ink run joins them at the left edge, or when barlines continue through the gap [OMR §2.5]. Classify the left-edge glyph: a brace is a tall curly component of width about 1–1.5 s; a bracket is a thick vertical bar with serifs at both ends. Record groupSymbol: 'brace' | 'bracket' | 'line' | 'none'.
7. Measures. Within each system, barlines that are present on every staff (or span the system) cut it into measures. A barline found on only some staves is kept only if it is consistent with the system's other barlines; otherwise it is reported as a candidate for recognize.js repair.

Failure modes. Staves that touch text or ornaments; lyric hyphens or extenders read as staff fragments; partial staves (ossia, cue staves); systems split across a gutter; very short systems (codas); barlines merged with stems in dense music. Confidence. Per staff: regularity of line spacing (coefficient of variation of the five gaps) and line coverage across the width; per system: group-symbol detection score.

4.3 symbols.js

Responsibility. Find and classify music symbols on each staff and attach a confidence to each.

Inputs/outputs. Page + StaffLayout → Symbol[] (§5.4) in page coordinates, each with staff index, pos (for heads, rests, dots and accidentals), class, bbox and confidence; plus a list of text-like components handed to lyrics.js.

Algorithms.

1. Connected components. Two-pass union-find with 8-connectivity on noStaff, recording bbox, area, centroid, density and moments in s units. A second pass on the inverted image inside each component's bbox finds enclosed holes (hollow heads, whole notes, flats, sharps, naturals) [OMR §2.7].
2. Text separation. Components outside the staff band (more than about 1.5 s above the top line or below the bottom line, excluding ledger-line regions) whose height is about 0.6–1.2 s and which line up on a common baseline are marked as text and masked out of music recognition [OMR §2.7].
3. Noteheads. Normalised cross-correlation of templates against the gray, un-removed image at the normalised s, restricted to staff positions (line or space, half-space quantisation) [OMR §2.8]. Templates for filled, hollow and whole heads (2–3 variants each) are rendered from Bravura or Leland (SIL OFL-1.1). Peaks above threshold, after non-maximum suppression, become head candidates. Hollow versus filled is checked by the hole test. Heads on ledger lines are found by extending the position grid above and below the staff where short horizontal runs of length about 1.5–2 s exist.
4. Stems. Vertical runs of length ≥ about 2.5 s and width ≤ about 0.3 s touching a head: stem at the head's right edge means stem up, at its left edge stem down. A head with stems on both sides, or a head with an up stem and a down stem, is marked stemDir: 'both' (a unison of two voices) [OMR §2.9; VOI §2.2]. A single stem with heads at both ends or a head at its middle is tested as two aligned stems (the known merge failure [VOI §2.7]).
5. Beams. After stems are removed from compound components, thick near-horizontal bands (height about 0.4–0.6 s) between stem ends. The number of beam lines crossing a stem gives the beam level (1 = eighth, 2 = sixteenth). Flags are separate components at the free end of a stem [OMR §2.9].
6. Augmentation dots. Small round components about 0.5–1 s right of a head, in a space; a dot for a head on a line sits in the space above. Dots above or below a head on the stem-free side are staccato and are ignored in v1 [OMR §2.9].
7. Rests, accidentals, clefs, digits. Template matching or a small feature classifier (zoned density, aspect ratio, hole count, Hu moments) on component crops [OMR §2.8]. Key signature = accidentals between the clef and the first note or time signature; time signature = stacked digits or the C / cut-C glyph.
8. Ties and slurs. Thin curved components whose ends lie near two heads. A curve between two heads at the same pitch is a tie, otherwise a slur [OMR §2.9]. Fermatas by template.
9. Optional ML assist behind the same output contract: a tiny CNN on component crops, or a full-page detector. To be decided by the design phase, candidates:
   • classical only (NCC templates plus feature classifier);
   • tiny CNN on 32×32 crops, 50–200k parameters, int8 ONNX about 0.1 MB, trained on LilyPond-rendered crops and DeepScoresV2 (CC BY 4.0) [OMR §2.8, §9.4];
   • copista-2m detector (Apache-2.0, 1.94M parameters, 7.8 MB fp32 .pt; needs ONNX export; predicts stem direction including both and a voice slot per head) used as an extra vote [OMR §3.1, §9.4];
   • oemer UNets (MIT; 70.8 MB and 38.4 MB fp32, need quantisation and chunking) [OMR §3.1]. The numbers that decide this will come from docs/spike-ml.md and docs/spike-classical.md, which have not been written yet.

Failure modes. Heads touching each other (seconds, unisons offset sideways); stems broken by staff removal; beams merged with slurs; accidentals touching heads; lyric hyphens read as ledger lines or ties [VOI §2.7]; fermatas read as heads; verse numbers read as fingering. Confidence. Each symbol carries its classifier score in [0,1]; heads also carry the NCC peak value and stems their length ratio.

4.4 recognize.js

Responsibility. Assemble symbols into notes, chords, rests and voices, assign pitch and duration, repair rhythm per measure, and keep parts continuous across systems and pages.

Inputs/outputs. Symbol[] + StaffLayout for every page → Score draft (§5.10) with measures, voices per staff, events with onsets and durations, and per-measure repair records.

Algorithms.

1. Pitch. Pitch = clef + staff position + key signature + accidentals in force in the measure. An accidental holds until the barline for that step and octave; ties carry it across the barline. A clef change applies from its x onwards. Octave clefs (G2 with an 8 below) set octaveChange = −1 [OMR §2.11].
2. Chords and slots. A stem with its heads forms a chord. Heads with stems on both sides are split into two mirrored chords that share a slot. Stemless heads are grouped by x. Slots are built per system measure: a shared stem means the same slot; the same beam group on different stems means different slots; otherwise x overlap within about 0.6 s [VOI §10 step 2; Audiveris SlotsRetriever described in VOI §2.5].
3. Voices per staff by stem direction. Two keys, U (upper) and L (lower) [VOI §10 step 3]:
   • stem up → U, stem down → L; a beam group takes the majority direction of its stems;
   • on a staff classified as two-voice, a single-stem chord with two heads is split top → U, bottom → L; extra middle heads are flagged divisi and attached as a chord to the nearer voice;
   • a single-stem single head with no opposite-stem event overlapping its span is a unison and is copied to both voices;
   • stemless heads: two in one slot go by pitch order; one goes to the voice that has no event in that span, else to both;
   • rests: ≥ +1 s above the middle line → U, ≤ −1 s below → L, centred → shared unless one voice already covers that time; a rest inside a beam group takes the beam's voice [VOI §2.3];
   • ties and beams keep their voice. Keyboard staves use the same steps without de-combining chords, with up to four voices per staff. Stem direction overrides pitch order, which preserves the 0.24 % (S/A) and 0.78 % (T/B) of slots where Bach chorale voices cross [VOI Appendix A].
4. Duration hypotheses. Each note gets a short list of hypotheses: head type × beam/flag count × dot present × tuplet membership, each with a prior from symbol confidence.
5. Rhythm repair as constraint search. Per staff and measure, choose one hypothesis per event and one resolution per ambiguous voice decision (shared or private rest, unison copy, divisi placement) so that:
   • hard: every voice's durations sum to the measure length (or to the pickup length in the first measure); onsets never decrease with x; events in one slot share an onset (Jin & Raphael's coincidence constraint [VOI §2.6]);
   • soft: maximise the summed log-confidence of the chosen hypotheses, penalise gaps, overlaps and onsets that are far from their x-column. Implementation: depth-first branch-and-bound ordered by confidence, with a budget of 200 states per measure [VOI §10 step 4]. Hymn measures have at most about 16 notes per voice, so exhaustive search inside the budget is cheap [OMR §2.10]. copisteria solves the same problem with a Viterbi search over 48ths of a quarter [OMR §2.10]; that remains the alternative if branch-and-bound proves too slow. The point of the search is that "one misread duration does not shift the rest of the bar".
6. Missing barlines. If a voice holds ≥ 1.75 bars of duration, look for a full-height vertical stroke near the point where the running sum crosses one bar, insert a barline and decode again [OMR §2.10].
7. Time signature inference. When none is printed, choose the bar length that the largest number of measures fit. Pickup: allow the first measure to be short and mark it implicit; in hymns the pickup and the final measure usually add up to one full measure [OMR §2.10].
8. Fallback. If no assignment satisfies the hard constraints within the budget, keep the best soft solution, pad each short voice with a <forward> (or an invisible rest), and flag the measure with confidence < 0.5 [VOI §11].
9. Cross-system continuity. Collate staff groups across systems into logical groups by staff count, clefs and labels; carry key, time and clef state forward; a system where a group is missing yields rests.

Failure modes. Missed dots and beams (the classic desynchronisation), unmarked triplets (copisteria gets 65 % of tuplet notes right when marks are removed against 96 % with them [OMR §2.10]), merged aligned stems, grace notes, cross-staff beams. Confidence. Per measure: 1.0 if the measure passed without repair, reduced by the cost of the repair, < 0.5 if padded. Per voice: share of its measures that passed without repair.

4.5 lyrics.js

Responsibility. Find lyric rows, OCR them, split into syllables, assign verses and syllabic, detect extenders, and align syllables to the reference voice's onsets. Detail in §7.

Inputs/outputs. Pass 1: Page + StaffLayout + text components → LyricRow[] (bands with bboxes and verse order). Pass 2: rows + env.ocr + Score draft → LyricLine[] of Syllable (§5.9) bound to slots of a reference voice, with conflict counts per receiving voice.

Algorithms (summary; §7 has the full design): text-row detection by horizontal projection of masked small components; crop with 0.3 s margin and upscale to x-height ≥ 20 px; tesseract single-line mode (tessedit_pageseg_mode = 7) with word boxes; verse numbers stripped; hyphen and gap tokenisation; geometric extender detection; monotonic DP alignment of syllables to onsets [OMR §6.3; VOI §7.2].

Failure modes. One-letter syllables ("a", "O") missed by OCR [OMR §3.3]; hyphens dropped; italic refrains; lyric rows touching low notes or ledger lines; verses 4 and later printed as stanzas below the music, which are not underlay [OMR §6.3]. Confidence. Per syllable: OCR word confidence × alignment cost factor; per part and verse: coverage (syllables attached / syllables OCR'd).

4.6 parts.js

Responsibility. Classify each logical staff group, decide whether to split voices into parts, name the parts, apply the lyric copy policy, and record provenance. Detail in §6.

Inputs/outputs. Score draft + LyricLine[] + label OCR → final Part[] (§5.8) with splitFrom provenance, names (read or inferred), part groups and per-part confidence.

Algorithms. The weighted vocal-score classifier and thresholds of [VOI §5], the decision procedure of [VOI §11], the naming procedure of [VOI §6], and the lyric copy rules of [VOI §7.2].

Failure modes. A hymn wrongly kept as one keyboard part; a piano accompaniment wrongly split; part count changing between systems; labels misread. Confidence. Per part: the minimum of classification confidence (sigmoid of the vocal score), voice-assignment confidence and stem-detection confidence [VOI §11].

4.7 musicxml.js

Responsibility. Serialise the final score model as XSD-valid MusicXML 4.0 score-partwise.

Inputs/outputs. Score → string. Pure and deterministic: the same input gives byte-identical output.

Algorithms.

1. Divisions. Internal durations are integer ticks at 48 per quarter (§5.5). Per part, divisions = 48 / gcd(48, all durations in the part); written in the first measure's <attributes> and only re-written if it changes.
2. Part list. <part-list> with <part-group type="start" number="n"> (group-name, group-symbol bracket for a split choir or brace for a keyboard part, group-barline yes), one <score-part> per part with <part-name> and <part-abbreviation>, and the matching type="stop" [VOI §8].
3. Attributes. Key (<fifths>, <mode> when known), time (<beats>, <beat-type>, or symbol="common" / "cut"), clefs (with number for multi-staff parts), <staves>2</staves> for grand staves. Split tenor parts use treble-8 (G, line 2, clef-octave-change −1) by default, with an option to keep the printed bass clef [VOI §8.1].
4. Voices. For each measure, write voice 1's events in order, then <backup> by the summed duration written so far, then voice 2, and so on. <backup> never crosses a measure boundary and its duration is positive [VOI §8]. Voice numbering on multi-staff parts: 1–4 for voices starting on staff 1, 5–8 on staff 2 [VOI §0 item 5]. Every note in a multi-staff part carries <staff>.
5. Notes. <pitch> (step, alter, octave) or <rest/> (with measure="yes" for whole-measure rests), <duration>, <tie type> plus <notations><tied>, <voice>, <type>, <dot/>, <time-modification> for tuplets, <stem> only in unsplit multi-voice parts, <chord/> for the second and later heads of a chord, <accidental> when printed, <notations><fermata/>. Padding uses <forward> with <voice>, or a rest with print-object="no".
6. Lyrics. <lyric number="v"> with <syllabic>, <text>, optional <elision>, and <extend type="start|stop"/>; melisma continuation notes get no lyric element [VOI §8].
7. Measures. number sequential from 1 (0 for a pickup), implicit="yes" on pickups, <barline> for repeats, double and final bars. <print new-system="yes"> and new-page="yes" mirror the scan's layout.
8. Escaping and identity. XML-escape all text; <identification><encoding> with software name and date; <work-title> from OCR of the largest text block on page 1 if confident, otherwise the file name.

Failure modes. Voice durations not matching after an upstream bug (guarded: the writer re-checks each voice sum and inserts <forward> with a warning rather than writing an inconsistent measure); invalid character data from OCR (stripped of control characters). Confidence. Not applicable; validity is checked in the harness with xmllint against the 4.0 XSD.

4.8 index.js

convert(input, {env, onProgress, options}) → {musicxml, score, pages: Overlay[], warnings, stats}. Runs stages in order, reports progress as {stage, page, pageCount, fraction}, records per-stage timings, contains errors (§3.5), and exposes reemit(score, overrides) so the UI can toggle a split decision and get new MusicXML without re-running recognition.

5. Data design

Conventions:

• Coordinates are in working-image pixels, origin top left, x right, y down, unless named src* (source image pixels). Each page stores the affine transform from working to source pixels for the overlay.
• Sizes and distances inside music logic are in staff spaces s unless the field name ends in Px.
• Staff position pos: half staff spaces from the middle line, positive up (§1.3).
• Time: integer ticks, TPQ = 48 ticks per quarter note. 48 covers dotted values down to 32nds and triplets down to 16th triplets; quintuplets and finer tuplets would need a change of TPQ (to be decided by the design phase if the corpus contains them).
• Confidence: number in [0,1] on every recognised object, field conf.
• IDs: strings, unique within the score, prefixed by type (st3, sy1, h204, n88).
• The shapes below are JSON with // comments added for reading; a string such as "Bitmap" or "up | down" names a type or lists the allowed values, and x0, y0 placeholders stand for numbers.

5.1 Bitmap and page

// Bitmap: the only image type that crosses module boundaries
{ "width": 2480, "height": 3508, "data": "Uint8ClampedArray(width*height), 0 = black, 255 = white" }

// Page
{
  "index": 0,
  "gray": "Bitmap",
  "binary": "Bitmap (0 = ink, 255 = paper)",
  "noStaff": "Bitmap | null (set by staff.js)",
  "srcWidth": 2550, "srcHeight": 3300, "sourceDpi": 300,
  "skewDeg": -0.35,
  "scale": 1.0,
  "toSource": [1, 0, 0, 1, 0, 0],
  "binarization": "sauvola | otsu",
  "conf": 0.92
}

5.2 Staff

{
  "id": "st3", "page": 0, "system": "sy1", "indexInSystem": 0,
  "lines": [[[120, 402.0], [600, 401.2], [2380, 399.8]], "... 5 polylines, top line first"],
  "s": 21.4, "t": 2.6,
  "x0": 118, "x1": 2382,
  "clefAtStart": { "sign": "G", "line": 2, "octaveChange": 0 },
  "lineCountDetected": 5,
  "conf": 0.97
}

5.3 System, staff group, barline, measure

// System
{
  "id": "sy1", "page": 0, "staves": ["st3", "st4"],
  "groups": [{ "id": "g1", "staves": ["st3", "st4"], "symbol": "brace | bracket | line | none", "conf": 0.8 }],
  "barlines": [{ "id": "bl7", "x": 640, "kind": "single | double | final | repeat-start | repeat-end", "staves": ["st3", "st4"], "conf": 0.95 }],
  "measures": ["m1", "m2", "m3"],
  "labels": [{ "text": "S. A.", "bbox": [20, 380, 100, 440], "staff": "st3", "conf": 0.7 }]
}

// Measure (geometric span, shared by all staves of the system)
{ "id": "m2", "system": "sy1", "number": 2, "x0": 640, "x1": 1180,
  "pickup": false, "timeTicks": 192, "repair": null, "conf": 1.0 }

5.4 Symbol

{
  "id": "h204", "page": 0, "staff": "st3",
  "kind": "head | stem | beam | flag | dot | rest | accidental | clef | keySig | timeSig | tie | slur | ledger | fermata | barline | text",
  "subtype": "filled | hollow | whole | quarter-rest | eighth-rest | sharp | flat | natural | G | F | C | 3 | ...",
  "bbox": [812, 371, 834, 391],
  "cx": 823, "cy": 381,
  "pos": 3,
  "stemDir": "up | down | both | none",
  "links": { "stem": "sm33", "dots": ["d9"], "accidental": "ac2", "beams": ["bm4"] },
  "conf": 0.88,
  "source": "ncc | features | cnn | detector"
}

5.5 Note, rest and chord

// Note
{
  "id": "n88", "head": "h204",
  "pitch": { "step": "E", "alter": 0, "octave": 5 },
  "accidentalPrinted": null,
  "durTicks": 96, "type": "half", "dots": 0,
  "tuplet": null,
  "tie": { "start": false, "stop": false },
  "fermata": false,
  "lyrics": ["Syllable refs"],
  "conf": 0.9
}

// Rest
{ "id": "r12", "durTicks": 48, "type": "quarter", "dots": 0, "measureRest": false,
  "vertical": "up | down | centre", "shared": false, "hidden": false, "conf": 0.85 }

// Chord: one onset, one stem (or stemless group), one voice
{ "id": "c40", "staff": "st3", "onsetTicks": 0, "durTicks": 96,
  "notes": ["n88"], "stemDir": "up", "beamGroup": "bg5", "slot": "sl3",
  "mirrorOf": null, "copiedFrom": null, "conf": 0.9 }

5.6 Voice

{
  "id": "v-st3-U", "staff": "st3", "key": "U | L | K1 | K2 | K3 | K4",
  "events": [{ "measure": "m2", "chord": "c40" }, { "measure": "m2", "rest": "r12" }],
  "unisonCopies": 3, "decombinedChords": 5, "divisiFlags": 0,
  "conf": 0.94
}

U/L are the upper and lower voices of a two-voice staff; K1–K4 are keyboard voices.

5.7 Measure content (per staff group, after repair)

{
  "measure": "m2", "group": "g1",
  "voices": {
    "v-st3-U": { "events": ["c40", "c41", "r12"], "sumTicks": 192, "padTicks": 0 },
    "v-st3-L": { "events": ["c42", "c43"], "sumTicks": 192, "padTicks": 0 }
  },
  "repair": { "applied": true, "states": 37, "changes": ["dot added to n91"], "cost": 1.6 },
  "conf": 0.78
}

5.8 Part

{
  "id": "P2",
  "name": "Alto", "abbreviation": "A.", "nameSource": "label | inferred",
  "kind": "vocal | keyboard | instrument | unknown",
  "staves": 1,
  "clef": { "sign": "G", "line": 2, "octaveChange": 0 },
  "voices": ["v-st3-L"],
  "splitFrom": { "logicalGroup": "LG1", "staffIndex": 0, "voiceKey": "L", "stem": "down",
                 "reason": "lyrics between staves; opposite stems in 92% of bars", "vocalScore": 7.0 },
  "group": "PG1",
  "lyricSource": "copied | read | none",
  "conf": 0.86
}

// Part group
{ "id": "PG1", "name": "Choir", "symbol": "bracket | brace", "barline": true, "parts": ["P1", "P2", "P3", "P4"] }

5.9 Lyric row, line and syllable

// LyricRow (pass 1, geometry only)
{ "id": "lr4", "system": "sy1", "aboveStaff": "st3", "belowStaff": "st4",
  "bbox": [130, 470, 2370, 505], "baselineY": 500, "xHeightPx": 13,
  "verseIndex": 1, "verseLabel": "1." }

// LyricLine (pass 2)
{ "id": "ll4", "row": "lr4", "verse": 1, "ocrText": "Praise God from whom all bless - ings flow",
  "ocrConf": 0.83, "syllables": ["sb51", "..."],
  "referenceVoice": "v-st3-U", "receivers": ["v-st3-U", "v-st3-L", "v-st4-U", "v-st4-L"],
  "conflicts": { "v-st3-L": 1, "v-st4-U": 0, "v-st4-L": 2 } }

// Syllable
{ "id": "sb51", "text": "bless", "syllabic": "begin", "verse": 1,
  "bbox": [1500, 478, 1560, 500], "cx": 1530,
  "extender": false, "elision": false,
  "slot": "sl14", "attachedTo": { "v-st3-U": "n120", "v-st3-L": "n121" },
  "conf": 0.79 }

5.10 Score

{
  "meta": { "title": "Old Hundredth", "titleConf": 0.6, "source": "upload.pdf", "pages": 1,
            "software": "scan2mxl 0.1", "created": "2026-11-08" },
  "pages": ["Page"], "staves": ["Staff"], "systems": ["System"], "measures": ["Measure"],
  "symbols": ["Symbol"], "chords": ["Chord"], "notes": ["Note"], "rests": ["Rest"],
  "voices": ["Voice"], "measureContent": ["MeasureContent"],
  "logicalGroups": [{ "id": "LG1", "staffCount": 2, "systems": ["sy1", "sy2"], "classification": "vocal-closed | keyboard | ambiguous | single", "vocalScore": 7.0, "features": {} }],
  "parts": ["Part"], "partGroups": ["PartGroup"],
  "lyricLines": ["LyricLine"], "syllables": ["Syllable"],
  "key": { "fifths": 1, "mode": "major" }, "time": { "beats": 4, "beatType": 4, "symbol": null },
  "tpq": 48,
  "overrides": { "LG1": "auto | split | keep" }
}

5.11 Overlay

Geometry for the UI, in source pixels so it can be drawn over the original scan.

{
  "page": 0, "width": 2550, "height": 3300,
  "layers": {
    "staves":   [{ "id": "st3", "polylines": [[[x, y], "..."]], "conf": 0.97 }],
    "systems":  [{ "id": "sy1", "bbox": [x0, y0, x1, y1], "groupSymbol": "brace" }],
    "barlines": [{ "x": 640, "y0": 380, "y1": 760 }],
    "heads":    [{ "bbox": [x0, y0, x1, y1], "part": "P2", "voiceKey": "L", "marker": "triangle-down", "conf": 0.88 }],
    "rests":    [{ "bbox": [x0, y0, x1, y1], "part": "P1", "conf": 0.8 }],
    "lyrics":   [{ "bbox": [x0, y0, x1, y1], "text": "bless", "verse": 1, "attached": true }],
    "flags":    [{ "bbox": [x0, y0, x1, y1], "measure": "m7", "reason": "padded", "severity": "warn" }]
  }
}

5.12 Warning

{
  "code": "MEASURE_PADDED | LYRICS_NOT_COPIED | SPLIT_AMBIGUOUS | NAME_INFERRED | FAINT_SCAN | STANZA_TEXT_UNATTACHED | DIVISI_KEPT_AS_CHORD | STAGE_FAILED | LOW_DPI_UPSCALED | PAGE_NO_MUSIC",
  "severity": "info | warn | error",
  "message": "Lyrics not applied to Tenor: rhythm differs in 4 places",
  "where": { "page": 0, "system": "sy2", "measure": "m9", "part": "P3" },
  "conf": 0.4
}

6. Part-splitting decision procedure

6.1 Evidence from the research

• Stem direction is the primary evidence for voices on a printed staff; pitch is a fallback [VOI §0, §2.1].
• In 368 four-part Bach chorales from the music21 10.5.2 corpus [VOI Appendix A]:

Consequences: unisons are common enough to need explicit copying; crossings are rare but real, so stem direction must win over pitch order; about 31 % of slots have an onset in only one voice, so lyric copying must align by slot, not by note index. Lyrics appear on the soprano only in 320 of the 368 files, so lower-voice lyric truth has to be derived [VOI §0].

• Consistency is the hard part: a greedy rule baseline gets only 55.1 % of piano measures with all voices matching, against 92.4 % for search guided by a model (BeadSolver, Table 7); Jin & Raphael's constrained graph search interprets 95.7 % of system measures correctly [VOI §0]. Closed hymns are easier, but the design uses the same tool: a hard bar-length constraint plus repair search (§4.4).

6.2 Vocal-score classifier

Computed per logical staff group, summed over all of its systems. Weights are the research recommendation and are tuned on the corpus [VOI §5]:

A brace alone must not veto a split, because hymnals often brace SA/TB staves so the short score can double as an organ part [VOI §5]. A typical unlabelled hymn scores about +7; a song accompaniment about −7.

6.3 Rules

for each logical staff group G (whole piece):
  s = vocalScore(G)
  if G.staves == 1:
     twoVoice = share of measures (where the staff has notes) with >= 2 concurrent streams
     names    = labelNames(G)
     if names.count >= 2 and (s >= 1 or twoVoice >= 0.2):    SPLIT into names
     elif twoVoice >= 0.5 and allStreamsMonophonic and s >= 3: SPLIT into "<Name> 1", "<Name> 2"
     else: KEEP one part, voices 1/2 (warn if twoVoice > 0)
  elif G.staves == 2:
     if s >= 3:   SPLIT: per staff k voices (k = max concurrent streams, usually 2; SAB gives 2+1)
     elif s <= -1: KEEP as keyboard part: <staves>2</staves>, voices 1-4 / 5-8
     else:        KEEP as closed score (two parts, voices 1/2) and flag SPLIT_AMBIGUOUS
  elif G.staves >= 3: KEEP (organ) unless every staff qualifies as vocal on its own
  part count is fixed for the piece; never split per measure;
  measures with fewer visible voices get unison copies or rests.

Source: [VOI §11]. Decisions are made once per logical group for the whole piece. Part layout is planned on the fullest system and every other system is mapped onto it, so a system where a voice drops out still yields the same parts, with rests [OMR §5.2 step 5].

6.4 Output forms

6.5 Ambiguous and edge cases

Source: [VOI §11]. The user toggle sets score.overrides[LG] = 'split' | 'keep' and calls reemit.

6.6 Naming

Label regions are text boxes left of the first barline that overlap a staff or sit centred on a group symbol. They are normalised (lowercase, diacritics and punctuation stripped, split on whitespace, /, &, ,, +, .) and looked up in a multilingual lexicon (soprano, alto, tenor, baritone, bass, collectives such as Women → S+A and Men → T+B, divisi markers, keyboard tokens). On a two-voice staff the first name goes to the upper voice; four names on a two-staff group map (1,2) to staff 1 and (3,4) to staff 2. Abbreviations on later systems are matched by prefix and never change the part count. Unlabelled vocal groups get S, A, T, B (treble + bass), Tenor I, Tenor II, Baritone, Bass (treble-8 + bass) or S, A, Baritone (SAB), and the UI shows "(inferred from layout)" [VOI §6].

7. Lyrics design

7.1 Locating lyric rows

1. After staff detection and music-symbol assembly, mask everything assigned to music. Text rows are found by horizontal projection of the remaining small components in each inter-staff gap and below the last staff of each system: x-height about 0.6–1.2 s, a common baseline, and horizontal extent over at least a quarter of the system width [OMR §6.3].
2. Several stacked rows in one gap are verses 1..n, top to bottom. A leading "1.", "2." gives the verse number and is stripped, as Audiveris does since 5.10 [OMR §3.3].
3. Each row belongs to the nearest staff above it in the same system; if there is no staff above, or the gap to it is more than twice the gap to the staff below, it belongs to the staff below [VOI §7.2 step 1].
4. Text far below the last system (stanzas printed as poetry) is not underlay. It is reported as unattached text with warning STANZA_TEXT_UNATTACHED; no underlay is invented [OMR §6.3].

7.2 OCR

• Engine: tesseract.js 7.0.0 (Apache-2.0) with tessdata_fast traineddata: eng 4.11 MB, lat 3.19 MB, deu 1.53 MB [OMR §0, §6.2]. tessdata_best eng (15.4 MB) is over the per-file cap. Since v6 only text output is on by default, so the call requests {blocks: true} for word boxes and confidences [OMR §6.2].
• Per row: crop with a 0.3 s margin, upscale so the x-height is ≥ 20 px, binarise, run with tessedit_pageseg_mode = 7 (single line), and restrict the character set to letters (including accented Latin), punctuation, hyphen, apostrophe and digits [OMR §6.3].
• Language: eng loaded by default; lat and deu loaded on demand when the UI language selector asks, or when the first row's OCR confidence in eng is low. Which languages ship by default is to be decided by the design phase from docs/spike-ocr.md (not yet written). Candidates: eng only (4.11 MB); eng + lat + deu (8.83 MB); PP-OCRv5 mobile Latin recogniser via ORT (det 4.8 MB + rec 8.0 MB, Apache-2.0) as the upgrade for degraded scans [OMR §6.2].

7.3 Syllables, hyphens, extenders

• Engravers separate syllables with centred hyphens ("glo - ry"); OCR returns them as separate - tokens or inside words ("glo-ry"). Word gaps are wider than syllable gaps. Tokens are split at hyphens, and word boundaries come from the gaps measured on the word boxes [OMR §6.3].
• syllabic: a word with no hyphen → single; first part → begin; inner parts → middle; last → end.
• Extenders are detected geometrically, because OCR usually drops them: a long thin horizontal component at baseline height after a word-final syllable, height ≤ 1.5 t and length ≥ 1 s [OMR §6.3].
• Hyphenation dictionaries (Liang/TeX patterns) are a last resort only, because the print already shows the splits.
• Hymn specifics: "A-men" codas, elisions (an undertie between syllables on one note → <elision>), apostrophes ("pray'r"), isolated punctuation merged into the previous syllable, italic refrains, capitals at verse starts [OMR §6.3, §3.3].

7.4 Alignment to notes

A monotonic dynamic-programming alignment runs between the syllables of one verse line and the onset slots of the reference voice, which is the upper voice of the staff above the row [VOI §7.2 step 3]:

• match cost: |syllable centre − notehead centre| in s, with a variant that allows a left-aligned first syllable of a word;
• skipping an onset (melisma) is cheap after a syllable that ends in an extender or hyphen and dearer otherwise; tied continuations are always skipped at no cost;
• an unmatched syllable is expensive.

Notes inside a melisma get no <text>. The syllable that starts a melisma gets <extend type="start"/> and the last note <extend type="stop"/> (after word-final syllables only).

This replaces Audiveris' nearest-chord rule, which on a closed staff tends to bind lyrics to the lower voice because its heads are closer to the text [VOI §7.1].

7.5 Copying to voices by onset

1. A row under a two-voice staff applies to both of its voices. In a closed SATB group with lyrics only between the staves, the block applies to all four voices; if a separate block exists under the bass staff, T and B take that block instead. In open score each staff carries its own text [VOI §7.2 step 2].
2. For each receiving voice: if the voice has an onset at the syllable's slot, attach a copy with the same syllabic and verse number. If the voice is sustaining a note there, it does not take the syllable and a conflict is counted. Onsets after a syllable with no syllable of their own are a melisma for that voice and get extender marks [VOI §7.2 step 4].
3. If more than 15 % of a verse's syllables conflict for a voice, the verse is not copied to that voice and a warning is raised: "Lyrics not applied to Alto: rhythm differs in N places" [VOI §7.2 step 5].
4. Provenance (read for the reference voice, copied otherwise) is stored per part for the lyrics preview.

The 31 % rhythm-divergence figure in Bach chorales (§6.1) is why copying is by slot rather than by note index.

8. UI design

8.1 Layout

One page, two columns above 900 px and one column below. Left: the scanned page with overlay. Right: tabs or stacked panels for the per-part summary, notation preview, lyrics preview, warnings and MusicXML. The visual language draws on engraving: staff-line rules as dividers, measure-number-style counters for stages, and terms such as "closed score", "divisi" and "staff space" in the summary. Fonts come from Google Fonts only. Colours are theme tokens defined on :root, redefined for dark mode, with the body background taken from a token.

8.2 States

8.3 Results panels

• Page view: the scan with layer toggles (staves, systems and barlines, noteheads coloured by part, lyrics boxes, flagged measures). Each part has a colour and a marker shape (for example circle for soprano, triangle-down for alto), and a legend names both. Page navigation for multi-page input. Zoom by buttons and pinch.
• Part summary: one row per part with name (and "(inferred from layout)" when applicable), clef, measure count, note count, confidence, and the split provenance ("Split from treble staff, stems down"). Ambiguous groups show a toggle "Split into parts / Keep together".
• Notation preview: OpenSheetMusicDisplay 2.3.0 (BSD-3) rendering of the emitted MusicXML.
• Lyrics preview: per verse and per part, the syllables joined into words, with copied syllables marked and conflicts listed.
• Warnings: grouped by severity; each item links to its page and measure in the page view.
• MusicXML: a read-only, selectable <textarea> and a Copy button. Copy uses navigator.clipboard.writeText inside the click handler; on failure it selects the text and says "Press Ctrl+C / Cmd+C to copy". There is no download link, because downloads are inert in the artifact.

8.4 Behaviour rules

No alert, confirm, prompt or print. Long work runs in the worker; the main thread only renders. The page opens in a working state by converting the bundled sample. Toggling a split decision re-emits MusicXML and re-renders the preview within about one second, without re-running recognition.

8.5 Accessibility

• All controls are native buttons, inputs and labels; the drop zone is also a button that opens the file picker, so drag-and-drop is never required.
• Visible focus rings with at least 3:1 contrast against both themes.
• The stage list is an aria-live="polite" region; errors use role="alert" text in the page (not dialogs).
• Colour is never the only carrier: parts have marker shapes and text labels; flagged measures have a hatched pattern and a listed warning.
• The overlay canvas has a text alternative: the part summary and the warning list describe what it shows.
• prefers-reduced-motion turns off progress animations and smooth scrolling.
• At 400 px width the layout is one column with no horizontal page scroll; the page view scrolls inside its own frame. Side gutters use max(16px, env(safe-area-inset-left)).

9. Deployment and artifact constraints

9.1 Packaging

build.mjs produces:

• dist/index.html: no <!doctype>, <html>, <head> or <body> tags; it starts with <title> (a two to four word name), <meta name="claude:viewport-fit-sides" content="cover"> and <style>; all app JS is inlined [WF].
• dist/assets/: everything fetched at run time, loaded by relative URL.

9.2 External scripts (pinned, script tags only)

Fetch, XHR, WASM and worker loads from CDNs are blocked by the artifact CSP, so all WASM, traineddata, models and worker scripts are self-hosted in assets/ [WF].

9.3 Assets and size budget

Sizes from [OMR §0, §6.2, §7.1, §9.3]. Phase 1 totals about 13–20 MB; phase 2 about 30–40 MB. The ORT WebGPU binary (28.3 MB) cannot ship as one file, so the WASM CPU execution provider with one thread is the plan; threads would need crossOriginIsolated, which the artifact frame is not expected to have [OMR §7.1].

build.mjs fails the build if any binary file exceeds 15 MB, any text file exceeds 16 MB, or the total exceeds 64 MB, and prints a size table.

9.4 Load sequence

1. Page renders immediately with the sample's name and the stage list.
2. Script tags for pdf.js, tesseract.js and OSMD load from the CDNs.
3. The OMR worker starts; tesseract creates its worker from assets/worker.min.js and its core from assets/ lazily, at the first lyric row, so the staff and symbol stages are not delayed.
4. The sample PDF is fetched from assets/ and converted.

9.5 Local verification

Serve dist/ with a CSP header that simulates the artifact (script-src the three CDN hosts plus 'unsafe-inline' 'wasm-unsafe-eval' blob:; connect-src 'self' blob: data:; worker-src 'self' blob:), and check in Playwright that the sample converts, overlay and preview render, lyrics appear, and the console has no errors or blocked loads [WF].

10. Testing and evaluation

10.1 Corpus

Synthetic degraded scans with exact truth (corpus/synthetic/<id>/, manifest corpus/manifest-synthetic.json). Being built at the time of writing: the generator tools exist in corpus/synthetic/_tools/synth/ (m21import.py, harmonize.py, dsl.py, lily.py, degrade.py, mxl.py, midicheck.py), and no items or manifest have been written yet. The plan [WF]: at least 14 items from public-domain or CC0 MusicXML (music21 Bach chorales, OpenScore Lieder, folk songs and hymns), engraved with LilyPond 2.24; closed-score SATB written with \voiceOne/\voiceTwo on two staves and \lyricsto, so the truth has four parts; rasterised at 150, 200 and 300 dpi; degraded with rotation, noise, blur, salt-and-pepper, JPEG, lighting gradients and slight curvature; wrapped as image-only PDFs (checked with pdffonts returning nothing). Coverage: single melody with lyrics, multi-verse hymns, closed SATB, open SATB, piano solo (must not split), voice and piano.

Real public-domain scans (corpus/real/<id>/, manifest corpus/manifest-real.json, not yet written). Eight Internet Archive pages are on disk, fetched by corpus/real/tools/fetch_scans.sh and wrapped unmodified into image-only PDFs:

The plan calls for hand-transcribed truth for at least three short items, verified by engraving the transcription and comparing it with the scan [WF].

Both manifests are merged into corpus/manifest.json with entries {id, title, source_url, license, origin: 'synthetic-scan' | 'real-scan', dpi, degradations, has_ground_truth, features, expected_parts}.

Negative set. OpenScore Lieder (CC0) voice-and-piano renders: the piano must never split and the voice staff must not split [VOI §12].

10.2 Metrics

The harness reports every metric split by origin (synthetic versus real) and by feature (lyrics, satb_closed, piano, …), plus an aggregate in [0,1]. The aggregate's weights are to be decided by the design phase; the workflow's improvement loop stops when the aggregate reaches 0.95 or fails to improve by more than 0.005 for two rounds [WF].

10.3 Harness CLI

node test/run.mjs [--out test/results.json] [--only id1,id2] [--filter satb_closed]
                  [--debug test/debug/] [--no-ocr] [--json]

• --out writes results to a named file so concurrent agents do not collide [WF];
• --only and --filter restrict items by id or feature;
• --debug writes overlay PNGs per page and stage;
• --no-ocr skips tesseract for fast symbol-only runs;
• exit code is non-zero on any crash or XSD failure.

Per-item results hold every metric, warnings, timings and the diff of parts; the aggregate block matches the workflow's metrics schema (aggregate, pitch, duration, lyrics, parts, xsd_valid_ratio, crashes, real_scan_aggregate, worst_items) [WF].

10.4 Unit fixtures

Each module owner keeps fixtures under test/fixtures/<module>/: recognise from symbol fixtures, parts from score fixtures (closed SATB, open SATB, piano, voice and piano, single melody, ambiguous cases), MusicXML fixtures validated against the XSD, lyric strips cropped from corpus scans [WF]. The voice logic is also tested on ground-truth symbols generated from the chorales, against the ≥ 99 % target [VOI §12].

10.5 Acceptance thresholds for v1

11. Licensing and third-party components

11.1 Shipped

11.2 Used offline only (harness, never shipped)

pdftoppm and pdfimages (poppler, GPL; run as external processes), LilyPond 2.24 (GPL; its output is not encumbered [OMR §4]), music21 (BSD-3), musicdiff (MIT), MV2H (MIT), Audiveris (AGPL-3.0; optional baseline run as a separate process).

11.3 Excluded

12. Risks and mitigations

13. Delivery plan

13.1 Track 1: fast-track v1 in mvp/

A classical pipeline built now, in mvp/, separate from the multi-agent build so it cannot collide with module owners' files. Scope: Sauvola, projection staff detection, run-length metrics, staff removal, connected components, NCC noteheads, stems and beams, barlines, a simple rhythm solver, stem-direction voices, the closed-score split rules of §6, tesseract lyrics with DP alignment, and the MusicXML writer, published as an artifact with the sample hymn. Its purpose is to give a working page early and to provide a baseline for the harness. It is replaced by Track 2's output when Track 2 beats it on the harness.

13.2 Track 2: full multi-agent build

The scan-pdf-to-musicxml workflow [WF] runs these phases:

13.3 Phases of capability

1. Phase 1 (ship): classical core, tesseract lyrics, SATB split rules, MusicXML writer, harness on synthetic hymn scans; targets in §10.5 [OMR §9.4].
2. Phase 2 (ML assist): tiny CNN for heads, rests, accidentals and clefs; copista-2m ONNX export spike as an extra vote on stem direction and voice slot; optional PP-OCRv5 for lyrics on degraded scans [OMR §9.4]. Each ships only if the harness shows a gain worth its size.
3. Not planned: end-to-end image-to-sequence models (SMT/SMT++, Zeus, LEGATO, TrOMR, homr) in the browser: weights of 40–430 MB, slow autoregressive decoding on single-thread WASM, no SATB part semantics, no lyrics [OMR §9.4].

14. Open questions