Copies/docs/final_output.md

# Final output: `A Rendre`

This documents the current implementation, as of 2026-09-12. The return folder
referred to as « À rendre » is named **`A Rendre`** on disk. JPEG files use the
extension **`.jpg`**, not `.jpeg`.

## Left space for Gemini comments

`config.py` (or the file selected by `COPIENATOR_CONFIG`) can override:

```python
ANNOTATION_MARGIN_LEFT = 300
ANNOTATION_MARGIN_POINTS = [(10, 450), (20, 400), (35, 300)]
```

The reference points are (measured margin in mm, added pixels at 200 dpi).
Each page fragment gets its own spacing: 450 px at or below 10 mm, 400 px at
20 mm, and 300 px at or above 35 mm, with linear interpolation between points.
Unknown pages retain their unknown status but use the arithmetic mean of the
copy's measured page margins, including during label-strip positioning.
Inferred values never contribute to that mean. If no page has a measurement,
`ANNOTATION_MARGIN_LEFT` supplies the default 300 px.
The old `ANNOTATION_LOW_MARGIN_LEFT` setting is replaced by the reference points.
The 600 px comment width is unchanged, so
comments can still overlap handwriting on pages with very little whitespace.

Automatic `crop-labels` saves per-page measurements in the Cutleft schema.
`split-answers` records source pages, crop rectangles and remaining margins in
`*.layout.json` files beside answer PDFs. Those files follow copies, manual
merges and vertical cuts. PDF fingerprints prevent using stale measurements
after files are replaced. Legacy schemas and manual/full-page Cutleft runs
without measurements keep the normal spacing.

For an existing evaluation, rerun automatic `crop-labels`, `split-answers`, and
annotation generation with overwrite to populate and use this metadata. Do
this before handwritten review: existing Bnot/BGnot/BRnot annotation batches
save their rendering geometry and reuse it when read back, even after settings
change. Older batches without saved geometry keep their original 300 px offset.
Do not overwrite a batch that still contains handwritten corrections to import.

Each fragment is positioned independently, with its annotation rectangles shifted
by the same amount. Rectangles crossing page boundaries are split at those boundaries.
Narrow pages receive any additional width on the right, so left padding does not
stack with the configured allowance. BGnot pads narrower answer
images with white without resizing handwriting; fitting the wider document to
screen or paper may reduce the displayed scale, and return-PDF page breaks can
change. The extra space is also used by Anot and Bnot.

## Files and their sources

After grouped correction and review:

```sh
python -m copienator read-grouped Interro
python -m copienator giving-names Interro BGnot
```

The expected layout for a copy is:

```text
Interro/A Rendre/
└── Student Name (01)/
    ├── Student Name.jpg
    ├── Student Name.pdf
    ├── score.json
    ├── info.json
    └── answers/                 # when individual answer export is enabled
        ├── 001 - Ex 1.jpg
        └── 002 - Ex 2.jpg
```

The name comes from `Copies/Copie01.json` (`name`), with filename sanitization.
The copy ID distinguishes folders even when several copies have the same name.
`giving-names` links the full JPEG, PDF and score file (or copies them when links
are unavailable). It writes `info.json` and optionally composes the individual
answer JPEGs.

| Return file | Source under `BGnot/Copie01/` | Contents |
| --- | --- | --- |
| `Student Name.jpg` | `Concat.jpg` | Full continuous image of the compiled answers and corrections. |
| `Student Name.pdf` | `Concat_F.pdf` | Filtered, paginated correction with context, questions and solutions. |
| `score.json` | `score.json` | Per-question scores, including questions omitted from the filtered PDF. |
| `info.json` | `info.json` | Answer presence, empty-answer classification, PDF membership and score. |
| `answers/*.jpg` | Final per-label JPEGs selected by `info.json` | One annotated answer, including empty scans, with optional supplementary material. |

## Enabling or disabling outputs

Set these independent options in `config.py` (the defaults also apply when
absent from an older personal configuration):

```python
RETURN_JPEG_ENABLED = True
RETURN_PDF_ENABLED = True
RETURN_ANSWERS_ENABLED = False
RETURN_ANSWERS_CONTEXT = False
RETURN_ANSWERS_QUESTION = True
RETURN_ANSWERS_SOLUTION = False
```

The personal `config.py` enables `RETURN_ANSWERS_ENABLED`; the distributed
default is `False`. Set a full-output option to `False`, then rerun `giving-names` to omit that file from
`A Rendre`. For each prepared copy, any existing named return file of a disabled
type is removed, including a symlink or fallback copy. Its annotation source
remains intact. These options control return publication, not intermediate
rendering or scoring. `score.json` and `info.json` are always included and have
no disabling options.

Cleanup allows the JPEG to be absent when disabled, still requires `score.json`,
and preserves return PDFs when present.

## Individual answer JPEGs

With `RETURN_ANSWERS_ENABLED = True`, `giving-names` generates an `answers/`
subdirectory inside each student's return folder. It includes **every present
compiled answer**, even a perfect answer omitted from the filtered PDF. Scans
marked `empty-answer` are included; labels without an answer are excluded. Every JPEG
contains the final annotated student answer, including retained feedback and
extracted handwriting.

The three supplementary options independently prepend, in this order:

1. Applicable context PDFs, if `RETURN_ANSWERS_CONTEXT` is enabled.
2. The question, if `RETURN_ANSWERS_QUESTION` is enabled (the default).
3. The model solution, if `RETURN_ANSWERS_SOLUTION` is enabled.
4. The annotated student answer, always.

These use the same `Text2`/`Sol2` sources as the filtered PDF. Missing supplements
are skipped; an unreadable existing file fails the export. They are concatenated
vertically on white, without PDF pagination or its black/blue borders. The
options affect only these individual images, not the full JPEG or filtered PDF.
Disabling all supplements produces just the annotated answer.

Filenames use natural label order, a three-digit minimum sequence number, and a
sanitized label (`001 - Ex 1.jpg`). Numbering prevents filename collisions when
labels differ only by characters forbidden in filenames. JSON keys retain exact
labels. The managed `answers/` directory is replaced on successful generation,
so removed answers do not leave stale images; failures preserve the previous
directory. Disabling the option clears this directory on the next `giving-names`
run. Separate storage keeps these images out of the personal final-mark stamping
step, which reads only JPEGs directly inside the student's folder.

Recompile annotations once before exporting old evaluations with this option:
the compiler now saves every final answer block and writes `info.json`
next to them. For the grouped workflow, run `read-grouped`, then `giving-names`.
This avoids reconstructing a reviewed answer from outdated correction data.

## `info.json`: per-question information

Every label in `score.json` has an object containing exactly four fields:

```json
{
  "Ex 1": {"present": true, "not_empty": true, "touched": false, "score": "4"},
  "Ex 2": {"present": true, "not_empty": true, "touched": true, "score": "2"},
  "Empty": {"present": true, "not_empty": false, "touched": false, "score": "0"},
  "Absent": {"present": false, "not_empty": false, "touched": false, "score": ""}
}
```

- `present`: an answer entry exists for this student and label in the compilation
  data. A supplied answer judged empty still has `present: true`. A body removed by
  manual annotation edits has `present: false`.
- `not_empty`: the answer was not marked `empty-answer` and was successfully
  compiled. Absent answers have `not_empty: false`. When individual export is
  enabled, every `present` answer gets a JPEG, including those with `not_empty: false`.
  Image retention alone does not change the empty classification. These fields
  describe the answer regardless of export settings.
- `touched`: the answer appears in the compiled filtered `Concat_F.pdf`, using
  the actual selection including handwriting and selective redo preservation.
  It is not a flag for human edits. Empty and absent answers have `false`.
- `score`: the same value as `score.json` (normally a numeric string, or `""`
  for an unpopulated score). Editing scores requires recompilation to update the
  images; return publication uses current `score.json` values for this field.

`info.json` is always exported, even when individual JPEGs or the named PDF are
disabled. `touched` tracks BGnot filtered-PDF membership. In `Anot` and
`Bnot` flows, all `touched` values remain false. Bnot also produces a filtered,
paginated PDF with context, questions and solutions, alongside its filtered JPEG.

This file replaces `touched.json` and the internal `answer_labels.json` manifest.
Recompile old annotations, then run `giving-names`; successful regeneration and
publication remove the obsolete files from their respective folders. Missing
or malformed metadata requires recompilation rather than guessing answer presence
from scores. Cleanup preserves `info.json` alongside `score.json`.

## JPEG: the full compiled correction

The JPEG stacks the rendered answer blocks vertically in natural label order
(for example, Ex 2 precedes Ex 10). Each block contains the scanned answer, its
label and score, retained global and local feedback, and detected handwritten
review annotations. Local feedback can include red rectangles and comments in
the left margin. Review checkboxes are applied as actions rather than reproduced
as controls; internal error labels are hidden during recompilation.

The result is one RGB image of variable height, with no page breaks. It contains
all successfully compiled answer blocks, including answers scored 4 with no
remaining feedback. “Full” refers to those answer blocks, not the original scan
pages or every question in the statement. Missing/unrenderable answers cannot
be included. Empty-answer scans are retained individually but excluded from
the combined and filtered documents.
The grouped compiler refuses to publish a new set when compilation is incomplete.

The JPEG does not prepend the question, context or model solution PDFs.

## PDF: a different selection and layout

**The PDF is not a PDF conversion of the JPEG.** During an ordinary full grouped
recompilation, an answer is omitted only when all three conditions hold:

- Its score is at least 4.
- Every feedback item is marked `to_delete` (also true for an empty feedback list).
- There are no significant detected handwritten annotations for that answer.

Thus, a 4/4 answer with retained feedback or handwriting still appears. Scores
for omitted answers remain in `score.json`, and their answer blocks remain in
the JPEG. Handwriting significance currently means more than 20 pixels with
alpha greater than 50 in the extracted annotation layer.

For each retained answer, the PDF stacks the following available material:

1. Applicable context PDFs from `Text2/CTXT first_label -> last_label.pdf`.
2. The question from `Text2/<label>.pdf`.
3. The model solution from `Sol2/<label>.pdf`.
4. The same compiled answer block used for the JPEG.

Missing supplementary PDFs are skipped. Each distinct context appears only once
per student PDF, before its first retained question. Duplicate detection uses
extracted PDF text with Unicode and whitespace normalization, including contexts
stored under different filenames or validity ranges. Numbers, mathematical
symbols and case remain significant. Contexts without extractable text use a
PDF-content hash instead, so unrelated image-only contexts are not collapsed.
The same rule applies to selective redo and manual annotation modifications.
Each question's complete group stays together on one page; groups
are packed until the next would exceed the target height. An oversized group
gets its own taller page, rather than being split. Pages are raster images saved
as PDF at 100 dpi, with variable heights, not fixed A4 sheets or searchable text.

The target height is `int(max_image_width * 1.414 * 1.25)` pixels. Small white
margins are added on the left and above/below the page. The question receives
a black border and the model solution a blue border. Context and student answer
blocks receive neither border, including when supplementary files are missing.

If nothing survives filtering, `read-grouped` removes old `Concat_F` outputs and
does not create a new PDF. During a selective `--refaire` merge, saved answer
images outside the selection are kept in the filtered output without reapplying
the perfect-answer filter; the resulting PDF can therefore retain more answers
than a full recompilation.

## JSON: per-question scores

`score.json` is a flat JSON object keyed by the exact question labels. For example
(illustrative data):

```json
{
  "Ex 1 : 1)": "4",
  "Ex 1 : 2)": "2.5",
  "Ex 2": ""
}
```

Values are **strings**, including numeric scores. The normal question scale is
0 to 4. `""` means no score was populated for that label; it is distinct from
`"0"`. The compiler initializes all labels from the evaluation's `labels` file
to `""`, then fills processed scores. This file contains no student identity,
feedback, annotation coordinates, grading weights, or overall final mark.

Scores incorporate review checkbox changes and, when requested, existing score
overrides via `read-grouped --update-score` (or `read-annotations --update-score`
for `Bnot`). Editing the JSON alone does not update the rendered scores. Overrides
are read from the annotation source folder: a return symlink points there, but
an independent fallback copy does not. Successful annotation rereads automatically
refresh individual `answers/` images in existing return folders, matching their
trailing copy ID and preserving manual names and other return files. This also
applies the selected supplementary-material options. Copies whose regeneration
fails retain their existing exports. Rerun `giving-names` to refresh other copied
return files.

With `--update-score`, both readers save a small content fingerprint per copy
after successful regeneration and answer publication. Subsequent runs skip
unchanged copies before scanning or rendering PDFs. A single edited `score.json`
therefore rebuilds only its copy. Changed shared documents or grouped annotation
inputs conservatively invalidate all copies; changed export options also trigger
regeneration. Missing generated files and failed runs are retried. The first run
establishes the fingerprints. Reads without `--update-score` always regenerate;
selective `--refaire` reads do not use this optimization.

`update-ods` reads these return JSON files; empty values become `NT` in the normal
per-question export. Its `--sum` option sums numeric values and skips nonnumeric
ones. Weighting and the final overall mark belong to the separate grading flow.

## Availability and later steps

- `giving-names` accepts `BGnot`, `Bnot`, or `Anot`. It selects the requested
  source if `score.json` and either `Concat.jpg` or `info.json` exist,
  otherwise it tries `Anot/CopieXX`. This also permits returns for an entirely
  empty copy. It links the PDF only if `Concat_F.pdf` exists. The normal
  `Bnot` reader produces both `Concat_F.jpg` and `Concat_F.pdf` for nonempty
  filtered answers. Simple annotation produces `Concat.jpg` without a filtered PDF.
- Preparing returns removes disabled named outputs, and removes older named
  outputs whose source is now absent, including broken symlinks.
- `add-final-score` writes to `FINAL_SCORE_OUTPUT_DIR/<evaluation>/`. It stamps
  the overall mark from the configured ODS in red at the JPEG's upper right,
  rounded down to one decimal place. It copies PDFs unchanged, together with
  `score.json`, `info.json` and the optional `answers/` directory. Alongside the
  student folders it writes `stat.json`, averaging each label across the current
  `A Rendre/*/score.json` files and rounding upward to one decimal place, stored
  as strings like the original scores (for example `"2.7"`). Blank
  and missing scores are excluded; labels without numeric scores retain `""`.
  It does not add the overall mark to the files inside `A Rendre`.
- The `clean` command retains return images (including individual answers), PDFs,
  `score.json` and `info.json`, materializing
  retained symlinks before deleting their sources.

## Implementation references

- [Naming and return links](../copienator/commands/giving_names.py)
- [Individual answer publication](../copienator/return_answers.py)
- [Grouped compilation, filtering and PDF pagination](../copienator/commands/reading_grouped_annotations.py)
- [Answer rendering](../copienator/commands/annotating.py)
- [Handwriting detection and per-copy compilation](../copienator/commands/reading_annotations.py)
- [Score and feedback actions](../copienator/annotation_actions.py)
- [Context, question and solution lookup](../copienator/utils.py)
- [ODS export](../copienator/commands/update_ods.py)
- [Final-mark stamping](../copienator/commands/add_final_score.py)
- [Cleanup retention](../copienator/commands/clean.py)


## Manual modifications after annotation reading

`modify-annotations EVALUATION --annotation-dir BGnot` (or `Bnot`) applies
`annotation_modifications.txt`. The GUI exposes it between annotation reading
and return preparation. Its `### Source: BGnot`/`Bnot` line must match the mode.
Instruction operators follow manual resolution syntax but act on reviewed raster
**bodies**, excluding headers and all supplementary pages. Headers and scores stay
with their labels. Missing target bodies are allowed; their new header identifies
the target and its existing score, without inventing feedback or a score.

Both readers record `annotation_baseline.json` and lossless hidden
`.annotation-base-*.png` images in each copy's annotation directory. The JSON
stores the exact header height, baseline answer metadata, and filtered membership.
Older outputs require one full annotation read before body editing. Legacy
selective redo still works, but cannot infer missing header boundaries.

Each application rebuilds from that baseline and publishes all affected copies
and the mode's `annotation_modifications_applied.txt` in one rollback-capable
transaction. Reapplying is idempotent; removing an instruction restores its prior
content and membership. Annotation rereads refresh the baseline and replay the
applied snapshot, not the draft. Selective redo preserves unaffected baseline
layers so previous transfers are never included twice.

The operation rebuilds per-label JPEGs, `info.json`, `Concat.jpg`, and the filtered
BGnot PDF or Bnot JPEG and PDF. Every changed non-empty body is included in the filtered
output, with its destination label's supplementary material. `score.json` is not
modified. The GUI marks downstream return preparation stale. Rerun `giving-names`
without `--update` to refresh both full outputs and individual answer exports.