Compare commits

..
9 Commits
Author SHA1 Message Date
sebastien 0a453403cf améliorations diverses 2026-09-17 22:19:05 +02:00
sebastien 5b8215e7f5 modest improvement to cropping (safer) 2026-09-15 15:13:41 +02:00
sebastien 9b22a8a137 Miscs improvements (Interro02) 2026-09-15 14:18:22 +02:00
sebastien 0a86403ca6 miscs (Interro02) : horizontal cutting resolution 2026-09-14 22:20:46 +02:00
sebastien 5080274e8f small changes to prompts 2026-09-14 09:49:01 +02:00
sebastien d60d5479d6 Miscs personal GUI improvement 2026-09-14 09:42:13 +02:00
sebastien e8b8a11c5b Configuration de l'output final 2026-09-12 23:56:07 +02:00
sebastien 060859ddef cropping summaries 2026-09-10 20:03:23 +02:00
sebastien dfce79f240 GUI documentation 2026-09-10 07:23:39 +02:00
61 changed files with 5012 additions and 369 deletions
+2
View File
@@ -1,4 +1,6 @@
OLD/
/Interro*/
/DS*/
__pycache__/
*.py[cod]
.venv/
+9 -3
View File
@@ -197,13 +197,17 @@ racine du projet). Dans la console, les boutons de copie et le clic droit
permettent de copier la sélection ou toute la sortie ; =Ctrl+C= et
=Ctrl+A= sont également disponibles (=Cmd= sous macOS).
Pendant =Découper la marge des labels=, =s= signale la copie en erreur
et passe à la suivante. =Entrée= valide et enregistre la découpe affichée.
Pendant =Découper une partie à gauche pour détection des labels=, =n= décale
la zone de 50 px vers la droite, =N= de 100 px, =t= de 50 px vers la
gauche et =l= l’élargit de 50 px. =1= utilise les pages entières. =s=
signale la copie en erreur et passe à la suivante ; =Entrée= valide et
enregistre la découpe affichée.
Une copie ignorée conserve ses anciennes découpes. Les signalements sont
conservés dans =.copienator/copy_errors.json=, même après fermeture.
Dans =Séparer et réordonner les pages=, =Traiter les copies signalées=
reprend ces copies à partir des originaux conservés. Le même bouton dans
=Découper la marge des labels= reprend leurs marges ; chaque signalement
=Découper une partie à gauche pour détection des labels= reprend leur
découpage ; chaque signalement
est effacé seulement après validation et enregistrement avec =Entrée=.
Fermer la fenêtre, appuyer à nouveau sur =s= ou rencontrer une erreur
conserve le signalement. Les commandes =page-split= et =crop-labels=
@@ -255,6 +259,8 @@ Les chemins des étapes personnelles peuvent être adaptés avec
* Documentation complémentaire
- [[file:docs/final_output.md][Fichiers finaux dans A Rendre]] : contenu du JPEG, sélection et
pagination du PDF, JPEG par réponse, =score.json=, =info.json= et diffusion.
- [[file:Script.org][Référence des étapes et des scripts]] : commandes, arguments,
prérequis, fichiers produits et parcours alternatifs.
- [[file:Architecture.org][Architecture et conventions de développement]] : API commune,
+28 -3
View File
@@ -290,8 +290,31 @@ OU
3. =python -m copienator giving-names Interro BGnot=
Crée un dossier =A Rendre= avec des liens symboliques vers
+ La copie à rendre
+ =<nom>.jpg= : la correction complète concaténée (=Concat.jpg=)
+ =<nom>.pdf=, si disponible : la correction filtrée, avec contexte,
énoncés et solutions dans le parcours =BGnot= (=Concat_F.pdf=)
+ un fichier =score.json= qui contient les notes par question
+ =info.json= : pour chaque label, =present= (réponse fournie),
=not_empty= (réponse non vide compilée), =touched= (présente dans le
PDF filtré) et =score= (même valeur que dans =score.json=)
+ =answers/*.jpg=, si =RETURN_ANSWERS_ENABLED= est activé : un JPEG
par réponse non vide, contenant toujours la réponse annotée
Le PDF n'est donc pas une conversion du JPEG. Voir la
[[file:docs/final_output.md][documentation des fichiers finaux]] pour les règles de sélection,
la pagination et les différences entre parcours.
=RETURN_JPEG_ENABLED= et =RETURN_PDF_ENABLED= dans =config.py=
permettent de désactiver ces sorties séparément (activées par défaut).
Relancer =giving-names= retire les fichiers nommés désactivés en
conservant leurs sources. =score.json= et =info.json= sont toujours inclus.
=RETURN_ANSWERS_ENABLED= est désactivé par défaut, activé dans la
configuration personnelle. =RETURN_ANSWERS_CONTEXT=,
=RETURN_ANSWERS_QUESTION= et =RETURN_ANSWERS_SOLUTION= choisissent les
documents ajoutés avant chaque réponse (seule la question est activée
par défaut). Recompiler les anciennes annotations une fois avant cet
export pour produire les images finales et leurs métadonnées.
Si un nom est =Unknown= : renommer à la main le dossier et le fichier dedans.
4. Éventuellement, faire des modifications manuelles aux =score.json=.
@@ -328,12 +351,14 @@ conserve que :
+ le résultat final =correction.json= ;
+ les journaux de =.copienator/logs= et les journaux placés à la
racine, comme =correction_log= ;
+ les images et fichiers =score.json= présents dans =A Rendre=.
+ les images (y compris =answers=), PDF et fichiers =score.json= et
=info.json= présents dans =A Rendre=.
Les liens symboliques conservés dans =A Rendre= sont remplacés par de
véritables fichiers avant la suppression de leurs cibles. La commande
refuse de démarrer si les copies traitées, =correction.json= ou une
image/un score d'élève sont absents. Elle affiche d'abord un résumé et
image/un score d'élève sont absents (l'image est facultative si
=RETURN_JPEG_ENABLED= vaut =False=). Elle affiche d'abord un résumé et
demande de saisir le nom de l'évaluation pour confirmer.
Dans le GUI, cette commande apparaît comme dernière étape facultative
+5 -2
View File
@@ -6,6 +6,7 @@ from pathlib import Path
from typing import Any
from .json_io import read_json
from .feedback_boxes import valid_feedback_box
Log = Callable[[str], None]
@@ -25,8 +26,10 @@ def apply_checkbox_actions(
continue
result = labels_data[label]["result"]
feedbacks = result.get("feedback", [])
global_feedbacks = [item for item in feedbacks if not item.get("box_2d")]
local_feedbacks = [item for item in feedbacks if item.get("box_2d")]
# Match the renderer's fallback for invalid boxes so checkbox indices
# still address the right comment when returned annotations are read.
global_feedbacks = [item for item in feedbacks if not valid_feedback_box(item.get("box_2d"))]
local_feedbacks = [item for item in feedbacks if valid_feedback_box(item.get("box_2d"))]
local_feedbacks.sort(key=lambda item: item["box_2d"][0])
for action in label_actions:
+2 -1
View File
@@ -8,6 +8,7 @@ from typing import Any
from PIL import Image
from .json_io import read_json
from .feedback_boxes import valid_feedback_box
from .workspace import EvaluationWorkspace
AnnotationData = dict[str, dict[str, dict[str, Any]]]
@@ -69,7 +70,7 @@ def _scaled_result(result: dict[str, Any], coordinates: GroupCoordinates | None)
return scaled
for feedback in scaled.get("feedback", []):
box = feedback.get("box_2d")
if not box or len(box) != 4:
if not box or not valid_feedback_box(box):
continue
box[0] = int(box[0] * coordinates.height) // 1000
box[2] = int(box[2] * coordinates.height) // 1000
+22
View File
@@ -0,0 +1,22 @@
from collections.abc import Collection, Mapping
from typing import Any
def build_answer_info(
scores: Mapping[str, str],
present_labels: Collection[str],
rendered_labels: Collection[str],
touched: Mapping[str, bool] | None = None,
) -> dict[str, dict[str, Any]]:
"""Describe every question using the answers actually compiled for a copy."""
present = set(present_labels)
rendered = set(rendered_labels)
return {
label: {
"present": label in present,
"not_empty": label in rendered,
"touched": (touched or {}).get(label, False),
"score": score,
}
for label, score in scores.items()
}
+64 -25
View File
@@ -7,11 +7,17 @@ from pathlib import Path
import pandas as pd
from PIL import Image, ImageDraw, ImageFont
from copienator.configuration import FINAL_SCORE_FONT_PATH, FINAL_SCORE_ODS_PATH, FINAL_SCORE_OUTPUT_DIR
from copienator.configuration import (
FINAL_SCORE_FONT_PATH,
FINAL_SCORE_HISTOGRAM_PATH,
FINAL_SCORE_ODS_PATH,
FINAL_SCORE_OUTPUT_DIR,
)
# Configuration constants
ODS_PATH = Path(FINAL_SCORE_ODS_PATH).expanduser()
OUTPUT_DIR = Path(FINAL_SCORE_OUTPUT_DIR).expanduser()
HISTOGRAM_PATH = Path(FINAL_SCORE_HISTOGRAM_PATH).expanduser()
def score_font(size):
@@ -33,6 +39,35 @@ def get_rounded_score(score):
except (ValueError, TypeError):
return None
def copy_return_artifacts(source_dir: Path, destination_dir: Path) -> None:
"""Copy the metadata and optional individual answers for one student."""
for filename in ("score.json", "info.json"):
source = source_dir / filename
if source.is_file():
shutil.copy2(source, destination_dir / filename)
else:
print(f"Warning: Missing '{source}'.")
answers_source = source_dir / "answers"
answers_destination = destination_dir / "answers"
if answers_destination.is_symlink():
answers_destination.unlink()
elif answers_destination.is_dir():
shutil.rmtree(answers_destination)
if answers_source.is_dir():
shutil.copytree(answers_source, answers_destination)
def copy_histogram(output_dir: Path) -> None:
"""Copy the score histogram beside the per-student output folders."""
if not HISTOGRAM_PATH.is_file():
print(f"Warning: Missing histogram '{HISTOGRAM_PATH}'.")
return
destination = output_dir / "histogramme.pdf"
shutil.copy2(HISTOGRAM_PATH, destination)
print(f"Copied histogram: {destination}")
def process_images(base_dir, output_dir):
# 1. Load Data
try:
@@ -56,60 +91,64 @@ def process_images(base_dir, output_dir):
print(f"Error: Directory '{search_path}' not found.")
sys.exit(1)
for img_path in sorted(search_path.glob("*/*.jpg")):
student_name = img_path.stem # Filename without extension
for student_source in sorted(path for path in search_path.iterdir() if path.is_dir()):
image_paths = sorted(student_source.glob("*.jpg"))
pdf_paths = sorted(student_source.glob("*.pdf"))
media_paths = image_paths or pdf_paths
if not media_paths:
print(f"Error: No JPG or PDF found in '{student_source}'.")
continue
student_name = media_paths[0].stem
student_output = output_dir / student_name
student_output.mkdir(parents=True, exist_ok=True)
# Remove files produced by the former flat output layout when migrating
# an existing export directory.
for suffix in (".jpg", ".pdf"):
legacy_output = output_dir / f"{student_name}{suffix}"
if legacy_output.is_file() or legacy_output.is_symlink():
legacy_output.unlink()
copy_return_artifacts(student_source, student_output)
# 4. Find Score
if student_name not in score_db:
print(f"Error: Student '{student_name}' not found in ODS file.")
continue
else:
raw_score = score_db[student_name]
score = get_rounded_score(raw_score)
if score is None:
print(f"Error: Invalid score '{raw_score}' for '{student_name}'.")
continue
# 5. Process Image
else:
# 5. Process Images
for img_path in image_paths:
try:
with Image.open(img_path) as img:
img = img.convert("RGB")
draw = ImageDraw.Draw(img)
width, height = img.size
width, _height = img.size
# Dynamic font size (15% of image height)
font_size = int(width * 0.08)
font = score_font(font_size)
text = str(score)
# Calculate text size and position (Top Right)
bbox = draw.textbbox((0, 0), text, font=font)
text_w = bbox[2] - bbox[0]
text_h = bbox[3] - bbox[1]
# 30px padding
# 30px padding, top right.
x = width - text_w - 30
y = 30
# Draw Text (Red)
draw.text((x, y), text, fill=(255, 0, 0), font=font)
# Save
save_path = output_dir / f"{student_name}.jpg"
img.save(save_path)
img.save(student_output / img_path.name)
print(f"Processed: {student_name} -> {score}")
except Exception as e:
print(f"Error processing image for '{student_name}': {e}")
for pdf_path in sorted(search_path.glob("*/*.pdf")):
student_name = pdf_path.stem # Filename without extension
save_path = output_dir / f"{student_name}.pdf"
for pdf_path in pdf_paths:
shutil.copy2(pdf_path, student_output / pdf_path.name)
shutil.copy(str(pdf_path), str(save_path))
copy_histogram(output_dir)
def main(argv=None):
+17
View File
@@ -30,7 +30,9 @@ from copienator import (
workspace_from_args,
)
from copienator.annotation_data import load_annotation_data
from copienator.answer_info import build_answer_info
from copienator.filesystem import staged_directory
from copienator.feedback_boxes import valid_feedback_box
from copienator.utils import natural_key
MARGIN_LEFT = 300
@@ -325,6 +327,16 @@ def compose_label_image(base_img, label, result, hmin,
# Filter deleted items (used by reading_annotations.py)
feedbacks = [f for f in feedbacks if "to_delete" not in f]
# Never guess where an invalid rectangle belongs, or lose its comment.
# Use a copy so rendering cannot mutate saved correction data.
normalized = []
for feedback in feedbacks:
box = feedback.get("box_2d")
if box is not None and not valid_feedback_box(box):
print(f"Warning: Copie{with_id or ''} {label}: invalid feedback box {box!r}; displaying the comment without a rectangle.")
feedback = {**feedback, "box_2d": None}
normalized.append(feedback)
feedbacks = normalized
global_fb = [f for f in feedbacks if not f.get('box_2d')]
local_fb = [f for f in feedbacks if f.get('box_2d')]
@@ -446,6 +458,7 @@ def process_student(student_id, labels_data, root_dir, all_labels, overwrite):
with staged_directory(output_dir) as staging:
d_notes = dict.fromkeys(all_labels, "")
label_images = []
answer_labels = []
sorted_labels = sorted(labels_data.items(), key=lambda item: natural_key(item[0]))
for label, content in sorted_labels:
@@ -475,8 +488,12 @@ def process_student(student_id, labels_data, root_dir, all_labels, overwrite):
final_img.save(staging / f"{label}.jpg")
if result.get('error', "") != "empty-answer":
label_images.append(final_img)
answer_labels.append(label)
atomic_write_json(staging / "score.json", d_notes)
atomic_write_json(staging / "info.json", build_answer_info(
d_notes, labels_data, answer_labels
))
if label_images:
max_w = max(image.width for image in label_images)
total_h = sum(image.height for image in label_images)
+45
View File
@@ -9,10 +9,13 @@ from google import genai
from copienator import configuration as config
from copienator import (
CliError,
EvaluationWorkspace,
ExitCode,
atomic_write_bytes,
execute,
read_json,
standard_parser,
workspace_from_args,
)
@@ -43,6 +46,41 @@ def list_jobs(*, client=None) -> ExitCode:
return ExitCode.SUCCESS
def check_evaluation_jobs(workspace: EvaluationWorkspace, *, client=None) -> ExitCode:
"""Only report readiness after checking every job recorded for this evaluation."""
if not workspace.batch_jobs_file.is_file():
print("Impossible de vérifier les batchs : batch_jobs.json est absent.")
return ExitCode.PARTIAL
manifest = read_json(workspace.batch_jobs_file)
jobs = manifest.get("jobs") if isinstance(manifest, dict) else None
if not isinstance(jobs, dict):
raise CliError(f"Invalid batch manifest: {workspace.batch_jobs_file}")
if not jobs:
print("Aucun job enregistré pour cette évaluation.")
return ExitCode.PARTIAL
if any(not isinstance(entry, dict) or not isinstance(entry.get("name"), str)
or not entry["name"].strip() for entry in jobs.values()):
raise CliError(f"Invalid batch job in {workspace.batch_jobs_file}")
client = client or _client()
ready = True
for tier, entry in jobs.items():
job = client.batches.get(name=entry["name"])
state = job.state.name if hasattr(job.state, "name") else job.state
print(f"{tier}{entry['name']}: {state}")
if state != "JOB_STATE_SUCCEEDED":
ready = False
if getattr(job, "error", None):
print(f" Erreur : {job.error}")
elif not getattr(getattr(job, "dest", None), "file_name", None):
ready = False
print(" Le fichier de résultats nest pas encore disponible.")
if ready:
print("Tous les batchs de l’évaluation ont réussi. Les résultats sont prêts à récupérer.")
return ExitCode.SUCCESS
print("Les résultats ne sont pas tous prêts. Consultez à nouveau cette étape plus tard.")
return ExitCode.PARTIAL
def download_job(
job_name: str,
*,
@@ -73,6 +111,8 @@ def build_parser() -> argparse.ArgumentParser:
parser = standard_parser("List or download Gemini correction batch jobs")
parser.add_argument("--download", metavar="JOB_NAME")
parser.add_argument("--output", type=Path, help="Downloaded JSONL destination")
parser.add_argument("--evaluation", type=Path,
help="Check readiness of jobs recorded in this evaluation's batch_jobs.json")
return parser
@@ -80,6 +120,11 @@ def main(argv: Sequence[str] | None = None) -> int:
parser = build_parser()
def handle(args: argparse.Namespace) -> ExitCode:
if args.evaluation is not None:
if args.download or args.output is not None:
raise CliError("--evaluation cannot be combined with --download or --output",
ExitCode.INVALID_ARGUMENTS)
return check_evaluation_jobs(workspace_from_args(args))
if args.output is not None and not args.download:
raise CliError("--output requires --download", ExitCode.INVALID_ARGUMENTS)
if args.download:
+7 -3
View File
@@ -13,6 +13,7 @@ from copienator import (
CliError,
EvaluationWorkspace,
ExitCode,
configuration,
evaluation_parser,
execute,
workspace_from_args,
@@ -83,15 +84,18 @@ def _return_artifacts(workspace: EvaluationWorkspace) -> list[Path]:
path for path in files if path.suffix.casefold() in IMAGE_SUFFIXES
]
scores = [path for path in files if path.name.casefold() == "score.json"]
if not images or not scores:
pdfs = [path for path in files if path.suffix.casefold() == ".pdf"]
if (configuration.RETURN_JPEG_ENABLED and not images) or not scores:
missing = []
if not images:
if configuration.RETURN_JPEG_ENABLED and not images:
missing.append("image")
if not scores:
missing.append("score.json")
incomplete.append(f"{directory.name} ({', '.join(missing)})")
artifacts.extend(images)
artifacts.extend(scores)
artifacts.extend(pdfs)
artifacts.extend(path for path in files if path.name.casefold() == "info.json")
if incomplete:
details = "\n".join(f" - {item}" for item in incomplete)
@@ -219,7 +223,7 @@ def print_plan(workspace: EvaluationWorkspace, plan: CleanupPlan, *, verbose: bo
print(f" - {statement_count} textual statement files")
print(" - correction.json")
print(f" - {log_count} log files")
print(f" - {return_count} image/score artifacts in A Rendre")
print(f" - {return_count} image/PDF/score artifacts in A Rendre")
print(
f"Will delete {len(plan.deleted_files)} files and "
f"{len(plan.deleted_directories)} directories "
+202 -25
View File
@@ -5,6 +5,7 @@ import base64
import concurrent.futures
import json
import os
import signal
import shlex
import shutil
import sys
@@ -18,6 +19,7 @@ from google import genai
from copienator import configuration as config
from copienator.commands import grouping
from copienator import prompting
from copienator.feedback_boxes import valid_feedback_box
from copienator import (
CliError,
EvaluationWorkspace,
@@ -44,15 +46,22 @@ COPIES_DIR = Path()
GROUPS_DIR = Path()
output_path = Path()
progress_path = Path()
pending_responses_path = Path()
tasks: list[tuple] = []
tasks_to_process: list[tuple] = []
results: dict = {}
completed_tasks: list = []
errors_summary: list = []
pending_responses: dict[str, str] = {}
overwrite = False
limit = None
client = None
start_time = 0.0
stop_requested = threading.Event()
class CorrectionStopRequested(Exception):
"""Raised in a worker before it starts another Gemini request."""
# --- Thread-safe Logging ---
log_lock = threading.Lock()
@@ -81,9 +90,30 @@ def flush_thread_log(tid=None):
f.write("\n".join(thread_logs[tid]) + "\n\n")
thread_logs[tid].clear()
def report_group_progress(completed: int, total: int) -> None:
"""Emit a stable, human-readable progress line for the GUI and CLI."""
print(
f"[Progression correction] Groupes traités : {completed}/{total}",
flush=True,
)
def request_graceful_stop(_signum=None, _frame=None) -> None:
"""Stop scheduling Gemini calls while allowing in-flight calls to finish."""
if not stop_requested.is_set():
stop_requested.set()
print(
"\n[Interruption] Arrêt des nouveaux appels Gemini demandé. "
"Attente des appels en cours et sauvegarde de leurs résultats…",
flush=True,
)
# --- Lock for thread-safe file writing ---
io_lock = threading.Lock()
pro_lock = threading.Lock()
group_index_lock = threading.Lock()
reserved_group_indices: dict[str, int] = {}
pro_count = 0
flash_count = 0
pro_quota_exhausted = False
@@ -137,6 +167,7 @@ def configure_runtime(
api_client=None,
) -> None:
global INPUT_DIR, COPIES_DIR, GROUPS_DIR, output_path, progress_path
global pending_responses_path, pending_responses
global tasks, tasks_to_process, results, completed_tasks, errors_summary
global overwrite, limit, client, start_time
global pro_count, flash_count, pro_quota_exhausted
@@ -146,17 +177,21 @@ def configure_runtime(
GROUPS_DIR = workspace.groups_dir
output_path = workspace.correction_file
progress_path = workspace.correction_progress_file
pending_responses_path = workspace.correction_pending_responses_file
tasks = list(discovered_tasks)
overwrite = bool(args.overwrite)
limit = args.limit
start_time = time.time()
errors_summary = []
pending_responses = {}
completed_tasks = []
results = {label: [] for _file, label in tasks}
thread_logs.clear()
reserved_group_indices.clear()
pro_count = 0
flash_count = 0
pro_quota_exhausted = False
stop_requested.clear()
if not overwrite:
if progress_path.is_file():
@@ -170,6 +205,20 @@ def configure_runtime(
raise TypeError("correction.json must contain a JSON object")
results = loaded_results
# A response saved during a graceful interruption is not a completed
# correction. Resume its auxiliary checks even with --overwrite instead of
# paying for the same primary request again.
if pending_responses_path.is_file():
loaded_pending = read_json(pending_responses_path)
if not isinstance(loaded_pending, dict) or not all(
isinstance(key, str) and isinstance(value, str)
for key, value in loaded_pending.items()
):
raise TypeError(
"correction_pending_responses.json must contain a JSON object"
)
pending_responses = loaded_pending
completed_set = {(str(file_path), label) for file_path, label in completed_tasks}
tasks_to_process = [
task for task in tasks if (str(task[0]), task[1]) not in completed_set
@@ -180,7 +229,11 @@ def configure_runtime(
def reset_workspace(workspace: EvaluationWorkspace) -> None:
"""Apply the explicitly requested correction reset."""
print("--- Running Reset ---")
for path in (workspace.correction_file, workspace.correction_progress_file):
for path in (
workspace.correction_file,
workspace.correction_progress_file,
workspace.correction_pending_responses_file,
):
if path.exists():
path.unlink()
print(f"Deleted: {path}")
@@ -209,6 +262,8 @@ def call_gemini_with_retries(model_id, contents, config,
delays = [60, 300]
for attempt in range(3):
if stop_requested.is_set():
raise CorrectionStopRequested
# Switch to fallback immediately if quota was exhausted by another thread
if model_id == MODEL_ID_pro and pro_quota_exhausted and fallback_model_id:
model_id = fallback_model_id
@@ -224,6 +279,8 @@ def call_gemini_with_retries(model_id, contents, config,
full_response_text += chunk.text
return full_response_text
except Exception as e:
if stop_requested.is_set():
raise CorrectionStopRequested from e
error_msg = str(e).lower()
is_quota_error = "429" in error_msg or "quota" in error_msg or "exhausted" in error_msg
is_minute_limit = "minute" in error_msg or "rpm" in error_msg or "tpm" in error_msg
@@ -235,7 +292,8 @@ def call_gemini_with_retries(model_id, contents, config,
wait_time = float(retry_match.group(1)) + 1.0 if retry_match else delays[attempt]
tprint(f"\tGemini Pro minute limit hit. Waiting {wait_time:.1f}s...")
time.sleep(wait_time)
if stop_requested.wait(wait_time):
raise CorrectionStopRequested
continue # Retry same model
# Immediately fallback to Flash without waiting if it's a Pro quota error
@@ -247,7 +305,8 @@ def call_gemini_with_retries(model_id, contents, config,
if attempt < 2:
tprint(f"\tGemini API failure: {e}. Retrying in {delays[attempt]} seconds...")
time.sleep(delays[attempt])
if stop_requested.wait(delays[attempt]):
raise CorrectionStopRequested
else:
tprint(f"\tGemini API failure: {e}. Maximum retries reached.")
raise
@@ -267,6 +326,9 @@ def correct_boxes_with_gemini(pid, label, pdf_path, original_feedbacks,
for f in corrected_feedbacks:
b = f.get("box_2d")
if b:
if not valid_feedback_box(b) or any(value < 0 or value > 1000 for value in b):
f["box_2d"] = None
continue
ymin_s, xmin_s, ymax_s, xmax_s = b
# Y mapping: Add the group Y-offset (yming), then normalize to total_height
@@ -290,6 +352,16 @@ def get_next_group_idx(label):
if not existing: return 0
return max([int(f.stem.split("_")[1]) for f in existing])
def reserve_next_group_idx(label: str) -> int:
"""Reserve a unique zero-based group index for this correction run."""
with group_index_lock:
if label not in reserved_group_indices:
reserved_group_indices[label] = get_next_group_idx(label)
idx = reserved_group_indices[label]
reserved_group_indices[label] = idx + 1
return idx
def handle_label_errors(pid, label, res, pdf_path):
"""Handles Gemini labeling errors, moves/copies files, and returns new tasks."""
new_tasks = []
@@ -330,7 +402,7 @@ def handle_label_errors(pid, label, res, pdf_path):
if pdf_path != old_pdf_path:
shutil.move(str(pdf_path), str(old_pdf_path))
idx = get_next_group_idx(new_label)
idx = reserve_next_group_idx(new_label)
height = grouping.get_pdf_height(str(new_pdf_path))
grouping.create_jpg(new_label, idx, [(pid, str(new_pdf_path), height)], GROUPS_DIR)
tprint(f"\t\tMaking {new_label} group {idx+1}")
@@ -342,6 +414,8 @@ def handle_label_errors(pid, label, res, pdf_path):
tprint(f"\tHandling additional-answer for {pid} {label}")
try:
add_labels = json.loads(call_gemini_with_retries(MODEL_ID_flash, contents, config))
except CorrectionStopRequested:
raise
except Exception: # noqa: BLE001 - invalid auxiliary model response
add_labels = []
@@ -362,7 +436,7 @@ def handle_label_errors(pid, label, res, pdf_path):
if not base_add_pdf_path.exists() and not add_pdf_path.exists():
shutil.copy(str(pdf_path), str(add_pdf_path))
tprint(f"\t\tCopying Copie{pid} : {label} -> {add_label}")
idx = get_next_group_idx(add_label)
idx = reserve_next_group_idx(add_label)
tprint(f"\t\tMaking {add_label} group {idx+1}")
height = grouping.get_pdf_height(str(add_pdf_path))
grouping.create_jpg(add_label, idx, [(pid, str(add_pdf_path), height)], GROUPS_DIR)
@@ -400,6 +474,9 @@ def process_single_task(task_tuple, precomputed_response=None):
total_height = group_data[-1][2]
use_flash = n >= 4 or total_height <= 500
if precomputed_response is None and stop_requested.is_set():
raise CorrectionStopRequested
# Only apply limits and counts if we are making a live call
if precomputed_response is None:
if not use_flash:
@@ -420,9 +497,11 @@ def process_single_task(task_tuple, precomputed_response=None):
model_to_use = MODEL_ID_flash if use_flash else MODEL_ID_pro
if precomputed_response:
tprint(f"Using batched response for: {label} {group_name}")
tprint(f"Using saved response for: {label} {group_name}")
full_response_text = precomputed_response
else:
if stop_requested.is_set():
raise CorrectionStopRequested
tprint(f"Asking Gemini {'Flash' if use_flash else 'Pro '}: {label} {group_name}")
full_response_text = call_gemini_with_retries(model_to_use, contents, config)
@@ -461,8 +540,28 @@ def process_single_task(task_tuple, precomputed_response=None):
if res["error"] != "":
tprint("\tError :", res["error"], "for Copie", pid, group_name)
if can_spawn_tasks and res.get("error") in ["wrong-label", "additional-answer"]:
new_tasks.extend(handle_label_errors(pid, label, res, pdf_path))
if can_spawn_tasks and res.get("error") in [
"wrong-label",
"additional-answer",
]:
if stop_requested.is_set():
with io_lock:
pending_responses[file_path] = json.dumps(json_data)
atomic_write_json(
pending_responses_path, pending_responses
)
raise CorrectionStopRequested
try:
new_tasks.extend(
handle_label_errors(pid, label, res, pdf_path)
)
except CorrectionStopRequested:
with io_lock:
pending_responses[file_path] = json.dumps(json_data)
atomic_write_json(
pending_responses_path, pending_responses
)
raise
# Si "wrong-label" a déplacé le fichier courant vers _old
if res.get("error", "").startswith("wrg-lbl-moved-to:"):
current_suffix = "_old"
@@ -475,6 +574,9 @@ def process_single_task(task_tuple, precomputed_response=None):
for (i,f) in enumerate(res["feedback"]):
b = f.get("box_2d")
if b:
if not valid_feedback_box(b):
needs_correction.append(i)
continue
ymin, _xmin, ymax, xmax = b
ymin = ymin * total_height // 1000
ymax = ymax * total_height // 1000
@@ -484,9 +586,11 @@ def process_single_task(task_tuple, precomputed_response=None):
pid, label, group_name)
continue
if (ymin < yming - 50 or ymax > ymaxg + 50 or xmax / 1000 > width_r):
if (ymin < yming - 50 or ymax > ymaxg + 50
or ymin > ymaxg + 50 or ymax < yming - 50
or _xmin < 0 or xmax / 1000 > width_r):
needs_correction.append(i)
break
continue
if ymin < yming - 5:
ymin = yming - 5
b[0] = ymin * 1000 // total_height
@@ -498,6 +602,8 @@ def process_single_task(task_tuple, precomputed_response=None):
if needs_correction:
tprint(f"\tBox anomalies detected for Copie {pid} {group_name}. \n\tRequesting isolated correction from Gemini Flash...")
try:
if stop_requested.is_set():
raise CorrectionStopRequested
# Pensez à passer pdf_path à la fonction modifiée !
res["feedback"] = correct_boxes_with_gemini(
pid, label, pdf_path, res["feedback"],
@@ -520,7 +626,12 @@ def process_single_task(task_tuple, precomputed_response=None):
# To track progress
completed_tasks.append((file_path, label))
atomic_write_json(progress_path, completed_tasks)
if file_path in pending_responses:
del pending_responses[file_path]
atomic_write_json(pending_responses_path, pending_responses)
except CorrectionStopRequested:
raise
except json.JSONDecodeError:
tprint(f"Error decoding JSON for {file_path}", file=sys.stderr)
with io_lock:
@@ -575,7 +686,7 @@ def resolve_delayed_moves():
if pdf_path != old_pdf_path:
shutil.move(str(pdf_path), str(old_pdf_path))
idx = get_next_group_idx(target_label)
idx = reserve_next_group_idx(target_label)
height = grouping.get_pdf_height(str(new_pdf_path))
grouping.create_jpg(target_label, idx, [(pid, str(new_pdf_path), height)], GROUPS_DIR)
new_tasks.append((str(GROUPS_DIR / target_label / f"Group_{idx+1}.jpg"), target_label, False))
@@ -593,7 +704,7 @@ def resolve_delayed_moves():
resolved_any = True
shutil.copy(str(pdf_path), str(add_pdf_path))
idx = get_next_group_idx(target_label)
idx = reserve_next_group_idx(target_label)
height = grouping.get_pdf_height(str(add_pdf_path))
grouping.create_jpg(target_label, idx, [(pid, str(add_pdf_path), height)], GROUPS_DIR)
new_tasks.append((str(GROUPS_DIR / target_label / f"Group_{idx+1}.jpg"), target_label, False))
@@ -679,7 +790,7 @@ def run_configured(args: argparse.Namespace) -> ExitCode:
# pdf_path = copie_dir / f"{label}_old.pdf"
if pdf_path.exists():
idx = get_next_group_idx(label)
idx = reserve_next_group_idx(label)
height = grouping.get_pdf_height(str(pdf_path))
grouping.create_jpg(label, idx, [(pid, str(pdf_path), height)], GROUPS_DIR)
new_group_path = str(GROUPS_DIR / label / f"Group_{idx+1}.jpg")
@@ -814,36 +925,82 @@ def run_configured(args: argparse.Namespace) -> ExitCode:
else:
print(f"Warning: Batch results file {batch_results_path} not found.", file=sys.stderr)
report_live_progress = not any(
(args.batch, args.batch_from, args.deal_with_batched, args.refaire)
)
progress_total = len(tasks)
progress_completed = max(0, progress_total - len(tasks_to_process))
if report_live_progress:
report_group_progress(progress_completed, progress_total)
made_progress = True
while tasks_to_process or made_progress:
if tasks_to_process:
print(f"Starting processing on {len(tasks_to_process)} tasks with {NB_THREADS} threads...")
with concurrent.futures.ThreadPoolExecutor(max_workers=NB_THREADS) as executor:
waiting_tasks = list(tasks_to_process)
futures = {}
for task in tasks_to_process:
file_path = task[0]
precomp = batched_responses.get(file_path)
futures[executor.submit(process_single_task, task, precomp)] = task
for future in concurrent.futures.as_completed(futures):
def submit_available_tasks() -> None:
while (
waiting_tasks
and len(futures) < NB_THREADS
and not stop_requested.is_set()
):
task = waiting_tasks.pop(0)
file_path = task[0]
precomp = pending_responses.get(
file_path, batched_responses.get(file_path)
)
futures[
executor.submit(process_single_task, task, precomp)
] = task
submit_available_tasks()
while futures:
completed_futures, _pending = concurrent.futures.wait(
tuple(futures),
return_when=concurrent.futures.FIRST_COMPLETED,
)
for future in completed_futures:
failed_task = futures.pop(future)
task_completed = False
try:
new_generated_tasks = future.result()
if new_generated_tasks:
for new_task in new_generated_tasks:
futures[executor.submit(process_single_task, new_task)] = new_task
task_completed = True
if new_generated_tasks and not stop_requested.is_set():
if report_live_progress:
progress_total += len(new_generated_tasks)
waiting_tasks.extend(new_generated_tasks)
except CorrectionStopRequested:
pass
except Exception as e: # noqa: BLE001 - future boundary
print(f"Exception during task execution: {e}", file=sys.stderr)
failed_task = futures[future]
print(
f"Exception during task execution: {e}",
file=sys.stderr,
)
with io_lock:
errors_summary.append((str(e), failed_task[0]))
if report_live_progress and task_completed:
progress_completed += 1
report_group_progress(
progress_completed, progress_total
)
submit_available_tasks()
tasks_to_process = [] # Vider la liste une fois traitée
# Après avoir traité toutes les tâches actuelles (live ou batched),
# on tente de débloquer les mouvements qui étaient en attente
if stop_requested.is_set():
break
delayed_tasks = resolve_delayed_moves()
if delayed_tasks:
print(f"Resolved {len(delayed_tasks)} delayed moves! Running executor for new tasks...")
if report_live_progress:
progress_total += len(delayed_tasks)
report_group_progress(progress_completed, progress_total)
tasks_to_process.extend(delayed_tasks)
made_progress = True
else:
@@ -852,7 +1009,7 @@ def run_configured(args: argparse.Namespace) -> ExitCode:
# Check for remaining unresolved delayed tasks
unresolved_delayed = []
with io_lock:
for label, batches in results.items():
for label, batches in sorted(results.items()):
for batch in batches:
for p in batch:
res = p.get("result", {})
@@ -869,7 +1026,7 @@ def run_configured(args: argparse.Namespace) -> ExitCode:
manual_path = INPUT_DIR / "manual_resolutions.txt"
atomic_write_text(
manual_path,
"### Use -> x>, -x, ss, sx, xx, xs\n"
"### Use -> x>, -x, ss, sx, xx, xs, c{43}1>, c{43}2x\n"
+ "\n".join(unresolved_delayed)
+ "\n",
)
@@ -885,6 +1042,12 @@ def run_configured(args: argparse.Namespace) -> ExitCode:
print(err, file=sys.stderr)
escaped_path = shlex.quote(str(file))
print(f"Run : python -m copienator correct {escaped_path}")
if stop_requested.is_set():
print(
"[Interruption] Appels en cours terminés ; résultats disponibles sauvegardés.",
flush=True,
)
return ExitCode.INTERRUPTED
return ExitCode.PARTIAL if errors_summary else ExitCode.SUCCESS
@@ -909,9 +1072,17 @@ def run(
configure_runtime(workspace, discovered, args, api_client=api_client)
if not discovered and not args.refaire:
return ExitCode.PARTIAL
previous_handlers = {}
for signal_name in ("SIGINT", "SIGBREAK"):
interrupt_signal = getattr(signal, signal_name, None)
if interrupt_signal is not None:
previous_handlers[interrupt_signal] = signal.getsignal(interrupt_signal)
signal.signal(interrupt_signal, request_graceful_stop)
try:
status = run_configured(args)
finally:
for interrupt_signal, previous_handler in previous_handlers.items():
signal.signal(interrupt_signal, previous_handler)
for thread_id in list(thread_logs):
flush_thread_log(thread_id)
if warnings and status == ExitCode.SUCCESS:
@@ -961,6 +1132,12 @@ def main(argv: Sequence[str] | None = None) -> int:
parser = build_parser()
def handle(args: argparse.Namespace) -> ExitCode:
if args.refaire and args.overwrite:
raise CliError(
"--overwrite cannot be used with --refaire; --refaire already "
"replaces the corrections selected in refaire.json",
ExitCode.INVALID_ARGUMENTS,
)
workspace, target = workspace_from_target(args)
targets = [target]
for additional in args.additional_targets:
+32 -3
View File
@@ -134,6 +134,27 @@ def _publish(
return backup_root
def crop_statistics(records: list[dict]) -> tuple[int, float]:
"""Return cropped exercise count and mean removed percentage per exercise."""
totals: dict[str, list[float]] = {}
cropped_files: set[str] = set()
for record in records:
total_height, removed_height = totals.setdefault(record["file"], [0.0, 0.0])
totals[record["file"]] = [
total_height + float(record["height_lines"]),
removed_height + float(record["bottom_removed_lines"]),
]
if record["status"] == "cropped":
cropped_files.add(record["file"])
percentages = [
min(100.0, totals[file_name][1] / totals[file_name][0] * 100)
for file_name in cropped_files
if totals[file_name][0] > 0
]
mean_percentage = sum(percentages) / len(percentages) if percentages else 0.0
return len(cropped_files), mean_percentage
def run(workspace: EvaluationWorkspace, target: Path, *, workers: int = 5) -> ExitCode:
if workers < 1:
raise CliError(
@@ -161,15 +182,23 @@ def run(workspace: EvaluationWorkspace, target: Path, *, workers: int = 5) -> Ex
raise CliError(
f"{source} a changé pendant lanalyse. Aucun PDF remplacé."
)
cropped_exercises, mean_percentage = crop_statistics(records)
if not changed_files:
print("Terminé : aucun PDF ne remplit les critères de rognage.", flush=True)
print("Terminé : aucun exercice ne remplit les critères de rognage.", flush=True)
print(
"Rognage moyen des exercices modifiés : 0.0 %.", flush=True
)
return ExitCode.SUCCESS
backup = _publish(workspace, changed_files, staging, records)
cropped_pages = sum(record["status"] == "cropped" for record in records)
print(f"Sauvegarde des PDF non rognés : {backup}", flush=True)
print(
f"Terminé : {cropped_pages} page(s) rognée(s) dans "
f"{len(changed_files)} PDF remplacé(s).",
f"Terminé : {cropped_exercises} exercice(s) rogné(s), soit "
f"{cropped_pages} page(s) dans {len(changed_files)} PDF remplacé(s).",
flush=True,
)
print(
f"Rognage moyen des exercices modifiés : {mean_percentage:.1f} %.",
flush=True,
)
return ExitCode.SUCCESS
+26 -1
View File
@@ -83,6 +83,26 @@ def process_copies(files: list[Path], staging: Path, workers: int) -> list[dict]
key=lambda row: (order[row["file"]], row["page"]))
def crop_statistics(records: list[dict]) -> tuple[int, float, int]:
"""Return cropped page count, their mean removed percentage, and >30% count."""
percentages: list[float] = []
for record in records:
removed_mm = record["top_removed_mm"] + record["bottom_removed_mm"]
if removed_mm <= 0:
continue
x0, y0, x1, y1 = record["original_cropbox"]
original_height_points = (
x1 - x0 if record.get("rotation", 0) % 180 else y1 - y0
)
if original_height_points <= 0:
continue
original_height_mm = original_height_points * 25.4 / 72
percentages.append(min(100.0, removed_mm / original_height_mm * 100))
mean_percentage = sum(percentages) / len(percentages) if percentages else 0.0
over_thirty = sum(percentage > 30 for percentage in percentages)
return len(percentages), mean_percentage, over_thirty
def run(workspace: EvaluationWorkspace, target: Path, *, workers: int = 5) -> ExitCode:
if workers < 1:
raise CliError("Le nombre de traitements parallèles doit être positif.",
@@ -109,9 +129,14 @@ def run(workspace: EvaluationWorkspace, target: Path, *, workers: int = 5) -> Ex
(backup/"report.json").write_text(json.dumps(records, ensure_ascii=False, indent=2),
encoding="utf-8")
print(f"Sauvegarde des PDF non rognés : {originals}", flush=True)
cropped = sum(row["top_removed_mm"]+row["bottom_removed_mm"] > 0 for row in records)
cropped, mean_percentage, over_thirty = crop_statistics(records)
print(f"Terminé : {cropped}/{len(records)} pages rognées ; "
f"{len(files)} PDF remplacés dans Copies.", flush=True)
print(
f"Rognage moyen des pages modifiées : {mean_percentage:.1f} % ; "
f"{over_thirty} page(s) rognée(s) de plus de 30 %.",
flush=True,
)
return ExitCode.SUCCESS
+34 -8
View File
@@ -28,6 +28,8 @@ from copienator.copy_errors import clear_copy_error, mark_copy_error, marked_cop
DELIMITER_WIDTH = 5
DELIMITER_COLOR = (0, 0, 0)
OUTPUT_SIZE = (1800, 1000)
CROP_SHIFT_STEP = 50
CROP_WIDTH_STEP = 50
pdf_cache_lock = threading.Lock()
@@ -65,7 +67,9 @@ def stitch_images(image_list: list[Image.Image]) -> Image.Image | None:
@lru_cache(maxsize=3)
def _get_pdf_pages_cached(pdf_path: Path) -> list[Image.Image]:
return convert_from_path(pdf_path)
# Label coordinates use the full, displayed MediaBox, including on PDFs
# previously cropped by crop-margins. Keep this in sync with split-answers.
return convert_from_path(pdf_path, use_cropbox=False)
def get_pdf_pages(pdf_path: Path) -> list[Image.Image]:
@@ -78,6 +82,7 @@ def process_single_pdf(
pdf_path: Path,
shift_offset: int = 0,
max_per_file: int = 5,
width_offset: int = 0,
) -> tuple[Image.Image, list[Image.Image], dict[str, object]] | None:
"""Convert one PDF into a preview, full-resolution splits and metadata."""
try:
@@ -88,7 +93,10 @@ def process_single_pdf(
left, right = 0, width
else:
left = max(0, 100 + shift_offset)
right = min(width, width // 3 + 100 + shift_offset)
right = min(
width,
width // 3 + 100 + shift_offset + max(0, width_offset),
)
if right > left:
cropped_images.append(image.crop((left, 0, right, height)))
if not cropped_images:
@@ -164,6 +172,7 @@ class ImageReviewer:
self.current_result = None
self.index = 0
self.current_shift = 0
self.current_width_offset = 0
self.default_max_per_file = default_max_per_file
self.current_max_per_file = default_max_per_file
self.current_preview: Image.Image | None = None
@@ -182,9 +191,10 @@ class ImageReviewer:
self.root.bind("<Return>", self.on_next)
self.root.bind("s", self.on_skip)
self.root.protocol("WM_DELETE_WINDOW", self.on_close)
self.root.bind("n", lambda _event: self.on_shift(50))
self.root.bind("N", lambda _event: self.on_shift(100))
self.root.bind("t", lambda _event: self.on_shift(-50))
self.root.bind("n", lambda _event: self.on_shift(CROP_SHIFT_STEP))
self.root.bind("N", lambda _event: self.on_shift(2 * CROP_SHIFT_STEP))
self.root.bind("t", lambda _event: self.on_shift(-CROP_SHIFT_STEP))
self.root.bind("l", lambda _event: self.on_enlarge(CROP_WIDTH_STEP))
self.root.bind("1", lambda _event: self.on_set_max_pages(1))
Thread(target=self.prefetch_worker, daemon=True).start()
@@ -220,6 +230,7 @@ class ImageReviewer:
return
self.is_processing = False
self.current_shift = 0
self.current_width_offset = 0
self.current_result = None
self.trigger_processing(self.files[self.index], self.current_shift)
@@ -232,7 +243,12 @@ class ImageReviewer:
def worker() -> None:
self.manual_queue.put(
process_single_pdf(pdf_path, shift, self.current_max_per_file)
process_single_pdf(
pdf_path,
shift,
self.current_max_per_file,
self.current_width_offset,
)
)
Thread(target=worker, daemon=True).start()
@@ -269,10 +285,12 @@ class ImageReviewer:
self.label_info.configure(
text=(
f"[{self.index + 1}/{len(self.files)}] {filename} | "
f"Shift: {self.current_shift}px\nFiles: {schema['number_of_files']} | "
f"Shift: {self.current_shift}px | "
f"Extra width: {self.current_width_offset}px\n"
f"Files: {schema['number_of_files']} | "
f"Cols: {schema['columns_per_file']}\n"
"Enter: Save and next | s: flag error and skip | n: +50 | N: +100 | t: -50 | "
"1: use single column"
"l: widen by 50 | 1: use full pages"
),
fg="black",
)
@@ -284,6 +302,13 @@ class ImageReviewer:
print(f"Applying shift: {self.current_shift}")
self.trigger_processing(self.files[self.index], self.current_shift)
def on_enlarge(self, amount: int) -> None:
if self.is_processing:
return
self.current_width_offset += amount
print(f"Applying extra width: {self.current_width_offset}")
self.trigger_processing(self.files[self.index], self.current_shift)
def on_next(self, _event: object) -> None:
if self.is_processing or self.current_result is None:
return
@@ -315,6 +340,7 @@ class ImageReviewer:
def _advance(self) -> None:
self.index += 1
self.current_shift = 0
self.current_width_offset = 0
self.current_max_per_file = self.default_max_per_file
self.load_current_image()
+5
View File
@@ -86,6 +86,11 @@ même si certains textes fournis sont dans une autre langue.
Conserve les formules mathématiques, les labels exacts des questions et les
clés JSON `rubrics`, `label` et `rubric_content` sans les traduire.
Chaque question DOIT être notée sur exactement 4 points. Propose une répartition logique de ces points.
Il est inutile d'indiquer dans `rubric_content` que le barème totalise 4 points :
ce total est toujours implicite.
N'utilise pas de caractères mathématiques Unicode dans `rubric_content`.
Écris les expressions mathématiques en LaTeX, par exemple
`$\\lfloor \\sqrt{k} \\rfloor$` plutôt qu'avec des symboles Unicode.
Par exemple :
- Au moins 2 points si le résultat est correct.
- Mettre la moitié des points si le raisonnement est correct mais pas le résultat.
+106 -23
View File
@@ -10,12 +10,14 @@ from pathlib import Path
from copienator import (
EvaluationWorkspace,
ExitCode,
configuration,
evaluation_parser,
execute,
read_json,
workspace_from_args,
)
from copienator.platform import replace_with_link_or_copy, safe_filename
from copienator.return_answers import publish_answer_returns
ANNOTATION_CHOICES = ("BGnot", "Bnot", "Anot")
@@ -27,9 +29,20 @@ def build_parser() -> argparse.ArgumentParser:
choices=ANNOTATION_CHOICES,
help="Annotation directory to use",
)
parser.add_argument(
"--update",
action="store_true",
help=(
"Update only the individual images in existing A Rendre/answers "
"directories, matching folders by their trailing copy ID"
),
)
return parser
RETURN_COPY_ID = re.compile(r"\((\d+)\)$")
def _read_expected_names(workspace: EvaluationWorkspace) -> set[str]:
names_path = workspace.names_file()
if not names_path.exists():
@@ -45,6 +58,70 @@ def _read_expected_names(workspace: EvaluationWorkspace) -> set[str]:
}
def _annotation_source(
workspace: EvaluationWorkspace,
annotation_dir_name: str,
copy_id: str,
) -> Path | None:
selected = workspace.root / annotation_dir_name / f"Copie{copy_id}"
fallback = workspace.annotation_dir("simple") / f"Copie{copy_id}"
for candidate in (selected, fallback):
if (candidate / "score.json").is_file() and (
(candidate / "Concat.jpg").is_file()
or (candidate / "info.json").is_file()
):
return candidate
return None
def update_named_return_answers(
workspace: EvaluationWorkspace,
annotation_dir_name: str,
) -> ExitCode:
"""Refresh only answers/ in existing returns, preserving manual names."""
workspace.require_directories(annotation_dir_name, "A Rendre")
had_errors = False
found = False
for destination in sorted(workspace.return_dir.iterdir()):
if not destination.is_dir():
continue
match = RETURN_COPY_ID.search(destination.name)
if match is None:
print(
f"Warning: cannot identify a copy ID in {destination.name!r}; skipped",
file=sys.stderr,
)
had_errors = True
continue
found = True
copy_id = match.group(1)
source_folder = _annotation_source(
workspace, annotation_dir_name, copy_id
)
if source_folder is None:
print(
f"Warning: no annotation source found for Copie{copy_id}; skipped",
file=sys.stderr,
)
had_errors = True
continue
try:
publish_answer_returns(
workspace.root,
source_folder,
destination,
answers_only=True,
)
print(f"Updated answers for {destination.name} from Copie{copy_id}")
except (OSError, TypeError, ValueError) as exc:
print(f"Error updating answers for {destination.name}: {exc}", file=sys.stderr)
had_errors = True
if not found:
print("Warning: no identifiable student folders found in A Rendre", file=sys.stderr)
had_errors = True
return ExitCode.PARTIAL if had_errors else ExitCode.SUCCESS
def prepare_named_returns(
workspace: EvaluationWorkspace,
annotation_dir_name: str,
@@ -72,9 +149,6 @@ def prepare_named_returns(
had_errors = True
assigned_names: set[str] = set()
selected_annotations = workspace.root / annotation_dir_name
fallback_annotations = workspace.annotation_dir("simple")
for name, copy_ids in copies_map.items():
if name == "Unknown":
print(
@@ -90,34 +164,40 @@ def prepare_named_returns(
safe_name = safe_filename(name)
for copy_id in copy_ids:
selected = selected_annotations / f"Copie{copy_id}"
fallback = fallback_annotations / f"Copie{copy_id}"
source_folder = None
for candidate in (selected, fallback):
if (candidate / "Concat.jpg").exists() and (
candidate / "score.json"
).exists():
source_folder = candidate
break
source_folder = _annotation_source(
workspace, annotation_dir_name, copy_id
)
if source_folder is None:
continue
assigned_names.add(name)
destination = workspace.return_dir / f"{safe_name} ({copy_id})"
destination.mkdir(parents=True, exist_ok=True)
links = (
("Concat.jpg", f"{safe_name}.jpg"),
("Concat_F.pdf", f"{safe_name}.pdf"),
("score.json", "score.json"),
)
for source_name, destination_name in links:
source = source_folder / source_name
if not source.exists():
continue
try:
publish_answer_returns(workspace.root, source_folder, destination)
except (OSError, TypeError, ValueError) as exc:
print(f"Error preparing answers for {destination.name}: {exc}", file=sys.stderr)
had_errors = True
continue
links = (
("Concat.jpg", f"{safe_name}.jpg", configuration.RETURN_JPEG_ENABLED),
("Concat_F.pdf", f"{safe_name}.pdf", configuration.RETURN_PDF_ENABLED),
("score.json", "score.json", True),
)
for source_name, destination_name, enabled in links:
source = source_folder / source_name
target = destination / destination_name
try:
if not enabled:
# Remove only the named return entry, never its link target.
target.unlink(missing_ok=True)
continue
if not source.exists():
target.unlink(missing_ok=True)
continue
method = replace_with_link_or_copy(
source,
destination / destination_name,
target,
prefer="symlink",
)
if method == "copy":
@@ -145,7 +225,10 @@ def run(
workspace: EvaluationWorkspace,
*,
annotation_dir: str,
update: bool = False,
) -> ExitCode:
if update:
return update_named_return_answers(workspace, annotation_dir)
return prepare_named_returns(workspace, annotation_dir)
@@ -157,10 +240,10 @@ def main(argv: Sequence[str] | None = None) -> int:
lambda args: run(
workspace_from_args(args, repository=Path.cwd()),
annotation_dir=args.annotation_dir,
update=args.update,
),
)
if __name__ == "__main__":
raise SystemExit(main())
+112 -21
View File
@@ -1,7 +1,9 @@
from __future__ import annotations
import argparse
import copy
import queue
import re
import threading
import tkinter as tk
from collections.abc import Sequence
@@ -24,6 +26,9 @@ from copienator.utils import natural_key, read_all_labels
# --- Configuration & Globals ---
padding = 60
MISSING_LABEL_COLOR = "orange"
COMMON_MISSING_LABEL_COLOR = "#403a00"
COMMON_MISSING_THRESHOLD = 0.66
try:
font = ImageFont.truetype("DejaVuSans.ttf", size=30)
@@ -81,13 +86,81 @@ def normalized_labels(entries):
if str(value["label"]) != "_"
]
def prepare_image(image_path: str, bounding_boxes, all_labels, nb_pages, last_label_index):
def frequently_missing_labels(
copies_dir: Path,
all_labels: list[str],
threshold: float = COMMON_MISSING_THRESHOLD,
) -> set[str]:
"""Return labels absent from at least ``threshold`` of detected copies."""
labels_by_copy: dict[str, set[str]] = {}
for json_path in copies_dir.glob("*.json"):
match = re.fullmatch(r"(.+)_\d+", json_path.stem)
if match is None:
continue
try:
data = read_json(json_path)
entries = data["list"]
if not isinstance(entries, list):
continue
present = set(normalized_labels(entries))
except (OSError, KeyError, TypeError, ValueError):
continue
labels_by_copy.setdefault(match.group(1), set()).update(present)
copy_count = len(labels_by_copy)
if copy_count == 0:
return set()
return {
label
for label in all_labels
if sum(label not in present for present in labels_by_copy.values())
>= threshold * copy_count
}
def label_color(
label: str | None,
all_labels: list[str],
last_label_index: int,
common_missing: set[str],
) -> tuple[str, int]:
"""Choose a label color and return the updated chronological index."""
color = "black"
if not label or label not in all_labels:
return color, last_label_index
current_index = all_labels.index(label)
if current_index < last_label_index or (
last_label_index == -1 and current_index != 0
):
color = "red"
elif current_index > last_label_index + 1:
only_previous_is_missing = current_index == last_label_index + 2
previous_label = all_labels[current_index - 1]
color = (
COMMON_MISSING_LABEL_COLOR
if only_previous_is_missing and previous_label in common_missing
else MISSING_LABEL_COLOR
)
return color, current_index
def prepare_image(
image_path: str,
bounding_boxes,
all_labels,
nb_pages,
last_label_index,
common_missing: set[str] | None = None,
):
im = Image.open(image_path)
im.load()
width, height = im.size
new_im = Image.new(im.mode, (width + padding, height), "white")
new_im.paste(im, (0, 0))
draw = ImageDraw.Draw(new_im)
common_missing = common_missing or set()
for bbox in sort_bounding_boxes(bounding_boxes, nb_pages):
raw_y_min = int(bbox["box_2d"][0] * height / 1000)
@@ -99,15 +172,13 @@ def prepare_image(image_path: str, bounding_boxes, all_labels, nb_pages, last_la
abs_y_max = min(height, raw_y_max + 10)
abs_x_max = min(width, raw_x_max + 10)
color = "black"
label = bbox.get("label")
if label and label in all_labels:
current_index = all_labels.index(label)
if current_index < last_label_index or (last_label_index == -1 and current_index != 0):
color = "red"
elif current_index > last_label_index + 1:
color = "orange"
last_label_index = current_index
color, last_label_index = label_color(
label,
all_labels,
last_label_index,
common_missing,
)
draw.rectangle(((abs_x_min, abs_y_min), (abs_x_max, abs_y_max)), outline=color, width=4)
if label:
@@ -126,6 +197,7 @@ def _worker_items(base_dir, files_to_process, all_labels, output_queue):
"""
previous_copie = None
last_label_index = None
common_missing = frequently_missing_labels(base_dir / "Copies", all_labels)
for img_path in files_to_process:
json_path = base_dir / "Copies" / f"{img_path.stem}.json"
copie_part = int(img_path.stem[-2:])
@@ -158,7 +230,14 @@ def _worker_items(base_dir, files_to_process, all_labels, output_queue):
try:
print(f"Buffering {img_path.name}...")
(pil_image, last_label_index) = \
prepare_image(str(img_path), bb_list, all_labels, nb_pages, last_label_index)
prepare_image(
str(img_path),
bb_list,
all_labels,
nb_pages,
last_label_index,
common_missing,
)
error_msg = None
except Exception as e: # noqa: BLE001 - keep the item editable in the GUI
@@ -271,7 +350,6 @@ class ImageViewer:
# Start new batch
self.active_copie_name = metadata["copie"]
self.accumulated_results = {"name": metadata["name"], "list": []}
self.history.clear()
self.display_image(pil_image, json_path, metadata)
except queue.Empty:
@@ -293,16 +371,23 @@ class ImageViewer:
def on_previous(self, event):
if self.is_viewing and self.history:
print("Going back to previous image...")
prev_pil, prev_json, prev_meta, num_added = self.history.pop()
# Undo the accumulation to prevent duplicates when we hit Enter again
if self.accumulated_results and num_added > 0:
self.accumulated_results["list"] = self.accumulated_results["list"][:-num_added]
(
prev_pil,
prev_json,
prev_meta,
previous_copie_name,
previous_results,
) = self.history.pop()
# Push current image to the forward stack so we don't lose it
self.forward_stack.append((self.current_pil_image,
self.current_json_path, self.current_meta))
# Restore the aggregation exactly as it was before validating the
# previous image. This also makes navigation across copies safe.
self.active_copie_name = previous_copie_name
self.accumulated_results = previous_results
# Display the previous image immediately
self.display_image(prev_pil, prev_json, prev_meta)
def display_image(self, pil_image, json_path, metadata):
@@ -329,7 +414,8 @@ class ImageViewer:
def on_enter(self, event):
if self.is_viewing:
print(f"Committing data for {self.current_json_path.name}...")
num_added = 0 # ADD THIS LINE
previous_copie_name = self.active_copie_name
previous_results = copy.deepcopy(self.accumulated_results)
try:
current_data = read_json(self.current_json_path)
@@ -352,8 +438,6 @@ class ImageViewer:
print(msg)
messagebox.showerror("Label Error", msg)
return
num_added = len(converted_items)
# Add to accumulator
if self.accumulated_results:
self.accumulated_results["list"].extend(converted_items)
@@ -368,8 +452,15 @@ class ImageViewer:
messagebox.showerror("JSON Error", msg)
return # Abort advancement
self.history.append((self.current_pil_image, self.current_json_path,
self.current_meta, num_added))
self.history.append(
(
self.current_pil_image,
self.current_json_path,
self.current_meta,
previous_copie_name,
previous_results,
)
)
# Advance UI
self.is_viewing = False
+51 -11
View File
@@ -9,6 +9,7 @@ from typing import Any
from copienator import (
EvaluationWorkspace,
ExitCode,
atomic_write_bytes,
atomic_write_json,
evaluation_parser,
execute,
@@ -18,6 +19,10 @@ from copienator import (
WORD_LIST_FILE = Path(__file__).resolve().parents[1] / "data" / "liste_francais.txt"
ACCENT_PATTERN = re.compile(r"[éèêëàâäîïôöùûüçœÉÈÊËÀÂÄÎÏÔÖÙÛÜÇŒ]")
MATH_PATTERN = re.compile(
r"(\$\$.*?\$\$|\$.*?\$|\\\(.*?\\\)|\\\[.*?\\\])",
re.DOTALL,
)
def build_parser() -> argparse.ArgumentParser:
@@ -25,22 +30,37 @@ def build_parser() -> argparse.ArgumentParser:
def escape_latex_underscores(text: str) -> str:
r"""Escape underscores outside LaTeX math environments."""
math_pattern = re.compile(
r"(\$\$.*?\$\$|\$.*?\$|\\\(.*?\\\)|\\\[.*?\\\])",
re.DOTALL,
)
r"""Escape underscores outside math without double-escaping existing ones."""
def escape_plain(value: str) -> str:
# Collapse any existing escape run as well, making cleanup idempotent.
return re.sub(r"\\*_", lambda _match: r"\_", value)
parts: list[str] = []
last_end = 0
for match in math_pattern.finditer(text):
for match in MATH_PATTERN.finditer(text):
start, end = match.span()
parts.append(text[last_end:start].replace("_", r"\_"))
parts.append(escape_plain(text[last_end:start]))
parts.append(match.group(0))
last_end = end
parts.append(text[last_end:].replace("_", r"\_"))
parts.append(escape_plain(text[last_end:]))
return "".join(parts)
def normalize_overescaped_latex_commands(text: str) -> str:
r"""Collapse doubled command escapes inside LaTeX math environments.
Model responses occasionally contain ``\\mathbb`` after JSON decoding where
LaTeX requires ``\mathbb``. A doubled backslash followed by whitespace is a
legitimate row break (for example in ``cases``), so it must be preserved.
"""
def normalize_math(match: re.Match[str]) -> str:
return re.sub(r"\\\\(?=[A-Za-z{}])", r"\\", match.group(0))
return MATH_PATTERN.sub(normalize_math, text)
def build_lookup_map(word_list_path: Path = WORD_LIST_FILE) -> dict[str, str]:
words = word_list_path.read_text(encoding="utf-8").splitlines()
lookup: dict[str, str] = {}
@@ -67,8 +87,23 @@ def fix_hex_corruption_safe(text: str) -> str:
)
def some_other_replacements(text: str) -> str:
return text.replace("\neq", "\\neq").replace("\not", "\\not")
def repair_json_escape_corruption(text: str) -> str:
r"""Restore observed LaTeX commands consumed as JSON control escapes."""
replacements = (
("\x0crac", r"\frac"),
("\x0ceuille", r"\equiv"),
("\theta", r"\theta"),
("\times", r"\times"),
("\textbackslash ", "\\"),
("\negthinspace", r"\negthinspace"),
("\neq", r"\neq"),
("\not", r"\not"),
("", r"\ensuremath{\in}"),
("", r"\ensuremath{\subset}"),
)
for broken, repaired in replacements:
text = text.replace(broken, repaired)
return text
def clean_string(text: str, lookup: dict[str, str]) -> str:
@@ -79,7 +114,9 @@ def clean_string(text: str, lookup: dict[str, str]) -> str:
text = re.sub(r" \x00{1,2} ", " à ", text)
if "\x00" in text:
text = fast_fix(text, lookup).replace("\x00", "")
return escape_latex_underscores(some_other_replacements(text))
text = repair_json_escape_corruption(text)
text = normalize_overescaped_latex_commands(text)
return escape_latex_underscores(text)
def clean_obj(value: Any, lookup: dict[str, str]) -> Any:
@@ -104,6 +141,9 @@ def run(
lookup = build_lookup_map(word_list_path)
data = read_json(workspace.correction_file)
cleaned = clean_obj(data, lookup)
backup = workspace.root / "correction_precleanup.json"
atomic_write_bytes(backup, workspace.correction_file.read_bytes())
print(f"Original JSON backed up to {backup}")
atomic_write_json(workspace.correction_file, cleaned)
print(f"Fixed JSON saved to {workspace.correction_file}")
return ExitCode.SUCCESS
+93 -18
View File
@@ -10,7 +10,7 @@ from pdf2image import convert_from_path
from PIL import Image, ImageChops, ImageDraw, ImageFilter
from copienator.commands import annotating
from copienator import utils
from copienator import configuration, utils
from copienator import (
EvaluationWorkspace,
ExitCode,
@@ -21,8 +21,10 @@ from copienator import (
workspace_from_args,
)
from copienator.annotation_actions import apply_checkbox_actions, apply_score_overrides
from copienator.answer_info import build_answer_info
from copienator.annotation_data import AnnotationData, load_annotation_data
from copienator.filesystem import staged_files
from copienator.return_answers import save_return_answer_options
Image.MAX_IMAGE_PIXELS = None
@@ -68,10 +70,11 @@ def detect_checks_and_notes(
print(f" Resizing annotated PDF from {user_image.size} to {reference.size}")
user_image = user_image.resize(reference.size, Image.Resampling.LANCZOS)
difference = np.abs(
np.array(reference).astype(int) - np.array(user_image).astype(int)
).astype(np.uint8)
difference_gray = np.mean(difference, axis=2)
# Keep the full-size difference in uint8. Converting both tall group images
# to the platform ``int`` dtype used several gigabytes per scan worker.
difference = np.asarray(
ImageChops.difference(reference, user_image), dtype=np.uint8
)
keep_mask = Image.new("L", reference.size, 255)
mask_draw = ImageDraw.Draw(keep_mask)
actions: list[dict[str, Any]] = []
@@ -82,10 +85,14 @@ def detect_checks_and_notes(
x1, y1, x2, y2 = map(int, raw_box["global_box"])
x1, y1 = max(0, x1), max(0, y1)
x2, y2 = min(reference.width, x2), min(reference.height, y2)
region = difference_gray[y1 + 5 : y2 - 5, x1 + 5 : x2 - 5]
region = difference[y1 + 5 : y2 - 5, x1 + 5 : x2 - 5]
if region.size == 0:
continue
density = np.sum(region > 30) / region.size
# Preserve the previous mean-across-RGB threshold, but allocate its
# temporary float array only for the small checkbox region.
density = np.count_nonzero(np.mean(region, axis=2) > 30) / (
region.shape[0] * region.shape[1]
)
if density > 0.05:
actions.append(raw_box)
mask_draw.rectangle([x1 - 15, y1 - 15, x2 + 15, y2 + 15], fill=0)
@@ -94,13 +101,17 @@ def detect_checks_and_notes(
if raw_box.get("type") == "score" and raw_box.get("value") == 0.0:
mask_draw.rectangle([0, y1 - 10, reference.width, y2 + 10], fill=0)
del difference
reference_blur = reference.filter(ImageFilter.GaussianBlur(2))
user_blur = user_image.filter(ImageFilter.GaussianBlur(2))
diff_image = ImageChops.difference(reference_blur, user_blur).convert("L")
alpha = np.where(np.array(diff_image) > 50, 255, 0).astype(np.uint8)
final_alpha = np.minimum(alpha, np.array(keep_mask))
del reference_blur, user_blur
alpha = np.asarray(diff_image, dtype=np.uint8).copy()
np.greater(alpha, 50, out=alpha)
alpha *= np.uint8(255)
np.minimum(alpha, np.asarray(keep_mask, dtype=np.uint8), out=alpha)
notes = user_image.convert("RGBA")
notes.putalpha(Image.fromarray(final_alpha))
notes.putalpha(Image.fromarray(alpha))
return actions, notes
@@ -148,11 +159,14 @@ def apply_actions_and_regenerate(
raise TypeError(f"Expected a JSON object in {bnote_path}")
labels_data = data[student_id]
dirty_labels = apply_checkbox_actions(labels_data, actions, print)
apply_checkbox_actions(labels_data, actions, print)
score_path = output_dir / "score.json"
preserve_score_file = update_score and score_path.is_file()
if update_score:
dirty_labels |= apply_score_overrides(labels_data, output_dir / "score.json", print)
apply_score_overrides(labels_data, score_path, print)
scores = dict.fromkeys(all_labels, "")
answer_labels: list[str] = []
dirty_images: dict[str, Image.Image] = {}
concatenated: list[Image.Image] = []
filtered: list[Image.Image] = []
@@ -169,6 +183,8 @@ def apply_actions_and_regenerate(
content = labels_data[label]
result = content["result"]
scores[label] = str(result.get("score", 0))
if result.get("error") == "empty-answer":
continue
sub_note = None
if notes_layer is not None:
@@ -204,30 +220,63 @@ def apply_actions_and_regenerate(
body = sub_note.crop((0, old_header_height, width, height))
final_image.paste(body, (0, new_header_height), mask=body)
if label in dirty_labels or has_notes:
dirty_images[label] = final_image
answer_labels.append(label)
concatenated.append(final_image)
if float(scores[label]) != 4.0 or result.get("feedback", []):
filtered.append(final_image)
concat_image = concatenate(concatenated)
filtered_image = concatenate(filtered)
with staged_files(output_dir) as staging:
with staged_files(output_dir, remove=("Concat.jpg", "Concat_F.jpg", "Concat_F.pdf",
"touched.json", "answer_labels.json")) as staging:
for label, image in dirty_images.items():
image.save(staging / f"{label}.jpg")
if not preserve_score_file:
atomic_write_json(staging / "score.json", scores)
atomic_write_json(staging / "info.json", build_answer_info(
scores, labels_data, answer_labels
))
if concat_image is not None:
concat_image.save(staging / "Concat.jpg")
if filtered_image is not None:
filtered_image.save(staging / "Concat_F.jpg")
if preserve_score_file:
print(f" Preserved existing score.json in {output_dir}")
print(f" Saved regenerated files in {output_dir}")
return ExitCode.PARTIAL if incomplete else ExitCode.SUCCESS
def run(workspace: EvaluationWorkspace, *, update_score: bool = False) -> ExitCode:
def run(
workspace: EvaluationWorkspace,
*,
update_score: bool = False,
return_answers_context: bool | None = None,
return_answers_question: bool | None = None,
return_answers_solution: bool | None = None,
) -> ExitCode:
workspace.require_files("labels", "correction.json")
workspace.require_directories("Copies", "Par label", "Bnot")
if configuration.RETURN_ANSWERS_ENABLED:
save_return_answer_options(
workspace.root,
context=(
configuration.RETURN_ANSWERS_CONTEXT
if return_answers_context is None
else return_answers_context
),
question=(
configuration.RETURN_ANSWERS_QUESTION
if return_answers_question is None
else return_answers_question
),
solution=(
configuration.RETURN_ANSWERS_SOLUTION
if return_answers_solution is None
else return_answers_solution
),
)
all_labels = utils.read_all_labels(workspace.root)
loaded = load_annotation_data(workspace)
for warning in loaded.warnings:
@@ -268,7 +317,28 @@ def build_parser() -> argparse.ArgumentParser:
parser.add_argument(
"--update-score",
action="store_true",
help="Override generated scores with values from existing score.json files",
help=(
"Regenerate images with current statement/solution PDFs while "
"preserving and applying existing score.json values"
),
)
parser.add_argument(
"--return-answers-context",
action=argparse.BooleanOptionalAction,
default=configuration.RETURN_ANSWERS_CONTEXT,
help="Include applicable context pages in individual answer exports",
)
parser.add_argument(
"--return-answers-question",
action=argparse.BooleanOptionalAction,
default=configuration.RETURN_ANSWERS_QUESTION,
help="Include the current question PDF in individual answer exports",
)
parser.add_argument(
"--return-answers-solution",
action=argparse.BooleanOptionalAction,
default=configuration.RETURN_ANSWERS_SOLUTION,
help="Include the current solution PDF in individual answer exports",
)
return parser
@@ -277,11 +347,16 @@ def main(argv: Sequence[str] | None = None) -> int:
parser = build_parser()
def handle(args: argparse.Namespace) -> ExitCode:
return run(workspace_from_args(args), update_score=args.update_score)
return run(
workspace_from_args(args),
update_score=args.update_score,
return_answers_context=args.return_answers_context,
return_answers_question=args.return_answers_question,
return_answers_solution=args.return_answers_solution,
)
return execute(parser, argv, handle)
if __name__ == "__main__":
raise SystemExit(main())
@@ -9,6 +9,7 @@ from typing import Any
from PIL import Image, ImageDraw
from copienator import configuration
from copienator import (
EvaluationWorkspace,
ExitCode,
@@ -20,6 +21,7 @@ from copienator import (
workspace_from_args,
)
from copienator.annotation_actions import apply_checkbox_actions, apply_score_overrides
from copienator.answer_info import build_answer_info
from copienator.annotation_data import AnnotationData, RefaireList, load_annotation_data
from copienator.commands import annotating
from copienator.commands.reading_annotations import (
@@ -28,9 +30,11 @@ from copienator.commands.reading_annotations import (
has_significant_notes,
)
from copienator.filesystem import staged_files
from copienator.return_answers import save_return_answer_options
LabelNotes = dict[str, dict[str, Any]]
ScanResult = tuple[dict[str, list[dict[str, Any]]], dict[str, LabelNotes]]
SCAN_WORKERS = 2
def get_extra_pdfs_as_images(
@@ -189,14 +193,15 @@ def apply_actions_and_regenerate_grouped(
logs = [f"\nProcessing compilation for: Copie{student_id}"]
output_dir = workspace.root / annotation_dir / f"Copie{student_id}"
labels_data = data.get(student_id, {})
dirty_labels = apply_checkbox_actions(labels_data, actions, logs.append)
apply_checkbox_actions(labels_data, actions, logs.append)
score_path = output_dir / "score.json"
preserve_score_file = update_score and score_path.is_file()
if update_score:
dirty_labels |= apply_score_overrides(
labels_data, output_dir / "score.json", logs.append
apply_score_overrides(
labels_data, score_path, logs.append
)
selected_labels = selected_labels if selected_labels is not None else set()
dirty_labels |= selected_labels
simple_layout = None
simple_annotated = None
if selected_labels and annotation_dir == "Anot":
@@ -239,6 +244,8 @@ def apply_actions_and_regenerate_grouped(
else {}
)
scores = dict.fromkeys(all_labels, "")
touched = dict.fromkeys(all_labels, False)
answer_labels: list[str] = []
dirty_images: dict[str, Image.Image] = {}
concat_images: list[Image.Image] = []
filtered_groups: list[list[Image.Image]] = []
@@ -255,6 +262,9 @@ def apply_actions_and_regenerate_grouped(
):
result["score"] = old_scores[label]
scores[label] = str(result.get("score", 0))
touched[label] = False
if result.get("error") == "empty-answer":
continue
saved_image = output_dir / f"{label}.jpg"
if selected_labels and label not in selected_labels and saved_image.is_file():
with Image.open(saved_image) as saved:
@@ -271,12 +281,14 @@ def apply_actions_and_regenerate_grouped(
dirty_images[label] = final_image
scores[label] = str(old_scores.get(label, scores[label]))
concat_images.append(final_image)
answer_labels.append(label)
# Keep previously reviewed content, including handwriting.
if annotation_dir == "BGnot":
extras = get_extra_pdfs_as_images(
workspace.root, label, annotating, all_labels
)
filtered_groups.append([*extras, final_image])
touched[label] = True
else:
filtered_groups.append([final_image])
continue
@@ -313,9 +325,9 @@ def apply_actions_and_regenerate_grouped(
body = sub_note.crop((0, old_header_height, width, height))
final_image.paste(body, (0, new_header_height), mask=body)
if label in dirty_labels or has_notes or selected_labels:
# Persist every final block, including unchanged answers, for returns.
dirty_images[label] = final_image
logs.append(f" Saved dirty image: {label}.jpg")
answer_labels.append(label)
concat_images.append(final_image)
feedbacks = result.get("feedback", [])
@@ -329,11 +341,13 @@ def apply_actions_and_regenerate_grouped(
else []
)
filtered_groups.append([*extras, final_image])
touched[label] = annotation_dir == "BGnot"
concat_image = concatenate(concat_images)
if incomplete:
return ExitCode.PARTIAL, "\n".join(logs)
with staged_files(output_dir, remove=("Concat_F.pdf", "Concat_F.jpg")) as staging:
with staged_files(output_dir, remove=("Concat.jpg", "Concat_F.pdf", "Concat_F.jpg",
"touched.json", "answer_labels.json")) as staging:
if simple_layout is not None:
simple_layout["replaced"] = sorted(
set(simple_layout["replaced"]) | selected_labels
@@ -341,7 +355,11 @@ def apply_actions_and_regenerate_grouped(
atomic_write_json(staging / "refaire_simple_layout.json", simple_layout)
for label, image in dirty_images.items():
image.save(staging / f"{label}.jpg")
if not preserve_score_file:
atomic_write_json(staging / "score.json", scores)
atomic_write_json(staging / "info.json", build_answer_info(
scores, labels_data, answer_labels, touched
))
if concat_image is not None:
concat_image.save(staging / "Concat.jpg")
if filtered_groups:
@@ -352,6 +370,8 @@ def apply_actions_and_regenerate_grouped(
[image for group in filtered_groups for image in group]
)
filtered_image.save(staging / "Concat_F.jpg")
if preserve_score_file:
logs.append(f" Preserved existing score.json in {output_dir}")
logs.append(f" Saved regenerated files in {output_dir}")
status = ExitCode.PARTIAL if incomplete else ExitCode.SUCCESS
return status, "\n".join(logs)
@@ -449,6 +469,9 @@ def run(
refaire: bool = False,
update_score: bool = False,
annotation_dir: str = "BGnot",
return_answers_context: bool | None = None,
return_answers_question: bool | None = None,
return_answers_solution: bool | None = None,
) -> ExitCode:
workspace.require_files("labels", "correction.json")
workspace.require_directories("Copies", "Par label", annotation_dir)
@@ -458,6 +481,25 @@ def run(
workspace.require_files("refaire.json")
workspace.require_directories("BRnot")
refaire_list, refaire_by_student = _read_refaire(workspace)
if configuration.RETURN_ANSWERS_ENABLED:
save_return_answer_options(
workspace.root,
context=(
configuration.RETURN_ANSWERS_CONTEXT
if return_answers_context is None
else return_answers_context
),
question=(
configuration.RETURN_ANSWERS_QUESTION
if return_answers_question is None
else return_answers_question
),
solution=(
configuration.RETURN_ANSWERS_SOLUTION
if return_answers_solution is None
else return_answers_solution
),
)
all_labels = utils.read_all_labels(workspace.root)
loaded = load_annotation_data(workspace)
@@ -482,7 +524,10 @@ def run(
and path.is_dir()
and not path.name.startswith("Copie")
]
with concurrent.futures.ThreadPoolExecutor(max_workers=6) as executor:
# Each worker decodes a full-height returned group and its reference image.
# Keep this stage deliberately narrow; answer regeneration below has its own
# parallel executor and a much smaller per-task memory footprint.
with concurrent.futures.ThreadPoolExecutor(max_workers=SCAN_WORKERS) as executor:
futures = [
executor.submit(_scan_annotation_directory, path, only_ids)
for path in group_dirs
@@ -587,7 +632,28 @@ def build_parser() -> argparse.ArgumentParser:
parser.add_argument(
"--update-score",
action="store_true",
help="Override generated scores with values from existing score.json files",
help=(
"Regenerate images with current statement/solution PDFs while "
"preserving and applying existing score.json values"
),
)
parser.add_argument(
"--return-answers-context",
action=argparse.BooleanOptionalAction,
default=configuration.RETURN_ANSWERS_CONTEXT,
help="Include applicable context pages in individual answer exports",
)
parser.add_argument(
"--return-answers-question",
action=argparse.BooleanOptionalAction,
default=configuration.RETURN_ANSWERS_QUESTION,
help="Include the current question PDF in individual answer exports",
)
parser.add_argument(
"--return-answers-solution",
action=argparse.BooleanOptionalAction,
default=configuration.RETURN_ANSWERS_SOLUTION,
help="Include the current solution PDF in individual answer exports",
)
return parser
@@ -603,6 +669,9 @@ def main(argv: Sequence[str] | None = None) -> int:
refaire=args.refaire,
update_score=args.update_score,
annotation_dir=args.annotation_dir,
return_answers_context=args.return_answers_context,
return_answers_question=args.return_answers_question,
return_answers_solution=args.return_answers_solution,
)
return execute(parser, argv, handle)
+48 -12
View File
@@ -9,6 +9,7 @@ from pathlib import Path
from typing import Any
from pypdf import PdfWriter
from copienator.pdf_cut import split_pdf
from copienator import (
CliError,
@@ -22,7 +23,8 @@ from copienator import (
)
OPERATORS = ("-x", "->", "x>", "ss", "sx", "xx", "xs")
OPERATOR_PATTERN = re.compile(r"\s+(-x|->|x>|ss|sx|xx|xs)\s+")
CUT_PATTERN = re.compile(r"c\{(\d+(?:\.\d+)?)\}([12])([x>])")
OPERATOR_PATTERN = re.compile(r"\s+(-x|->|x>|ss|sx|xx|xs|c\{\d+(?:\.\d+)?\}[12][x>])\s+")
COPY_PATTERN = re.compile(r"Copie(\d+)\s+(.+)")
@@ -34,6 +36,11 @@ class ManualInstruction:
new_label: str
pipe_first: bool
@property
def cut(self) -> tuple[float, int] | None:
match = CUT_PATTERN.fullmatch(self.operator)
return (float(match[1]), int(match[2])) if match else None
@property
def should_merge(self) -> bool:
return self.operator.endswith(">")
@@ -48,10 +55,14 @@ def build_parser() -> argparse.ArgumentParser:
def parse_instructions(path: Path) -> list[ManualInstruction]:
return parse_instruction_text(path.read_text(encoding="utf-8"))
def parse_instruction_text(text: str) -> list[ManualInstruction]:
instructions: list[ManualInstruction] = []
malformed: list[int] = []
for line_number, raw_line in enumerate(
path.read_text(encoding="utf-8").splitlines(), start=1
text.splitlines(), start=1
):
line = raw_line.strip()
if not line or line.startswith("###"):
@@ -64,7 +75,10 @@ def parse_instructions(path: Path) -> list[ManualInstruction]:
right = line[operator_match.end() :].strip()
copy_match = COPY_PATTERN.fullmatch(left)
new_label = right.strip("|").strip()
if copy_match is None or not new_label:
cut_match = CUT_PATTERN.fullmatch(operator_match.group(1))
if (copy_match is None or not new_label
or (cut_match and (not 0 < float(cut_match[1]) < 100
or copy_match.group(2).strip() == new_label))):
malformed.append(line_number)
continue
instructions.append(
@@ -97,15 +111,25 @@ def set_suffix_and_clean_error(
item["result"]["suffix"] = suffix
error = item["result"].get("error", "")
if new_label_target:
if f"wrg-lbl:{new_label_target}?delayed" in error:
item["result"]["error"] = (
f"wrg-lbl-moved-to:{new_label_target}"
)
if f"(delayed){new_label_target}" in error:
item["result"]["error"] = error.replace(
f"(delayed){new_label_target}",
f"(->){new_label_target}",
)
# This instruction acknowledges this source/target conflict,
# including decisions to keep/discard PDFs without merging.
# Leave other pending targets (and other copies) untouched.
result = item["result"]
if "delayed" in result:
pending = [entry for entry in result["delayed"]
if entry not in (["wrong-label", new_label_target],
["add-label", new_label_target])]
if pending:
result["delayed"] = pending
else:
result.pop("delayed")
if error in {f"wrg-lbl:{new_label_target}?",
f"wrg-lbl:{new_label_target}?delayed",
f"wrg-lbl:{new_label_target}?exists"}:
error = f"wrg-lbl-moved-to:{new_label_target}"
error = error.replace(f"(delayed){new_label_target}", f"(->){new_label_target}")
error = error.replace(f"(->){new_label_target}?", f"(->){new_label_target}")
result["error"] = error
def get_actual_pdf(copies_dir: Path, copy_id: str, label: str) -> Path:
@@ -154,6 +178,9 @@ def resolve_manual(workspace: EvaluationWorkspace) -> ExitCode:
raise CliError("correction.json must contain a JSON object")
results: dict[str, Any] = loaded
instructions = parse_instructions(workspace.manual_resolutions_file)
cut_sources = [(item.copy_id, item.old_label) for item in instructions if item.cut]
if len(set(cut_sources)) != len(cut_sources):
raise CliError("Une seule coupe par label source est autorisée dans une résolution.")
initial_paths: dict[tuple[str, str], Path] = {}
current_paths: dict[tuple[str, str], Path] = {}
@@ -174,6 +201,15 @@ def resolve_manual(workspace: EvaluationWorkspace) -> ExitCode:
key_new = (instruction.copy_id, instruction.new_label)
source = initial_paths[key_old]
destination = current_paths[key_new]
if instruction.cut:
percent, keep = instruction.cut
first = source.parent / f"temp_{len(temp_files)}.pdf"
second = source.parent / f"temp_{len(temp_files) + 1}.pdf"
temp_files.extend((first, second))
split_pdf(source, percent, first, second)
retained, source = (first, second) if keep == 1 else (second, first)
current_paths[key_old] = retained
files_to_old.add(initial_paths[key_old])
temp_output = (
workspace.copies_dir
/ f"Copie{instruction.copy_id}"
+44 -18
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
import argparse
import math
import shutil
import tempfile
from collections import defaultdict
@@ -23,6 +24,7 @@ from copienator import (
from copienator.filesystem import staged_directory
SQUARE = 1000 // 38
ANSWER_TOP_PADDING_POINTS = 4 * 72 / 25.4
Coordinate = tuple[str, int, int, int, int, int]
ParsedCoordinate = tuple[str, str, int, int, int, int, int]
@@ -80,15 +82,9 @@ def _save_cropped_page(
y1: float,
output_path: Path,
) -> None:
page = document[page_number]
rotated_rectangle = page.rect * page.transformation_matrix
visual_crop = pymupdf.Rect(
rotated_rectangle.x0 + x0,
y0,
rotated_rectangle.x0 + x1,
y1,
)
unrotated_clip = visual_crop * page.derotation_matrix
# The source has been normalized by _prepare_split_pages: no rotation or
# CropBox translation remains in the coordinates passed to show_pdf_page.
visual_crop = pymupdf.Rect(x0, y0, x1, y1)
cropped = pymupdf.open()
try:
target_page = cropped.new_page(width=visual_crop.width, height=visual_crop.height)
@@ -96,14 +92,32 @@ def _save_cropped_page(
target_page.rect,
document,
page_number,
rotate=-page.rotation,
clip=unrotated_clip,
clip=visual_crop,
)
cropped.save(output_path)
finally:
cropped.close()
def _prepare_split_pages(document: pymupdf.Document) -> list[pymupdf.Rect]:
"""Use the label preview's full-page coordinates; retain visible bounds.
Work only on the in-memory document. Baking rotation into the content after
restoring the MediaBox avoids show_pdf_page's rotated CropBox offsets.
Intersecting with the saved bounds later preserves prior margin cropping.
"""
visible_bounds = []
for page in document:
crop = page.cropbox
media = page.mediabox
full_crop = pymupdf.Rect(media.x0, 0, media.x1, media.height)
page.set_cropbox(full_crop)
crop -= (full_crop.x0, full_crop.y0, full_crop.x0, full_crop.y0)
visible_bounds.append(crop * page.rotation_matrix)
page.remove_rotation()
return visible_bounds
def _render_split_outputs(
input_pdf: Path,
coords_list: list[Coordinate],
@@ -112,6 +126,7 @@ def _render_split_outputs(
"""Render every current answer into an otherwise empty staging directory."""
document = pymupdf.open(input_pdf)
try:
visible_bounds = _prepare_split_pages(document)
parsed = _parse_coordinates(coords_list)
parts_by_label: defaultdict[str, list[Path]] = defaultdict(list)
with tempfile.TemporaryDirectory(prefix="copienator-split-") as temp_directory:
@@ -158,18 +173,30 @@ def _render_split_outputs(
for page_number in range(start_page, end_page + 1):
page = document[page_number]
y0 = (y_start / 1000) * page.rect.height if page_number == start_page else 0
y0 = (
math.floor(max(
0,
(y_start / 1000) * page.rect.height
- ANSWER_TOP_PADDING_POINTS,
))
if page_number == start_page
else 0
)
y1 = (end_y / 1000) * page.rect.height if page_number == end_page else page.rect.height
if y1 <= y0 + 1:
clip = pymupdf.Rect(
fraction_x0 * page.rect.width, y0,
fraction_x1 * page.rect.width, y1,
) & visible_bounds[page_number] & page.rect
if clip.is_empty or clip.height <= 1 or clip.width <= 1:
continue
part_path = temporary / f"part-{index}-{page_number}.pdf"
_save_cropped_page(
document,
page_number,
fraction_x0 * page.rect.width,
y0,
fraction_x1 * page.rect.width,
y1,
clip.x0,
clip.y0,
clip.x1,
clip.y1,
part_path,
)
parts_by_label[clean_label].append(part_path)
@@ -264,4 +291,3 @@ def main(argv: Sequence[str] | None = None) -> int:
if __name__ == "__main__":
raise SystemExit(main())
+7
View File
@@ -33,6 +33,13 @@ else:
# Keep new optional settings compatible with older personal configuration files.
ALWAYS_CROP = False
RETURN_JPEG_ENABLED = True
RETURN_PDF_ENABLED = True
RETURN_ANSWERS_ENABLED = False
RETURN_ANSWERS_CONTEXT = False
RETURN_ANSWERS_QUESTION = True
RETURN_ANSWERS_SOLUTION = False
FINAL_SCORE_HISTOGRAM_PATH = Path("histogramme.pdf")
for _name in dir(_configuration):
if not _name.startswith("_"):
globals()[_name] = getattr(_configuration, _name)
+2 -1
View File
@@ -110,7 +110,8 @@ def main() -> None:
flush=True)
(args.output/'report.json').write_text(json.dumps(rows, indent=2)+'\n')
with (args.output/'report.csv').open('w') as stream:
writer = csv.DictWriter(stream, fieldnames=list(rows[0]))
fieldnames = list(dict.fromkeys(key for row in rows for key in row))
writer = csv.DictWriter(stream, fieldnames=fieldnames)
writer.writeheader()
writer.writerows(rows)
cards = []
+13
View File
@@ -0,0 +1,13 @@
"""Geometry checks shared by correction and annotation rendering."""
import math
from numbers import Real
def valid_feedback_box(box) -> bool:
return (
isinstance(box, (list, tuple)) and len(box) == 4
and all(isinstance(value, Real) and not isinstance(value, bool)
and math.isfinite(value) for value in box)
and box[0] < box[2] and box[1] < box[3]
)
+17 -4
View File
@@ -65,13 +65,26 @@ def _large_blank_ink(gray: np.ndarray, clean: np.ndarray, dpi: float):
n, labels, stats, centers = cv2.connectedComponentsWithStats(
(residual > 35).astype(np.uint8), 8)
keep = np.zeros(n, bool)
xx = stats[1:, 0]
ww, hh, area = stats[1:, 2], stats[1:, 3], stats[1:, 4]
keep[1:] = ((area >= .8*px*px) & (np.minimum(ww, hh) >= .6*px)
& (area / (ww*hh) > .3)
& (np.maximum(ww, hh) < 4*np.minimum(ww, hh)))
density = area / (ww*hh)
shortest, longest = np.minimum(ww, hh), np.maximum(ww, hh)
compact_ink = ((area >= .8*px*px) & (shortest >= .6*px)
& (density > .3) & (longest < 4*shortest))
# Ruling suppression can fragment faint pencil handwriting into sparse,
# elongated components. Admit those moderately more readily in the central
# 80% of the sheet. The outermost 9% deliberately uses a stricter filter:
# punched holes, torn binding edges, and page numbers usually occur there.
center_x = xx + ww/2
central = (center_x > width*.1) & (center_x < width*.9)
central_ink = (central & (area >= .55*px*px) & (shortest >= .45*px)
& (density > .18) & (longest < 6*shortest))
outer = (center_x < width*.09) | (center_x > width*.91)
outer_ink = (outer & (area >= 1.2*px*px) & (shortest >= .8*px)
& (density >= .4) & (longest < 3*shortest))
keep[1:] = np.where(outer, outer_ink, compact_ink | central_ink)
# Use side columns as a prior, then require repeated size and alignment. An
# isolated note in the same column remains eligible to protect the margin.
xx = stats[1:, 0]
candidates = np.flatnonzero(
((xx < 15*px) | (xx+ww > width-15*px))
& (ww > px) & (ww < 9*px) & (hh > px) & (hh < 12*px)) + 1
+57
View File
@@ -0,0 +1,57 @@
"""Split a PDF along its cumulative visible page height, in reading order."""
import math
from pathlib import Path
import pymupdf
from copienator.commands.splitting_int import _prepare_split_pages
# Percentages emitted by the GUI have six decimal places. Recover exact page
# boundaries despite rounding, without turning nearby in-page cuts into breaks.
BOUNDARY_TOLERANCE_PERCENT = 0.000001
def cut_position(heights: list[float], percent: float) -> tuple[int, float]:
"""Return (page index, offset); offset zero denotes an exact page break."""
if not heights or any(height <= 0 for height in heights):
raise ValueError("Le PDF doit contenir des pages non vides.")
if not math.isfinite(percent) or not 0 < percent < 100:
raise ValueError("Le pourcentage de coupe doit être strictement entre 0 et 100.")
total = sum(heights)
position = total * percent / 100
start = 0.0
for index, height in enumerate(heights):
if index and abs(percent - start / total * 100) <= BOUNDARY_TOLERANCE_PERCENT:
return index, 0.0
end = start + height
if index + 1 < len(heights) and abs(percent - end / total * 100) <= BOUNDARY_TOLERANCE_PERCENT:
return index + 1, 0.0
if position < end:
return index, position - start
start = end
raise ValueError("Coupe hors du document.")
def split_pdf(source: Path, percent: float, first_path: Path, second_path: Path) -> None:
"""Preserve whole pages; clip only the page actually crossed by the cut."""
with pymupdf.open(source) as document, pymupdf.open() as first, pymupdf.open() as second:
index, offset = cut_position([page.rect.height for page in document], percent)
if index:
first.insert_pdf(document, from_page=0, to_page=index - 1)
if offset == 0:
second.insert_pdf(document, from_page=index)
else:
with pymupdf.open() as page_document:
page_document.insert_pdf(document, from_page=index, to_page=index)
visible = _prepare_split_pages(page_document)[0]
for target, clip in (
(first, pymupdf.Rect(visible.x0, visible.y0, visible.x1, visible.y0 + offset)),
(second, pymupdf.Rect(visible.x0, visible.y0 + offset, visible.x1, visible.y1)),
):
page = target.new_page(width=clip.width, height=clip.height)
page.show_pdf_page(page.rect, page_document, 0, clip=clip)
if index + 1 < len(document):
second.insert_pdf(document, from_page=index + 1)
first.save(first_path)
second.save(second_path)
+12 -1
View File
@@ -2,6 +2,11 @@ from pathlib import Path
import io
from . import utils
PERSPECTIVE_GUIDANCE = (
"Ce barème est indicatif, si une réponse est entièrement correcte mais utilise "
"une méthode différente, elle mérite quand même tous les points."
)
main_prompt = """Je te fournis une image contenant plusieurs réponses manuscrites à un examen.
Chaque réponse est séparée de la précédente par une ligne horizontale noire.
@@ -103,7 +108,13 @@ def make_prompt(input_dir,full_label):
# print("Debug : l/t/c/p", full_label, text, corr, persp)
if persp:
persp = "\n\nVoici des consignes de notation complémentaires : \n\n```\n" + persp +"\n```\n"
persp = (
"\n\nVoici des consignes de notation complémentaires :\n\n"
+ PERSPECTIVE_GUIDANCE
+ "\n\n```\n"
+ persp
+ "\n```\n"
)
return main_prompt.replace("<<text>>", text).replace("<<corr>>", corr).replace("<<persp>>", persp).replace("<<label>>", full_label)
+125
View File
@@ -0,0 +1,125 @@
from __future__ import annotations
from pathlib import Path
from PIL import Image
from copienator import atomic_write_json, configuration, read_json, utils
from copienator.filesystem import staged_directory
from copienator.platform import safe_filename
RETURN_ANSWER_OPTIONS_FILE = Path(".copienator") / "return_answers.json"
def configured_return_answer_options() -> dict[str, bool]:
return {
"context": bool(configuration.RETURN_ANSWERS_CONTEXT),
"question": bool(configuration.RETURN_ANSWERS_QUESTION),
"solution": bool(configuration.RETURN_ANSWERS_SOLUTION),
}
def save_return_answer_options(
root: Path,
*,
context: bool,
question: bool,
solution: bool,
) -> None:
path = Path(root) / RETURN_ANSWER_OPTIONS_FILE
path.parent.mkdir(parents=True, exist_ok=True)
atomic_write_json(
path,
{"context": context, "question": question, "solution": solution},
)
def load_return_answer_options(root: Path) -> dict[str, bool]:
options = configured_return_answer_options()
path = Path(root) / RETURN_ANSWER_OPTIONS_FILE
if not path.is_file():
return options
loaded = read_json(path)
if not isinstance(loaded, dict):
raise ValueError(f"Expected a return-answer options object in {path}")
for name in options:
if name in loaded:
if type(loaded[name]) is not bool:
raise ValueError(f"Expected a boolean for {name!r} in {path}")
options[name] = loaded[name]
return options
def publish_answer_returns(
root: Path,
source: Path,
destination: Path,
*,
answers_only: bool = False,
) -> None:
"""Publish reviewed answers, optionally without touching return metadata."""
scores = read_json(source / "score.json")
if not isinstance(scores, dict):
raise ValueError(f"Expected a score object in {source}")
info_path = source / "info.json"
if not info_path.is_file():
raise ValueError(f"Missing {info_path}; recompile annotations before giving-names")
info = read_json(info_path)
if not isinstance(info, dict) or set(info) != set(scores):
raise ValueError(f"Invalid question information in {info_path}; recompile annotations")
for label, entry in info.items():
if (
not isinstance(entry, dict)
or set(entry) != {"present", "not_empty", "touched", "score"}
or any(type(entry[key]) is not bool for key in ("present", "not_empty", "touched"))
or (entry["not_empty"] and not entry["present"])
or (entry["touched"] and not entry["not_empty"])
):
raise ValueError(f"Invalid question information in {info_path}; recompile annotations")
# Match score.json, including manual score edits awaiting recompilation.
entry["score"] = scores[label]
answers_dir = destination / "answers"
if answers_dir.is_symlink():
raise ValueError(f"Expected a real answer directory: {answers_dir}")
if configuration.RETURN_ANSWERS_ENABLED:
options = load_return_answer_options(root)
labels = [label for label, entry in info.items() if entry["present"] and entry["not_empty"]]
# Import the rendering backend only when individual images are requested.
from copienator.commands.annotating import make_base_image
from copienator.commands.reading_annotations import concatenate
all_labels = utils.read_all_labels(root)
with staged_directory(answers_dir) as staging:
for label in sorted(labels, key=utils.natural_key):
paths = []
if options["context"]:
paths.extend(utils.pdf_images_of_contexts(root, label, all_labels))
if options["question"]:
paths.append(utils.pdf_image_of_enonce(root, label))
if options["solution"]:
paths.append(utils.pdf_image_of_solution(root, label))
images = []
for path in paths:
if path:
supplement, _, _ = make_base_image(path)
if supplement is None:
raise ValueError(f"Could not render {path}")
images.append(supplement)
with Image.open(source / f"{label}.jpg") as answer:
images.append(answer.convert("RGB"))
image = concatenate(images)
output = staging / f"{safe_filename(label)}.jpg"
if output.exists():
raise ValueError(
f"Answer labels produce the same filename in {answers_dir}: {label}"
)
image.save(output)
elif answers_dir.exists():
# Replace the managed directory with an empty one to remove stale exports.
with staged_directory(answers_dir):
pass
if not answers_only:
atomic_write_json(destination / "info.json", info)
(destination / "touched.json").unlink(missing_ok=True)
+21 -6
View File
@@ -160,23 +160,38 @@ def compile_to_pdf(text, output_pdf_path):
# env['TEXINPUTS'] = f".:{current_dir}:"
try:
subprocess.run(
result = subprocess.run(
['pdflatex', '-interaction=nonstopmode', tex_filename],
cwd=temp_dir,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
check=False
)
if "minted" in text:
subprocess.run(
result = subprocess.run(
['pdflatex', '-interaction=nonstopmode', tex_filename],
cwd=temp_dir,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
check=False)
if result.returncode != 0:
error_lines = [
line.strip() for line in result.stdout.splitlines()
if line.lstrip().startswith("!")
]
detail = f": {error_lines[0]}" if error_lines else ""
print(
f"Warning: LaTeX compilation failed for {output_pdf_path} "
f"(exit code {result.returncode}){detail}"
)
generated_pdf = os.path.join(temp_dir, pdf_filename)
if os.path.exists(generated_pdf):
shutil.move(generated_pdf, output_pdf_path)
else:
print(f"Warning: LaTeX compilation produced no PDF for {output_pdf_path}")
except Exception as e:
print(f"Compilation error for {output_pdf_path}: {e}")
+4
View File
@@ -109,6 +109,10 @@ class EvaluationWorkspace:
def correction_progress_file(self) -> Path:
return self.root / "correction_progress.json"
@property
def correction_pending_responses_file(self) -> Path:
return self.root / "correction_pending_responses.json"
@property
def batch_jobs_file(self) -> Path:
return self.root / "batch_jobs.json"
+436 -56
View File
@@ -2,6 +2,8 @@ from __future__ import annotations
import os
import queue
import re
import shutil
import tkinter as tk
from dataclasses import replace
from pathlib import Path
@@ -10,6 +12,7 @@ from typing import Any
from copienator import CliError, EvaluationWorkspace, ExitCode, atomic_write_json
from copienator.copy_errors import copy_errors
from copienator.filesystem import staged_files
from copienator.platform import (
WindowsLabelError,
add_platform_executable_paths,
@@ -18,6 +21,10 @@ from copienator.platform import (
)
from .diagnostics import collect_diagnostics
from .batch_monitor import BatchMonitor
from .giving_names import GivingNamesPanel, find_name_issues
from .notifications import notify_desktop
from .manual_resolution import ManualResolutionPanel
from .refaire import SECTION as REFAIRE_SECTION
from .refaire import (
RefaireSelection,
@@ -52,12 +59,106 @@ STATUS_LABELS = {
}
DEFAULT_HTTPS_PROXY = "http://10.0.0.1:3128"
CORRECTION_PROGRESS_RE = re.compile(
r"\[Progression correction\] Groupes traités : (\d+)/(\d+)"
)
PERSONAL_INTERRO_SOURCE = Path("/home/sebastien/Prépa/Staging/Interro")
ANNOTATION_DIRECTORIES = ("BGnot", "Bnot", "Anot")
ANNOTATION_VARIANT_DIRECTORIES = {
"simple": "Anot",
"checks": "Bnot",
"grouped": "BGnot",
}
ANNOTATION_VARIANT_READERS = {
"checks": "standard",
"grouped": "grouped",
}
def get_personal_interro_files(
evaluation: Path, source_directory: Path = PERSONAL_INTERRO_SOURCE
) -> tuple[Path, Path, Path]:
"""Copy an Interro project's statement sources into the evaluation."""
match = re.fullmatch(r"Interro(\d+)", evaluation.name)
if match is None:
raise ValueError("Le dossier d’évaluation doit sappeler Interro{id}, avec un identifiant numérique.")
project_name = evaluation.name
source_to_destination = (
(source_directory / f"{project_name}.pdf", "enonce.pdf"),
(source_directory / f"{project_name}.tex", "enonce.tex"),
(source_directory / f"{project_name}c.tex", "correction.tex"),
)
missing = [source.name for source, _destination in source_to_destination if not source.is_file()]
if missing:
raise FileNotFoundError(
"Fichier(s) source introuvable(s) dans "
f"{source_directory} : {', '.join(missing)}"
)
with staged_files(evaluation) as staging:
for source, destination_name in source_to_destination:
shutil.copy2(source, staging / destination_name)
return tuple(evaluation / name for _source, name in source_to_destination)
class Tooltip:
"""Small delayed tooltip for Tk and ttk widgets."""
def __init__(self, widget: tk.Widget, text: str, delay_ms: int = 450) -> None:
self.widget = widget
self.text = text
self.delay_ms = delay_ms
self._after_id: str | None = None
self.window: tk.Toplevel | None = None
widget.bind("<Enter>", self._schedule, add="+")
widget.bind("<Leave>", self.hide, add="+")
widget.bind("<ButtonPress>", self.hide, add="+")
def _schedule(self, _event: tk.Event | None = None) -> None:
self.hide()
self._after_id = self.widget.after(self.delay_ms, self.show)
def show(self) -> None:
self._after_id = None
if self.window is not None or not self.widget.winfo_exists():
return
self.window = tk.Toplevel(self.widget)
self.window.wm_overrideredirect(True)
self.window.attributes("-topmost", True)
x = self.widget.winfo_rootx() + 12
y = self.widget.winfo_rooty() + self.widget.winfo_height() + 6
self.window.wm_geometry(f"+{x}+{y}")
tk.Label(
self.window,
text=self.text,
justify="left",
wraplength=430,
background="#fffbd8",
foreground="#202020",
relief="solid",
borderwidth=1,
padx=8,
pady=5,
).pack()
def hide(self, _event: tk.Event | None = None) -> None:
if self._after_id is not None:
try:
self.widget.after_cancel(self._after_id)
except tk.TclError:
pass
self._after_id = None
if self.window is not None:
self.window.destroy()
self.window = None
def attach_tooltip(widget: tk.Widget, text: str) -> Tooltip:
tooltip = Tooltip(widget, text)
# Keep the tooltip easy to inspect and alive for as long as its widget.
widget._copienator_tooltip = tooltip # type: ignore[attr-defined]
return tooltip
def copy_pdf_paths(evaluation: Path) -> list[Path]:
@@ -166,8 +267,11 @@ class CopienatorApp(tk.Tk):
self.copy_paths: dict[str, Path] = {}
self._rendering = False
self.refaire_panel: RefaireSelection | None = None
self.giving_names_panel: GivingNamesPanel | None = None
self.complete_giving_names_button: ttk.Button | None = None
self.pending_refaire_commands: list[list[str]] = []
self.refaire_command_index = 0
self._correction_progress_buffer = ""
self.title("Copienator — assistant de correction")
self.geometry("1180x820")
@@ -184,6 +288,10 @@ class CopienatorApp(tk.Tk):
self.extra_var = tk.StringVar()
self.command_var = tk.StringVar(value="Sélectionnez une étape.")
self.info_var = tk.StringVar(value="Choisissez un dossier d’évaluation.")
self.batch_watch_status = tk.StringVar(value="Vérification automatique arrêtée.")
self.batch_monitor = BatchMonitor(
self, self._batch_results_ready, self._batch_watch_status_changed, self._append_console
)
self._build_ui(show_personal_steps)
self.extra_var.trace_add("write", lambda *_args: self._update_command_preview())
@@ -231,19 +339,33 @@ class CopienatorApp(tk.Tk):
)
environment = ttk.Frame(top)
environment.grid(row=1, column=2, columnspan=5, sticky="e", pady=(7, 0))
ttk.Checkbutton(
proxy_toggle = ttk.Checkbutton(
environment,
text="Utiliser le proxy HTTPS",
variable=self.use_proxy_var,
command=self._toggle_proxy,
).pack(side="left", padx=(0, 6))
)
proxy_toggle.pack(side="left", padx=(0, 6))
attach_tooltip(
proxy_toggle,
"Active le proxy HTTPS configuré dans le champ voisin pour les commandes lancées par le GUI.",
)
self.proxy_entry = ttk.Entry(environment, textvariable=self.proxy_var, width=28, state="disabled")
self.proxy_entry.pack(side="left")
ttk.Checkbutton(
attach_tooltip(
self.proxy_entry,
"Adresse du proxy HTTPS transmise aux commandes lorsque loption de proxy est activée.",
)
verbose_toggle = ttk.Checkbutton(
environment,
text="Afficher les détails en cas derreur (--verbose)",
text="Détails derreur ⓘ",
variable=self.verbose_var,
).pack(side="left", padx=(10, 0))
)
verbose_toggle.pack(side="left", padx=(10, 0))
attach_tooltip(
verbose_toggle,
"Ajoute --verbose aux commandes compatibles afin dafficher la trace complète lorsquune erreur inattendue survient.",
)
profile = "standard + personnel" if show_personal_steps else "standard"
ttk.Label(environment, text=f"Profil : {profile}").pack(side="left", padx=(12, 0))
@@ -265,13 +387,23 @@ class CopienatorApp(tk.Tk):
self.detail = ttk.Frame(main_pane, padding=(12, 6, 4, 4))
self.detail.columnconfigure(0, weight=1)
self.detail.rowconfigure(2, weight=1)
self.detail.rowconfigure(3, weight=1)
self.title_label = ttk.Label(self.detail, text="Sélectionnez une étape", font=("TkDefaultFont", 15, "bold"))
self.title_label.grid(row=0, column=0, sticky="w")
self.description_label = ttk.Label(self.detail, text="", wraplength=680, justify="left")
self.description_label.grid(row=1, column=0, sticky="ew", pady=(5, 8))
ttk.Style(self).configure("MissingPrerequisite.TLabel", foreground="#c62828")
self.missing_requirements_label = ttk.Label(
self.detail,
text="",
style="MissingPrerequisite.TLabel",
wraplength=680,
justify="left",
)
self.missing_requirements_label.grid(row=2, column=0, sticky="ew", pady=(0, 8))
self.missing_requirements_label.grid_remove()
form_container = ttk.Frame(self.detail)
form_container.grid(row=2, column=0, sticky="nsew")
form_container.grid(row=3, column=0, sticky="nsew")
form_container.columnconfigure(0, weight=1)
form_container.rowconfigure(0, weight=1)
self.form_canvas = tk.Canvas(form_container, highlightthickness=0, width=1, height=1)
@@ -286,7 +418,7 @@ class CopienatorApp(tk.Tk):
self.form.columnconfigure(1, weight=1)
command_box = ttk.LabelFrame(self.detail, text="Commande", padding=6)
command_box.grid(row=3, column=0, sticky="ew", pady=(8, 5))
command_box.grid(row=4, column=0, sticky="ew", pady=(8, 5))
command_box.columnconfigure(0, weight=1)
ttk.Label(command_box, textvariable=self.command_var, wraplength=690, justify="left").grid(
row=0, column=0, sticky="ew"
@@ -295,7 +427,7 @@ class CopienatorApp(tk.Tk):
self.copy_command_button.grid(row=1, column=0, sticky="w", pady=(5, 0))
buttons = ttk.Frame(self.detail)
buttons.grid(row=4, column=0, sticky="ew", pady=(5, 0))
buttons.grid(row=5, column=0, sticky="ew", pady=(5, 0))
self.previous_button = ttk.Button(buttons, text="← Précédente", command=lambda: self._move_selection(-1))
self.previous_button.pack(side="left")
self.next_button = ttk.Button(buttons, text="Suivante →", command=lambda: self._move_selection(1))
@@ -429,6 +561,7 @@ class CopienatorApp(tk.Tk):
if not evaluation or not evaluation.is_dir():
messagebox.showerror("Dossier invalide", "Choisissez un dossier d’évaluation existant.")
return
self.batch_monitor.stop()
self._save_current_form()
self.state_store.load(evaluation)
for step in self.steps:
@@ -530,6 +663,40 @@ class CopienatorApp(tk.Tk):
self._render_step()
self.info_var.set("Fichiers revérifiés — " + ("manquants : " + ", ".join(missing) if missing else "tous les fichiers dentrée sont présents."))
def _get_input_files(self) -> None:
evaluation = self.evaluation
if (
not self.show_personal_steps
or not evaluation
or not self.state_store.evaluation
or self.active_step_id
or self.runner.running
):
return
try:
copied = get_personal_interro_files(evaluation)
except (OSError, ValueError) as exc:
messagebox.showerror("Récupération impossible", str(exc))
return
missing = self._missing_requirements(self.step_by_id["inputs"])
if missing:
self.state_store.update_step("inputs", status="ready")
self._populate_tree()
self._render_step()
messagebox.showwarning(
"Prérequis encore manquant",
"Les trois fichiers ont été copiés, mais l’étape ne peut pas encore "
"être terminée :\n\n" + "\n".join(missing),
)
return
self._mark_step("success", automatic=True, reason="Fichiers dentrée récupérés")
self.info_var.set(
"Fichiers récupérés : " + ", ".join(path.name for path in copied)
)
self._move_selection_from("inputs", 1)
def _on_tree_select(self, _event: tk.Event[Any] | None = None) -> None:
selection = self.tree.selection()
if not selection or selection[0].startswith("section:"):
@@ -596,7 +763,15 @@ class CopienatorApp(tk.Tk):
return
self._rendering = True
redo = step.section == REFAIRE_SECTION
expanded_form = redo or step.id in {"page_splitter", "crop_blank_margins", "cutleft", "labels"}
expanded_form = redo or step.id in {
"page_splitter",
"crop_blank_margins",
"cutleft",
"labels",
"plotting",
"manual_resolution",
"giving_names",
}
self.console.configure(height=8 if expanded_form else 14)
self.rowconfigure(1, weight=4 if expanded_form else 3)
self.rowconfigure(2, weight=1 if expanded_form else 2)
@@ -607,6 +782,8 @@ class CopienatorApp(tk.Tk):
child.destroy()
self.arg_vars.clear()
self.refaire_panel = None
self.giving_names_panel = None
self.complete_giving_names_button = None
entry = self.state_store.step(step.id) if self.state_store.evaluation else {}
saved_variant = entry.get("variant", step.variants[0].id)
if saved_variant not in {variant.id for variant in step.variants}:
@@ -619,10 +796,15 @@ class CopienatorApp(tk.Tk):
if step.id == "statement" and saved_variant == "personal":
description = ("Lit les blocs SHEETINFO de enonce.tex et récupère les énoncés, solutions "
"et barèmes du service personnel (localhost:8080). Crée un groupe par exercice. "
"Les actions Gemini ci-dessous restent facultatives.")
if missing:
description += "\nPrérequis non détectés : " + ", ".join(missing)
"Les deux étapes Gemini suivantes restent facultatives.")
self.description_label.configure(text=description)
if missing:
self.missing_requirements_label.configure(
text="Prérequis non détectés : " + ", ".join(missing)
)
self.missing_requirements_label.grid()
else:
self.missing_requirements_label.grid_remove()
row = self._render_context_controls(step, 0)
if len(step.variants) > 1:
@@ -647,7 +829,9 @@ class CopienatorApp(tk.Tk):
choices = spec.choices
if spec.name == "annotation_dir" and step.id in {"export", "import"}:
choices, value = self._annotation_directory_choices(step.id)
ttk.Label(self.form, text=spec.label).grid(row=row, column=0, sticky="nw", pady=4, padx=(0, 8))
argument_label = ttk.Label(self.form, text=f"{spec.label}")
argument_label.grid(row=row, column=0, sticky="nw", pady=4, padx=(0, 8))
attach_tooltip(argument_label, spec.help)
if spec.kind == "bool":
variable: tk.Variable = tk.BooleanVar(value=bool(value))
widget = ttk.Checkbutton(self.form, variable=variable)
@@ -672,11 +856,19 @@ class CopienatorApp(tk.Tk):
)
self.arg_vars[spec.name] = variable
variable.trace_add("write", lambda *_args: self._update_command_preview())
if spec.help:
ttk.Label(self.form, text=spec.help, foreground="#666666", wraplength=540).grid(
row=row + 1, column=1, sticky="w"
)
attach_tooltip(widget, spec.help)
row += 1
if step.id == "manual_resolution":
def manual_evaluation():
target = self.arg_vars.get("target")
value = target.get().strip() if target else ""
return (self.repository / value).resolve() if value else self.evaluation
self.manual_panel = ManualResolutionPanel(self.form, manual_evaluation)
self.manual_panel.grid(row=row, column=0, columnspan=2, sticky="nsew", pady=8)
if "target" in self.arg_vars:
self.arg_vars["target"].trace_add("write", lambda *_args: self.manual_panel.reload())
row += 1
show_extra = bool(step.extra_arguments_help) and (
@@ -684,27 +876,15 @@ class CopienatorApp(tk.Tk):
or variant.id in step.extra_arguments_variants
)
if show_extra and not step.is_manual and step.section != REFAIRE_SECTION:
ttk.Label(self.form, text="Arguments supplémentaires").grid(
extra_label = ttk.Label(self.form, text="Arguments supplémentaires")
extra_label.grid(
row=row, column=0, sticky="w", pady=(10, 4), padx=(0, 8)
)
ttk.Entry(self.form, textvariable=self.extra_var).grid(row=row, column=1, sticky="ew", pady=(10, 4))
extra_entry = ttk.Entry(self.form, textvariable=self.extra_var)
extra_entry.grid(row=row, column=1, sticky="ew", pady=(10, 4))
attach_tooltip(extra_label, step.extra_arguments_help)
attach_tooltip(extra_entry, step.extra_arguments_help)
row += 1
ttk.Label(
self.form,
text=step.extra_arguments_help,
foreground="#666666",
wraplength=540,
justify="left",
).grid(row=row, column=1, sticky="w")
row += 1
if self.show_personal_steps and step.id == "statement":
actions = ttk.LabelFrame(self.form, text="Après génération — facultatif", padding=7)
actions.grid(row=row, column=0, columnspan=2, sticky="ew", pady=(10, 8))
for ident, title in (("statement_groups", "Regrouper avec Gemini…"),
("statement_persp", "Remplacer Persp avec Gemini…")):
ttk.Button(actions, text=title, command=lambda target=ident: self._select_statement_action(target)).pack(
side="left", padx=(0, 6))
self.run_button.configure(text="Enregistrer la sélection" if step.id == "refaire_selection" else ("Marquer terminée" if step.is_manual else "Exécuter"))
self.skip_button.configure(state="normal" if step.optional else "disabled")
@@ -713,6 +893,23 @@ class CopienatorApp(tk.Tk):
self._update_controls()
def _render_context_controls(self, step: StepDefinition, row: int) -> int:
if step.id == "batch_status":
controls = ttk.Frame(self.form)
controls.grid(row=row, column=0, columnspan=2, sticky="w", pady=8)
self.watch_batches_button = ttk.Button(
controls, text="Vérifier toutes les 5 minutes", command=self._start_batch_watch
)
self.watch_batches_button.pack(side="left")
self.stop_watch_batches_button = ttk.Button(
controls, text="Arrêter les vérifications", command=self.batch_monitor.stop
)
self.stop_watch_batches_button.pack(side="left", padx=8)
ttk.Label(self.form, textvariable=self.batch_watch_status).grid(
row=row + 1, column=0, columnspan=2, sticky="w"
)
ttk.Label(self.form, text="Gardez Copienator ouvert. Une notification de bureau sera envoyée lorsque tous les résultats seront prêts.",
wraplength=650).grid(row=row + 2, column=0, columnspan=2, sticky="w", pady=8)
return row + 3
if step.id == "rename":
self.validate_rename_button = ttk.Button(
self.form, text="Valider sans renommer", command=self._validate_rename_step
@@ -723,7 +920,41 @@ class CopienatorApp(tk.Tk):
ttk.Button(self.form, text="Recharger — vérifier à nouveau", command=self._reload_inputs).grid(
row=row, column=0, columnspan=2, sticky="w", pady=(0, 8)
)
return row + 1
row += 1
if self.show_personal_steps:
self.get_input_files_button = ttk.Button(
self.form, text="Get the files", command=self._get_input_files
)
self.get_input_files_button.grid(
row=row, column=0, columnspan=2, sticky="w", pady=(0, 8)
)
row += 1
return row
if step.id == "giving_names" and self.evaluation:
self.giving_names_panel = GivingNamesPanel(
self.form, self.evaluation
)
self.giving_names_panel.grid(
row=row,
column=0,
columnspan=2,
sticky="ew",
pady=(0, 8),
)
row += 1
self.complete_giving_names_button = ttk.Button(
self.form,
text="Marquer terminée",
command=self._complete_giving_names,
)
self.complete_giving_names_button.grid(
row=row,
column=0,
columnspan=2,
sticky="w",
pady=(0, 8),
)
row += 1
if step.section == REFAIRE_SECTION:
evaluation = self.evaluation
if step.id in {"refaire_selection", "refaire_correct"}:
@@ -750,7 +981,14 @@ class CopienatorApp(tk.Tk):
return row + 1
if step.id in {"review_persp", "correction"}:
row = self._correction_folder_buttons(row)
if step.id in {"page_splitter", "crop_blank_margins", "cutleft", "labels"}:
copy_selection_steps = {
"page_splitter",
"crop_blank_margins",
"cutleft",
"labels",
"plotting",
}
if step.id in copy_selection_steps:
evaluation = self.evaluation
paths = copy_pdf_paths(evaluation) if evaluation else []
self.copy_paths = {path.name: path for path in paths}
@@ -781,7 +1019,7 @@ class CopienatorApp(tk.Tk):
command=self._open_selected_copy,
state="normal" if names else "disabled",
).grid(row=1, column=1, padx=(6, 0), pady=(7, 0))
if step.id in {"page_splitter", "crop_blank_margins", "cutleft", "labels"}:
if step.id in copy_selection_steps:
actions = ttk.Frame(copies)
actions.grid(row=2, column=0, columnspan=2, sticky="w", pady=(7, 0))
label = "Refaire la copie sélectionnée" if step.id == "page_splitter" else "Cibler la copie sélectionnée"
@@ -793,6 +1031,8 @@ class CopienatorApp(tk.Tk):
if step.id == "page_splitter"
else "Seule la copie ciblée sera analysée par Gemini. "
if step.id == "labels"
else "Seule la copie ciblée sera vérifiée. "
if step.id == "plotting"
else "Seule la copie ciblée sera traitée. "
)
ttk.Label(copies, text=help_text + "Vérifiez la commande puis cliquez sur Exécuter.",
@@ -824,14 +1064,6 @@ class CopienatorApp(tk.Tk):
row += 1
return row
def _select_statement_action(self, step_id: str) -> None:
self._save_current_form()
target = self.arg_vars.get("target")
if target:
self.state_store.update_step(step_id, values={"target": target.get()})
self.tree.selection_set(step_id)
self.tree.see(step_id)
def _refaire_restart_controls(self, row: int) -> int:
controls = ttk.Frame(self.form)
controls.grid(row=row, column=0, columnspan=2, sticky="w", pady=(0, 5))
@@ -1159,6 +1391,8 @@ class CopienatorApp(tk.Tk):
if not step or not evaluation or not self.state_store.evaluation:
messagebox.showerror("Évaluation absente", "Chargez dabord un dossier d’évaluation.")
return
if step.id == "batch_status" and self.batch_monitor.active:
return
if self.active_step_id or self.runner.running:
messagebox.showwarning("Traitement en cours", "Interrompez le traitement actuel avant den lancer un autre.")
return
@@ -1173,7 +1407,6 @@ class CopienatorApp(tk.Tk):
return
if step.is_manual:
self._mark_step("success")
if step.section == REFAIRE_SECTION:
self._move_selection_from(step.id, 1)
return
if os.name == "nt" and step.id != "statement":
@@ -1221,6 +1454,9 @@ class CopienatorApp(tk.Tk):
last_run_values=run_values,
)
self.active_step_id = step.id
if step.id == "correction":
self._correction_progress_buffer = ""
self.info_var.set("Correction : préparation des groupes…")
workspace = self.state_store.workspace
assert workspace is not None
log_path = workspace.log_path(step.id)
@@ -1284,6 +1520,33 @@ class CopienatorApp(tk.Tk):
self._populate_tree()
self.info_var.set("Renommage validé — aucun fichier modifié.")
def _complete_giving_names(self) -> None:
if (
not self.current_step
or self.current_step.id != "giving_names"
or not self.evaluation
or self.active_step_id
or self.runner.running
):
return
return_dir = EvaluationWorkspace(self.evaluation).return_dir
if not return_dir.is_dir():
messagebox.showerror(
"A Rendre absent",
"Exécutez dabord lattribution des noms.",
)
return
issues = find_name_issues(self.evaluation)
if issues and not messagebox.askyesno(
"Noms encore à vérifier",
f"{len(issues)} copie(s) ont encore un nom inconnu ou dupliqué. "
"Marquer quand même l’étape comme terminée ?",
icon="warning",
):
return
self._mark_step("success")
self._move_selection_from("giving_names", 1)
def _skip_step(self) -> None:
if self.current_step and self.current_step.optional:
step_id = self.current_step.id
@@ -1298,7 +1561,9 @@ class CopienatorApp(tk.Tk):
except queue.Empty:
break
if event == "output":
self._append_console(str(payload))
output = str(payload)
self._track_correction_progress(output)
self._append_console(output)
elif event == "input_echo":
self._append_console(f"> {payload}")
elif event == "runner_error":
@@ -1308,6 +1573,57 @@ class CopienatorApp(tk.Tk):
self._finish_process(int(return_code), bool(interrupted))
self.after(60, self._poll_runner)
def _track_correction_progress(self, output: str) -> None:
if self.active_step_id != "correction":
return
self._correction_progress_buffer = (
self._correction_progress_buffer + output
)[-512:]
matches = list(CORRECTION_PROGRESS_RE.finditer(
self._correction_progress_buffer
))
if not matches:
return
completed, total = (int(value) for value in matches[-1].groups())
percent = round(100 * completed / total) if total else 100
self.info_var.set(
f"Correction : {completed} groupe(s) traité(s) sur {total} "
f"({percent} %)."
)
def _batch_watch_status_changed(self, message: str) -> None:
self.batch_watch_status.set(message)
self._update_controls()
def _start_batch_watch(self) -> None:
if self.active_step_id or self.runner.running or not self.state_store.evaluation:
return
step = self.step_by_id["batch_status"]
command = build_command(self.repository, step, step.variants[0], {},
str(self.state_store.evaluation))
environment = build_runner_environment(
os.environ, self.api_key_var.get(), self.proxy_var.get(), self.use_proxy_var.get()
)
self.batch_monitor.start(command, self.repository, environment,
self.state_store.workspace.log_path("batch_status_watch"))
def _batch_results_ready(self) -> None:
evaluation = self.state_store.evaluation
if evaluation is None:
return
self.state_store.update_step("batch_status", status="success", return_code=0)
self._populate_tree()
message = f"{evaluation.name} : tous les résultats batch sont prêts à récupérer."
self.info_var.set(message)
try:
notify_desktop("Copienator — batchs terminés", message)
except (OSError, RuntimeError) as exc:
self.bell()
self._append_console(f"Notification de bureau indisponible : {exc}\n{message}\n")
if (self.current_step and self.current_step.id == "batch_status"
and not self.active_step_id and not self.runner.running):
self._move_selection_from("batch_status", 1)
def _finish_process(self, return_code: int, interrupted: bool) -> None:
step_id = self.active_step_id
if not step_id:
@@ -1353,7 +1669,25 @@ class CopienatorApp(tk.Tk):
"été conservés.",
)
return
self.state_store.update_step(step_id, status=status, return_code=return_code)
stored_status = (
"detected"
if step_id == "giving_names" and status == "success"
else status
)
self.state_store.update_step(
step_id, status=stored_status, return_code=return_code
)
if step_id == "annotation" and status == "success":
generation_variant = self.state_store.step("annotation").get(
"last_run_variant"
)
reader_variant = ANNOTATION_VARIANT_READERS.get(
str(generation_variant)
)
if reader_variant:
self.state_store.update_step(
"read_annotations", variant=reader_variant
)
self.state_store.add_history(
{
"step": step_id,
@@ -1368,8 +1702,27 @@ class CopienatorApp(tk.Tk):
self.active_step_id = None
self._populate_tree()
self._update_controls()
if step_id == "giving_names" and status == "success":
if self.giving_names_panel:
self.giving_names_panel.reload()
self.info_var.set(
"A Rendre a été préparé. Vérifiez les noms, puis cliquez "
"sur « Marquer terminée »."
)
return
if status == "success":
self.after_idle(lambda completed_id=step_id: self._move_selection_from(completed_id, 1))
if step_id == "batch_status" and self.batch_monitor.active:
self.batch_monitor.stop()
self._batch_results_ready()
return
if step_id == "manual_resolution":
self.after_idle(self._select_refaire_correction)
else:
self.after_idle(
lambda completed_id=step_id: self._move_selection_from(
completed_id, 1
)
)
def _send_input(self) -> None:
text = self.stdin_var.get()
@@ -1381,12 +1734,21 @@ class CopienatorApp(tk.Tk):
self.stdin_var.set("")
def _interrupt(self) -> None:
if self.runner.running and messagebox.askyesno(
"Interrompre", "Envoyer une interruption au script en cours ?"
):
graceful_correction = self.active_step_id == "correction"
question = (
"Arrêter de lancer de nouveaux appels Gemini ? Les appels déjà en "
"cours pourront se terminer et leurs résultats seront sauvegardés."
if graceful_correction
else "Envoyer une interruption au script en cours ?"
)
if self.runner.running and messagebox.askyesno("Interrompre", question):
try:
self.runner.interrupt()
self.info_var.set("Interruption demandée. Utilisez « Forcer larrêt » si le script ne répond pas.")
self.info_var.set(
"Arrêt demandé : attente des appels Gemini en cours…"
if graceful_correction
else "Interruption demandée. Utilisez « Forcer larrêt » si le script ne répond pas."
)
except OSError as exc:
messagebox.showerror("Interruption impossible", str(exc))
@@ -1412,18 +1774,27 @@ class CopienatorApp(tk.Tk):
def _update_controls(self) -> None:
running = bool(self.active_step_id) or self.runner.running
if self.current_step and self.current_step.id == "batch_status":
self.watch_batches_button.configure(state="disabled" if running or self.batch_monitor.active or not self.state_store.evaluation else "normal")
self.stop_watch_batches_button.configure(state="normal" if self.batch_monitor.active else "disabled")
if running and self.current_step and self.current_step.id in {"page_splitter", "cutleft"}:
self.marked_copies_button.configure(state="disabled")
if self.current_step and self.current_step.id == "rename":
self.validate_rename_button.configure(
state="disabled" if running or not self.state_store.evaluation else "normal"
)
self.run_button.configure(state="disabled" if running or not self.current_step else "normal")
watching_current = bool(self.current_step and self.current_step.id == "batch_status"
and self.batch_monitor.active)
self.run_button.configure(state="disabled" if running or not self.current_step or watching_current else "normal")
self.skip_button.configure(state="normal" if not running and self.current_step and self.current_step.optional else "disabled")
self.interrupt_button.configure(state="normal" if running else "disabled")
self.force_button.configure(state="normal" if running else "disabled")
self.send_button.configure(state="normal" if running else "disabled")
self.stdin_entry.configure(state="normal" if running else "disabled")
if self.complete_giving_names_button is not None:
self.complete_giving_names_button.configure(
state="disabled" if running else "normal"
)
def _move_selection(self, delta: int) -> None:
if not self.current_step:
@@ -1442,6 +1813,14 @@ class CopienatorApp(tk.Tk):
self.tree.selection_set(ids[target])
self.tree.see(ids[target])
def _select_refaire_correction(self) -> None:
"""Return from manual conflict resolution to the main redo correction."""
if self.runner.running or self.active_step_id or not self.state_store.evaluation:
return
self.state_store.update_step("correction", variant="refaire")
self.tree.selection_set("correction")
self.tree.see("correction")
def _on_close(self) -> None:
if self.runner.running:
if not messagebox.askyesno(
@@ -1453,4 +1832,5 @@ class CopienatorApp(tk.Tk):
except OSError:
pass
self._save_current_form()
self.batch_monitor.stop()
self.destroy()
+81
View File
@@ -0,0 +1,81 @@
"""Non-blocking, cancellable batch checks while the desktop GUI is open."""
import queue
from .runner import ProcessRunner
CHECK_INTERVAL_MS = 5 * 60 * 1000
class BatchMonitor:
def __init__(self, scheduler, on_ready, on_status, on_output):
self.scheduler = scheduler
self.on_ready = on_ready
self.on_status = on_status
self.on_output = on_output
self.active = False
self.timer = None
self.runner = None
def start(self, command, cwd, environment, log_path):
if self.active:
return
self.arguments = (command, cwd, environment, log_path)
self.active = True
self._check()
def stop(self):
self.active = False
if self.timer is not None:
self.scheduler.after_cancel(self.timer)
self.timer = None
if self.runner is not None:
try:
self.runner.force_stop()
except OSError as exc:
self.on_output(f"Arrêt de la vérification : {exc}\n")
self.runner = None
self.on_status("Vérification automatique arrêtée.")
def _later(self):
self.timer = self.scheduler.after(CHECK_INTERVAL_MS, self._check)
def _check(self):
self.timer = None
if not self.active:
return
self.runner = ProcessRunner()
self.on_status("Vérification des batchs en cours…")
try:
self.runner.start(*self.arguments)
except (OSError, RuntimeError) as exc:
self.on_output(f"Vérification impossible : {exc}\n")
self.runner = None
self.on_status("Échec de la vérification. Nouvel essai dans 5 minutes.")
self._later()
return
self.timer = self.scheduler.after(100, self._poll)
def _poll(self):
self.timer = None
if not self.active or self.runner is None:
return
while True:
try:
event, payload = self.runner.events.get_nowait()
except queue.Empty:
break
if event in {"output", "runner_error"}:
self.on_output(str(payload))
elif event == "finished":
code, interrupted = payload
self.runner = None
if code == 0 and not interrupted:
self.active = False
self.on_status("Tous les résultats batch sont prêts.")
self.on_ready()
else:
self.on_status("Résultats pas encore prêts. Nouvelle vérification dans 5 minutes.")
self._later()
return
self.timer = self.scheduler.after(100, self._poll)
+131
View File
@@ -0,0 +1,131 @@
from __future__ import annotations
import tkinter as tk
from pathlib import Path
from tkinter import messagebox, ttk
import pymupdf
from PIL import Image, ImageTk
from copienator.pdf_cut import cut_position
PAGE_GAP = 28
SNAP_PIXELS = 12
def percentage_at_y(y: float, heights: list[float], scale: float) -> float:
"""Convert canvas y to document height, snapping across inter-page gaps."""
top = 0.0
cumulative = 0.0
total = sum(heights)
for index, height in enumerate(heights):
bottom = top + height * scale
if index + 1 < len(heights) and bottom - SNAP_PIXELS <= y <= bottom + PAGE_GAP + SNAP_PIXELS:
return (cumulative + height) / total * 100
if y <= bottom:
return max(0.0, min(100.0, (cumulative + (y - top) / scale) / total * 100))
cumulative += height
top = bottom + PAGE_GAP
return 100.0
def y_at_percentage(percent: float, heights: list[float], scale: float) -> float:
if percent <= 0:
return 0.0
if percent >= 100:
return sum(heights) * scale + (len(heights) - 1) * PAGE_GAP
index, offset = cut_position(heights, percent)
top = sum(heights[:index]) * scale + index * PAGE_GAP
return top - PAGE_GAP / 2 if index and offset == 0 else top + offset * scale
def cut_operator(percent: float, keep: int, mode: str) -> str:
value = f"{percent:.6f}".rstrip("0").rstrip(".")
return f"c{{{value}}}{keep}{mode}"
class CutHelper(tk.Toplevel):
def __init__(self, parent, path: Path, on_accept, initial=None):
# Load the source before creating a window so invalid PDFs leave no dialog.
with pymupdf.open(path) as document:
if not len(document):
raise ValueError("Le PDF est vide.")
self.heights = [page.rect.height for page in document]
width = min(850, parent.winfo_screenwidth() - 100)
self.scale = min(1.5, width / max(page.rect.width for page in document))
rendered = []
for page in document:
pix = page.get_pixmap(matrix=pymupdf.Matrix(self.scale, self.scale), alpha=False,
colorspace=pymupdf.csRGB)
rendered.append(Image.frombytes("RGB", (pix.width, pix.height), pix.samples))
super().__init__(parent)
self.title(f"Cut — {path.name}")
self.geometry(f"{width + 45}x{min(850, parent.winfo_screenheight() - 100)}")
self.transient(parent.winfo_toplevel())
self.on_accept = on_accept
self.percent = initial[0] if initial else 50.0
self.keep = tk.IntVar(value=initial[1] if initial else 1)
self.mode = tk.StringVar(value=initial[2] if initial else ">")
self.caption = tk.StringVar()
ttk.Label(self, text="Déplacez la barre rouge. Entrée : afficher la commande ; Échap : annuler.",
wraplength=width).pack(anchor="w", padx=8, pady=5)
controls = ttk.Frame(self)
controls.pack(fill="x", padx=8)
ttk.Label(controls, text="Conserver à la source :").pack(side="left")
ttk.Radiobutton(controls, text="1 — début", variable=self.keep, value=1).pack(side="left")
ttk.Radiobutton(controls, text="2 — fin", variable=self.keep, value=2).pack(side="left")
ttk.Radiobutton(controls, text="Ajouter à la cible", variable=self.mode, value=">").pack(side="left")
ttk.Radiobutton(controls, text="Remplacer", variable=self.mode, value="x").pack(side="left")
ttk.Label(self, textvariable=self.caption).pack(anchor="w", padx=8, pady=5)
viewport = ttk.Frame(self)
viewport.pack(fill="both", expand=True)
self.canvas = tk.Canvas(viewport, background="#555555", highlightthickness=0)
scrollbar = ttk.Scrollbar(viewport, command=self.canvas.yview)
self.canvas.configure(yscrollcommand=scrollbar.set)
scrollbar.pack(side="right", fill="y")
self.canvas.pack(fill="both", expand=True)
self.images = [ImageTk.PhotoImage(image, master=self) for image in rendered]
top = 0.0
self.width = width
for index, image in enumerate(self.images):
self.canvas.create_image(0, top, anchor="nw", image=image)
top += self.heights[index] * self.scale
if index + 1 < len(self.images):
top += PAGE_GAP
self.canvas.configure(scrollregion=(0, 0, width, top))
self.bar = self.canvas.create_line(0, 0, width, 0, fill="#ff3030", width=4)
self.canvas.bind("<Button-1>", self.move_bar)
self.canvas.bind("<B1-Motion>", self.move_bar)
self.canvas.bind("<Button-4>", lambda event: self.canvas.yview_scroll(-3, "units"))
self.canvas.bind("<Button-5>", lambda event: self.canvas.yview_scroll(3, "units"))
self.canvas.bind("<MouseWheel>", lambda event: self.canvas.yview_scroll(-1 if event.delta > 0 else 1, "units"))
self.bind("<Return>", self.accept)
self.bind("<Escape>", lambda event: self.destroy())
self.keep.trace_add("write", lambda *_: self.draw_bar())
self.mode.trace_add("write", lambda *_: self.draw_bar())
self.draw_bar()
self.canvas.yview_moveto(max(0, (y_at_percentage(self.percent, self.heights, self.scale) - 200) / top))
self.focus_set()
self.grab_set()
def move_bar(self, event):
self.percent = percentage_at_y(self.canvas.canvasy(event.y), self.heights, self.scale)
self.draw_bar()
def draw_bar(self):
y = y_at_percentage(self.percent, self.heights, self.scale)
self.canvas.coords(self.bar, 0, y, self.width, y)
text = cut_operator(self.percent, self.keep.get(), self.mode.get())
if 0 < self.percent < 100:
index, offset = cut_position(self.heights, self.percent)
text += f" — entre les pages {index} et {index + 1}" if offset == 0 else f" — page {index + 1}"
self.caption.set(text)
def accept(self, event=None):
operator = cut_operator(self.percent, self.keep.get(), self.mode.get())
rounded = float(operator.split("{")[1].split("}")[0])
if not 0 < rounded < 100:
messagebox.showerror("Coupe invalide", "Chaque partie doit contenir une portion du PDF.", parent=self)
return
self.destroy()
self.on_accept(operator)
+154
View File
@@ -0,0 +1,154 @@
from __future__ import annotations
import tkinter as tk
from pathlib import Path
from tkinter import messagebox, ttk
from copienator import CliError
from copienator.commands.resolve_manual import get_actual_pdf, parse_instruction_text
from copienator.platform import open_path
from .cut_helper import CutHelper
HELP = """La correction propose x> pour un mauvais label et -> pour une réponse supplémentaire lorsque le PDF cible existe déjà. Vérifiez les deux PDF avant de choisir.
Format : Copie01 Label source OP Label cible|
Conservez Copie suivi du numéro et les labels exacts (sans .pdf). Les espaces dans les labels sont acceptés ; entourez lopérateur despaces.
-> : fusionner dans la cible, conserver la source.
x> : fusionner dans la cible, archiver la source.
-x : remplacer la cible par une copie de la source, conserver la source.
xx : remplacer la cible par une copie de la source, archiver la source.
ss : conserver les deux PDF sans fusion.
sx : conserver la source, archiver la cible, sans fusion.
xs : archiver la source, conserver la cible, sans fusion.
c{43}1> : couper la source à 43 %, conserver le début et ajouter la fin à la cible.
c{43}2x : couper la source à 43 %, conserver la fin et remplacer la cible par le début.
1 conserve le début, 2 conserve la fin ; > fusionne lautre partie avec la cible, x la remplace. Le pourcentage porte sur la hauteur cumulée des pages visibles. Les décimales sont acceptées. Une seule coupe par label source est autorisée.
Cut permet de placer la coupe, avec accrochage entre les pages. Entrée ferme laperçu et affiche la commande à recopier ci-dessus : aucun fichier nest modifié. Une séparation entre pages conserve les pages entières. La source complète est archivée en _old ; les deux labels modifiés sont générés en _new et ajoutés à refaire.json.
Pour les fusions : « Source x> Cible| » place la cible avant la source ; « Source x> |Cible » place la source avant la cible. Même règle avec -> et c{}1>/c{}2> (pour la partie transférée) ; sans |, la cible vient en premier.
Vous pouvez changer lopérateur, déplacer |, corriger les labels ou retirer une instruction. Les lignes vides et celles commençant par ### sont ignorées. Retirer/commenter une ligne ne résout pas son conflit.
Enregistrez dans léditeur, puis cliquez sur Recharger et enfin sur Exécuter. Seul le fichier enregistré est appliqué. Larchivage utilise le suffixe _old ; les PDF créés utilisent _new. Après succès, manual_resolutions.txt est supprimé et correction.json est mis à jour. Si des PDF sont créés, refaire.json est généré : relancez la correction avec --refaire."""
class ManualResolutionPanel(ttk.Frame):
def __init__(self, parent, get_evaluation):
super().__init__(parent)
self.get_evaluation = get_evaluation
self.pdf_buttons = []
self.cut_buttons = []
actions = ttk.Frame(self)
actions.pack(fill="x")
self.editor_button = ttk.Button(
actions, text="Ouvrir dans un éditeur de texte", command=self.open_editor
)
self.editor_button.pack(side="left")
ttk.Button(actions, text="Recharger", command=self.reload).pack(side="left", padx=6)
self.cut_result = tk.StringVar()
result = ttk.Frame(self)
result.pack(fill="x", pady=4)
ttk.Entry(result, textvariable=self.cut_result, state="readonly").pack(side="left", fill="x", expand=True)
ttk.Button(result, text="Copier la commande Cut", command=self.copy_cut_command).pack(side="left", padx=6)
self.status = ttk.Label(self, wraplength=650)
self.status.pack(fill="x", pady=4)
preview = ttk.Frame(self)
preview.pack(fill="both", expand=True)
self.text = tk.Text(preview, height=10, width=50, wrap="word", state="disabled")
scroll = ttk.Scrollbar(preview, command=self.text.yview)
self.text.configure(yscrollcommand=scroll.set)
scroll.pack(side="right", fill="y")
self.text.pack(fill="both", expand=True)
ttk.Label(self, text=HELP, wraplength=650, justify="left").pack(fill="x", pady=8)
self.reload()
def manual_path(self) -> Path | None:
evaluation = self.get_evaluation()
return evaluation / "manual_resolutions.txt" if evaluation else None
def open_editor(self):
path = self.manual_path()
if path is None or not path.is_file():
self.reload()
return
try:
# .txt is opened in the desktop's associated text editor.
open_path(path)
except (OSError, RuntimeError) as exc:
messagebox.showerror("Ouverture impossible", str(exc))
def open_pdf(self, evaluation, copy_id, label):
path = get_actual_pdf(evaluation / "Copies", copy_id, label)
try:
if not path.is_file():
raise FileNotFoundError(f"PDF introuvable : {path}")
open_path(path)
except (OSError, RuntimeError) as exc:
messagebox.showerror("Ouverture du PDF", str(exc))
def reload(self):
for button in self.pdf_buttons + self.cut_buttons:
button.destroy()
self.pdf_buttons.clear()
self.cut_buttons.clear()
self.cut_result.set("")
self.text.configure(state="normal")
self.text.delete("1.0", "end")
path = self.manual_path()
self.editor_button.configure(state="disabled")
try:
if path is None:
raise FileNotFoundError("Chargez une évaluation.")
content = path.read_text(encoding="utf-8")
except (OSError, UnicodeError) as exc:
self.status.configure(text=f"manual_resolutions.txt indisponible : {exc}")
else:
self.editor_button.configure(state="normal")
malformed = []
for number, raw in enumerate(content.splitlines(keepends=True), 1):
self.text.insert("end", raw.rstrip("\r\n"))
try:
instructions = parse_instruction_text(raw)
except CliError:
malformed.append(str(number))
else:
if instructions:
instruction = instructions[0]
for title, label in (("PDF source", instruction.old_label),
("PDF cible", instruction.new_label)):
button = ttk.Button(
self.text, text=title,
command=lambda label=label, copy_id=instruction.copy_id, root=path.parent: self.open_pdf(root, copy_id, label),
)
self.pdf_buttons.append(button)
self.text.window_create("end", window=button, padx=8)
button = ttk.Button(self.text, text="Cut",
command=lambda item=instruction, root=path.parent: self.open_cut(root, item))
self.cut_buttons.append(button)
self.text.window_create("end", window=button, padx=8)
if raw.endswith("\n"):
self.text.insert("end", "\n")
detail = (" — lignes invalides : " + ", ".join(malformed)) if malformed else (
f"{len(self.pdf_buttons) // 2} instruction(s)"
)
self.status.configure(text=str(path) + detail)
finally:
self.text.configure(state="disabled")
def copy_cut_command(self):
if self.cut_result.get():
self.clipboard_clear()
self.clipboard_append(self.cut_result.get())
def open_cut(self, evaluation, instruction):
source = get_actual_pdf(evaluation / "Copies", instruction.copy_id, instruction.old_label)
def accepted(operator):
target = ("|" + instruction.new_label) if instruction.pipe_first else (instruction.new_label + "|")
self.cut_result.set(f"Copie{instruction.copy_id} {instruction.old_label} {operator} {target}")
try:
initial = (*instruction.cut, instruction.operator[-1]) if instruction.cut else None
CutHelper(self, source, accepted, initial)
except (OSError, RuntimeError, ValueError) as exc:
messagebox.showerror("Ouverture du PDF", str(exc))
+37
View File
@@ -0,0 +1,37 @@
"""Launch native desktop notifications without blocking Tk."""
import base64
import json
import os
import subprocess
import sys
from copienator.platform import find_executable
def notify_desktop(title: str, message: str) -> None:
if sys.platform.startswith("linux"):
executable = find_executable("notify-send")
if not executable:
raise RuntimeError("Installez notify-send (libnotify) pour les notifications de bureau.")
command = [executable, "--app-name=Copienator", "--", title, message]
elif sys.platform == "darwin":
command = ["/usr/bin/osascript", "-e",
f"display notification {json.dumps(message, ensure_ascii=False)} with title {json.dumps(title, ensure_ascii=False)}"]
elif os.name == "nt":
# Encode the script and quote strings as literals; no shell interpolation.
quote = lambda value: "'" + value.replace("'", "''") + "'"
script = (
"Add-Type -AssemblyName System.Windows.Forms;"
"$notice = New-Object System.Windows.Forms.NotifyIcon;"
"$notice.Icon = [System.Drawing.SystemIcons]::Information;"
"$notice.Visible = $true;"
f"$notice.ShowBalloonTip(10000, {quote(title)}, {quote(message)}, "
"[System.Windows.Forms.ToolTipIcon]::Info);"
"Start-Sleep -Seconds 12; $notice.Dispose()"
)
command = ["powershell.exe", "-NoProfile", "-NonInteractive", "-WindowStyle", "Hidden",
"-EncodedCommand", base64.b64encode(script.encode("utf-16-le")).decode("ascii")]
else:
raise RuntimeError("Notifications de bureau indisponibles sur ce système.")
subprocess.Popen(command, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+222 -72
View File
@@ -7,6 +7,7 @@ from collections.abc import Iterable
from dataclasses import dataclass
from pathlib import Path
from copienator import configuration
from copienator.configuration import ALWAYS_CROP
EVALUATION = "${evaluation}"
@@ -18,6 +19,7 @@ class ArgumentSpec:
label: str
kind: str = "text" # text, int, bool, choice, path
flag: str | None = None
false_flag: str | None = None
default: object = ""
choices: tuple[str, ...] = ()
help: str = ""
@@ -61,14 +63,14 @@ class StepDefinition:
return all(variant.kind == "manual" for variant in self.variants)
def arg_target(help_text: str = "Dossier d’évaluation ou fichier à traiter") -> ArgumentSpec:
def arg_target(help_text: str = "un dossier d’évaluation ou un fichier à traiter") -> ArgumentSpec:
return ArgumentSpec(
"target",
"Cible",
kind="path",
default=EVALUATION,
positional=True,
help=help_text,
help=f"La cible peut être {help_text.rstrip('.')}.",
)
@@ -87,6 +89,38 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
ident, label, None, "manual"
)
return_answer_arguments = ()
if configuration.RETURN_ANSWERS_ENABLED:
return_answer_arguments = (
ArgumentSpec(
"return_answers_context",
"Inclure le contexte dans answers",
"bool",
"--return-answers-context",
"--no-return-answers-context",
default=configuration.RETURN_ANSWERS_CONTEXT,
help="Inclut les pages de contexte applicables avant chaque réponse individuelle publiée dans answers.",
),
ArgumentSpec(
"return_answers_question",
"Inclure l’énoncé dans answers",
"bool",
"--return-answers-question",
"--no-return-answers-question",
default=configuration.RETURN_ANSWERS_QUESTION,
help="Inclut l’énoncé actuel avant chaque réponse individuelle publiée dans answers.",
),
ArgumentSpec(
"return_answers_solution",
"Inclure la correction dans answers",
"bool",
"--return-answers-solution",
"--no-return-answers-solution",
default=configuration.RETURN_ANSWERS_SOLUTION,
help="Inclut la correction actuelle avant chaque réponse individuelle publiée dans answers.",
),
)
steps = [
StepDefinition(
"inputs",
@@ -102,17 +136,17 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"Prétraitement de l’énoncé",
"Analyser l’énoncé",
"Détecte les questions, leurs groupes, le corrigé et les indications de barème.",
(
python("gemini", "Analyse avec Gemini", "statement"),
) + ((python("personal", "Énoncés et solutions personnels (SHEETINFO)", "statement-personal"),)
if show_personal_steps else ()),
((python("personal", "Énoncés et solutions personnels (SHEETINFO)", "statement-personal"),)
if show_personal_steps else ())
+ (python("gemini", "Analyse avec Gemini", "statement"),),
arguments=(
arg_target("Dossier de l’évaluation"),
arg_target("le dossier de l’évaluation"),
ArgumentSpec(
"restart",
"Ignorer le cache (--restart)",
kind="bool",
flag="--restart",
help="Ignore les résultats Gemini mis en cache et recommence entièrement lanalyse de l’énoncé.",
variants=("gemini",),
),
),
@@ -124,7 +158,7 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"Facultatif après la génération : remplace les groupes par exercice par des groupes "
"proposés par Gemini, en conservant les labels, les énoncés, les solutions et les barèmes.",
(python("default", "Groupes Gemini", "statement", fixed_args=("--groups-only",)),),
arguments=(arg_target("Dossier de l’évaluation"),),
arguments=(arg_target("le dossier de l’évaluation"),),
optional=True, personal=True, requires=("labels", "Text2", "Sol2"),
),
StepDefinition(
@@ -132,7 +166,7 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"Facultatif : remplace Persp par des barèmes Gemini sur 4 points, pour les groupes actuels. "
"Les énoncés et les solutions personnels sont conservés.",
(python("default", "Barèmes Gemini", "statement", fixed_args=("--persp-only",)),),
arguments=(arg_target("Dossier de l’évaluation"),),
arguments=(arg_target("le dossier de l’évaluation"),),
optional=True, personal=True, requires=("labels", "label_groups", "Text2", "Sol2"),
),
StepDefinition(
@@ -160,7 +194,7 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
supports_verbose=True,
),
),
arguments=(arg_target("Dossier de l’évaluation"),),
arguments=(arg_target("le dossier de l’évaluation"),),
optional=True,
),
StepDefinition(
@@ -179,7 +213,7 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
supports_verbose=True,
),
),
arguments=(arg_target("Dossier de l’évaluation"),),
arguments=(arg_target("le dossier de l’évaluation"),),
),
StepDefinition(
"page_splitter",
@@ -187,7 +221,16 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"Séparer et réordonner les pages",
"Ouvre loutil interactif de découpage A3 vers A4. La cible peut être un dossier ou un PDF.",
(python("default", "Séparation des pages", "page-split"),),
arguments=(arg_target(), ArgumentSpec("marked", "Copies signalées uniquement", "bool", "--marked")),
arguments=(
arg_target(),
ArgumentSpec(
"marked",
"Copies signalées uniquement",
"bool",
"--marked",
help="Limite lopération aux copies signalées dans linterface au lieu de traiter toutes les copies.",
),
),
artifacts=("Copies", "Copies Originales"),
),
StepDefinition(
@@ -198,8 +241,17 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"puis remplace les PDF dans Copies. Les versions non rognées sont sauvegardées. "
"À effectuer avant la détection des labels. Traite plusieurs copies en parallèle.",
(python("default", "Rognage des zones vides", "crop-margins"),),
arguments=(arg_target("Dossier de l’évaluation ou PDF dans Copies"),
ArgumentSpec("workers", "Copies traitées en parallèle", "int", "--workers", default=5)),
arguments=(
arg_target("le dossier de l’évaluation ou un PDF du dossier Copies"),
ArgumentSpec(
"workers",
"Copies traitées en parallèle",
"int",
"--workers",
default=5,
help="Fixe le nombre maximal de copies rognées simultanément. Une valeur élevée accélère le traitement si la machine possède assez de cœurs et de mémoire.",
),
),
optional=True,
requires=("Copies",),
auto_start_first_visit=ALWAYS_CROP,
@@ -207,13 +259,28 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
StepDefinition(
"cutleft",
"Prétraitement des copies",
"Découper la marge des labels",
"Produit les images de la partie gauche des copies. Une copie précise peut être ciblée.",
"Découper une partie à gauche pour détection des labels",
"Produit les images de la partie gauche des copies. Dans la fenêtre de découpe : "
"n décale la zone de 50 px vers la droite, N de 100 px, t de 50 px vers la gauche, "
"l l’élargit de 50 px, 1 utilise les pages entières, Entrée valide et s signale une erreur. "
"Une copie précise peut être ciblée.",
(python("default", "Découpe", "crop-labels"),),
arguments=(
arg_target(),
ArgumentSpec("fullpage", "Toujours utiliser la page entière", "bool", "--fullpage"),
ArgumentSpec("marked", "Copies signalées uniquement", "bool", "--marked"),
ArgumentSpec(
"fullpage",
"Toujours utiliser la page entière",
"bool",
"--fullpage",
help="Désactive la découpe habituelle de la marge gauche et transmet chaque page entière à la détection des labels.",
),
ArgumentSpec(
"marked",
"Copies signalées uniquement",
"bool",
"--marked",
help="Limite lopération aux copies signalées dans linterface au lieu de traiter toutes les copies.",
),
),
requires=("Copies",),
artifacts=("Cutleft",),
@@ -226,7 +293,13 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
(python("default", "Détection des labels", "labels"),),
arguments=(
arg_target(),
ArgumentSpec("overwrite", "Régénérer les résultats", "bool", "--overwrite"),
ArgumentSpec(
"overwrite",
"Régénérer les résultats",
"bool",
"--overwrite",
help="Relance la détection même lorsquun fichier JSON de labels existe déjà pour la copie.",
),
),
requires=("labels", "Copies", "Cutleft"),
artifacts=("Copies/*.json",),
@@ -239,7 +312,8 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"plotting",
"Labels et regroupement",
"Vérifier visuellement les labels",
"Ouvre la fenêtre de vérification. La cible peut être toute l’évaluation ou une copie.",
"Ouvre la fenêtre de vérification. La cible peut être toute l’évaluation ou une copie. "
"Le raccourci p revient à limage précédente, y compris dans la copie précédente.",
(python("default", "Vérification", "review-labels"),),
arguments=(arg_target(),),
requires=("labels", "Cutleft"),
@@ -265,13 +339,14 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"sauvegardés et plusieurs fichiers sont analysés en parallèle.",
(python("default", "Rognage du bas", "crop-answer-bottoms"),),
arguments=(
arg_target("Dossier de l’évaluation"),
arg_target("le dossier de l’évaluation"),
ArgumentSpec(
"workers",
"PDF traités en parallèle",
"int",
"--workers",
default=5,
help="Fixe le nombre maximal de PDF de réponse analysés simultanément. Une valeur élevée accélère le traitement si la machine possède assez de cœurs et de mémoire.",
),
),
optional=True,
@@ -284,7 +359,7 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"Regrouper les réponses",
"Regroupe les réponses portant le même label pour préparer les requêtes.",
(python("default", "Regroupement", "group-answers"),),
arguments=(arg_target("Dossier de l’évaluation"),),
arguments=(arg_target("le dossier de l’évaluation"),),
requires=("Copies",),
artifacts=("Par label",),
auto_start_first_visit=True,
@@ -293,7 +368,10 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"correction",
"Correction",
"Lancer ou intégrer la correction",
"Choisir une correction immédiate, batch, hybride, une recorrection, ou lintégration dun batch.",
"Choisir une correction immédiate, batch, hybride, une recorrection, ou lintégration dun batch. "
"En correction immédiate, la barre d’état affiche le nombre de groupes traités et le total. "
"Interrompre bloque les nouveaux appels Gemini, attend ceux déjà en cours, sauvegarde leurs résultats puis arrête la commande. "
"Au prochain lancement, les validations encore nécessaires (mauvais label ou contenu supplémentaire) reprennent sans refaire la requête principale.",
(
python("live", "Correction immédiate", "correct"),
python("batch", "Préparer toutes les requêtes batch", "correct", fixed_args=("--batch",)),
@@ -308,22 +386,31 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
python("reset", "Réinitialiser les corrections", "correct", fixed_args=("--reset",), dangerous=True),
),
arguments=(
arg_target("Évaluation ou image Group_X.jpg"),
arg_target("le dossier de l’évaluation ou une image Group_X.jpg"),
ArgumentSpec(
"overwrite",
"Écraser les corrections existantes",
"bool",
"--overwrite",
variants=("live", "batch", "hybrid", "refaire"),
help="Relance les corrections demandées même lorsquun résultat existe déjà.",
variants=("live", "batch", "hybrid", "integrate"),
),
ArgumentSpec(
"limit",
"Limite dappels Pro",
"int",
"--limit",
help="Limite le nombre dappels au modèle Pro pendant cette exécution. Laissez ce champ vide pour ne pas imposer de limite.",
variants=("live", "hybrid", "refaire"),
),
ArgumentSpec("batch_from", "Premier label envoyé en batch", "text", "--batch-from", variants=("hybrid",)),
ArgumentSpec(
"batch_from",
"Premier label envoyé en batch",
"text",
"--batch-from",
help="Indique le premier label traité en batch ; les labels précédents sont corrigés immédiatement.",
variants=("hybrid",),
),
),
requires=("Par label", "Persp", "labels"),
artifacts=("correction.json", "batch_requests_*.jsonl"),
@@ -339,7 +426,7 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"Envoyer les batchs",
"Envoie à Gemini les fichiers JSONL produits par le mode batch.",
(python("default", "Envoi", "batch-submit"),),
arguments=(arg_target("Dossier de l’évaluation"),),
arguments=(arg_target("le dossier de l’évaluation"),),
optional=True,
artifacts=("batch_jobs.json",),
skip_for_live_correction=True,
@@ -348,18 +435,8 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"batch_status",
"Correction",
"Consulter l’état des batchs",
"Affiche les jobs Gemini en cours. Lidentifiant de téléchargement est facultatif.",
"Vérifie les jobs enregistrés pour l’évaluation. Passe à la récupération uniquement lorsque tous ont réussi et que leurs résultats sont disponibles ; sinon, reste sur cette étape.",
(python("default", "État des batchs", "batch-status"),),
arguments=(
ArgumentSpec("download", "Télécharger le job", "text", "--download"),
ArgumentSpec(
"output",
"Fichier JSONL de destination",
"path",
"--output",
help="Utilisé avec un identifiant de téléchargement.",
),
),
optional=True,
skip_for_live_correction=True,
),
@@ -369,30 +446,31 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"Récupérer les résultats batch",
"Télécharge et rassemble les réponses des jobs terminés.",
(python("default", "Récupération", "batch-fetch"),),
arguments=(arg_target("Dossier de l’évaluation"),),
arguments=(arg_target("le dossier de l’évaluation"),),
optional=True,
skip_for_live_correction=True,
),
StepDefinition(
"post_correction",
"Correction",
"Nettoyer la correction",
"Corrige certains problèmes dencodage et prépare le texte pour LaTeX.",
(python("default", "Post-correction", "post-correction"),),
arguments=(arg_target("Dossier de l’évaluation"),),
requires=("correction.json",),
),
StepDefinition(
"manual_resolution",
"Correction",
"Résoudre les conflits manuels",
"Étape conditionnelle, uniquement si manual_resolutions.txt contient des conflits à traiter.",
"Étape conditionnelle, uniquement si manual_resolutions.txt contient des conflits à traiter. "
"Après succès, le GUI revient à la correction avec « Recorrection depuis refaire.json » sélectionné.",
(python("default", "Résolution", "resolve-manual"),),
arguments=(arg_target("Dossier de l’évaluation"),),
arguments=(arg_target("le dossier de l’évaluation"),),
optional=True,
requires=("manual_resolutions.txt", "correction.json"),
skip_without_manual_conflicts=True,
),
StepDefinition(
"post_correction",
"Correction",
"Nettoyer la correction",
"Sauvegarde correction.json dans correction_precleanup.json, puis corrige certains problèmes dencodage et prépare le texte pour LaTeX. La sauvegarde est remplacée à chaque nettoyage.",
(python("default", "Post-correction", "post-correction"),),
arguments=(arg_target("le dossier de l’évaluation"),),
requires=("correction.json",),
),
StepDefinition(
"annotation",
"Génération des annotations",
@@ -404,9 +482,22 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
python("grouped", "Annotations groupées (BGnot)", "annotate-grouped"),
),
arguments=(
arg_target("Dossier de l’évaluation"),
ArgumentSpec("overwrite", "Écraser les sorties", "bool", "--overwrite"),
ArgumentSpec("refaire", "Mode refaire", "bool", "--refaire", variants=("checks", "grouped")),
arg_target("le dossier de l’évaluation"),
ArgumentSpec(
"overwrite",
"Écraser les sorties",
"bool",
"--overwrite",
help="Remplace les annotations déjà générées dans le dossier de sortie sélectionné.",
),
ArgumentSpec(
"refaire",
"Mode refaire",
"bool",
"--refaire",
help="Génère uniquement les copies et questions inscrites dans refaire.json, dans le dossier réservé à la reprise.",
variants=("checks", "grouped"),
),
),
requires=("correction.json",),
artifacts=("Anot", "Bnot", "BGnot"),
@@ -418,16 +509,23 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"Exporte les annotations vers le dossier EXPORT_DIR défini dans config.py.",
(python("default", "Export", "export"),),
arguments=(
arg_target("Dossier de l’évaluation"),
arg_target("le dossier de l’évaluation"),
ArgumentSpec(
"annotation_dir",
"Dossier dannotations",
"choice",
default="BGnot",
choices=("BGnot", "Bnot", "Anot"),
help="Choisissez le dossier dannotations à exporter : groupées, avec cases, ou simples.",
positional=True,
),
ArgumentSpec("refaire", "Mode refaire", "bool", "--refaire"),
ArgumentSpec(
"refaire",
"Mode refaire",
"bool",
"--refaire",
help="Exporte les annotations du passage de reprise BRnot au lieu du dossier principal.",
),
),
optional=True,
),
@@ -445,16 +543,23 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"Copie les PDF présents dans IMPORT_DIR vers l’évaluation.",
(python("default", "Import", "import"),),
arguments=(
arg_target("Dossier de l’évaluation"),
arg_target("le dossier de l’évaluation"),
ArgumentSpec(
"annotation_dir",
"Dossier dannotations",
"choice",
default="BGnot",
choices=("BGnot", "Bnot", "Anot"),
help="Choisissez le dossier principal dans lequel importer les annotations manuscrites.",
positional=True,
),
ArgumentSpec("refaire", "Mode refaire", "bool", "--refaire"),
ArgumentSpec(
"refaire",
"Mode refaire",
"bool",
"--refaire",
help="Importe les annotations manuscrites dans BRnot pour le passage de reprise.",
),
),
),
StepDefinition(
@@ -467,9 +572,22 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
python("grouped", "Lecture BGnot", "read-grouped"),
),
arguments=(
arg_target("Dossier de l’évaluation"),
ArgumentSpec("update_score", "Réappliquer les score.json", "bool", "--update-score"),
ArgumentSpec("refaire", "Mode refaire", "bool", "--refaire", variants=("grouped",)),
arg_target("le dossier de l’évaluation"),
ArgumentSpec(
"update_score",
"Régénérer avec les nouveaux énoncés/corrigés et les score.json",
"bool",
"--update-score",
help="Régénère les images avec les PDF actuels d’énoncé et de correction ; pour chaque label, le score.json existant prévaut sur le score relu dans lannotation manuscrite.",
),
ArgumentSpec(
"refaire",
"Mode refaire",
"bool",
"--refaire",
help="Lit les annotations du passage de reprise BRnot et les fusionne avec les copies principales.",
variants=("grouped",),
),
ArgumentSpec(
"annotation_dir",
"Passage principal du mode refaire",
@@ -477,26 +595,37 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"--annotation-dir",
default="BGnot",
choices=("BGnot", "Bnot", "Anot"),
help="Indique le dossier dannotations du passage principal dans lequel intégrer les questions refaites.",
variants=("grouped",),
),
),
) + return_answer_arguments,
),
StepDefinition(
"giving_names",
"Finalisation",
"Attribuer les noms et préparer A Rendre",
"Crée le dossier A Rendre à partir du dossier dannotations choisi.",
"Crée le dossier A Rendre à partir du dossier dannotations choisi, sans passer automatiquement à l’étape suivante. "
"Après lexécution, vous pouvez renommer chaque dossier, ainsi que son fichier .jpg et son fichier .pdf, avec un autre nom ; le suffixe (id) du dossier est conservé. "
"Les outils affichés permettent didentifier et corriger les noms Unknown ou attribués à plusieurs copies. Cliquez ensuite sur « Marquer terminée ».",
(python("default", "Attribution des noms", "giving-names"),),
arguments=(
arg_target("Dossier de l’évaluation"),
arg_target("le dossier de l’évaluation"),
ArgumentSpec(
"annotation_dir",
"Dossier dannotations",
"choice",
default="BGnot",
choices=("BGnot", "Bnot", "Anot"),
help="Choisissez le dossier dannotations utilisé pour construire les fichiers nommés dans A Rendre.",
positional=True,
),
ArgumentSpec(
"update",
"Mettre à jour uniquement les images de answers",
"bool",
"--update",
help="Met à jour uniquement les images du dossier answers de chaque élève existant, en identifiant la copie par le numéro final entre parenthèses et sans modifier le nom du dossier ni les autres fichiers.",
),
),
artifacts=("A Rendre",),
),
@@ -523,8 +652,14 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"Transfère les scores, avec la possibilité de n’écrire que leur somme.",
(python("default", "Mise à jour ODS", "update-ods", supports_verbose=False),),
arguments=(
arg_target("Dossier de l’évaluation"),
ArgumentSpec("sum", "Écrire seulement la somme", "bool", "--sum"),
arg_target("le dossier de l’évaluation"),
ArgumentSpec(
"sum",
"Écrire seulement la somme",
"bool",
"--sum",
help="Écrit uniquement la note totale de chaque élève dans le fichier ODS, sans détailler les scores par question.",
),
),
personal=True,
),
@@ -548,9 +683,11 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"final_score",
"Étapes personnelles",
"Ajouter le score final",
"Génère les fichiers de diffusion avec le score final.",
"Prérequis immédiat : gestion_classe wse doit avoir été exécuté juste avant. "
"Génère ensuite les dossiers de diffusion avec le score final, puis copie "
"gestion_classe/Staging/histogramme.pdf dans le dossier Server/copies de l’évaluation.",
(python("default", "Score final", "add-final-score", supports_verbose=False),),
arguments=(arg_target("Dossier de l’évaluation"),),
arguments=(arg_target("le dossier de l’évaluation"),),
personal=True,
),
StepDefinition(
@@ -576,7 +713,7 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"Nettoyer les fichiers intermédiaires",
"Supprime définitivement les fichiers permettant de reprendre le parcours. "
"Conserve les PDF traités, les fichiers textuels de l’énoncé, correction.json, "
"les journaux, ainsi que les images et score.json de A Rendre.",
"les journaux, ainsi que les images, PDF, score.json et info.json de A Rendre.",
(
python(
"default",
@@ -590,8 +727,8 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
"seront supprimés. Il ne sera plus possible de reprendre une "
"étape sans régénérer ses données.\n\n"
"Les PDF traités, les fichiers textuels de l’énoncé, "
"correction.json, les journaux, ainsi que les images et "
"score.json de A Rendre seront conservés.\n\n"
"correction.json, les journaux, ainsi que les images, PDF et "
"score.json et info.json de A Rendre seront conservés.\n\n"
"Continuer ?"
),
),
@@ -602,7 +739,7 @@ def build_workflow(show_personal_steps: bool) -> list[StepDefinition]:
fixed_args=("--dry-run",),
),
),
arguments=(arg_target("Dossier de l’évaluation"),),
arguments=(arg_target("le dossier de l’évaluation"),),
optional=True,
requires=("Copies", "correction.json", "A Rendre"),
),
@@ -635,7 +772,15 @@ def build_refaire_workflow() -> list[StepDefinition]:
for suffix, title, description, program, flags, optional in definitions:
arguments = (arg_target(),) if program else ()
if suffix == "merge":
arguments += (ArgumentSpec("annotation_dir", "Passage principal", "choice", "--annotation-dir", default="BGnot", choices=("BGnot", "Bnot", "Anot")),)
arguments += (ArgumentSpec(
"annotation_dir",
"Passage principal",
"choice",
"--annotation-dir",
default="BGnot",
choices=("BGnot", "Bnot", "Anot"),
help="Indique le dossier dannotations du passage principal dans lequel intégrer les questions refaites.",
),)
requirements = ("refaire.json", "Copies", "labels", "correction.json")
if suffix in {"export", "tablet", "import", "merge"}:
requirements += ("BRnot",)
@@ -680,6 +825,9 @@ def build_command(
positionals: list[str] = []
options: list[str] = []
if step.id == "batch_status":
# Use the loaded evaluation without introducing another input field.
options.extend(("--evaluation", evaluation_arg))
for spec in step.arguments:
if spec.variants and variant.id not in spec.variants:
continue
@@ -687,6 +835,8 @@ def build_command(
if spec.kind == "bool":
if bool(value) and spec.flag:
options.append(spec.flag)
elif not bool(value) and spec.false_flag:
options.append(spec.false_flag)
continue
if value is None or str(value).strip() == "":
continue
+10
View File
@@ -6,6 +6,14 @@ API_KEY = os.environ.get("GEMINI_API_KEY")
EXPORT_DIR = Path("Export")
IMPORT_DIR = Path("Import")
# Fichiers à inclure dans A Rendre (les sources d'annotations sont conservées).
RETURN_JPEG_ENABLED = True
RETURN_PDF_ENABLED = True
RETURN_ANSWERS_ENABLED = False
RETURN_ANSWERS_CONTEXT = False
RETURN_ANSWERS_QUESTION = True
RETURN_ANSWERS_SOLUTION = False
# Les étapes gestion_classe, ODS et publication sont masquées par défaut.
SHOW_PERSONAL_STEPS = False
@@ -16,6 +24,7 @@ ALWAYS_CROP = False
CURRENT_SCORE_ODS_PATH = Path("current_eval.ods")
FINAL_SCORE_ODS_PATH = Path("simple_eval.ods")
FINAL_SCORE_OUTPUT_DIR = Path("Server") / "copies"
FINAL_SCORE_HISTOGRAM_PATH = Path("histogramme.pdf")
FINAL_SCORE_FONT_PATH = None
# Modèle pour des choses très légères
@@ -74,6 +83,7 @@ LATEX_BEFORE = r"""\documentclass[varwidth=24.8cm,margin=0.4cm]{standalone}
\usepackage{minted}
\usepackage{graphicx}
\usepackage{enumitem}
\usepackage{multicol}
\begin{document}
\begin{minipage}{24.8cm}
"""
+102
View File
@@ -0,0 +1,102 @@
# Scanned PDF margin cropping
Copienator has one automatic crop detector. It finds strongly coloured or dark
ink, recovers nearby weaker strokes, and removes substantial blank areas above
and below the detected content. It is designed for scanned student work on
plain, lined, or gridded paper, including mildly skewed pages and recurring
punched-hole artifacts.
## Review utility
Run from the repository root:
```sh
python -m copienator.crop_blank_margins Interro01/Copies tmp/cropped-copies
```
The input can be a directory or one PDF. Directory processing includes only
PDFs directly inside that directory. The utility writes processed PDFs, an HTML
comparison gallery, JPEG previews, and JSON/CSV reports into the output
directory. Source PDFs are never modified.
Available options:
- `--dpi 200`: analysis resolution.
- `--padding-mm 6`: space retained around detected content.
- `--min-crop-mm 5`: minimum worthwhile removal at either edge.
The output retains filenames, page order, page count, colour, rotation, and the
embedded scan data. Cropping changes the PDF CropBox rather than rasterizing the
page. Red shading in `index.html` shows the removed part of each original page.
A `review-*` status records uncertainty; one edge can still be cropped while the
other remains unchanged.
## Detection
The detector uses colour and darkness as strong ink seeds. It recovers connected
weak strokes within a 2 mm neighbourhood using directional contrast, which
limits growth along paper lines. Two seed thresholds are compared so unstable
boundaries can be flagged for review.
For dark neutral paper, it deskews the scan and confirms repeated horizontal or
vertical ruling before suppressing paper-line pixels. It uses short directional
openings to tolerate broken or bent grid lines. Very dark fraction bars and
diagram axes remain protected. Repeated components with similar size and
alignment in the outer 15 mm are treated as punched holes only when at least
three span a substantial part of the page. Writing in the same side column still
protects its margin.
On confirmed ruled paper, faint sparse strokes in the central 80% of the page
use a moderately more sensitive component filter. The outermost 9% on each side
uses a stricter filter because punched holes, torn binding edges, and page
numbers normally appear there.
The large-blank refinement changes an edge only when it finds at least 30 mm of
additional empty paper. A 2 mm recovery neighbourhood is applied before the
normal padding. Apparently blank pages and pages without reliable ink seeds are
kept at full height.
This remains a heuristic. Extremely faint isolated pencil marks, unusually
damaged ruling, and repeated handwriting shaped like hole artifacts can be
ambiguous. Review crops before generating answer coordinates.
## Optional GUI step
After **Séparer et réordonner les pages**, the GUI offers **Rogner les zones
vides**. It can process the whole evaluation or a selected PDF. It runs at 200
dpi with 6 mm padding and uses five worker processes by default. The CLI form is:
```sh
python -m copienator crop-margins EVALUATION --workers 5
```
The batch is fully prepared before any source is replaced. Detection failures
and interruptions leave the working PDFs intact; replacement errors roll back.
Each successful run saves the untrimmed PDFs and its report under
`.copienator/runs/crop-margins-*/`. The **Archivage** step removes these backups
and reports while keeping the cropped copies and execution logs.
Cropping must run before label detection. If a selected PDF already has a
same-named JSON coordinate file, the command stops before changing any PDFs.
When `ALWAYS_CROP` is true in `config.py`, this facultative step starts
automatically when first reached; the default configuration keeps it manual.
## Performance
Separate worker processes isolate MuPDF and each worker uses one OpenCV thread.
The detector uses native channel operations, vectorized component filtering,
cached separable background filtering, and a coarser Hough voting step for skew
candidates. Report rows remain ordered by copy and page regardless of worker
completion order.
On the Ryzen 7 PRO 7840U, an end-to-end benchmark took 56.48 seconds for 48 PDFs
of 10 pages, including rendering, detection, PDF writing, backups, and
replacement. The fixture uses Interro01 and DS08VA scans and cycles pages in
shorter copies, so it does not contain 480 distinct scans. Runtime depends on the
CPU, storage, and scan content.
Run the focused checks with:
```sh
python -m unittest tests.test_crop_blank_margins tests.test_ink_detection tests.test_crop_margins_command -v
```
+42
View File
@@ -0,0 +1,42 @@
# Bottom cropping after exercise splitting
This review utility processes the exercise PDFs stored directly under
`Copies/CopieXX/`. It never changes the source files. Modified PDFs are written
to a matching tree under the chosen output directory; unchanged PDFs are not
copied.
```sh
python -m copienator.crop_exercise_bottoms Interro01 tmp/exercise-bottom-crop
```
One line is 1/36 of the uncropped full-page height recorded by the matching
`Copies/CopieXX.pdf`. Each exercise PDF page is considered independently:
1. Pages shorter than 10 lines are skipped.
2. The bottom 0.75 line is excluded from detection so a fragment of the next
label cannot keep a large blank area.
3. The existing scan detector locates the last ink above that strip and keeps
6 mm of padding.
4. The bottom CropBox changes only if the proposed removal is at least 4 lines.
The top CropBox is always retained.
The ignored 0.75-line strip is therefore not removed on its own. It is included
in the result only when the complete proposed crop passes the four-line
threshold.
The output contains `index.html`, previews with removed areas shaded red, a
plain `cropped-files.txt` list, `report.json`, and `report.csv`. The command uses
five worker processes by default; `--workers`, `--dpi`, and `--padding-mm` are
configurable.
## GUI integration
After **Découper les réponses par question**, the GUI offers the facultative
step **Rogner le bas des réponses**. It runs the same thresholds at 200 dpi and
uses five worker processes by default. Only PDFs with an accepted crop are
replaced. Their unmodified versions and the complete report are stored under
`.copienator/runs/crop-exercise-bottoms-*/`; a failure or interruption before
publication leaves every exercise PDF unchanged. The **Archivage** step removes
these retained originals and reports. When `ALWAYS_CROP` is true in `config.py`,
this facultative step starts automatically when first reached; the default
configuration keeps it manual.
+258
View File
@@ -0,0 +1,258 @@
# Final output: `A Rendre`
This documents the current implementation, as of 2026-09-12. The return folder
referred to as « À rendre » is named **`A Rendre`** on disk. JPEG files use the
extension **`.jpg`**, not `.jpeg`.
## Files and their sources
After grouped correction and review:
```sh
python -m copienator read-grouped Interro
python -m copienator giving-names Interro BGnot
```
The expected layout for a copy is:
```text
Interro/A Rendre/
└── Student Name (01)/
├── Student Name.jpg
├── Student Name.pdf
├── score.json
├── info.json
└── answers/ # when individual answer export is enabled
├── 001 - Ex 1.jpg
└── 002 - Ex 2.jpg
```
The name comes from `Copies/Copie01.json` (`name`), with filename sanitization.
The copy ID distinguishes folders even when several copies have the same name.
`giving-names` links the full JPEG, PDF and score file (or copies them when links
are unavailable). It writes `info.json` and optionally composes the individual
answer JPEGs.
| Return file | Source under `BGnot/Copie01/` | Contents |
| --- | --- | --- |
| `Student Name.jpg` | `Concat.jpg` | Full continuous image of the compiled answers and corrections. |
| `Student Name.pdf` | `Concat_F.pdf` | Filtered, paginated correction with context, questions and solutions. |
| `score.json` | `score.json` | Per-question scores, including questions omitted from the filtered PDF. |
| `info.json` | `info.json` | Answer presence, empty-answer classification, PDF membership and score. |
| `answers/*.jpg` | Final per-label JPEGs selected by `info.json` | One annotated non-empty answer, with optional supplementary material. |
## Enabling or disabling outputs
Set these independent options in `config.py` (the defaults also apply when
absent from an older personal configuration):
```python
RETURN_JPEG_ENABLED = True
RETURN_PDF_ENABLED = True
RETURN_ANSWERS_ENABLED = False
RETURN_ANSWERS_CONTEXT = False
RETURN_ANSWERS_QUESTION = True
RETURN_ANSWERS_SOLUTION = False
```
The personal `config.py` enables `RETURN_ANSWERS_ENABLED`; the distributed
default is `False`. Set a full-output option to `False`, then rerun `giving-names` to omit that file from
`A Rendre`. For each prepared copy, any existing named return file of a disabled
type is removed, including a symlink or fallback copy. Its annotation source
remains intact. These options control return publication, not intermediate
rendering or scoring. `score.json` and `info.json` are always included and have
no disabling options.
Cleanup allows the JPEG to be absent when disabled, still requires `score.json`,
and preserves return PDFs when present.
## Individual answer JPEGs
With `RETURN_ANSWERS_ENABLED = True`, `giving-names` generates an `answers/`
subdirectory inside each student's return folder. It includes **every non-empty
compiled answer**, even a perfect answer omitted from the filtered PDF. Labels
marked `empty-answer` and labels without an answer are excluded. Every JPEG
contains the final annotated student answer, including retained feedback and
extracted handwriting.
The three supplementary options independently prepend, in this order:
1. Applicable context PDFs, if `RETURN_ANSWERS_CONTEXT` is enabled.
2. The question, if `RETURN_ANSWERS_QUESTION` is enabled (the default).
3. The model solution, if `RETURN_ANSWERS_SOLUTION` is enabled.
4. The annotated student answer, always.
These use the same `Text2`/`Sol2` sources as the filtered PDF. Missing supplements
are skipped; an unreadable existing file fails the export. They are concatenated
vertically on white, without PDF pagination or its black/blue borders. The
options affect only these individual images, not the full JPEG or filtered PDF.
Disabling all supplements produces just the annotated answer.
Filenames use natural label order, a three-digit minimum sequence number, and a
sanitized label (`001 - Ex 1.jpg`). Numbering prevents filename collisions when
labels differ only by characters forbidden in filenames. JSON keys retain exact
labels. The managed `answers/` directory is replaced on successful generation,
so removed/empty answers do not leave stale images; failures preserve the previous
directory. Disabling the option clears this directory on the next `giving-names`
run. Separate storage keeps these images out of the personal final-mark stamping
step, which reads only JPEGs directly inside the student's folder.
Recompile annotations once before exporting old evaluations with this option:
the compiler now saves every final answer block and writes `info.json`
next to them. For the grouped workflow, run `read-grouped`, then `giving-names`.
This avoids reconstructing a reviewed answer from outdated correction data.
## `info.json`: per-question information
Every label in `score.json` has an object containing exactly four fields:
```json
{
"Ex 1": {"present": true, "not_empty": true, "touched": false, "score": "4"},
"Ex 2": {"present": true, "not_empty": true, "touched": true, "score": "2"},
"Empty": {"present": true, "not_empty": false, "touched": false, "score": "0"},
"Absent": {"present": false, "not_empty": false, "touched": false, "score": ""}
}
```
- `present`: an answer entry exists for this student and label in the compilation
data. A supplied answer judged empty still has `present: true`.
- `not_empty`: the answer was not marked `empty-answer` and was successfully
compiled. Absent answers have `not_empty: false`. When individual export is
enabled, a JPEG is generated if and only if both `present` and `not_empty` are
true. These fields describe the answer regardless of export settings.
- `touched`: the answer appears in the compiled filtered `Concat_F.pdf`, using
the actual selection including handwriting and selective redo preservation.
It is not a flag for human edits. Empty and absent answers have `false`.
- `score`: the same value as `score.json` (normally a numeric string, or `""`
for an unpopulated score). Editing scores requires recompilation to update the
images; return publication uses current `score.json` values for this field.
`info.json` is always exported, even when individual JPEGs or the named PDF are
disabled. `touched` describes the source filtered PDF. In normal `Anot` and
`Bnot` flows, no filtered PDF is produced and all `touched` values are false.
This file replaces `touched.json` and the internal `answer_labels.json` manifest.
Recompile old annotations, then run `giving-names`; successful regeneration and
publication remove the obsolete files from their respective folders. Missing
or malformed metadata requires recompilation rather than guessing answer presence
from scores. Cleanup preserves `info.json` alongside `score.json`.
## JPEG: the full compiled correction
The JPEG stacks the rendered answer blocks vertically in natural label order
(for example, Ex 2 precedes Ex 10). Each block contains the scanned answer, its
label and score, retained global and local feedback, and detected handwritten
review annotations. Local feedback can include red rectangles and comments in
the left margin. Review checkboxes are applied as actions rather than reproduced
as controls; internal error labels are hidden during recompilation.
The result is one RGB image of variable height, with no page breaks. It contains
all successfully compiled answer blocks, including answers scored 4 with no
remaining feedback. “Full” refers to those answer blocks, not the original scan
pages or every question in the statement. Missing/unrenderable answers cannot
be included, and the renderer normally suppresses `empty-answer` results.
The grouped compiler refuses to publish a new set when compilation is incomplete.
The JPEG does not prepend the question, context or model solution PDFs.
## PDF: a different selection and layout
**The PDF is not a PDF conversion of the JPEG.** During an ordinary full grouped
recompilation, an answer is omitted only when all three conditions hold:
- Its score is at least 4.
- Every feedback item is marked `to_delete` (also true for an empty feedback list).
- There are no significant detected handwritten annotations for that answer.
Thus, a 4/4 answer with retained feedback or handwriting still appears. Scores
for omitted answers remain in `score.json`, and their answer blocks remain in
the JPEG. Handwriting significance currently means more than 20 pixels with
alpha greater than 50 in the extracted annotation layer.
For each retained answer, the PDF stacks the following available material:
1. Applicable context PDFs from `Text2/CTXT first_label -> last_label.pdf`.
2. The question from `Text2/<label>.pdf`.
3. The model solution from `Sol2/<label>.pdf`.
4. The same compiled answer block used for the JPEG.
Missing supplementary PDFs are skipped. Contexts can repeat for successive
questions. Each question's complete group stays together on one page; groups
are packed until the next would exceed the target height. An oversized group
gets its own taller page, rather than being split. Pages are raster images saved
as PDF at 100 dpi, with variable heights, not fixed A4 sheets or searchable text.
The target height is `int(max_image_width * 1.414 * 1.25)` pixels. Small white
margins are added on the left and above/below the page. The first image of each
group receives a black border and the second a blue border. These borders are
assigned by position, so they do not consistently identify question and solution
when contexts are present or supplementary files are missing.
If nothing survives filtering, `read-grouped` removes old `Concat_F` outputs and
does not create a new PDF. During a selective `--refaire` merge, saved answer
images outside the selection are kept in the filtered output without reapplying
the perfect-answer filter; the resulting PDF can therefore retain more answers
than a full recompilation.
## JSON: per-question scores
`score.json` is a flat JSON object keyed by the exact question labels. For example
(illustrative data):
```json
{
"Ex 1 : 1)": "4",
"Ex 1 : 2)": "2.5",
"Ex 2": ""
}
```
Values are **strings**, including numeric scores. The normal question scale is
0 to 4. `""` means no score was populated for that label; it is distinct from
`"0"`. The compiler initializes all labels from the evaluation's `labels` file
to `""`, then fills processed scores. This file contains no student identity,
feedback, annotation coordinates, grading weights, or overall final mark.
Scores incorporate review checkbox changes and, when requested, existing score
overrides via `read-grouped --update-score` (or `read-annotations --update-score`
for `Bnot`). Editing the JSON alone does not update the rendered scores. Overrides
are read from the annotation source folder: a return symlink points there, but
an independent fallback copy does not. After regeneration, rerun `giving-names`
to refresh copied return files.
`update-ods` reads these return JSON files; empty values become `NT` in the normal
per-question export. Its `--sum` option sums numeric values and skips nonnumeric
ones. Weighting and the final overall mark belong to the separate grading flow.
## Availability and later steps
- `giving-names` accepts `BGnot`, `Bnot`, or `Anot`. It selects the requested
source if `score.json` and either `Concat.jpg` or `info.json` exist,
otherwise it tries `Anot/CopieXX`. This also permits returns for an entirely
empty copy. It links the PDF only if `Concat_F.pdf` exists. The normal
`Bnot` reader produces a filtered `Concat_F.jpg`, and simple annotation produces
`Concat.jpg`; these paths do not guarantee a filtered PDF.
- Preparing returns removes disabled named outputs, and removes older named
outputs whose source is now absent, including broken symlinks.
- `add-final-score` writes to `FINAL_SCORE_OUTPUT_DIR/<evaluation>/`. It stamps
the overall mark from the configured ODS in red at the JPEG's upper right,
rounded down to one decimal place. It copies PDFs unchanged and does not export
either JSON file or the `answers/` directory. It does not add that mark to the
files inside `A Rendre`.
- The `clean` command retains return images (including individual answers), PDFs,
`score.json` and `info.json`, materializing
retained symlinks before deleting their sources.
## Implementation references
- [Naming and return links](../copienator/commands/giving_names.py)
- [Individual answer publication](../copienator/return_answers.py)
- [Grouped compilation, filtering and PDF pagination](../copienator/commands/reading_grouped_annotations.py)
- [Answer rendering](../copienator/commands/annotating.py)
- [Handwriting detection and per-copy compilation](../copienator/commands/reading_annotations.py)
- [Score and feedback actions](../copienator/annotation_actions.py)
- [Context, question and solution lookup](../copienator/utils.py)
- [ODS export](../copienator/commands/update_ods.py)
- [Final-mark stamping](../copienator/commands/add_final_score.py)
- [Cleanup retention](../copienator/commands/clean.py)
+99
View File
@@ -0,0 +1,99 @@
import contextlib
import io
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
import pandas as pd
from PIL import Image
from copienator.commands import add_final_score
class AddFinalScoreTests(unittest.TestCase):
def test_creates_complete_student_directory(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
source = root / "A Rendre" / "Student (01)"
answers = source / "answers"
answers.mkdir(parents=True)
Image.new("RGB", (300, 200), "white").save(source / "Student.jpg")
(source / "Student.pdf").write_bytes(b"pdf")
(source / "score.json").write_bytes(b'{"Ex 1": "4"}')
(source / "info.json").write_bytes(b'{"Ex 1": {}}')
(answers / "Ex 1.jpg").write_bytes(b"answer")
output = root / "output"
output.mkdir()
(output / "Student.jpg").write_bytes(b"legacy jpeg")
(output / "Student.pdf").write_bytes(b"legacy pdf")
stale_answers = output / "Student" / "answers"
stale_answers.mkdir(parents=True)
(stale_answers / "001 - Ex 1.jpg").write_bytes(b"stale")
scores = pd.DataFrame({0: ["Student"], 1: [12.39]})
histogram = root / "histogramme.pdf"
histogram.write_bytes(b"histogram")
with patch.object(add_final_score.pd, "read_excel", return_value=scores), \
patch.object(add_final_score, "HISTOGRAM_PATH", histogram), \
contextlib.redirect_stdout(io.StringIO()):
add_final_score.process_images(root, output)
student = output / "Student"
self.assertEqual(
{path.name for path in student.iterdir()},
{"Student.jpg", "Student.pdf", "score.json", "info.json", "answers"},
)
self.assertEqual((student / "Student.pdf").read_bytes(), b"pdf")
self.assertEqual((student / "score.json").read_bytes(), b'{"Ex 1": "4"}')
self.assertEqual((student / "info.json").read_bytes(), b'{"Ex 1": {}}')
self.assertEqual(
(student / "answers" / "Ex 1.jpg").read_bytes(), b"answer"
)
self.assertFalse((output / "Student.jpg").exists())
self.assertFalse((output / "Student.pdf").exists())
self.assertFalse((student / "answers" / "001 - Ex 1.jpg").exists())
self.assertEqual((output / "histogramme.pdf").read_bytes(), b"histogram")
def test_omits_answers_directory_when_it_was_not_generated(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
source = root / "A Rendre" / "Student (01)"
source.mkdir(parents=True)
Image.new("RGB", (300, 200), "white").save(source / "Student.jpg")
(source / "Student.pdf").write_bytes(b"pdf")
(source / "score.json").write_text("{}")
(source / "info.json").write_text("{}")
output = root / "output"
scores = pd.DataFrame({0: ["Student"], 1: [10]})
with patch.object(add_final_score.pd, "read_excel", return_value=scores), \
contextlib.redirect_stdout(io.StringIO()):
add_final_score.process_images(root, output)
self.assertFalse((output / "Student" / "answers").exists())
def test_missing_histogram_warns_without_discarding_student_outputs(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
source = root / "A Rendre" / "Student (01)"
source.mkdir(parents=True)
Image.new("RGB", (300, 200), "white").save(source / "Student.jpg")
scores = pd.DataFrame({0: ["Student"], 1: [10]})
output = root / "output"
messages = io.StringIO()
with patch.object(
add_final_score.pd, "read_excel", return_value=scores
), patch.object(
add_final_score, "HISTOGRAM_PATH", root / "missing.pdf"
), contextlib.redirect_stdout(messages):
add_final_score.process_images(root, output)
self.assertTrue((output / "Student" / "Student.jpg").is_file())
self.assertFalse((output / "histogramme.pdf").exists())
self.assertIn("Missing histogram", messages.getvalue())
if __name__ == "__main__":
unittest.main()
+343
View File
@@ -0,0 +1,343 @@
import contextlib
import io
import itertools
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from PIL import Image
from copienator import EvaluationWorkspace, atomic_write_json, configuration, read_json
from copienator.commands import annotating, giving_names
from copienator.commands import reading_grouped_annotations as reader
from copienator.return_answers import (
publish_answer_returns,
save_return_answer_options,
)
class AnswerReturnTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.source = self.root / "BGnot" / "Copie01"
self.source.mkdir(parents=True)
self.destination = self.root / "A Rendre" / "Student (01)"
self.destination.mkdir(parents=True)
(self.root / "labels").write_text("Ex 1\nEx 2\nEmpty\n")
atomic_write_json(self.source / "score.json", {"Ex 1": "4", "Ex 2": "2", "Empty": "0"})
atomic_write_json(self.source / "info.json", {
"Ex 1": {"present": True, "not_empty": True, "touched": False, "score": "4"},
"Ex 2": {"present": True, "not_empty": True, "touched": True, "score": "2"},
"Empty": {"present": True, "not_empty": False, "touched": False, "score": "0"},
})
for label in ("Ex 1", "Ex 2", "Empty"):
Image.new("RGB", (100, 30), "red").save(self.source / f"{label}.jpg")
for folder in ("Text2", "Sol2"):
(self.root / folder).mkdir()
for label in ("Ex 1", "Ex 2"):
(self.root / "Text2" / f"{label}.pdf").touch()
(self.root / "Sol2" / f"{label}.pdf").touch()
(self.root / "Text2" / "CTXT Ex 1 -> Ex 2.pdf").touch()
@staticmethod
def supplement(path):
path = Path(path)
color = "blue" if path.name.startswith("CTXT") else "green" if path.parent.name == "Text2" else "yellow"
return Image.new("RGB", (100, 20), color), 0, 0
def test_all_supplement_combinations_always_include_annotated_answer(self):
for context, question, solution in itertools.product((False, True), repeat=3):
with self.subTest(context=context, question=question, solution=solution), patch.multiple(
configuration, RETURN_ANSWERS_ENABLED=True,
RETURN_ANSWERS_CONTEXT=context, RETURN_ANSWERS_QUESTION=question,
RETURN_ANSWERS_SOLUTION=solution,
), patch.object(annotating, "make_base_image", side_effect=self.supplement):
publish_answer_returns(self.root, self.source, self.destination)
files = sorted((self.destination / "answers").glob("*.jpg"))
self.assertEqual([p.name for p in files], ["Ex 1.jpg", "Ex 2.jpg"])
with Image.open(files[0]) as image:
self.assertEqual(image.size, (100, 30 + 20 * sum((context, question, solution))))
colors = []
if context:
colors.append((0, 0, 255))
if question:
colors.append((0, 128, 0))
if solution:
colors.append((255, 255, 0))
colors.append((255, 0, 0))
for index, color in enumerate(colors):
pixel = image.getpixel((50, index * 20 + 10))
self.assertTrue(all(abs(a - b) < 10 for a, b in zip(pixel, color)))
self.assertEqual(read_json(self.destination / "info.json"), read_json(self.source / "info.json"))
def test_saved_reading_options_override_configuration_for_answers(self):
save_return_answer_options(
self.root,
context=True,
question=False,
solution=True,
)
with patch.multiple(
configuration,
RETURN_ANSWERS_ENABLED=True,
RETURN_ANSWERS_CONTEXT=False,
RETURN_ANSWERS_QUESTION=True,
RETURN_ANSWERS_SOLUTION=False,
), patch.object(
annotating, "make_base_image", side_effect=self.supplement
):
publish_answer_returns(self.root, self.source, self.destination)
with Image.open(self.destination / "answers" / "Ex 1.jpg") as image:
self.assertEqual(image.size, (100, 70))
for y, color in (
(10, (0, 0, 255)),
(30, (255, 255, 0)),
(50, (255, 0, 0)),
):
self.assertTrue(
all(
abs(actual - expected) < 10
for actual, expected in zip(image.getpixel((50, y)), color)
)
)
def test_missing_supplement_is_optional_and_failure_preserves_old_answers(self):
with patch.multiple(configuration, RETURN_ANSWERS_ENABLED=True,
RETURN_ANSWERS_CONTEXT=False, RETURN_ANSWERS_QUESTION=True,
RETURN_ANSWERS_SOLUTION=False), patch.object(
annotating, "make_base_image", side_effect=self.supplement
):
(self.root / "Text2" / "Ex 1.pdf").unlink()
publish_answer_returns(self.root, self.source, self.destination)
path = self.destination / "answers" / "Ex 1.jpg"
with Image.open(path) as image:
self.assertEqual(image.size, (100, 30))
original = path.read_bytes()
(self.source / "Ex 2.jpg").unlink()
with self.assertRaises(FileNotFoundError):
publish_answer_returns(self.root, self.source, self.destination)
self.assertEqual(path.read_bytes(), original)
self.assertTrue((self.destination / "answers" / "Ex 2.jpg").exists())
def test_disabling_clears_individual_images_but_retains_info(self):
with patch.multiple(configuration, RETURN_ANSWERS_ENABLED=True,
RETURN_ANSWERS_CONTEXT=False, RETURN_ANSWERS_QUESTION=False,
RETURN_ANSWERS_SOLUTION=False):
publish_answer_returns(self.root, self.source, self.destination)
with patch.object(configuration, "RETURN_ANSWERS_ENABLED", False):
publish_answer_returns(self.root, self.source, self.destination)
self.assertEqual(list((self.destination / "answers").iterdir()), [])
self.assertTrue(read_json(self.destination / "info.json")["Ex 2"]["touched"])
def test_missing_info_requires_recompilation_instead_of_guessing(self):
(self.source / "info.json").unlink()
(self.source / "Concat_F.pdf").touch()
with self.assertRaisesRegex(ValueError, "recompile"):
publish_answer_returns(self.root, self.source, self.destination)
def test_info_replaces_touched_and_tracks_manual_score_edits(self):
atomic_write_json(self.destination / "touched.json", {"obsolete": True})
atomic_write_json(self.source / "score.json", {"Ex 1": "3.5", "Ex 2": "2", "Empty": "0"})
with patch.object(configuration, "RETURN_ANSWERS_ENABLED", False):
publish_answer_returns(self.root, self.source, self.destination)
self.assertFalse((self.destination / "touched.json").exists())
self.assertEqual(read_json(self.destination / "info.json")["Ex 1"], {
"present": True, "not_empty": True, "touched": False, "score": "3.5"
})
def test_invalid_info_preserves_previous_return(self):
atomic_write_json(self.destination / "info.json", {"previous": "keep"})
info = read_json(self.source / "info.json")
info["Empty"]["touched"] = True
atomic_write_json(self.source / "info.json", info)
with self.assertRaisesRegex(ValueError, "Invalid question information"):
publish_answer_returns(self.root, self.source, self.destination)
self.assertEqual(read_json(self.destination / "info.json"), {"previous": "keep"})
def test_publication_is_independent_of_full_jpeg_and_pdf_options(self):
workspace = EvaluationWorkspace(self.root)
workspace.copies_dir.mkdir()
atomic_write_json(workspace.copies_dir / "Copie01.json", {"name": "Student"})
(self.root / "names").write_text("Student\n")
with patch.multiple(configuration, RETURN_ANSWERS_ENABLED=True,
RETURN_ANSWERS_CONTEXT=False, RETURN_ANSWERS_QUESTION=False,
RETURN_ANSWERS_SOLUTION=False, RETURN_JPEG_ENABLED=False,
RETURN_PDF_ENABLED=False), contextlib.redirect_stdout(io.StringIO()):
self.assertEqual(giving_names.run(workspace, annotation_dir="BGnot"), 0)
self.assertEqual({p.name for p in self.destination.iterdir()}, {"answers", "score.json", "info.json"})
def test_update_uses_copy_id_from_renamed_folder_and_touches_only_answers(self):
workspace = EvaluationWorkspace(self.root)
workspace.copies_dir.mkdir()
atomic_write_json(workspace.copies_dir / "Copie01.json", {"name": "Student"})
renamed = self.destination.with_name("Nom modifié manuellement (01)")
self.destination.rename(renamed)
(renamed / "Nom personnalisé.jpg").write_bytes(b"keep-jpeg")
(renamed / "Nom personnalisé.pdf").write_bytes(b"keep-pdf")
(renamed / "score.json").write_bytes(b"keep-score")
(renamed / "info.json").write_bytes(b"keep-info")
answers = renamed / "answers"
answers.mkdir()
(answers / "obsolete.jpg").write_bytes(b"obsolete")
preserved = {
path.name: path.read_bytes()
for path in renamed.iterdir()
if path.is_file()
}
with patch.multiple(
configuration,
RETURN_ANSWERS_ENABLED=True,
RETURN_ANSWERS_CONTEXT=False,
RETURN_ANSWERS_QUESTION=False,
RETURN_ANSWERS_SOLUTION=False,
), contextlib.redirect_stdout(io.StringIO()):
self.assertEqual(
giving_names.run(workspace, annotation_dir="BGnot", update=True),
0,
)
self.assertFalse((workspace.return_dir / "Student (01)").exists())
self.assertEqual(
{
path.name: path.read_bytes()
for path in renamed.iterdir()
if path.is_file()
},
preserved,
)
self.assertEqual(
sorted(path.name for path in answers.iterdir()),
["Ex 1.jpg", "Ex 2.jpg"],
)
class CompiledMembershipTests(unittest.TestCase):
def test_update_score_preserves_file_and_manual_value_wins(self):
with tempfile.TemporaryDirectory() as directory:
workspace = EvaluationWorkspace(Path(directory))
output = workspace.root / "BGnot" / "Copie01"
output.mkdir(parents=True)
original_score = b'{\n "Ex 1": "3.5"\n}\n'
(output / "score.json").write_bytes(original_score)
answer = workspace.root / "answer.pdf"
answer.touch()
data = {
"01": {
"Ex 1": {
"result": {"score": 1, "feedback": []},
"pdf_path": answer,
"coordinates": (0, 0),
}
}
}
rendered_scores = []
def compose(base, label, result, *args, **kwargs):
rendered_scores.append(result["score"])
return Image.new("RGB", (100, 50), "white"), 0
with patch.object(
annotating, "make_base_image", return_value=(None, 0, 0)
), patch.object(
annotating, "compose_label_image", side_effect=compose
), patch.object(
reader, "get_extra_pdfs_as_images", return_value=[]
), patch.object(reader, "save_paginated_pdf"):
status, _ = reader.apply_actions_and_regenerate_grouped(
workspace,
data,
"01",
[{"label": "Ex 1", "type": "score", "value": "2"}],
{},
["Ex 1"],
update_score=True,
)
self.assertEqual(status, 0)
self.assertEqual(rendered_scores, ["3.5"])
self.assertEqual((output / "score.json").read_bytes(), original_score)
self.assertEqual(read_json(output / "info.json")["Ex 1"]["score"], "3.5")
def test_membership_matches_actual_groups_and_saves_every_nonempty_answer(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
workspace = EvaluationWorkspace(root)
output = root / "BGnot" / "Copie01"
output.mkdir(parents=True)
answer = root / "answer.pdf"
answer.touch()
results = {
"Perfect": {"score": 4, "feedback": []},
"Low": {"score": 2, "feedback": []},
"Feedback": {"score": 4, "feedback": [{"text": "keep"}]},
"Deleted": {"score": 4, "feedback": [{"text": "delete", "to_delete": True}]},
"Handwriting": {"score": 4, "feedback": []},
"Empty": {"score": 0, "error": "empty-answer"},
}
data = {"01": {label: {"result": result, "pdf_path": answer, "coordinates": (0, 0)}
for label, result in results.items()}}
all_labels = [*results, "Absent"]
rendered = []
def compose(base, label, *args, **kwargs):
rendered.append(label)
return Image.new("RGB", (100, 50), "white"), 0
notes = {"Handwriting": {"img": Image.new("RGBA", (100, 50), "red"), "old_header_h": 0}}
with patch.object(annotating, "make_base_image", return_value=(None, 0, 0)), patch.object(
annotating, "compose_label_image", side_effect=compose
), patch.object(reader, "get_extra_pdfs_as_images", return_value=[]), patch.object(
reader, "save_paginated_pdf"
) as save_pdf:
status, _ = reader.apply_actions_and_regenerate_grouped(workspace, data, "01", [], notes, all_labels)
self.assertEqual(status, 0)
info = read_json(output / "info.json")
touched = {label: entry["touched"] for label, entry in info.items()}
self.assertEqual({label for label, value in touched.items() if value}, {"Low", "Feedback", "Handwriting"})
self.assertEqual(len(save_pdf.call_args.args[0]), sum(touched.values()))
self.assertEqual({label for label, entry in info.items() if entry["present"] and entry["not_empty"]}, set(results) - {"Empty"})
self.assertEqual(info["Empty"], {"present": True, "not_empty": False, "touched": False, "score": "0"})
self.assertEqual(info["Absent"], {"present": False, "not_empty": False, "touched": False, "score": ""})
self.assertEqual(info["Perfect"], {"present": True, "not_empty": True, "touched": False, "score": "4"})
self.assertNotIn("Empty", rendered)
for label in rendered:
self.assertTrue((output / f"{label}.jpg").is_file())
# Selective redo keeps unselected saved answers in the PDF,
# including previously perfect answers; touched must follow it.
status, _ = reader.apply_actions_and_regenerate_grouped(
workspace, data, "01", [], {}, all_labels, selected_labels={"Low"}
)
self.assertEqual(status, 0)
self.assertTrue(read_json(output / "info.json")["Perfect"]["touched"])
self.assertFalse(read_json(output / "info.json")["Empty"]["not_empty"])
def test_all_empty_removes_stale_concat_and_records_false(self):
with tempfile.TemporaryDirectory() as directory:
workspace = EvaluationWorkspace(Path(directory))
output = workspace.root / "BGnot" / "Copie01"
output.mkdir(parents=True)
for name in ("Concat.jpg", "Concat_F.pdf"):
(output / name).write_bytes(b"stale")
atomic_write_json(output / "touched.json", {"Empty": True})
atomic_write_json(output / "answer_labels.json", ["Empty"])
status, _ = reader.apply_actions_and_regenerate_grouped(
workspace, {"01": {"Empty": {"result": {"score": 0, "error": "empty-answer"}}}},
"01", [], {}, ["Empty"]
)
self.assertEqual(status, 0)
self.assertEqual(read_json(output / "info.json"), {
"Empty": {"present": True, "not_empty": False, "touched": False, "score": "0"}
})
self.assertFalse((output / "Concat.jpg").exists())
self.assertFalse((output / "Concat_F.pdf").exists())
self.assertFalse((output / "touched.json").exists())
self.assertFalse((output / "answer_labels.json").exists())
if __name__ == "__main__":
unittest.main()
+69
View File
@@ -0,0 +1,69 @@
import queue
import unittest
from unittest.mock import Mock, patch
from copienator_gui.batch_monitor import BatchMonitor, CHECK_INTERVAL_MS
from copienator_gui.notifications import notify_desktop
class BatchMonitorTests(unittest.TestCase):
def setUp(self):
self.scheduler = Mock()
self.scheduler.after.side_effect = lambda delay, callback: (delay, callback)
self.ready = Mock()
self.status = Mock()
self.monitor = BatchMonitor(self.scheduler, self.ready, self.status, Mock())
self.runner = Mock()
self.runner.events = queue.Queue()
self.factory = patch("copienator_gui.batch_monitor.ProcessRunner", return_value=self.runner)
self.factory.start()
self.addCleanup(self.factory.stop)
def start(self):
self.monitor.start(["check"], "/tmp", {}, None)
def test_checks_immediately_retries_in_five_minutes_and_notifies_once(self):
self.start()
self.runner.start.assert_called_once()
self.start()
self.runner.start.assert_called_once()
self.runner.events.put(("finished", (4, False)))
self.monitor._poll()
self.assertEqual(self.monitor.timer[0], CHECK_INTERVAL_MS)
self.assertEqual(CHECK_INTERVAL_MS, 300000)
self.ready.assert_not_called()
self.monitor.timer[1]()
self.assertEqual(self.runner.start.call_count, 2)
self.runner.events.put(("finished", (0, False)))
self.monitor._poll()
self.assertFalse(self.monitor.active)
self.assertIsNone(self.monitor.timer)
self.ready.assert_called_once()
self.monitor._poll()
self.ready.assert_called_once()
def test_stop_cancels_timer_and_running_check_without_notification(self):
self.start()
timer = self.monitor.timer
self.monitor.stop()
self.scheduler.after_cancel.assert_called_once_with(timer)
self.runner.force_stop.assert_called_once()
self.runner.events.put(("finished", (0, False)))
self.monitor._poll()
self.monitor._check()
self.ready.assert_not_called()
self.runner.start.assert_called_once()
def test_start_failure_retries_without_reporting_readiness(self):
self.runner.start.side_effect = OSError("unavailable")
self.start()
self.assertEqual(self.monitor.timer[0], CHECK_INTERVAL_MS)
self.ready.assert_not_called()
def test_linux_notification_passes_text_as_arguments(self):
with patch("copienator_gui.notifications.sys.platform", "linux"), patch(
"copienator_gui.notifications.find_executable", return_value="/usr/bin/notify-send"
), patch("copienator_gui.notifications.subprocess.Popen") as launch:
notify_desktop("Copienator", "Interro02 : résultats prêts")
self.assertEqual(launch.call_args.args[0], ["/usr/bin/notify-send", "--app-name=Copienator",
"--", "Copienator", "Interro02 : résultats prêts"])
+54
View File
@@ -0,0 +1,54 @@
import tempfile
import unittest
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import Mock, call, patch
from copienator import CliError, EvaluationWorkspace, ExitCode, atomic_write_json
from copienator.commands.batch_status import check_evaluation_jobs, main
class BatchReadinessTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.workspace = EvaluationWorkspace(Path(self.temp.name))
self.client = Mock()
def manifest(self, jobs):
atomic_write_json(self.workspace.batch_jobs_file, {"jobs": jobs})
def test_only_recorded_jobs_are_checked_and_all_must_have_results(self):
self.manifest({"flash": {"name": "batches/flash"}, "pro": {"name": "batches/pro"}})
succeeded = SimpleNamespace(state="JOB_STATE_SUCCEEDED",
dest=SimpleNamespace(file_name="files/result"))
for state in ("JOB_STATE_PENDING", "JOB_STATE_RUNNING", "JOB_STATE_FAILED",
"JOB_STATE_CANCELLED", "JOB_STATE_EXPIRED", "UNKNOWN", "JOB_STATE_SUCCEEDED"):
with self.subTest(state=state):
self.client.reset_mock()
self.client.batches.get.side_effect = [succeeded, SimpleNamespace(
state=SimpleNamespace(name=state), dest=SimpleNamespace(file_name="files/pro"))]
result = check_evaluation_jobs(self.workspace, client=self.client)
self.assertEqual(result, ExitCode.SUCCESS if state == "JOB_STATE_SUCCEEDED" else ExitCode.PARTIAL)
self.assertEqual(self.client.batches.get.call_args_list,
[call(name="batches/flash"), call(name="batches/pro")])
self.client.batches.list.assert_not_called()
self.client.files.download.assert_not_called()
self.client.batches.get.side_effect = [succeeded, SimpleNamespace(
state="JOB_STATE_SUCCEEDED", dest=None)]
self.assertEqual(check_evaluation_jobs(self.workspace, client=self.client), ExitCode.PARTIAL)
def test_missing_empty_and_invalid_manifest_cannot_report_success(self):
self.assertEqual(check_evaluation_jobs(self.workspace, client=self.client), ExitCode.PARTIAL)
self.manifest({})
self.assertEqual(check_evaluation_jobs(self.workspace, client=self.client), ExitCode.PARTIAL)
for jobs in ([], {"flash": {}}, {"flash": {"name": ""}}, {"flash": None}):
self.manifest(jobs)
with self.assertRaises(CliError):
check_evaluation_jobs(self.workspace, client=self.client)
self.client.batches.get.assert_not_called()
def test_cli_returns_readiness_code_for_selected_evaluation(self):
with patch("copienator.commands.batch_status.check_evaluation_jobs", return_value=ExitCode.PARTIAL) as check:
self.assertEqual(main(["--evaluation", str(self.workspace.root)]), ExitCode.PARTIAL)
self.assertEqual(check.call_args.args[0].root, self.workspace.root)
@@ -49,6 +49,8 @@ class CropExerciseBottomsCommandTests(unittest.TestCase):
0,
)
self.assertIn("1 PDF remplacé", log.getvalue())
self.assertIn("1 exercice(s) rogné(s)", log.getvalue())
self.assertIn("Rognage moyen des exercices modifiés", log.getvalue())
with pymupdf.open(self.large) as cropped:
self.assertLess(cropped[0].rect.height, 250)
self.assertEqual(self.short.read_bytes(), self.short_original)
@@ -68,6 +70,37 @@ class CropExerciseBottomsCommandTests(unittest.TestCase):
)
self.assertTrue((backups[0].parents[2] / "report.json").is_file())
def test_crop_statistics_average_percentages_by_exercise(self):
records = [
{
"file": "Copies/Copie01/Ex 1.pdf",
"height_lines": 10.0,
"bottom_removed_lines": 4.0,
"status": "cropped",
},
{
"file": "Copies/Copie01/Ex 1.pdf",
"height_lines": 10.0,
"bottom_removed_lines": 0.0,
"status": "skipped-short",
},
{
"file": "Copies/Copie01/Ex 2.pdf",
"height_lines": 10.0,
"bottom_removed_lines": 5.0,
"status": "cropped",
},
{
"file": "Copies/Copie01/Ex 3.pdf",
"height_lines": 10.0,
"bottom_removed_lines": 0.0,
"status": "unchanged-small-crop",
},
]
count, average = crop_exercise_bottoms.crop_statistics(records)
self.assertEqual(count, 2)
self.assertAlmostEqual(average, 35.0)
def test_detection_failure_does_not_publish_an_earlier_result(self):
broken = self.answers / "Ex 3.pdf"
broken.write_bytes(b"not a PDF")
+29
View File
@@ -37,6 +37,9 @@ class CropMarginsCommandTests(unittest.TestCase):
with contextlib.redirect_stdout(io.StringIO()) as log:
self.assertEqual(main(["crop-margins", str(self.workspace.root)]), 0)
self.assertIn("Page 2/2", log.getvalue())
self.assertIn("1/2 pages rognées", log.getvalue())
self.assertIn("Rognage moyen des pages modifiées", log.getvalue())
self.assertIn("page(s) rognée(s) de plus de 30 %", log.getvalue())
with pymupdf.open(self.source) as result:
self.assertEqual(len(result), 2)
self.assertLess(result[0].rect.height, 150)
@@ -50,6 +53,32 @@ class CropMarginsCommandTests(unittest.TestCase):
self.assertEqual(backups[0].read_bytes(), self.original)
self.assertTrue((backups[0].parent.parent/"report.json").is_file())
def test_crop_statistics_use_only_modified_pages(self):
records = [
{
"top_removed_mm": 10.0,
"bottom_removed_mm": 20.0,
"original_cropbox": [0, 0, 200, 300],
"rotation": 0,
},
{
"top_removed_mm": 40.0,
"bottom_removed_mm": 0.0,
"original_cropbox": [0, 0, 200, 300],
"rotation": 90,
},
{
"top_removed_mm": 0.0,
"bottom_removed_mm": 0.0,
"original_cropbox": [0, 0, 200, 300],
"rotation": 0,
},
]
count, average, over_thirty = crop_margins.crop_statistics(records)
self.assertEqual(count, 2)
self.assertAlmostEqual(average, (28.3465 + 56.6929) / 2, places=3)
self.assertEqual(over_thirty, 1)
def test_failure_or_interruption_never_publishes_partial_batch(self):
second = self.workspace.copies_dir/"Copie02.pdf"
second.write_bytes(self.original)
+82
View File
@@ -0,0 +1,82 @@
import copy
import json
import tempfile
import unittest
from pathlib import Path
from unittest.mock import Mock, patch
from PIL import Image
from copienator import EvaluationWorkspace, atomic_write_json
from copienator.annotation_data import GroupCoordinates, _scaled_result
from copienator.annotation_actions import apply_checkbox_actions
from copienator.commands import annotating, correction
from copienator.feedback_boxes import valid_feedback_box
class FeedbackBoxTests(unittest.TestCase):
def test_checkbox_for_promoted_feedback_deletes_the_correct_comment(self):
feedback = [{"text": "Invalid local", "box_2d": [753, 680, 287, 946]},
{"text": "Existing global", "box_2d": None}]
apply_checkbox_actions({"Ex 5": {"result": {"feedback": feedback}}},
[{"label": "Ex 5", "type": "del_global", "index": 0}], lambda _: None)
self.assertTrue(feedback[0]["to_delete"])
self.assertNotIn("to_delete", feedback[1])
def test_bad_boxes_keep_their_comment_as_global_feedback_without_mutating_data(self):
for box in ([753, 680, 287, 946], [10, 50, 20, 30], [10, 20, 10, 30],
[1, 2, 3], [None, 0, 10, 20], [float("nan"), 0, 10, 20]):
with self.subTest(box=box):
result = {"score": 2, "feedback": [{"text": "Important comment", "box_2d": box}]}
original_box = result["feedback"][0]["box_2d"]
scaled = _scaled_result(result, GroupCoordinates(2415, 3297, 1655, 3297))
callback = Mock()
render = Mock(return_value=Image.new("RGBA", (200, 40), "white"))
with patch.object(annotating, "render_score_text", return_value=Image.new("RGBA", (200, 40))):
image, _ = annotating.compose_label_image(Image.new("RGBA", (800, 100)), "Ex 5", scaled,
2415, render_fn=render, draw_callback=callback)
self.assertIsNotNone(image)
render.assert_called_once_with("Important comment", unittest.mock.ANY)
self.assertFalse(any(call.args[0] == "local_rect" for call in callback.call_args_list))
self.assertIs(result["feedback"][0]["box_2d"], original_box)
def test_valid_boxes_remain_local(self):
result = {"feedback": [{"text": "Local", "box_2d": [10, 20, 30, 40]}]}
before = copy.deepcopy(result)
callback = Mock()
with patch.object(annotating, "render_score_text", return_value=Image.new("RGBA", (200, 40))):
annotating.compose_label_image(Image.new("RGBA", (800, 100)), "Ex 5", result, 0,
render_fn=Mock(return_value=Image.new("RGBA", (200, 40))),
draw_callback=callback)
self.assertTrue(any(call.args[0] == "local_rect" for call in callback.call_args_list))
self.assertEqual(result, before)
self.assertTrue(valid_feedback_box([10, 20, 30, 40]))
def test_invalid_auxiliary_response_loses_only_its_rectangle(self):
returned = [{"text": "Keep this", "box_2d": [753, 680, 287, 946]}]
with patch.object(correction.prompting, "request_for_box_correction", return_value=([], {})), patch.object(
correction, "call_gemini_with_retries", return_value=json.dumps(returned)
):
feedback = correction.correct_boxes_with_gemini("26", "Ex 5", Path("unused.pdf"), [], 0, 1000, 1, 1000)
self.assertEqual(feedback, [{"text": "Keep this", "box_2d": None}])
def test_correction_requests_repair_for_inverted_boxes_and_falls_back_without_losing_comments(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
group = root / "Par label" / "Ex 5" / "Group_1.jpg"
group.parent.mkdir(parents=True)
group.touch()
atomic_write_json(group.with_suffix(".json"), [["26", 0, 1000, 1, "Ex 5"]])
args = correction.build_parser().parse_args([str(root)])
correction.configure_runtime(EvaluationWorkspace(root), [(str(group), "Ex 5")], args, api_client=Mock())
response = [{"id": "26", "result": {"score": 2, "error": "", "feedback": [
{"text": "First", "box_2d": [753, 680, 287, 946]},
{"text": "Second", "box_2d": [100, 900, 200, 100]}]}}]
with patch.object(correction.prompting, "generate_request", return_value=([], {})), patch.object(
correction, "correct_boxes_with_gemini", side_effect=RuntimeError("repair failed")
) as repair:
correction.process_single_task((str(group), "Ex 5"), json.dumps(response))
repair.assert_called_once()
feedback = correction.results["Ex 5"][0][0]["result"]["feedback"]
self.assertEqual([f["text"] for f in feedback], ["First", "Second"])
self.assertTrue(all(f["box_2d"] is None for f in feedback))
+170 -4
View File
@@ -52,7 +52,11 @@ class GuiConvenienceTests(unittest.TestCase):
(self.evaluation / "enonce.pdf").unlink()
self.app._reload_inputs()
self.assertEqual(self.app.state_store.step("inputs")["status"], "ready")
self.assertIn("enonce.pdf", self.app.description_label.cget("text"))
self.assertIn("enonce.pdf", self.app.missing_requirements_label.cget("text"))
style = ttk.Style(self.app)
self.assertEqual(
style.lookup("MissingPrerequisite.TLabel", "foreground"), "#c62828"
)
def test_compact_evaluation_input_and_open_folder_button(self):
self.assertEqual(int(self.app.evaluation_entry.cget("width")), 42)
@@ -64,6 +68,113 @@ class GuiConvenienceTests(unittest.TestCase):
self.app.open_evaluation_button.invoke()
opened.assert_called_once_with(self.evaluation)
def test_manual_resolution_panel_follows_command_target(self):
manual = self.evaluation / "manual_resolutions.txt"
manual.write_text("Copie01 A -> B|\n", encoding="utf-8")
self.app.tree.selection_set("manual_resolution")
self.app.update()
self.assertEqual(len(self.app.manual_panel.pdf_buttons), 2)
other = self.repository / "Other"
other.mkdir()
other_manual = other / "manual_resolutions.txt"
other_manual.write_text("Copie02 C xs D\n", encoding="utf-8")
self.app.arg_vars["target"].set(str(other))
self.app.update()
self.assertIn("Copie02", self.app.manual_panel.text.get("1.0", "end"))
with patch("copienator_gui.manual_resolution.open_path") as opened:
self.app.manual_panel.editor_button.invoke()
opened.assert_called_once_with(other_manual)
def test_successful_manual_resolution_returns_to_refaire_correction(self):
manual = self.evaluation / "manual_resolutions.txt"
manual.write_text("Copie01 A -> B|\n", encoding="utf-8")
self.app.state_store.update_step("manual_resolution", visited=True)
self.app.tree.selection_set("manual_resolution")
self.app.update()
self.app.active_step_id = "manual_resolution"
self.app._finish_process(0, False)
self.app.update()
self.assertEqual(self.app.current_step.id, "correction")
self.assertEqual(self.app.variant_var.get(), "refaire")
self.assertEqual(
self.app.state_store.step("correction")["variant"], "refaire"
)
self.assertNotIn("overwrite", self.app.arg_vars)
self.assertIn("--refaire", self.app._make_command())
def test_giving_names_waits_for_explicit_manual_completion(self):
return_dir = self.evaluation / "A Rendre" / "Student (01)"
return_dir.mkdir(parents=True)
(return_dir / "Student.jpg").write_bytes(b"jpg")
(return_dir / "Student.pdf").write_bytes(b"pdf")
self.app.tree.selection_set("giving_names")
self.app.update()
self.app.active_step_id = "giving_names"
self.app._finish_process(0, False)
self.app.update()
self.assertEqual(self.app.current_step.id, "giving_names")
self.assertEqual(
self.app.state_store.step("giving_names")["status"], "detected"
)
self.assertEqual(
self.app.complete_giving_names_button.cget("text"),
"Marquer terminée",
)
self.app.complete_giving_names_button.invoke()
self.app.update()
self.assertEqual(
self.app.state_store.step("giving_names")["status"], "success"
)
self.assertNotEqual(self.app.current_step.id, "giving_names")
def test_batch_status_waits_until_all_jobs_are_ready(self):
self.app.state_store.update_step("correction", variant="batch")
self.app.state_store.update_step("batch_status", visited=True)
self.app.tree.selection_set("batch_status")
self.app.update()
command = self.app._make_command()
self.assertIn("--evaluation", command)
self.assertFalse(self.app.arg_vars)
self.app.active_step_id = "batch_status"
self.app._finish_process(4, False)
self.app.update()
self.assertEqual(self.app.current_step.id, "batch_status")
self.app.active_step_id = "batch_status"
self.app._finish_process(0, False)
self.app.update()
self.assertEqual(self.app.current_step.id, "fetch_batches")
def test_batch_watch_buttons_use_loaded_evaluation_and_stop_on_reload(self):
self.app.state_store.update_step("correction", variant="batch")
self.app.tree.selection_set("batch_status")
self.app.update()
with patch.object(self.app.batch_monitor, "start") as start:
self.app.watch_batches_button.invoke()
self.assertEqual(start.call_args.args[0][-2:], ["--evaluation", str(self.evaluation)])
self.app.batch_monitor.active = True
self.app._update_controls()
self.assertEqual(str(self.app.stop_watch_batches_button.cget("state")), "normal")
self.app.stop_watch_batches_button.invoke()
self.assertFalse(self.app.batch_monitor.active)
self.app.batch_monitor.active = True
self.app._load_evaluation()
self.assertFalse(self.app.batch_monitor.active)
def test_background_batch_readiness_notifies_without_changing_other_step(self):
self.app.tree.selection_set("inputs")
self.app.update()
with patch("copienator_gui.app.notify_desktop") as notify:
self.app._batch_results_ready()
notify.assert_called_once()
self.assertIn(self.evaluation.name, notify.call_args.args[1])
self.assertEqual(self.app.current_step.id, "inputs")
self.assertEqual(self.app.state_store.step("batch_status")["status"], "success")
def test_optional_blank_crop_can_target_a_copy_or_be_skipped(self):
copies = self.evaluation/"Copies"
copies.mkdir()
@@ -149,16 +260,71 @@ class GuiConvenienceTests(unittest.TestCase):
self.assertIn("Cibler la copie sélectionnée", controls)
self.assertIn("Cibler tout le dossier", controls)
def test_label_review_can_show_and_target_one_copy(self):
copies = self.evaluation / "Copies"
copies.mkdir()
for name in ("Copie01.pdf", "Copie02.pdf"):
(copies / name).touch()
self.app.tree.selection_set("plotting")
self.app.update()
self.app.copy_var.set("Copie02.pdf")
with patch("copienator_gui.app.open_path") as opened:
self.app._open_selected_copy()
opened.assert_called_once_with(copies / "Copie02.pdf")
self.app._target_selected_copy()
command = self.app._make_command()
self.assertEqual(Path(command[-1]), copies / "Copie02.pdf")
labels = self._label_texts(self.app.form)
self.assertTrue(
any("Seule la copie ciblée sera vérifiée" in text for text in labels)
)
def test_free_form_arguments_are_shown_only_when_documented(self):
self.app.tree.selection_set("labels")
self.app.update()
labels_text = self._label_texts(self.app.form)
self.assertIn("Arguments supplémentaires", labels_text)
self.assertTrue(any("Cutleft" in text for text in labels_text))
extra_label = next(
child
for child in self.app.form.winfo_children()
if isinstance(child, ttk.Label)
and str(child.cget("text")).startswith("Arguments supplémentaires")
)
tooltip = extra_label._copienator_tooltip
self.assertIn("Cutleft", tooltip.text)
self.assertFalse(any("Cutleft" in text for text in labels_text))
tooltip.show()
self.app.update()
self.assertIsNotNone(tooltip.window)
self.assertIn("Cutleft", tooltip.window.winfo_children()[0].cget("text"))
tooltip.hide()
self.app.tree.selection_set("plotting")
self.app.update()
self.assertNotIn("Arguments supplémentaires", self._label_texts(self.app.form))
self.assertFalse(
any(
text.startswith("Arguments supplémentaires")
for text in self._label_texts(self.app.form)
)
)
def test_each_visible_argument_label_has_a_tooltip(self):
self.app.tree.selection_set("correction")
self.app.update()
argument_labels = [
child
for child in self.app.form.winfo_children()
if isinstance(child, ttk.Label) and str(child.cget("text")).endswith("")
]
self.assertGreaterEqual(len(argument_labels), 4)
for label in argument_labels:
with self.subTest(label=label.cget("text")):
tooltip = getattr(label, "_copienator_tooltip", None)
self.assertIsNotNone(tooltip)
self.assertTrue(tooltip.text.strip())
def test_verbose_checkbox_updates_supported_commands(self):
self.app.tree.selection_set("labels")
+623 -8
View File
@@ -38,13 +38,20 @@ from copienator_gui.app import (
build_runner_environment,
copy_pdf_paths,
detected_annotation_directories,
get_personal_interro_files,
has_manual_conflicts,
plotting_shortcut_lines,
process_status,
)
from copienator_gui.giving_names import (
find_name_issues,
original_copy_path,
rename_return_copy,
)
from copienator_gui.diagnostics import collect_diagnostics
from copienator_gui.runner import ProcessRunner
from copienator_gui.state import StateStore
from copienator_gui import workflow as workflow_module
from copienator_gui.workflow import build_command, build_workflow, evaluation_argument
from copienator.commands.copies_tools import rename_all, rotate_all
from copienator.platform import (
@@ -489,7 +496,7 @@ class StandardCliTests(unittest.TestCase):
"giving_names": (
"giving_names",
"default",
{"target": evaluation, "annotation_dir": "BGnot"},
{"target": evaluation, "annotation_dir": "BGnot", "update": True},
),
"grouping": ("grouping", "default", {"target": evaluation}),
"post_correction": (
@@ -502,11 +509,6 @@ class StandardCliTests(unittest.TestCase):
"default",
{"target": evaluation},
),
"verify_groups": (
"verify_groups",
"default",
{"target": evaluation},
),
"annotating": (
"annotation",
"simple",
@@ -690,6 +692,10 @@ class StandardCliTests(unittest.TestCase):
annotations.mkdir(parents=True)
atomic_write_json(copies / "Copie01.json", {"name": "Élève Test"})
atomic_write_json(annotations / "score.json", {"total": 10})
atomic_write_json(annotations / "info.json", {
"total": {"present": False, "not_empty": False, "touched": False, "score": 10}
})
(evaluation / "labels").write_text("total\n")
(annotations / "Concat.jpg").write_bytes(b"image")
(evaluation / "names").write_text("Élève Test\n", encoding="utf-8")
@@ -910,6 +916,43 @@ class StandardCliTests(unittest.TestCase):
)
self.assertIsNone(viewer.accumulated_results)
def test_plotting_detects_labels_missing_from_two_thirds_of_copies(self) -> None:
module = self.modules["plotting"]
with tempfile.TemporaryDirectory() as directory:
copies = Path(directory)
detected = {
"Copie01_01.json": ["Ex 1", "Ex 3"],
"Copie02_01.json": ["Ex 1", "Ex 3"],
"Copie03_01.json": ["Ex 1", "Ex 2", "Ex 3"],
}
for filename, labels in detected.items():
atomic_write_json(
copies / filename,
{"list": [{"label": label} for label in labels]},
)
missing = module.frequently_missing_labels(
copies,
["Ex 1", "Ex 2", "Ex 3", "Ex 4"],
)
self.assertEqual(missing, {"Ex 2", "Ex 4"})
def test_plotting_mutes_only_a_single_commonly_missing_label(self) -> None:
module = self.modules["plotting"]
labels = ["Ex 1", "Ex 2", "Ex 3", "Ex 4"]
muted, index = module.label_color("Ex 3", labels, 0, {"Ex 2"})
ordinary, _ = module.label_color("Ex 3", labels, 0, set())
two_missing, _ = module.label_color(
"Ex 4", labels, 0, {"Ex 2", "Ex 3"}
)
self.assertEqual(muted, module.COMMON_MISSING_LABEL_COLOR)
self.assertEqual(index, 2)
self.assertEqual(ordinary, module.MISSING_LABEL_COLOR)
self.assertEqual(two_missing, module.MISSING_LABEL_COLOR)
def test_plotting_validation_preserves_manually_edited_order(self) -> None:
module = self.modules["plotting"]
with tempfile.TemporaryDirectory() as directory:
@@ -930,6 +973,7 @@ class StandardCliTests(unittest.TestCase):
"part": 1,
}
viewer.valid_labels = {"Ex 1", "Ex 2"}
viewer.active_copie_name = "Copie01"
viewer.accumulated_results = {"name": "Student", "list": []}
viewer.history = []
viewer.current_pil_image = Mock()
@@ -943,6 +987,49 @@ class StandardCliTests(unittest.TestCase):
["Ex 2", "Ex 1"],
)
def test_plotting_previous_restores_state_across_copies(self) -> None:
module = self.modules["plotting"]
viewer = module.ImageViewer.__new__(module.ImageViewer)
previous_image = Mock()
previous_json = Path("Copie01_02.json")
previous_meta = {"copie": "Copie01"}
previous_results = {
"name": "Student 1",
"list": [{"label": "Ex 1"}],
}
current_image = Mock()
current_json = Path("Copie02_01.json")
current_meta = {"copie": "Copie02"}
viewer.is_viewing = True
viewer.history = [
(
previous_image,
previous_json,
previous_meta,
"Copie01",
previous_results,
)
]
viewer.forward_stack = []
viewer.current_pil_image = current_image
viewer.current_json_path = current_json
viewer.current_meta = current_meta
viewer.active_copie_name = "Copie02"
viewer.accumulated_results = {"name": "Student 2", "list": []}
viewer.display_image = Mock()
viewer.on_previous(None)
self.assertEqual(viewer.active_copie_name, "Copie01")
self.assertEqual(viewer.accumulated_results, previous_results)
self.assertEqual(
viewer.forward_stack,
[(current_image, current_json, current_meta)],
)
viewer.display_image.assert_called_once_with(
previous_image, previous_json, previous_meta
)
def test_plotting_validates_plain_and_directional_labels(self) -> None:
module = self.modules["plotting"]
self.assertEqual(
@@ -1225,6 +1312,293 @@ class StandardCliTests(unittest.TestCase):
self.assertEqual(module.results, {"Ex 1": []})
self.assertEqual(module.tasks_to_process, [("new.jpg", "Ex 1")])
def test_correction_rejects_overwrite_with_refaire_without_writing(self) -> None:
module = self.modules["correction"]
with tempfile.TemporaryDirectory() as directory:
evaluation = Path(directory) / "Exam"
evaluation.mkdir()
correction = evaluation / "correction.json"
original = {"Ex 1": [[{"id": "01", "result": {"feedback": []}}]]}
atomic_write_json(correction, original)
errors = io.StringIO()
with redirect_stderr(errors):
status = module.main(
[str(evaluation), "--refaire", "--overwrite"]
)
self.assertEqual(status, module.ExitCode.INVALID_ARGUMENTS)
self.assertIn(
"--overwrite cannot be used with --refaire", errors.getvalue()
)
self.assertEqual(read_json(correction), original)
def test_correction_reserves_unique_group_indices_concurrently(self) -> None:
module = self.modules["correction"]
with tempfile.TemporaryDirectory() as directory:
groups = Path(directory) / "Par label"
label_dir = groups / "Ex 8 : 2)"
label_dir.mkdir(parents=True)
(label_dir / "Group_3.jpg").write_bytes(b"existing")
with patch.object(module, "GROUPS_DIR", groups):
module.reserved_group_indices.clear()
with ThreadPoolExecutor(max_workers=8) as executor:
indices = list(
executor.map(
module.reserve_next_group_idx,
["Ex 8 : 2)"] * 8,
)
)
self.assertEqual(sorted(indices), list(range(3, 11)))
self.assertEqual(len(indices), len(set(indices)))
def test_live_correction_reports_progress_for_generated_groups(self) -> None:
module = self.modules["correction"]
with tempfile.TemporaryDirectory() as directory:
workspace = EvaluationWorkspace(Path(directory))
args = module.build_parser().parse_args(
[str(workspace.root), "--overwrite"]
)
initial_tasks = [("group-1.jpg", "Ex 1"), ("group-2.jpg", "Ex 2")]
module.configure_runtime(
workspace,
initial_tasks,
args,
api_client=Mock(),
)
processed = []
def process(task, _precomputed=None):
processed.append(task[0])
if task[0] == "group-1.jpg":
return [("group-3.jpg", "Ex 3", False)]
return []
output = io.StringIO()
with patch.object(
module, "process_single_task", side_effect=process
), patch.object(
module, "resolve_delayed_moves", return_value=[]
), redirect_stdout(output):
self.assertEqual(module.run_configured(args), 0)
self.assertCountEqual(
processed,
["group-1.jpg", "group-2.jpg", "group-3.jpg"],
)
progress = [
line
for line in output.getvalue().splitlines()
if line.startswith("[Progression correction]")
]
self.assertEqual(progress[0], "[Progression correction] Groupes traités : 0/2")
self.assertEqual(progress[-1], "[Progression correction] Groupes traités : 3/3")
def test_live_correction_stops_scheduling_and_waits_for_active_task(self) -> None:
module = self.modules["correction"]
self.addCleanup(module.stop_requested.clear)
with tempfile.TemporaryDirectory() as directory:
workspace = EvaluationWorkspace(Path(directory))
args = module.build_parser().parse_args(
[str(workspace.root), "--overwrite"]
)
module.configure_runtime(
workspace,
[("group-1.jpg", "Ex 1"), ("group-2.jpg", "Ex 2")],
args,
api_client=Mock(),
)
processed = []
def process(task, _precomputed=None):
processed.append(task[0])
module.request_graceful_stop()
(workspace.root / "saved-result").write_text(task[0])
return []
with patch.object(module, "NB_THREADS", 1), patch.object(
module, "process_single_task", side_effect=process
), patch.object(
module, "resolve_delayed_moves", return_value=[]
):
status = module.run_configured(args)
saved_result = (workspace.root / "saved-result").read_text()
self.assertEqual(status, module.ExitCode.INTERRUPTED)
self.assertEqual(processed, ["group-1.jpg"])
self.assertEqual(saved_result, "group-1.jpg")
def test_gemini_call_finishes_inflight_stream_but_does_not_retry(self) -> None:
module = self.modules["correction"]
self.addCleanup(module.stop_requested.clear)
module.stop_requested.clear()
api_client = Mock()
def stream(**_kwargs):
yield SimpleNamespace(text="first")
module.stop_requested.set()
yield SimpleNamespace(text="-second")
api_client.models.generate_content_stream.side_effect = stream
with patch.object(module, "client", api_client):
self.assertEqual(
module.call_gemini_with_retries("model", [], Mock()),
"first-second",
)
with self.assertRaises(module.CorrectionStopRequested):
module.call_gemini_with_retries("model", [], Mock())
self.assertEqual(api_client.models.generate_content_stream.call_count, 1)
def test_interrupted_live_correction_resumes_only_remaining_groups(self) -> None:
module = self.modules["correction"]
self.addCleanup(module.stop_requested.clear)
with tempfile.TemporaryDirectory() as directory:
workspace = EvaluationWorkspace(Path(directory))
workspace.copies_dir.mkdir()
tasks = []
for index, label in enumerate(("Ex 1", "Ex 2"), start=1):
group_dir = workspace.groups_dir / label
group_dir.mkdir(parents=True)
image = group_dir / f"Group_{index}.jpg"
image.write_bytes(b"image")
atomic_write_json(
image.with_suffix(".json"),
[["01", 0, 400, 1.0, label]],
)
tasks.append((str(image), label))
response = json.dumps(
[{"id": "01", "result": {"error": "", "feedback": []}}]
)
call_count = 0
def stream(**_kwargs):
nonlocal call_count
call_count += 1
if call_count == 1:
module.stop_requested.set()
yield SimpleNamespace(text=response)
api_client = Mock()
api_client.models.generate_content_stream.side_effect = stream
args = module.build_parser().parse_args([str(workspace.root)])
with patch.object(module, "NB_THREADS", 1), patch.object(
module.prompting, "generate_request", return_value=([], Mock())
):
module.configure_runtime(
workspace, tasks, args, api_client=api_client
)
first_status = module.run_configured(args)
first_progress = read_json(workspace.correction_progress_file)
first_results = read_json(workspace.correction_file)
module.configure_runtime(
workspace, tasks, args, api_client=api_client
)
pending_on_resume = list(module.tasks_to_process)
second_status = module.run_configured(args)
final_progress = read_json(workspace.correction_progress_file)
final_results = read_json(workspace.correction_file)
self.assertEqual(first_status, module.ExitCode.INTERRUPTED)
self.assertEqual(first_progress, [list(tasks[0])])
self.assertEqual(len(first_results["Ex 1"]), 1)
self.assertEqual(first_results["Ex 2"], [])
self.assertEqual(pending_on_resume, [tasks[1]])
self.assertEqual(second_status, module.ExitCode.SUCCESS)
self.assertEqual(final_progress, [list(tasks[0]), list(tasks[1])])
self.assertEqual(len(final_results["Ex 1"]), 1)
self.assertEqual(len(final_results["Ex 2"]), 1)
self.assertEqual(call_count, 2)
def test_interrupted_label_errors_resume_auxiliary_requests(self) -> None:
module = self.modules["correction"]
self.addCleanup(module.stop_requested.clear)
for error_type in ("wrong-label", "additional-answer"):
with self.subTest(error_type=error_type), tempfile.TemporaryDirectory() as directory:
workspace = EvaluationWorkspace(Path(directory))
workspace.copies_dir.mkdir()
group_dir = workspace.groups_dir / "Ex 1"
group_dir.mkdir(parents=True)
image = group_dir / "Group_1.jpg"
image.write_bytes(b"image")
atomic_write_json(
image.with_suffix(".json"),
[["01", 0, 400, 1.0, "Ex 1"]],
)
task = (str(image), "Ex 1")
response = json.dumps(
[
{
"id": "01",
"result": {"error": error_type, "feedback": []},
}
]
)
primary_call_count = 0
def stream(**_kwargs):
nonlocal primary_call_count
primary_call_count += 1
module.stop_requested.set()
yield SimpleNamespace(text=response)
def resolve_error(_pid, _label, result, _pdf_path):
self.assertEqual(result["error"], error_type)
result["error"] = ""
return []
api_client = Mock()
api_client.models.generate_content_stream.side_effect = stream
args = module.build_parser().parse_args(
[str(workspace.root), "--overwrite"]
)
with patch.object(module, "NB_THREADS", 1), patch.object(
module.prompting, "generate_request", return_value=([], Mock())
), patch.object(
module, "handle_label_errors", side_effect=resolve_error
) as handler:
module.configure_runtime(
workspace, [task], args, api_client=api_client
)
first_status = module.run_configured(args)
saved_pending = read_json(
workspace.correction_pending_responses_file
)
first_progress_exists = (
workspace.correction_progress_file.exists()
)
first_correction_exists = workspace.correction_file.exists()
module.configure_runtime(
workspace, [task], args, api_client=api_client
)
second_status = module.run_configured(args)
self.assertEqual(first_status, module.ExitCode.INTERRUPTED)
self.assertFalse(first_progress_exists)
self.assertFalse(first_correction_exists)
self.assertEqual(
json.loads(saved_pending[str(image)]), json.loads(response)
)
self.assertEqual(second_status, module.ExitCode.SUCCESS)
self.assertEqual(primary_call_count, 1)
handler.assert_called_once()
self.assertEqual(
read_json(workspace.correction_progress_file), [list(task)]
)
corrected = read_json(workspace.correction_file)
self.assertEqual(corrected["Ex 1"][0][0]["result"]["error"], "")
self.assertEqual(
read_json(workspace.correction_pending_responses_file), {}
)
def test_correction_reset_restores_old_and_deletes_new_files(self) -> None:
module = self.modules["correction"]
with tempfile.TemporaryDirectory() as directory:
@@ -1233,6 +1607,9 @@ class StandardCliTests(unittest.TestCase):
copy_dir.mkdir(parents=True)
atomic_write_json(evaluation / "correction.json", {"old": True})
atomic_write_json(evaluation / "correction_progress.json", ["old"])
atomic_write_json(
evaluation / "correction_pending_responses.json", {"group": "response"}
)
(copy_dir / "Ex 1.pdf").write_bytes(b"current")
(copy_dir / "Ex 1_old.pdf").write_bytes(b"original")
(copy_dir / "Ex 2_new.pdf").write_bytes(b"generated")
@@ -1240,6 +1617,9 @@ class StandardCliTests(unittest.TestCase):
self.assertEqual(module.main([str(evaluation), "--reset"]), 0)
self.assertFalse((evaluation / "correction.json").exists())
self.assertFalse((evaluation / "correction_progress.json").exists())
self.assertFalse(
(evaluation / "correction_pending_responses.json").exists()
)
self.assertEqual((copy_dir / "Ex 1.pdf").read_bytes(), b"original")
self.assertFalse((copy_dir / "Ex 1_old.pdf").exists())
self.assertFalse((copy_dir / "Ex 2_new.pdf").exists())
@@ -1333,6 +1713,43 @@ class StandardCliTests(unittest.TestCase):
def test_post_correction_main_cleans_json_atomically(self) -> None:
module = self.modules["post_correction"]
cleaned_text = module.clean_string(
r"outside_name already\_escaped; "
r"bad $\\mathbb{U}_n = \\{z \\in \\mathbb{C}\\}$; "
r"valid $\begin{cases} A=0 \\ B=1 \end{cases}$",
{},
)
self.assertEqual(
cleaned_text,
r"outside\_name already\_escaped; "
r"bad $\mathbb{U}_n = \{z \in \mathbb{C}\}$; "
r"valid $\begin{cases} A=0 \\ B=1 \end{cases}$",
)
self.assertEqual(module.clean_string(cleaned_text, {}), cleaned_text)
corrupted_json_escapes = (
"Formulas $"
+ "\x0c"
+ "rac{1}{2}$, $"
+ "\t"
+ "heta "
+ "\x0c"
+ "euille 0 [2"
+ "\t"
+ "extbackslash pi]$, $e "
+ "\t"
+ "imes x$, and ["
+ "\n"
+ "egthinspace[0,n]"
)
self.assertEqual(
module.clean_string(corrupted_json_escapes, {}),
r"Formulas $\frac{1}{2}$, $\theta \equiv 0 [2\pi]$, "
r"$e \times x$, and [\negthinspace[0,n]",
)
self.assertEqual(
module.clean_string("x ∈ A and B ⊂ C", {}),
r"x \ensuremath{\in} A and B \ensuremath{\subset} C",
)
with tempfile.TemporaryDirectory() as directory:
evaluation = Path(directory) / "Exam"
evaluation.mkdir()
@@ -1340,11 +1757,20 @@ class StandardCliTests(unittest.TestCase):
evaluation / "correction.json",
{"text": "outside_name and $math_name$", "suffix": "_new"},
)
original = (evaluation / "correction.json").read_bytes()
backup = evaluation / "correction_precleanup.json"
backup.write_bytes(b"previous backup")
self.assertEqual(module.main([str(evaluation)]), 0)
self.assertEqual(backup.read_bytes(), original)
self.assertEqual(
read_json(evaluation / "correction.json"),
{"text": r"outside\_name and $math_name$", "suffix": "_new"},
)
cleaned = (evaluation / "correction.json").read_bytes()
with patch.object(module, "atomic_write_bytes", side_effect=OSError("backup failed")):
self.assertNotEqual(module.main([str(evaluation)]), 0)
self.assertEqual((evaluation / "correction.json").read_bytes(), cleaned)
self.assertEqual(backup.read_bytes(), original)
def test_manual_resolution_with_no_actions_is_safe(self) -> None:
module = self.modules["resolve_manual"]
@@ -1661,6 +2087,53 @@ class WorkflowTests(unittest.TestCase):
self.assertNotIn("update_ods", standard_ids)
self.assertIn("update_ods", personal_ids)
def test_get_personal_interro_files_copies_the_three_expected_sources(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
source = root / "source"
evaluation = root / "Interro07"
source.mkdir()
evaluation.mkdir()
contents = {
"Interro07.pdf": b"pdf",
"Interro07.tex": b"statement",
"Interro07c.tex": b"correction",
}
for name, content in contents.items():
(source / name).write_bytes(content)
copied = get_personal_interro_files(evaluation, source)
self.assertEqual(
copied,
(
evaluation / "enonce.pdf",
evaluation / "enonce.tex",
evaluation / "correction.tex",
),
)
self.assertEqual((evaluation / "enonce.pdf").read_bytes(), b"pdf")
self.assertEqual((evaluation / "enonce.tex").read_bytes(), b"statement")
self.assertEqual((evaluation / "correction.tex").read_bytes(), b"correction")
def test_get_personal_interro_files_validates_name_and_all_sources_first(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
source = root / "source"
source.mkdir()
invalid = root / "DS07"
invalid.mkdir()
with self.assertRaisesRegex(ValueError, "Interro"):
get_personal_interro_files(invalid, source)
evaluation = root / "Interro07"
evaluation.mkdir()
(evaluation / "enonce.pdf").write_bytes(b"existing")
(source / "Interro07.pdf").write_bytes(b"new")
with self.assertRaisesRegex(FileNotFoundError, "Interro07.tex"):
get_personal_interro_files(evaluation, source)
self.assertEqual((evaluation / "enonce.pdf").read_bytes(), b"existing")
def test_first_visit_automation_is_declared_on_expected_steps(self) -> None:
auto_start = {
step.id for step in self.steps.values() if step.auto_start_first_visit
@@ -1683,6 +2156,13 @@ class WorkflowTests(unittest.TestCase):
)
self.assertEqual(skip_without_conflicts, {"manual_resolution"})
def test_manual_resolution_precedes_post_correction(self) -> None:
ordered_ids = [step.id for step in build_workflow(False)]
self.assertLess(
ordered_ids.index("manual_resolution"),
ordered_ids.index("post_correction"),
)
def test_crop_auto_start_follows_always_crop_configuration(self) -> None:
for enabled in (False, True):
with self.subTest(enabled=enabled), patch(
@@ -1708,6 +2188,16 @@ class WorkflowTests(unittest.TestCase):
def test_review_persp_has_shorter_title(self) -> None:
self.assertEqual(self.steps["review_persp"].title, "Relire les barèmes")
def test_cutleft_title_and_shortcuts_are_documented(self) -> None:
step = self.steps["cutleft"]
self.assertEqual(
step.title,
"Découper une partie à gauche pour détection des labels",
)
for shortcut in ("n", "N", "t", "l", "1", "Entrée", "s"):
with self.subTest(shortcut=shortcut):
self.assertIn(shortcut, step.description)
def test_export_has_shorter_title_and_annotation_argument(self) -> None:
self.assertEqual(self.steps["export"].title, "Exporter")
command = self.command(
@@ -1742,6 +2232,47 @@ class WorkflowTests(unittest.TestCase):
detected_annotation_directories(evaluation), ("Bnot", "Anot")
)
def test_giving_names_utilities_find_and_rename_problematic_copies(self) -> None:
with tempfile.TemporaryDirectory() as directory:
evaluation = Path(directory)
return_dir = evaluation / "A Rendre"
original_dir = evaluation / "Copies Originales"
return_dir.mkdir()
original_dir.mkdir()
(original_dir / "Copie01.pdf").write_bytes(b"original")
folders = {
"Unknown (01)": (b"unknown-jpg", b"unknown-pdf"),
"Dupont (02)": (b"two-jpg", b"two-pdf"),
"Dupont (03)": (b"three-jpg", b"three-pdf"),
"Unique (04)": (b"four-jpg", b"four-pdf"),
}
for folder_name, (jpg, pdf) in folders.items():
folder = return_dir / folder_name
folder.mkdir()
base_name = folder_name.rsplit(" (", 1)[0]
(folder / f"{base_name}.jpg").write_bytes(jpg)
(folder / f"{base_name}.pdf").write_bytes(pdf)
(folder / "score.json").write_text("{}", encoding="utf-8")
(folder / "answers").mkdir()
issues = find_name_issues(evaluation)
self.assertEqual([issue.copy_id for issue in issues], ["01", "02", "03"])
self.assertEqual(
original_copy_path(evaluation, "01"),
original_dir / "Copie01.pdf",
)
renamed = rename_return_copy(return_dir / "Unknown (01)", "Alice")
self.assertEqual(renamed.name, "Alice (01)")
self.assertEqual((renamed / "Alice.jpg").read_bytes(), b"unknown-jpg")
self.assertEqual((renamed / "Alice.pdf").read_bytes(), b"unknown-pdf")
self.assertTrue((renamed / "score.json").is_file())
self.assertTrue((renamed / "answers").is_dir())
self.assertEqual(
[issue.copy_id for issue in find_name_issues(evaluation)],
["02", "03"],
)
def test_export_and_import_defaults_follow_previous_runs(self) -> None:
with tempfile.TemporaryDirectory() as directory:
evaluation = Path(directory)
@@ -1768,10 +2299,27 @@ class WorkflowTests(unittest.TestCase):
def test_plotting_shortcuts_describe_open_actions(self) -> None:
rendered = "\n".join(plotting_shortcut_lines())
self.assertIn("p : revenir à la précédente", rendered)
self.assertIn("ouvrir l’énoncé", rendered)
self.assertIn("ouvrir la copie traitée", rendered)
self.assertIn("ouvrir la copie originale", rendered)
def test_gui_tracks_correction_progress_across_output_chunks(self) -> None:
app = object.__new__(CopienatorApp)
app.active_step_id = "correction"
app._correction_progress_buffer = ""
app.info_var = Mock()
app._track_correction_progress(
"[Progression correction] Groupes trai"
)
app.info_var.set.assert_not_called()
app._track_correction_progress("tés : 3/8\n")
app.info_var.set.assert_called_once_with(
"Correction : 3 groupe(s) traité(s) sur 8 (38 %)."
)
def test_proxy_is_opt_in_and_prefilled(self) -> None:
base = {"HTTPS_PROXY": "http://system-proxy", "OTHER": "kept"}
without_proxy = build_runner_environment(
@@ -1807,6 +2355,21 @@ class WorkflowTests(unittest.TestCase):
)
self.assertEqual(command[4], "correct")
def test_refaire_correction_does_not_offer_overwrite(self) -> None:
step = self.steps["correction"]
overwrite = next(
argument for argument in step.arguments if argument.name == "overwrite"
)
self.assertNotIn("refaire", overwrite.variants)
command = self.command(
"correction",
"refaire",
{"target": self.evaluation, "overwrite": True},
)
self.assertIn("--refaire", command)
self.assertNotIn("--overwrite", command)
def test_hybrid_correction_arguments(self) -> None:
command = self.command(
"correction", "hybrid", {"target": self.evaluation, "batch_from": "Ex 4"}
@@ -1829,6 +2392,19 @@ class WorkflowTests(unittest.TestCase):
("live", "batch", "hybrid", "refaire"),
)
def test_every_graphical_argument_has_tooltip_documentation(self) -> None:
for step in self.steps.values():
for spec in step.arguments:
with self.subTest(step=step.id, argument=spec.name):
self.assertTrue(spec.help.strip())
self.assertTrue(spec.help.rstrip().endswith("."))
def test_final_score_documents_immediate_gestion_classe_prerequisite(self) -> None:
description = self.steps["final_score"].description
self.assertIn("gestion_classe wse", description)
self.assertIn("juste avant", description)
self.assertIn("histogramme.pdf", description)
def test_options_previously_requiring_free_form_arguments_have_controls(self) -> None:
status = self.command(
"batch_status",
@@ -1837,7 +2413,7 @@ class WorkflowTests(unittest.TestCase):
)
self.assertEqual(
command_arguments(status),
["--download", "files/job-1", "--output", "/tmp/result.jsonl"],
["--evaluation", self.evaluation],
)
grouped = self.command(
@@ -1851,7 +2427,46 @@ class WorkflowTests(unittest.TestCase):
)
self.assertEqual(
command_arguments(grouped),
[self.evaluation, "--refaire", "--annotation-dir", "Bnot"],
[
self.evaluation,
"--refaire",
"--annotation-dir",
"Bnot",
"--no-return-answers-context",
"--return-answers-question",
"--return-answers-solution",
],
)
def test_return_answer_controls_follow_configuration_and_can_be_hidden(self) -> None:
with patch.multiple(
workflow_module.configuration,
RETURN_ANSWERS_ENABLED=True,
RETURN_ANSWERS_CONTEXT=True,
RETURN_ANSWERS_QUESTION=False,
RETURN_ANSWERS_SOLUTION=True,
):
step = next(
item for item in build_workflow(True) if item.id == "read_annotations"
)
specs = {spec.name: spec for spec in step.arguments}
self.assertIs(specs["return_answers_context"].default, True)
self.assertIs(specs["return_answers_question"].default, False)
self.assertIs(specs["return_answers_solution"].default, True)
with patch.object(
workflow_module.configuration, "RETURN_ANSWERS_ENABLED", False
):
hidden_step = next(
item for item in build_workflow(True) if item.id == "read_annotations"
)
self.assertFalse(
{
"return_answers_context",
"return_answers_question",
"return_answers_solution",
}
& {spec.name for spec in hidden_step.arguments}
)
def test_verbose_is_added_only_to_commands_that_support_it(self) -> None:
+56
View File
@@ -0,0 +1,56 @@
import os
import tempfile
import tkinter as tk
import unittest
from pathlib import Path
from types import SimpleNamespace
import pymupdf
from copienator_gui.cut_helper import CutHelper, PAGE_GAP
from copienator_gui.manual_resolution import ManualResolutionPanel
@unittest.skipUnless(os.environ.get("DISPLAY"), "requires a display")
class CutHelperTests(unittest.TestCase):
def test_cut_button_drag_to_page_gap_and_enter_only_produces_command(self):
with tempfile.TemporaryDirectory() as temporary:
directory = Path(temporary)
source = directory / "Copies" / "Copie16" / "A.pdf"
source.parent.mkdir(parents=True)
with pymupdf.open() as document:
for height in (200, 400):
page = document.new_page(width=300, height=height)
page.insert_text((20, 40), "Student answer")
document.save(source)
manual = directory / "manual_resolutions.txt"
manual.write_text("Copie16 A -> B|\n")
original = source.read_bytes()
root = tk.Tk()
try:
panel = ManualResolutionPanel(root, lambda: directory)
panel.pack()
root.update()
self.assertEqual(len(panel.cut_buttons), 1)
panel.cut_buttons[0].invoke()
helper = next(child for child in panel.winfo_children() if isinstance(child, CutHelper))
helper.canvas.yview_moveto(0)
root.update()
helper.move_bar(SimpleNamespace(y=200 * helper.scale + PAGE_GAP / 2))
self.assertIn("entre les pages 1 et 2", helper.caption.get())
helper.keep.set(2)
helper.mode.set("x")
helper.focus_force()
root.update()
helper.event_generate("<Return>")
root.update()
self.assertFalse(helper.winfo_exists())
self.assertEqual(panel.cut_result.get(), "Copie16 A c{33.333333}2x B|")
panel.copy_cut_command()
self.assertEqual(root.clipboard_get(), panel.cut_result.get())
self.assertEqual(source.read_bytes(), original)
self.assertEqual(manual.read_text(), "Copie16 A -> B|\n")
panel.reload()
self.assertEqual(len(panel.cut_buttons), 1)
finally:
root.destroy()
+74
View File
@@ -0,0 +1,74 @@
import os
import tempfile
import tkinter as tk
import unittest
from pathlib import Path
from unittest.mock import call, patch
from copienator_gui.manual_resolution import ManualResolutionPanel
@unittest.skipUnless(os.environ.get("DISPLAY"), "requires a display")
class ManualResolutionPanelTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.evaluation = Path(self.temp.name)
self.root = tk.Tk()
self.addCleanup(self.root.destroy)
self.path = self.evaluation / "manual_resolutions.txt"
def panel(self):
panel = ManualResolutionPanel(self.root, lambda: self.evaluation)
panel.pack()
self.root.update()
return panel
def test_preview_editor_and_each_pdf_pair_use_shared_resolution_rules(self):
content = "### Instructions\nCopie01 Ex 1 x> |Ex 2\n\nCopie02 Ex 3 ss Ex 4|\n"
self.path.write_text(content, encoding="utf-8")
paths = []
for copy, label in (("01", "Ex 1_new"), ("01", "Ex 2_old"),
("02", "Ex 3"), ("02", "Ex 4")):
path = self.evaluation / "Copies" / f"Copie{copy}" / f"{label}.pdf"
path.parent.mkdir(parents=True, exist_ok=True)
path.touch()
paths.append(path)
panel = self.panel()
self.assertEqual(panel.text.get("1.0", "end-1c"), content)
self.assertEqual(len(panel.pdf_buttons), 4)
self.assertEqual([button.cget("text") for button in panel.pdf_buttons],
["PDF source", "PDF cible"] * 2)
with patch("copienator_gui.manual_resolution.open_path") as opened:
panel.editor_button.invoke()
for button in panel.pdf_buttons:
button.invoke()
self.assertEqual(opened.call_args_list, [call(self.path), *map(call, paths)])
def test_reload_keeps_invalid_lines_visible_and_removes_stale_actions(self):
self.path.write_text("Copie01 A -> B|\n", encoding="utf-8")
panel = self.panel()
self.path.write_text("bad instruction\nCopie02 C sx D\n", encoding="utf-8")
panel.reload()
self.assertIn("bad instruction", panel.text.get("1.0", "end"))
self.assertIn("lignes invalides : 1", panel.status.cget("text"))
self.assertEqual(len(panel.pdf_buttons), 2)
self.path.unlink()
panel.reload()
self.assertFalse(panel.pdf_buttons)
self.assertEqual(str(panel.editor_button.cget("state")), "disabled")
def test_missing_source_does_not_prevent_opening_destination(self):
self.path.write_text("Copie01 A -> B|\n", encoding="utf-8")
destination = self.evaluation / "Copies" / "Copie01" / "B.pdf"
destination.parent.mkdir(parents=True)
destination.touch()
panel = self.panel()
with patch("copienator_gui.manual_resolution.open_path") as opened, patch(
"copienator_gui.manual_resolution.messagebox.showerror"
) as error:
panel.pdf_buttons[0].invoke()
opened.assert_not_called()
panel.pdf_buttons[1].invoke()
opened.assert_called_once_with(destination)
self.assertIn("A.pdf", error.call_args.args[1])
+14
View File
@@ -108,6 +108,14 @@ class InkDetectionTests(unittest.TestCase):
.7,(190,190,190),2)
self.assertGreater(detect_bounds(image,dpi=150)['bottom_px'],1100)
def test_sparse_central_pencil_on_dark_grid_survives(self):
image = self.dark_grid()
# Thin, pale handwriting crossing the ruling is split into sparse
# components. Its central position distinguishes it from page holes.
cv2.putText(image,'result',(330,1090),cv2.FONT_HERSHEY_SCRIPT_SIMPLEX,
.85,(155,155,155),1)
self.assertGreater(detect_bounds(image,dpi=150)['bottom_px'],1090)
def test_large_unruled_diagram_is_not_paper(self):
image = np.full((1200,850,3),255,np.uint8)
cv2.rectangle(image,(100,100),(750,1100),(20,20,20),3)
@@ -151,6 +159,12 @@ class InkDetectionTests(unittest.TestCase):
r = detect_bounds(image,dpi=150)
self.assertGreater(r['bottom_px'],1060)
def test_faint_page_number_in_outer_band_does_not_block_crop(self):
image = self.dark_grid()
cv2.putText(image,'4/',(3,1160),cv2.FONT_HERSHEY_SIMPLEX,
.55,(130,130,130),1)
self.assertLess(detect_bounds(image,dpi=150)['bottom_px'],850)
def test_faded_neutral_grid(self):
image = self.dark_grid()
image[np.all(image == 65,axis=2)] = 145
+125
View File
@@ -0,0 +1,125 @@
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
import numpy as np
import pymupdf
from copienator import CliError, EvaluationWorkspace, atomic_write_json, read_json
from copienator.commands.resolve_manual import parse_instruction_text, resolve_manual
from copienator.pdf_cut import cut_position, split_pdf
from copienator_gui.cut_helper import PAGE_GAP, cut_operator, percentage_at_y, y_at_percentage
def make_pdf(path, heights=(300, 700), rotation=0, cropped=False):
with pymupdf.open() as document:
for index, height in enumerate(heights):
page = document.new_page(width=200, height=height)
page.draw_rect(pymupdf.Rect(0, 0, 200, height / 2), color=None, fill=(1, 0, 0))
page.draw_rect(pymupdf.Rect(0, height / 2, 200, height), color=None, fill=(0, 0, 1))
page.insert_text((30, 30), f"Page {index + 1}")
if cropped:
page.set_cropbox(pymupdf.Rect(10, 20, 180, height - 30))
page.set_rotation(rotation)
document.save(path)
class ManualCutTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
def test_parser_accepts_new_operators_and_rejects_invalid_cuts(self):
for keep in (1, 2):
for action in (">", "x"):
item = parse_instruction_text(f"Copie16 Ex 4 : 1) c{{43.125}}{keep}{action} |Ex 4 : 2)")[0]
self.assertEqual(item.cut, (43.125, keep))
self.assertEqual(item.should_merge, action == ">")
self.assertTrue(item.pipe_first)
for operator in ("c{0}1>", "c{100}1>", "c{-1}1>", "c{101}1>", "c{43}3>",
"c{43}1s", "c{NaN}1>"):
with self.assertRaises(CliError):
parse_instruction_text(f"Copie16 A {operator} B")
with self.assertRaises(CliError):
parse_instruction_text("Copie16 A c{43}1> A")
def test_page_boundary_preserves_whole_pages_without_clipping(self):
source, first, second = [self.root / name for name in ("source.pdf", "first.pdf", "second.pdf")]
make_pdf(source, heights=(100, 200), rotation=180)
before = source.read_bytes()
with patch("pymupdf.Page.show_pdf_page", side_effect=AssertionError("must not clip whole pages")):
split_pdf(source, 33.333333, first, second)
for path, height in ((first, 100), (second, 200)):
with pymupdf.open(path) as pdf:
self.assertEqual(len(pdf), 1)
self.assertEqual(pdf[0].rect.height, height)
self.assertEqual(pdf[0].rotation, 180)
self.assertEqual(source.read_bytes(), before)
def test_in_page_cut_preserves_visible_pixels_for_cropped_rotated_pages(self):
source, first, second = [self.root / name for name in ("source.pdf", "first.pdf", "second.pdf")]
for rotation in (0, 90, 180, 270):
with self.subTest(rotation=rotation):
make_pdf(source, heights=(400,), rotation=rotation, cropped=True)
with pymupdf.open(source) as pdf:
pix = pdf[0].get_pixmap()
expected = np.frombuffer(pix.samples, np.uint8).reshape(pix.height, pix.width, 3)
split_pdf(source, 50, first, second)
for path, pixels in ((first, expected[:len(expected)//2]), (second, expected[len(expected)//2:])):
with pymupdf.open(path) as pdf:
pix = pdf[0].get_pixmap()
actual = np.frombuffer(pix.samples, np.uint8).reshape(pix.height, pix.width, 3)
self.assertEqual(actual.shape, pixels.shape)
self.assertLess(np.abs(actual.astype(float) - pixels).mean(), 0.1)
def test_cut_within_page_preserves_subsequent_pages(self):
source, first, second = [self.root / name for name in ("source.pdf", "first.pdf", "second.pdf")]
make_pdf(source)
split_pdf(source, 15, first, second)
with pymupdf.open(first) as pdf:
self.assertEqual([page.rect.height for page in pdf], [150])
with pymupdf.open(second) as pdf:
self.assertEqual([page.rect.height for page in pdf], [150, 700])
def test_resolver_archives_source_and_recorrrects_both_labels(self):
for keep in (1, 2):
for mode in (">", "x"):
for pipe_first in (False, True):
with self.subTest(keep=keep, mode=mode, pipe_first=pipe_first):
root = self.root / f"{keep}{mode == '>'}{pipe_first}"
copies = root / "Copies" / "Copie16"
copies.mkdir(parents=True)
source, target = copies / "A.pdf", copies / "B.pdf"
make_pdf(source)
make_pdf(target, heights=(80,))
original, destination = source.read_bytes(), target.read_bytes()
atomic_write_json(root / "correction.json", {
label: [[{"id": "16", "result": {}}]] for label in ("A", "B")
})
new_label = "|B" if pipe_first else "B|"
(root / "manual_resolutions.txt").write_text(f"Copie16 A c{{30}}{keep}{mode} {new_label}\n")
self.assertEqual(resolve_manual(EvaluationWorkspace(root)), 0)
self.assertEqual((copies / "A_old.pdf").read_bytes(), original)
self.assertEqual((copies / "B_old.pdf").read_bytes(), destination)
retained, moved = (300, 700) if keep == 1 else (700, 300)
with pymupdf.open(copies / "A_new.pdf") as pdf:
self.assertEqual([page.rect.height for page in pdf], [retained])
with pymupdf.open(copies / "B_new.pdf") as pdf:
expected = ([moved, 80] if pipe_first else [80, moved]) if mode == ">" else [moved]
self.assertEqual([page.rect.height for page in pdf], expected)
self.assertEqual(read_json(root / "refaire.json"), [["Copie16", ["A", "B"]]])
self.assertTrue(all(read_json(root / "correction.json")[label][0][0]["result"]["suffix"] == "_new" for label in ("A", "B")))
self.assertFalse(list(copies.glob("temp_*.pdf")))
def test_helper_snaps_to_gap_and_round_trips_percentage(self):
heights = [100, 200]
for y in (95, 100, 100 + PAGE_GAP / 2, 100 + PAGE_GAP + 5):
percent = percentage_at_y(y, heights, 1)
self.assertEqual(cut_position(heights, percent), (1, 0))
self.assertEqual(y_at_percentage(percent, heights, 1), 100 + PAGE_GAP / 2)
self.assertEqual(cut_operator(43, 1, ">"), "c{43}1>")
self.assertEqual(cut_operator(100/3, 2, "x"), "c{33.333333}2x")
self.assertEqual(cut_position(heights, 33.333333), (1, 0))
self.assertNotEqual(cut_position(heights, 33.3)[1], 0)
+65
View File
@@ -0,0 +1,65 @@
import copy
import tempfile
import unittest
from pathlib import Path
from unittest.mock import Mock, patch
import pymupdf
from copienator import EvaluationWorkspace, atomic_write_json, read_json
from copienator.commands import correction, resolve_manual
class ManualResolutionStateTests(unittest.TestCase):
def test_acknowledgement_clears_only_this_target_and_copy(self):
for error in ("wrg-lbl:B?", "wrg-lbl:B?delayed", "wrg-lbl:B?exists", "al:(->)B?(->)C?", "al:(delayed)B"):
with self.subTest(error=error):
source = {"id": "01", "result": {"error": error, "delayed": [
["wrong-label", "B"], ["add-label", "B"], ["add-label", "C"]]}}
other = {"id": "02", "result": {"error": error, "delayed": [["wrong-label", "B"]]}}
before_other = copy.deepcopy(other)
results = {"A": [[source, other]]}
resolve_manual.set_suffix_and_clean_error(results, "01", "A", None, "B")
self.assertEqual(source["result"]["delayed"], [["add-label", "C"]])
self.assertNotIn("B?", source["result"]["error"])
self.assertEqual(other, before_other)
resolve_manual.set_suffix_and_clean_error(results, "01", "A", None, "C")
self.assertNotIn("delayed", source["result"])
def fixture(self, root, operator):
copies = root / "Copies" / "Copie01"
copies.mkdir(parents=True)
for label in ("A", "B"):
with pymupdf.open() as doc:
page = doc.new_page(width=200, height=200)
page.insert_text((20, 40), label)
doc.save(copies / f"{label}.pdf")
atomic_write_json(root / "correction.json", {
"A": [[{"id": "01", "result": {"error": "wrg-lbl:B?", "delayed": [["wrong-label", "B"]]}}]],
"B": [[{"id": "01", "result": {"error": ""}}]],
})
(root / "manual_resolutions.txt").write_text(f"Copie01 A {operator} B|\n")
return EvaluationWorkspace(root)
def test_every_resolution_acknowledges_pending_conflict(self):
for operator in (*resolve_manual.OPERATORS, "c{50}1>", "c{50}2x"):
with self.subTest(operator=operator), tempfile.TemporaryDirectory() as temporary:
workspace = self.fixture(Path(temporary), operator)
resolve_manual.resolve_manual(workspace)
result = read_json(workspace.correction_file)["A"][0][0]["result"]
self.assertNotIn("delayed", result)
self.assertFalse(workspace.manual_resolutions_file.exists())
def test_refaire_does_not_recreate_a_resolved_source_conflict(self):
with tempfile.TemporaryDirectory() as temporary:
workspace = self.fixture(Path(temporary), "x>")
resolve_manual.resolve_manual(workspace)
(workspace.groups_dir / "B").mkdir(parents=True)
args = correction.build_parser().parse_args([str(workspace.root), "--refaire"])
correction.configure_runtime(workspace, [], args, api_client=Mock())
with patch.object(correction.grouping, "get_pdf_height", return_value=400), patch.object(
correction.grouping, "create_jpg"
), patch.object(correction, "process_single_task", return_value=[]):
self.assertEqual(correction.run_configured(args), 0)
self.assertFalse(workspace.manual_resolutions_file.exists())
self.assertNotIn("delayed", correction.results["A"][0][0]["result"])
+26
View File
@@ -36,6 +36,7 @@ class MarkedCopiesTests(unittest.TestCase):
review.had_errors = False
review.completed = False
review.current_shift = 50
review.current_width_offset = 0
review.default_max_per_file = 5
review.current_max_per_file = 1
review.root = Mock()
@@ -70,6 +71,7 @@ class MarkedCopiesTests(unittest.TestCase):
self.assertIn("Copie01.pdf", copy_errors(self.workspace))
self.assertEqual(review.index, 1)
self.assertEqual(review.current_max_per_file, 5)
self.assertEqual(review.current_width_offset, 0)
self.assertTrue(review.had_errors)
review.load_current_image.assert_called_once_with()
@@ -96,6 +98,30 @@ class MarkedCopiesTests(unittest.TestCase):
self.assertTrue(review.stop_prefetch.is_set())
self.assertIn("Copie01.pdf", copy_errors(self.workspace))
def test_enlarge_reprocesses_with_a_wider_selection(self):
review = self.reviewer()
review.trigger_processing = Mock()
review.on_enlarge(cutleft.CROP_WIDTH_STEP)
self.assertEqual(review.current_width_offset, 50)
review.trigger_processing.assert_called_once_with(self.files[0], 50)
def test_process_single_pdf_enlarges_crop_and_caps_it_at_page_edge(self):
page = Image.new("RGB", (900, 300), "white")
with patch.object(cutleft, "get_pdf_pages", return_value=[page]):
regular = cutleft.process_single_pdf(self.files[0])
enlarged = cutleft.process_single_pdf(self.files[0], width_offset=200)
capped = cutleft.process_single_pdf(self.files[0], width_offset=1000)
fullpage = cutleft.process_single_pdf(
self.files[0], max_per_file=1, width_offset=200
)
self.assertEqual(regular[1][0].size, (300, 300))
self.assertEqual(enlarged[1][0].size, (500, 300))
self.assertEqual(capped[1][0].size, (800, 300))
self.assertEqual(fullpage[1][0].size, (900, 300))
def test_processing_blocks_skip_and_failed_conversion_is_flagged(self):
review = self.reviewer()
review.is_processing = True
+33
View File
@@ -0,0 +1,33 @@
import unittest
from unittest.mock import patch
from copienator import prompting
class PromptingTests(unittest.TestCase):
def test_perspective_is_inserted_with_alternative_method_guidance(self) -> None:
with patch.object(
prompting, "get_label_text_content", return_value="Question"
), patch.object(
prompting, "get_label_sol_content", return_value="Solution"
), patch.object(
prompting, "get_label_persp_content", return_value="Barème détaillé"
):
prompt = prompting.make_prompt("evaluation", "Ex 1")
self.assertIn(prompting.PERSPECTIVE_GUIDANCE, prompt)
self.assertIn("Barème détaillé", prompt)
def test_alternative_method_guidance_is_omitted_without_perspective(self) -> None:
with patch.object(
prompting, "get_label_text_content", return_value="Question"
), patch.object(
prompting, "get_label_sol_content", return_value="Solution"
), patch.object(prompting, "get_label_persp_content", return_value=None):
prompt = prompting.make_prompt("evaluation", "Ex 1")
self.assertNotIn(prompting.PERSPECTIVE_GUIDANCE, prompt)
if __name__ == "__main__":
unittest.main()
+124
View File
@@ -0,0 +1,124 @@
import contextlib
import io
import itertools
import runpy
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from copienator import EvaluationWorkspace, atomic_write_json, configuration
from copienator.commands import clean, giving_names
class ReturnOutputTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.workspace = EvaluationWorkspace(Path(self.temp.name))
self.workspace.copies_dir.mkdir()
atomic_write_json(self.workspace.copies_dir / "Copie01.json", {"name": "Student"})
(self.workspace.copies_dir / "Copie01.pdf").write_bytes(b"original")
atomic_write_json(self.workspace.correction_file, {})
(self.workspace.root / "names").write_text("Student\n")
self.source = self.workspace.root / "BGnot" / "Copie01"
self.source.mkdir(parents=True)
(self.source / "Concat.jpg").write_bytes(b"jpeg")
(self.source / "Concat_F.pdf").write_bytes(b"pdf")
atomic_write_json(self.source / "score.json", {"Ex 1": "4"})
atomic_write_json(self.source / "info.json", {
"Ex 1": {"present": True, "not_empty": True, "touched": True, "score": "4"}
})
self.answer_option = patch.object(configuration, "RETURN_ANSWERS_ENABLED", False)
self.answer_option.start()
self.addCleanup(self.answer_option.stop)
self.destination = self.workspace.return_dir / "Student (01)"
def prepare(self, jpeg=True, pdf=True):
with patch.object(configuration, "RETURN_JPEG_ENABLED", jpeg), patch.object(
configuration, "RETURN_PDF_ENABLED", pdf
), contextlib.redirect_stdout(io.StringIO()):
self.assertEqual(giving_names.run(self.workspace, annotation_dir="BGnot"), 0)
def test_all_combinations_and_reenable_preserve_sources_and_scores(self):
for jpeg, pdf in itertools.product((True, False), repeat=2):
with self.subTest(jpeg=jpeg, pdf=pdf):
self.prepare()
self.prepare(jpeg, pdf)
expected = {"score.json", "info.json"}
if jpeg:
expected.add("Student.jpg")
if pdf:
expected.add("Student.pdf")
self.assertEqual({p.name for p in self.destination.iterdir()}, expected)
self.assertEqual((self.source / "Concat.jpg").read_bytes(), b"jpeg")
self.assertEqual((self.source / "Concat_F.pdf").read_bytes(), b"pdf")
self.assertEqual(
(self.destination / "score.json").read_bytes(),
(self.source / "score.json").read_bytes(),
)
self.prepare()
self.assertEqual((self.destination / "Student.jpg").read_bytes(), b"jpeg")
self.assertEqual((self.destination / "Student.pdf").read_bytes(), b"pdf")
def test_disabling_removes_regular_files_and_broken_links_only(self):
self.prepare()
jpg = self.destination / "Student.jpg"
jpg.unlink()
jpg.write_bytes(b"old fallback copy")
pdf = self.destination / "Student.pdf"
pdf.unlink()
pdf.symlink_to(self.source / "missing.pdf")
unrelated = self.destination / "notes.txt"
unrelated.write_text("keep")
self.prepare(False, False)
self.assertEqual({p.name for p in self.destination.iterdir()}, {"score.json", "info.json", "notes.txt"})
self.assertTrue((self.source / "Concat_F.pdf").is_file())
def test_fallback_source_respects_options(self):
self.source.parent.rename(self.workspace.root / "Anot")
(self.workspace.root / "BGnot").mkdir()
self.prepare(False, True)
self.assertFalse((self.destination / "Student.jpg").exists())
self.assertEqual((self.destination / "Student.pdf").read_bytes(), b"pdf")
self.assertTrue((self.destination / "score.json").is_file())
self.assertTrue((self.destination / "info.json").is_file())
def test_cleanup_preserves_enabled_returns_without_jpeg(self):
self.prepare(False, True)
with patch.object(configuration, "RETURN_JPEG_ENABLED", False):
plan = clean.build_cleanup_plan(self.workspace)
with contextlib.redirect_stdout(io.StringIO()):
clean.apply_cleanup(self.workspace, plan)
self.assertFalse(self.source.exists())
self.assertEqual((self.destination / "Student.pdf").read_bytes(), b"pdf")
self.assertFalse((self.destination / "Student.pdf").is_symlink())
self.assertTrue((self.destination / "score.json").is_file())
self.assertTrue((self.destination / "info.json").is_file())
def test_cleanup_accepts_scores_only_but_still_requires_scores(self):
self.prepare(False, False)
with patch.object(configuration, "RETURN_JPEG_ENABLED", False):
plan = clean.build_cleanup_plan(self.workspace)
self.assertIn(self.destination / "score.json", plan.kept_files)
(self.destination / "score.json").unlink()
with self.assertRaisesRegex(clean.CliError, "score.json"):
clean.build_cleanup_plan(self.workspace)
def test_older_config_defaults_to_enabled(self):
old_config = self.workspace.root / "old_config.py"
old_config.write_text("ALWAYS_CROP = False\n")
with patch.dict("os.environ", {"COPIENATOR_CONFIG": str(old_config)}):
loaded = runpy.run_path(configuration.__file__)
self.assertIs(loaded["RETURN_JPEG_ENABLED"], True)
self.assertIs(loaded["RETURN_PDF_ENABLED"], True)
self.assertIs(loaded["RETURN_ANSWERS_ENABLED"], False)
self.assertIs(loaded["RETURN_ANSWERS_CONTEXT"], False)
self.assertIs(loaded["RETURN_ANSWERS_QUESTION"], True)
self.assertIs(loaded["RETURN_ANSWERS_SOLUTION"], False)
self.assertNotIn("RETURN_JSON_ENABLED", loaded)
if __name__ == "__main__":
unittest.main()
+96
View File
@@ -0,0 +1,96 @@
"""Compare extracted answers against the same Poppler view used for labels."""
import tempfile
import unittest
from pathlib import Path
import numpy as np
import pymupdf
from pdf2image import convert_from_path
from copienator.commands.splitting_int import (
ANSWER_TOP_PADDING_POINTS,
_render_split_outputs,
)
class SplittingGeometryTests(unittest.TestCase):
def make_source(self, path, rotation, cropped=True):
with pymupdf.open() as document:
page = document.new_page(width=800, height=1000)
# Asymmetric colours exercise both translation and orientation.
for row in range(20):
for column in range(8):
colour = (row / 20, column / 8, (row + column) % 7 / 7)
page.draw_rect(
pymupdf.Rect(column * 100, row * 50,
(column + 1) * 100, (row + 1) * 50),
color=None, fill=colour,
)
page.insert_text((200, 500), "Middle of the answer")
if cropped:
page.set_cropbox(pymupdf.Rect(20, 80, 760, 920))
page.set_rotation(rotation)
document.save(path)
def assert_matches_preview(self, answer, preview, bounds):
actual = np.asarray(convert_from_path(answer, dpi=72)[0]).astype(float)
expected = np.asarray(preview.crop(bounds)).astype(float)
self.assertEqual(actual.shape, expected.shape)
self.assertLess(np.abs(actual - expected).mean(), 0.5)
def test_cropped_and_uncropped_pages_at_every_rotation(self):
visible = {
0: (20, 80, 760, 920),
90: (80, 20, 920, 760),
180: (40, 80, 780, 920),
270: (80, 40, 920, 780),
}
for rotation in (0, 90, 180, 270):
for cropped in (False, True):
for full_answer in (False, True):
with self.subTest(rotation=rotation, cropped=cropped, full=full_answer):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
source = root / "copy.pdf"
self.make_source(source, rotation, cropped)
before = source.read_bytes()
preview = convert_from_path(source, dpi=72, use_cropbox=False)[0]
width, height = preview.size
bounds = visible[rotation] if cropped else (0, 0, width, height)
if full_answer:
coordinates = [("A", 0, 0, 10, 0, 100)]
else:
coordinates = [("A", 0, 250, 260, 0, 100),
("_", 0, 711, 721, 0, 100)]
answer_top = int(height // 4 - ANSWER_TOP_PADDING_POINTS)
bounds = (bounds[0], max(bounds[1], answer_top),
bounds[2], min(bounds[3], height * 3 // 4))
_render_split_outputs(source, coordinates, root)
self.assert_matches_preview(root / "A.pdf", preview, bounds)
self.assertEqual(source.read_bytes(), before)
def test_answer_continues_to_bottom_then_next_page(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
source = root / "copy.pdf"
first = root / "first.pdf"
second = root / "second.pdf"
self.make_source(first, 180)
self.make_source(second, 0)
with pymupdf.open() as document:
for path in (first, second):
with pymupdf.open(path) as part:
document.insert_pdf(part)
document.save(source)
previews = convert_from_path(source, dpi=72, use_cropbox=False)
_render_split_outputs(source, [("A", 0, 700, 720, 0, 100),
("_", 1, 211, 230, 500, 600)], root)
rendered = convert_from_path(root / "A.pdf", dpi=72)
self.assertEqual(len(rendered), 2)
for actual, preview, bounds in zip(rendered, previews,
[(40, 688, 780, 920), (20, 80, 760, 250)]):
expected = np.asarray(preview.crop(bounds)).astype(float)
actual = np.asarray(actual).astype(float)
self.assertEqual(actual.shape, expected.shape)
self.assertLess(np.abs(actual - expected).mean(), 0.5)
+59 -4
View File
@@ -4,13 +4,14 @@ import os
import tempfile
import unittest
from pathlib import Path
from tkinter import ttk
from types import SimpleNamespace
from unittest.mock import Mock, patch
from copienator import CliError, EvaluationWorkspace, ExitCode
from copienator.commands import enonce_info as personal
from copienator.commands import gemini_for_enonce as gemini
from copienator_gui.app import CopienatorApp
from copienator_gui.app import CopienatorApp, get_personal_interro_files
from copienator_gui.workflow import build_workflow
@@ -53,7 +54,7 @@ class PersonalStatementTests(unittest.TestCase):
enabled = {step.id: step for step in build_workflow(True)}
self.assertEqual([variant.id for variant in standard["statement"].variants], ["gemini"])
self.assertEqual([variant.program for variant in enabled["statement"].variants],
["statement", "statement-personal"])
["statement-personal", "statement"])
for ident in ("statement_groups", "statement_persp"):
self.assertNotIn(ident, standard)
self.assertTrue(enabled[ident].optional)
@@ -120,6 +121,11 @@ class SelectiveGeminiTests(unittest.TestCase):
self.assertEqual(call.kwargs["contents"][0].parts[0].text, gemini.PROMPT_4)
self.assertTrue(call.kwargs["config"].automatic_function_calling.disable)
def test_rubric_prompt_omits_the_assumed_total_and_requires_latex(self):
self.assertIn("total est toujours implicite", gemini.PROMPT_4)
self.assertIn("caractères mathématiques Unicode", gemini.PROMPT_4)
self.assertIn(r"$\lfloor \sqrt{k} \rfloor$", gemini.PROMPT_4)
def test_incomplete_or_failed_rubrics_preserve_entire_persp(self):
before = self.snapshot()
for last in (self.response({"rubrics": []}), RuntimeError("API unavailable")):
@@ -140,6 +146,48 @@ class SelectiveGeminiTests(unittest.TestCase):
@unittest.skipUnless(os.environ.get("DISPLAY"), "Tk requires a display")
class PersonalStatementGuiTests(unittest.TestCase):
def test_get_file_button_completes_inputs_and_advances(self):
with tempfile.TemporaryDirectory() as temporary:
repository = Path(temporary)
evaluation = repository / "Interro12"
source = repository / "source"
evaluation.mkdir()
source.mkdir()
(repository / "names").touch()
for name, content in (
("Interro12.pdf", b"pdf"),
("Interro12.tex", b"statement"),
("Interro12c.tex", b"correction"),
):
(source / name).write_bytes(content)
app = CopienatorApp(repository, True, evaluation)
try:
app.update()
buttons = [
child
for child in app.form.winfo_children()
if isinstance(child, ttk.Button)
]
get_button = next(
button for button in buttons if button.cget("text") == "Get the files"
)
with patch(
"copienator_gui.app.get_personal_interro_files",
side_effect=lambda target: get_personal_interro_files(target, source),
):
get_button.invoke()
app.update()
self.assertEqual(app.state_store.step("inputs")["status"], "success")
self.assertEqual(app.current_step.id, "statement")
self.assertEqual((evaluation / "enonce.pdf").read_bytes(), b"pdf")
self.assertEqual((evaluation / "enonce.tex").read_bytes(), b"statement")
self.assertEqual((evaluation / "correction.tex").read_bytes(), b"correction")
finally:
for callback in app.tk.splitlist(app.tk.call("after", "info")):
app.after_cancel(callback)
app.destroy()
def test_personal_requirements_and_optional_button_command_previews(self):
with tempfile.TemporaryDirectory() as temporary:
evaluation = Path(temporary)
@@ -149,12 +197,19 @@ class PersonalStatementGuiTests(unittest.TestCase):
app.update()
app.tree.selection_set("statement")
app.update()
app._select_variant(1)
self.assertEqual(app.variant_var.get(), "personal")
self.assertIn("statement-personal", app.command_var.get())
self.assertEqual(app._missing_requirements(app.current_step), [])
self.assertIn("SHEETINFO", app.description_label.cget("text"))
self.assertFalse(
any(
isinstance(child, ttk.LabelFrame)
and child.cget("text") == "Après génération — facultatif"
for child in app.form.winfo_children()
)
)
for ident, flag in (("statement_groups", "--groups-only"), ("statement_persp", "--persp-only")):
app._select_statement_action(ident)
app.tree.selection_set(ident)
app.update()
self.assertEqual(app.current_step.id, ident)
self.assertIn(flag, app.command_var.get())
+48
View File
@@ -0,0 +1,48 @@
import contextlib
import io
import subprocess
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from copienator import utils
class CompileToPdfTests(unittest.TestCase):
def test_warns_when_latex_fails_even_if_partial_pdf_exists(self):
with tempfile.TemporaryDirectory() as temporary:
output = Path(temporary) / "result.pdf"
def failed_run(*_args, **kwargs):
(Path(kwargs["cwd"]) / "text.pdf").write_bytes(b"partial")
return subprocess.CompletedProcess(
args=[], returncode=1,
stdout="! LaTeX Error: Something's wrong--perhaps a missing \\item.\n",
)
console = io.StringIO()
with patch.object(utils.subprocess, "run", side_effect=failed_run), \
contextlib.redirect_stdout(console):
utils.compile_to_pdf("broken", output)
self.assertEqual(output.read_bytes(), b"partial")
self.assertIn("Warning: LaTeX compilation failed", console.getvalue())
self.assertIn("missing \\item", console.getvalue())
def test_warns_when_no_pdf_is_produced(self):
with tempfile.TemporaryDirectory() as temporary:
output = Path(temporary) / "result.pdf"
result = subprocess.CompletedProcess(args=[], returncode=0, stdout="")
console = io.StringIO()
with patch.object(utils.subprocess, "run", return_value=result), \
contextlib.redirect_stdout(console):
utils.compile_to_pdf("valid", output)
self.assertFalse(output.exists())
self.assertIn("produced no PDF", console.getvalue())
if __name__ == "__main__":
unittest.main()