fix: a presentation that runs out of memory says so, and the service survives it
On a 4 GB VM the kernel killed the import service mid-conversion. Measured: LibreOffice needs 0.2 GB for a text deck and 1.4-2.1 GB for decks of many different photographs, for about twenty seconds — a spike, not a leak. What made it a problem was the rest: OOMPolicy=stop took the watcher down with LibreOffice, it restarted and resumed the same conversion with no limit, and the library said "Getting this ready" throughout. The service is capped at 40% of memory, is first in line when memory runs short everywhere, and survives a killed LibreOffice (OOMPolicy=continue). A killed conversion is a failed import of kind "memory"; an import interrupted twice is not tried a third time; the card says the computer needed more memory, to close other programs and choose Try again; an import nobody has worked on for 20 minutes shows the same card; and an exclusive lock keeps it to one import at a time. Lab, the real service and LibreOffice: at a 4 GB-machine ceiling the 2.1 GB deck still imported; with no swap and a 300 MB ceiling LibreOffice was killed, the watcher kept its PID and the card offered Try again, which worked; the whole service killed during two attempts gave up after the second; three decks at once never ran two LibreOffices together. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
940f63e0e5
commit
3855989029
@@ -288,6 +288,44 @@ class Presentation:
|
||||
def failed(self):
|
||||
return self.state == "failed"
|
||||
|
||||
@property
|
||||
def failure(self):
|
||||
"""Why it failed, as one word a teacher's card can be chosen by.
|
||||
|
||||
"memory": the conversion was killed for want of memory, or was
|
||||
interrupted too often to keep trying — worth trying again once
|
||||
other programs are closed. "unreadable": the file itself would not
|
||||
convert, which trying again will not change.
|
||||
"""
|
||||
if self.stalled:
|
||||
return "memory"
|
||||
|
||||
if not self.failed:
|
||||
return None
|
||||
|
||||
return self.data.get("error_kind") or "unreadable"
|
||||
|
||||
@property
|
||||
def stalled(self):
|
||||
"""Still "getting ready" long after any import could have finished.
|
||||
|
||||
The importer's own limits end every attempt within
|
||||
CONVERT_TIMEOUT + RENDER_TIMEOUT. Past that, with no answer, the
|
||||
importer is not running at all — and a card that says "Getting
|
||||
this ready" for ever is the thing a teacher must never be left
|
||||
looking at.
|
||||
"""
|
||||
if self.state not in ("importing", "retry"):
|
||||
return False
|
||||
|
||||
started = parse_stamp(self.data.get("attempt_started_at")
|
||||
or self.data.get("imported_at"))
|
||||
|
||||
if started is None:
|
||||
return False
|
||||
|
||||
return (now() - started).total_seconds() > STALLED_SECONDS
|
||||
|
||||
@property
|
||||
def error(self):
|
||||
return self.data.get("error") or ""
|
||||
@@ -458,6 +496,24 @@ def library(root=None):
|
||||
return found
|
||||
|
||||
|
||||
def request_retry(presentation):
|
||||
"""A teacher's Try again: the importer takes it up on its next pass.
|
||||
|
||||
Only a presentation that failed for want of memory, or stalled; a file
|
||||
that will not convert is not going to convert on the second try.
|
||||
"""
|
||||
if presentation.failure != "memory":
|
||||
return False
|
||||
|
||||
presentation.data["state"] = "retry"
|
||||
presentation.data["error"] = None
|
||||
presentation.data["error_kind"] = None
|
||||
presentation.data["attempts"] = 0
|
||||
presentation.data["attempt_started_at"] = stamp()
|
||||
presentation.save()
|
||||
return True
|
||||
|
||||
|
||||
def remove(presentation):
|
||||
"""Delete exactly one managed entry, unless it is on screen now."""
|
||||
try:
|
||||
@@ -642,6 +698,21 @@ PDFTOTEXT = "pdftotext"
|
||||
CONVERT_TIMEOUT = 300
|
||||
RENDER_TIMEOUT = 600
|
||||
|
||||
# Past this, a presentation still "getting ready" has no importer working
|
||||
# on it; see Presentation.stalled.
|
||||
STALLED_SECONDS = CONVERT_TIMEOUT + RENDER_TIMEOUT + 300
|
||||
|
||||
# What convert_to_pdf and render_pages say when the kernel killed the
|
||||
# program doing the work. Measured: a deck of 30 different photographs
|
||||
# takes LibreOffice 1.4 GB for about twenty seconds, 20 larger ones 2.1 GB,
|
||||
# and the importer's service is capped at 40% of the machine's memory.
|
||||
KILLED = "killed for want of memory"
|
||||
|
||||
|
||||
def killed(returncode):
|
||||
"""Did the kernel end this process? SIGKILL, directly or via a shell."""
|
||||
return returncode in (-9, 137)
|
||||
|
||||
|
||||
def _run(command, timeout, **kwargs):
|
||||
import subprocess
|
||||
@@ -704,6 +775,9 @@ def convert_to_pdf(source, destination):
|
||||
shutil.rmtree(profile, ignore_errors=True)
|
||||
shutil.rmtree(workspace, ignore_errors=True)
|
||||
|
||||
if killed(result.returncode):
|
||||
return False, KILLED
|
||||
|
||||
return False, (result.stderr or result.stdout or
|
||||
"the file could not be opened").strip()
|
||||
|
||||
@@ -775,6 +849,9 @@ def render_pages(pdf, directory, width=SLIDE_WIDTH, progress=None):
|
||||
time.sleep(0.4)
|
||||
|
||||
if process.returncode != 0:
|
||||
if killed(process.returncode):
|
||||
return 0, KILLED
|
||||
|
||||
return 0, (process.stderr.read() if process.stderr else
|
||||
"the slides could not be rendered")
|
||||
|
||||
|
||||
Reference in New Issue
Block a user