One function
QualityJudge in open-video-ai/open-video
The author described this change as “fix(judge): stringly-typed VLM booleans no longer create spurious issues”. It counts as a record because the check below fails on the code as it stood at a3b80bfd0 and passes on 25bb411ab, with nothing else changed between the two runs.
Projectopen-video-ai/open-video
Fix saved2026-08-07
Sharing licenceApache-2.0 · LICENSE
Change size+10 −3
What the code was meant to do, written into the code itself as a docstring
Extracts frames → vision-assesses vs prompt intent + quality bar → verdict + diagnosis.
The change
| 38 | 38 | paths.append(str(dst)) | |
| 39 | 39 | return paths | |
| 40 | 40 | ||
| 41 | + | @staticmethod | |
| 42 | + | def _flag(value) -> bool: | |
| 43 | + | """VLMs return booleans as strings — treat 'false'/'none'/'no'/'' as False.""" | |
| 44 | + | if isinstance(value, str): | |
| 45 | + | return value.strip().lower() not in ("", "false", "none", "no", "0") | |
| 46 | + | return bool(value) | |
| 47 | + | ||
| 41 | 48 | def diagnose(self, vision_result: dict, prompt: str) -> list: | |
| 42 | 49 | """Parse vision-model output into structured issues + fixes.""" | |
| 43 | 50 | issues = [] | |
| ⋯ | |||
| 45 | 52 | for elem in vision_result.get("missing_elements", []): | |
| 46 | 53 | issues.append(Issue(type="dropped_element", detail=f"'{elem}' from prompt not visible", | |
| 47 | 54 | fix=f"emphasize '{elem}' earlier + more explicitly in the prompt")) | |
| 48 | - | if vision_result.get("artifacts"): | |
| 49 | - | issues.append(Issue(type="artifact", detail=vision_result["artifacts"], | |
| 55 | + | if self._flag(vision_result.get("artifacts")): | |
| 56 | + | issues.append(Issue(type="artifact", detail=str(vision_result["artifacts"]), | |
| 50 | 57 | fix="try different seed or add resolution/stability constraint")) | |
| 51 | 58 | if vision_result.get("motion_quality") == "poor": | |
| 52 | 59 | issues.append(Issue(type="bad_motion", detail="motion is choppy/unnatural", | |
| 53 | 60 | fix="increase steps to 25 or adjust camera amplitude/speed")) | |
| 54 | - | if vision_result.get("incoherence"): | |
| 61 | + | if self._flag(vision_result.get("incoherence")): | |
| 55 | 62 | issues.append(Issue(type="incoherence", detail="frames don't flow coherently", | |
| 56 | 63 | fix="simplify the shot to single-action; reduce cut count")) | |
| 57 | 64 | return issues | |
The check that tells the two apart
fail→pass·tests/test_judge_vlm.py::test_diagnose_ignores_stringly_false_flags
Check file tests/test_judge_vlm.py, taken without changes from the fix and copied onto the older code, so the exact same check runs against both versions.
Origin and history
The code before ita3b80bfd0cbe3e8382ac9031d177af43116bf11e
Broken version dated2026-08-07
Modulecore.judge
Units changedQualityJudge
Fingerprint98f52e4f0b3a2b46
Checked2026-08-18 by goldset/0.1
Every field above is generated by our program. None of it is written by hand.