Commits
Schemes with AI, polite ingestion and new public frontend
Sectoral schemes proposed by AI, historical reprocessing, anti-plagiarism controls,
The longest night of the phase: closing external knowledge end-to-end and, along the way, rebuilding the entire public frontend.
Sectoral schemes proposed by AI
From the panel I can choose between 3 and 10 documents and ask the AI for validated sectoral fields: name, type, order, activation, expiry and revision, all visually editable. Saving creates a new version without destroying previous ones, and subsequent ingestion only applies the active schema and discards invented keys or types. The real E2E proposed 12 fields from three controlled documents and saved the version correctly.
Historical reprocessing and autoposts with support
A new schema version can reprocess up to 20 current documents without touching the historical ones: parallel revisions per schema, reused vectors and idempotent execution. Autoposts receive a diversified dossier by domain with facts, dates and provenance, filter by source and confidence and demand a minimum of independent sources: if support is lacking, draft or cancellation. Two anti-plagiarism controls: textual match per phrase and cosine similarity per paragraph. With this, the external knowledge phase is closed with complete E2E in production and an operational guide.
Three review bugs
The thread audit surfaced three real problems: run() was re-executing runs already in progress (overlapping cron and manual trigger processed the same article twice; now a running run is returned untouched), the dossier traveled as own_content and the AI received it as own content (now it's a separate block: "contrast, don't copy", with a 16k limit), and the rate limits accumulated keys forever (memory without cap; added a limit with purge of the oldest). Regression test and 38 tests in green.
Polite batch ingestion
To avoid being blocked: maximum 5 new documents and 10 downloads per execution, random pause of 1.5–3.5 seconds, robots.txt respected (Disallow and Crawl-delay with cap of 10 s), conditional requests with ETag/Last-Modified and immediate stop on 403/429/503. Two scares along the way: a migration with columns without stamped version (resolved with alembic stamp) and an indentation that broke the SourceDiscovery class without a syntax error — the tests caught it. 41 tests.
The public website, from scratch
The landing goes from demo to sales page: dark with lime accent, live terminal narrating the pipeline, real panel screenshots and an honest comparison with WordPress. The rest of the public frontend rises to the same level: editorial blog, terminal chat, legal pages and a real 404. Three battles: the .rich-text was sizing to its max-width inside flex (capped with min(100%, …)), tables were breaking on mobile (overflow-x: auto) and the key one: :global() in a plain CSS silenced every entire rule due to an invalid selector — weeks of invisible bugs avoided by understanding it's scoped component syntax. The /[lang] vs /[seccion] collision was closed with a single product homepage: languages live in the blog.
At dawn, a list annotation was hiding the built-in type and the API wouldn't start: from __future__ import annotations and production healthy. In the morning, second public iteration: vertical rhythm, own diagrams, comparison without invented figures and chat in terminal format. Production verified at 1440/768/390, without overflow or JS errors.