Technology
Every technology choice is intentional. This page explains not just what is used, but why — and the architectural boundaries that keep the system decoupled and resilient.
Why Two Completely Separate Systems?
Python + Node.js video pipeline running on AWS EC2 with PostgreSQL. Orchestrates all production work. Runs on a cron schedule. Pins 100% CPU during render — fundamentally incompatible with serverless function timeout limits.
This Next.js website hosted on Vercel. Reads only from MongoDB Atlas — a presentation-layer copy of pipeline output. The website stays up even when EC2 is offline. MongoDB credentials never touch EC2; EC2 Postgres credentials never touch Vercel.
Runtime
All collectors, ingestion pipeline, evaluation, editorial engine, uploaders, and S3 bridge. Chosen for ecosystem breadth and async/thread-pool capabilities.
Remotion and the batch orchestrator (batch_generate_and_upload.js) run on Node. Mixed runtime because Remotion is a Node/React library.
AI & Editorial
Primary LLM. Used for both Pass 1 mission checks (parallel chunks) and Pass 2 relative ranking. Temperature fixed at 0.0 for deterministic output. JSON schema enforced.
Automatic fallback model when the primary returns a malformed response after retries. A third fallback (gemini-3.1-flash-lite) is also configured.
Microsoft's neural voice synthesis. Produces natural-sounding audio with sentence-level timing metadata, which drives exact Remotion scene-cut boundaries.
Video Production
React-based programmatic video renderer. Chosen because it treats video as a React component — version-controllable, composable, and debuggable like any UI code.
Remotion's rendering engine. Spring physics (useSpring), animated transitions, and dynamic layouts are expressed as CSS-in-JS, not After Effects keyframes.
Compile-time safety for the Remotion template layer. Script schema errors are caught before render, not during it.
Token-based styling in Remotion templates. Same stack as the website, enabling design consistency.
Audio/video utility layer. Used for duration probing and final MP4 metadata tagging.
Cloud & Publishing
The pipeline runs on a compute instance — not serverless. Remotion pins 100% of CPU cores during render, which violates serverless timeout limits. EC2 is the right host.
Ephemeral bridge storage. Video is uploaded once, a presigned URL is generated (2-hour expiry), broadcast to Meta, then auto-deleted. Zero permanent storage cost.
OAuth 2.0 + resumable 2 MB chunked uploads. Private staging with immediate publication. Token cached in token.json for headless cron execution.
Instagram Reels container lifecycle (init → poll → publish) and Facebook Page Video endpoint. Both receive the S3 presigned URL concurrently.
Web & Presentation
This site. SSG for all public Explore pages. Server Components with auth-gated MongoDB reads for the Control Center. Deployed on Vercel.
Presentation archive — one document per published video. Written by the EC2 pipeline after each publish, read by this site. Deliberately decoupled from PostgreSQL so the website works even when EC2 is offline.
Pipeline's internal source of truth — candidates, evaluation results, full run history. Never queried directly by this website.
V1 vs V2 — What Changed
| Aspect | V1 | V2 (Current) |
|---|---|---|
| Sources | GitHub Trending only | HN + Blogs + GitHub Releases + Product Hunt |
| Selection | Random trending repo | Two-pass evidence-driven Gemini editorial |
| Filtering | None | Signal → Quality → Dedup → Staleness gates |
| Output | 1 video, sequential | Up to 5 videos, concurrent pre-production |
| Publishing | Manual upload | YouTube + Instagram + Facebook automatically |
| Storage | Local files accumulated | Ephemeral S3 bridge — zero permanent storage |
| Concurrency | Single-threaded | ThreadPoolExecutor + 100% CPU Remotion render |