How HackFinder Works

A technical deep-dive into the architecture powering hackathon discovery at the edge.

Architecture

Data flows from external sources to the browser in five hops.

Sources→Worker→D1→Pages→Browser

A Cloudflare Worker fetches from Devpost, MLH, and HackerEarth on a cron schedule. Normalized data lands in D1 (SQLite at the edge). Astro on Cloudflare Pages reads D1 at request time and serves HTML.

Data Pipeline

Every hour, the aggregation worker runs five steps:

  1. 1

    Cron fires

    A Cloudflare Cron Trigger wakes the worker every hour.

  2. 2

    Fetch in parallel

    All enabled source adapters run concurrently. If one fails, the rest still succeed.

  3. 3

    Normalize

    Raw responses are mapped to a shared schema — dates standardized, HTML stripped, fields validated.

  4. 4

    Deduplicate

    A unique index on (title, start_date) prevents duplicates. Existing records are merged, not overwritten.

  5. 5

    Store

    Data is upserted into D1. Triggers keep the FTS5 index in sync automatically.

Search

Full-text search powered by SQLite FTS5 with BM25 relevance ranking.

  • ●FTS5 + BM25: Queries hit a virtual table with porter tokenizer and unicode61 — typo-tolerant, language-aware ranking out of the box.
  • ●Filter composition: Format, status, and date filters compose with search queries via SQL WHERE clauses — no post-filtering.
  • ●Sub-300ms responses: D1 runs on the same edge as Pages. No network round-trip to a remote database.

Frontend

Fast by default, interactive where it matters.

CI/CD

Two deployment paths, both automated.

  • ●Pages: Git push to main auto-deploys via Cloudflare's git integration. Preview deploys on every branch.
  • ●Worker: A GitHub Action deploys the aggregation worker on push to main. Runs wrangler deploy from the worker directory.
  • ●Secrets: API tokens and credentials are stored in GitHub Secrets — never committed, injected at deploy time.

Built With Kiro

This entire platform was built using Kiro's spec-driven workflow — from requirements gathering through design, implementation, and deployment. 83 tasks executed, 230 tests passing.

Learn about Kiro