Skip to content

Setup & Installation

Requirements

  • Python ≥ 3.11
  • uv package manager

Install

uv sync

Running the core command

The core workflow is the leads command, which discovers and enriches rental buildings for a given ZIP code or city:

uv run buildings-scraper leads --zip 60614
uv run buildings-scraper leads --city "Chicago, IL"

Optional extras

Extras add optional dependencies for specific capabilities and are installed with uv sync --extra <name>.

Extra Adds Enables Install
browser playwright>=1.44 Rendering JS-heavy sites (e.g. Zillow) uv sync --extra browser && uv run playwright install chromium
ai anthropic>=0.40 Live AI enrichment via leads --ai-enrich uv sync --extra ai
outreach openai>=1.30, dnspython>=2.4 LLM-drafted outreach replies via an NVIDIA NIM- or OpenRouter-compatible API uv sync --extra outreach
db psycopg[binary]>=3.1 The Postgres warehouse; ships its own libpq, so cron/CI boxes need no system package uv sync --extra db

Core dependencies

Always installed, regardless of extras: httpx, pyyaml, structlog, beautifulsoup4, lxml>=6.1.1, openpyxl, tldextract, curl-cffi, pillow.

Environment

Copy .env.example to .env and fill in secrets from the team password manager. Key secrets:

Variable Purpose
NVIDIA_API_KEY / GEMINI_API_KEY (or OPENROUTER_API_KEY) LLM outreach drafting
DATABASE_URL Postgres warehouse DSN (DigitalOcean Managed Postgres) — required in practice; there is no CSV-only fallback
LINKEDIN_SESSIONS / WHATSAPP_WEB_SESSIONS / GOOGLE_VOICE_SESSIONS Operator-driven outreach channels

Non-secret configuration

Non-secret configuration lives in config/*.yaml/*.json. These files are gitignored; .example templates are provided.

File Purpose
sites.yaml Aggregator HTTP settings
proxies.yaml Bright Data unblocker backend
search.yaml Bright Data SERP
senders.csv Outreach sender pool
outreach.json Outreach configuration
warehouse.json Retention configuration

Warehouse setup

Install the db extra, set DATABASE_URL, then run the migration command:

uv sync --extra db
uv run buildings-scraper db-migrate

Warehouse is DigitalOcean Managed Postgres, not Supabase

Supabase was retired as the warehouse when the system cut over to DigitalOcean Managed Postgres on 2026-07-25. Never point DATABASE_URL at a Supabase host — writers refuse a Supabase DSN outright.

Next steps