Miami Public Record Backbone¶

Miami-area (Miami-Dade / Broward / Palm Beach) building discovery starts from authoritative government public records rather than aggregator scraping, then resolves official rental websites and extracts inventory from there. This pipeline writes only into the existing buildings table (see Warehouse Schema) — there is no separate source table — with source-specific evidence merged into the buildings.sources field.

Active public sources¶

Scope	Source	Admission rule
Miami-Dade	Property Appraiser ArcGIS parcel layer	DOR 03-family, 10+ unit apartment classes
Broward	BCPA ArcGIS tax roll	`USE_CODE='03'`, multifamily 10+ units
Palm Beach	PAPA ArcGIS parcel layer	explicit 10+ unit multifamily classes
Tri-County	Florida DBPR active lodging extracts	active `NAPT` license, 10+ rental units
Miami-Dade	County permit ArcGIS layer	new residential `BLDG`/`MBLD` permits, 10+ units
Broward	HCED permit ArcGIS layer	project name explicitly references apartment/multifamily

Key concepts¶

The assessor and DBPR licensing feeds form the countywide "building backbone." Permit observations add construction-lifecycle signal but do not by themselves prove a building is complete, leasing, has available units, or pays locator commission.
Palm Beach also publishes an aggregate permit-activity report and an address-by-address open-permit search (unincorporated areas only). Neither is admitted as bulk property evidence — both are used only for spot lookups.

Merge/write contract¶

Existing curated building fields are fill-only from this pipeline — a public-records refresh can never clear a field that is already populated.
Every observation carries an official source URL, an observed timestamp, a source identity, and its semantics, all recorded in buildings.sources.
Importers explicitly stamp metro_slug='miami', because Palm Beach's parcel layer has no reliable situs ZIP to infer it from.
County permit-only observations stay tagged as pipeline-stage evidence unless the official record itself supplies a completion signal.
Commission terms, website, contact form, and rent claims each still require their own downstream evidence — a public-records row alone can never promote any of those fields.
Form discovery from this pipeline is read-only; it never submits forms or sends outreach.

Note

This pipeline is deliberately conservative about what a government record is allowed to imply. A permit or licensing row establishes that a building exists and meets an admission threshold — it does not establish leasing status, contact information, or commission terms, all of which come from downstream, source-specific evidence.

Running it¶

uv run --env-file .env --extra db python scripts/miami_autoresearch_loop.py --once

Individual idempotent refreshers can also be run standalone:

refresh_miami_public_backbone.py
refresh_florida_apartment_licenses.py
refresh_miami_dade_multifamily_permits.py
refresh_broward_multifamily_permits.py

Each building/provenance batch commits in a single transaction, so a dropped database connection never leaves a partial refresh.

Scheduling¶

This pipeline runs daily via a GitHub Actions workflow (.github/workflows/miami-public-backbone.yml), plus manual dispatch, using an encrypted DATABASE_URL secret. It performs a bounded connection preflight before downloading any county feed, so a firewall or network-policy block fails the run cleanly before any partial import.