Map crawler

serviceupdated 2026-10-09edit the YAML

Crawls the three sites and the Substack archive into one graph of pages, posts, repositories, threads and tags, with body links kept apart from navigation. Runs locally and in every deploy.

Owner: Stefan Coetzee
Tier: Tier 3 Tooling a reader does not meet directly. Waits for the next working session.
Lifecycle: Production
System: Publishing
Address: machinebehavior.io/map
Repository: uncovertechtalent/machinebehavior.io/blob/main/scripts/crawl_map.py
Current state: on the status page

Docs

Map crawler

Runbooks

No runbook yet.

SRE principles applied

Each principle, how this platform applies it and where it falls short: Principles in practice.

Service level objectives

None defined. Health is read from the records listed on the status page. Why that is a gap and what closes it: Service level objectives.

Dependencies

Depends on

Outside the catalog: Substack.

Used by

Service catalog · built from services/map-crawler.yml by build_inside.py · services.json