Cloud VM Scraping Stack

Config Generator · Flatcar Ignition 3.4.0 & Ubuntu cloud-init

VM Settings

Hostname

SSH Public Keys

One key per line

API Keys & Environment Variables → /etc/scraper/.env

Services

Scrapy Project → cloned on first boot

Git repository URL

Private repo? Embed a token: https://<token>@github.com/…

Scrapy: Container installs Scrapy at startup then stays alive. Connect with docker exec -it scraper-scrapy-1 scrapy shell https://example.com