site-scraperv1.4.6

CASOON Open Source

Take a website with you.

site-scraper follows internal links, downloads pages and assets, and rewrites references into a self-contained local copy. Use headless Chrome when the page needs JavaScript, or capture full-page screenshots without mirroring anything.

curl -fsSL https://raw.githubusercontent.com/casoon/site-scraper/main/install.sh | bash
cargo install --git https://github.com/casoon/site-scraper
MITv1.4.6Rust CLILinux + macOS
site-scraper https://www.casoon.de --max-depth 1 --no-sitemap --placeholder local
ℹ Crawling https://www.casoon.de/
✔ https://www.casoon.de/ → index.html
✔ https://www.casoon.de/ratgeber/ → ratgeber/index.html
✔ https://www.casoon.de/joern-seidel/ → joern-seidel/index.html
✔ https://www.casoon.de/arbeitsweise/ → arbeitsweise/index.html
✔ https://www.casoon.de/ki/ → ki/index.html
✔ https://www.casoon.de/hotellerie/rostock/ → hotellerie/rostock/index.html
✔ https://www.casoon.de/impressum/ → impressum/index.html
✔ https://www.casoon.de/tool-engineering/ → tool-engineering/index.html
✔ https://www.casoon.de/technologien/ → technologien/index.html
✔ https://www.casoon.de/handwerker/rostock/ → handwerker/rostock/index.html
✔ https://www.casoon.de/praxis/rostock/ → praxis/rostock/index.html
✔ https://www.casoon.de/ki-automatisierung/ → ki-automatisierung/index.html
✔ https://www.casoon.de/kanzlei/rostock/ → kanzlei/rostock/index.html
✔ https://www.casoon.de/kollaboration/ → kollaboration/index.html
✔ https://www.casoon.de/agb/ → agb/index.html
✔ https://www.casoon.de/leistungskatalog/ → leistungskatalog/index.html
✔ https://www.casoon.de/kontakt/ → kontakt/index.html
✔ https://www.casoon.de/strategieberatung/ → strategieberatung/index.html
✔ https://www.casoon.de/existenzgruendung-webdesign-foerderung/ → existenzgruendung-webdesign-foerderung/index.html
✔ https://www.casoon.de/cloud-entwicklung/ → cloud-entwicklung/index.html
✔ https://www.casoon.de/gastronomie/rostock/ → gastronomie/rostock/index.html
✔ https://www.casoon.de/usp/ → usp/index.html
✔ https://www.casoon.de/dienstleister/rostock/ → dienstleister/rostock/index.html
✔ https://www.casoon.de/datenschutz/ → datenschutz/index.html
✔ https://www.casoon.de/webentwicklung/ → webentwicklung/index.html
✔ https://www.casoon.de/e-commerce/ → e-commerce/index.html
✔ https://www.casoon.de/mittelstand/rostock/ → mittelstand/rostock/index.html
✔ https://www.casoon.de/wartung-und-support/ → wartung-und-support/index.html
✔ https://www.casoon.de/ferienwohnung/rostock/ → ferienwohnung/rostock/index.html
✔ https://www.casoon.de/plattform-apps/ → plattform-apps/index.html
✔ https://www.casoon.de/picks/ → picks/index.html
✔ https://www.casoon.de/ferienwohnung/mecklenburg-vorpommern/ → ferienwohnung/mecklenburg-vorpommern/index.html
✔ https://www.casoon.de/seo-marketing/ → seo-marketing/index.html
✔ https://www.casoon.de/ → output/www.casoon.de

  Pages: 33   Failed: 0   Warnings: 0   Duration: 11s
image strategies
3
crawl engines: HTTP and headless Chrome
2
prebuilt release targets
4
command for batch screenshots
1

What it does

  1. Self-contained static copies

    HTML, CSS, JavaScript and fonts are saved locally. References are rewritten to relative paths so the result can be opened without the original CMS.

  2. Choose how images travel

    Download originals, generate local gray placeholders, or replace images with external placehold.co URLs.

  3. Render JavaScript-heavy pages

    The optional headless mode launches local Chrome or Chromium, waits for frameworks, scrolls the page, and saves the rendered HTML.

  4. Screenshot one URL or a list

    The screenshot subcommand captures full-page PNGs with deterministic filenames and bounded browser concurrency.

Generated at build time

All examples →
site-scraper screenshot --help
Only take full-page screenshots of one URL or a list of URLs (requires Chrome)

Usage: site-scraper screenshot [OPTIONS] [URL]

Arguments:
  [URL]  URL to screenshot

Options:
      --file <FILE>                Text file with one URL per line (blank lines and # comments are ignored)
      --output <OUTPUT>            Output directory (existing screenshots are kept) [default: screenshots]
      --concurrency <CONCURRENCY>  Number of parallel browser pages [default: 2]
  -h, --help                       Print help

The command reference shown here is captured from the current CLI, and the same fixture is used by the showcase.

examples/screenshot-help.txt

Quickstart

Three steps from a live URL to a local copy. The documentation covers every crawl mode and option.

  1. Install the latest release or build the Rust binary from source.
  2. Run site-scraper with a URL; add flags to skip the interactive setup.
  3. Open the generated files under output/<domain>/.
Read the documentation
terminalShell
# Mirror a site with browser-like request headers
site-scraper https://www.example.com \
  --max-depth 2 \
  --placeholder real

# Open the generated copy
open output/www.example.com/index.html