site-scraperv1.4.6

Changelog

Generated from CHANGELOG.md in the project repository. Never maintained twice.

1.4.6

15 September 2026

  • changedSecurity: update rustls to 0.23.45 (RUSTSEC-2026-0285)
  • changedSitemap: also read Sitemap: entries from robots.txt and /wp-sitemap.xml, unpack gzipped sitemaps and decode XML entities in <loc>
  • changedHeadless: treat an HTTP error with an empty body as its status, so stale sitemap entries only warn
  • changedUpdate scraper to 0.27, rand to 0.9 and anyhow to 1.0.104 (clears the rand/fxhash cargo-audit warnings)
  • changedUpdate runemark to 0.3.3: status lines are no longer lost when TERM is unset or dumb
  • changedPages: show real crawl output of casoon.de on the project site

1.4.5

12 September 2026

  • changedScreenshot: capture pages with an HTTP error status (e.g. custom 404) instead of failing
  • changedHeadless: raise the default browser navigation timeout to 60s and make it configurable with --timeout
  • changedHeadless: default to a 1440px desktop viewport; add --device desktop|tablet|mobile
  • changedUpdate README: document headless mode, extended interactive setup, new options

1.4.4

2 May 2026

  • changedHeadless: wait 800ms after load before scrolling

1.4.3

2 May 2026

  • changedHeadless: strip <script> tags before saving rendered HTML

1.4.2

2 May 2026

  • changedRevert canvas freeze: toDataURL fails on WebGL/tainted canvases

1.4.1

2 May 2026

  • changedHeadless: freeze canvas elements as static images before capture

1.4.0

2 May 2026

  • changedHeadless: capture HTML while scrolled to preserve scroll-driven styles

1.3.9

2 May 2026

  • changedHeadless: only reveal opacity-0 elements that also have a translate class

1.3.8

2 May 2026

  • changedHeadless: remove opacity-0 and paired translate classes before capture

1.3.7

2 May 2026

  • changedHeadless: scroll page after load to trigger IntersectionObserver animations

1.3.6

2 May 2026

  • changedinstall.sh: fail loudly on download error instead of extracting HTML
  • changedBuild release binaries with --features headless so --headless works out of the box

1.3.5

2 May 2026

  • changedAdd extended interactive setup mode

1.3.4

2 May 2026

  • changedAdd --headless mode with optional chromiumoxide feature

1.3.3

29 April 2026

  • changedResolve redirect before crawling so www → non-www sites are fully crawled
  • changedUpdate README: document interactive mode and prompt behaviour

1.3.2

29 April 2026

  • changedSwitch interactive prompts to English

1.3.1

29 April 2026

  • changedInternal fixes, no user-facing change

1.3.0

29 April 2026

  • changedAdd interactive mode with dialoguer prompts (depth, images, bot) when only a URL is given

1.2.5

27 April 2026

  • changedFix --placeholder real being silently ignored (was mapped to external in cli.rs)

1.2.4

27 April 2026

  • changedHandle lazy-loaded images (data-src/data-lazy-src) and skip data URI placeholders

1.2.3

27 April 2026

  • changedMap query-string URLs to unique filenames (fixes WordPress ?page_id= sites)
  • changedOpt into Node.js 24 for GitHub Actions to silence deprecation warnings

1.2.2

27 April 2026

  • changedChange default install dir to ~/.local/bin to avoid sudo prompt

1.2.1

27 April 2026

  • changedAdd --placeholder real mode to download original images
  • changedDocument --placeholder real in README
  • changedFix formatting and clippy warnings

1.2.0

7 April 2026

  • changedRewrite from TypeScript to Rust
  • changedRefactor codebase into modular structure
  • changedAdd request headers/delay config and graceful asset error handling
  • changedAdd Puppeteer support for Cloudflare-protected sites
  • changedFix CSS url() path calculation — use CSS file path instead of HTML referer
  • changedAdd release workflow for automated binary builds
  • changedSwitch to rustls-tls and use cross for aarch64-linux builds
  • changedAdd Biome for linting/formatting and GitHub Actions CI
  • changedTranslate README to English