R

robotstxtr

Project information

Engine-aware robots.txt policy for R with Google's matcher algorithm.

R rstats r-package robots-txt robots-exclu... rfc9309 crawler web-crawling googlebot seo technical-seo

182 Commits
1 Branch
3 Tags

README
MIT License
CHANGELOG
CONTRIBUTING
GitLab Pages

Created on

August 01, 2026

Loading