There are some browser plugins that try to guess what technologies are used by t... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		pzmarzly on Oct 24, 2024 \| parent \| context \| favorite \| on: Ask HN: Website with 6^16 subpages and 80k+ daily ... There are some browser plugins that try to guess what technologies are used by the website you are visiting. I hope the better ones can guess it by just looking at HTML and HTTP headers, but wouldn't be surprised if others were querying some known endpoints.

etiennebausson on Oct 24, 2024 [–]

Then they are, by definition, bots scriping the site for informations, and should start by the robots.txt

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact