I’m trying to understand the bot problem in the internet and finding more ways to defend myself. One thing that I can’t seem to understand is why most bots, scrapers and crawlers seem to have residential IPs.

• Is it that ISPs are being paid by tech-bros to assign them these IPs?
• Is it that residential devices have been hacked /contain malware that does this?
• Is it trivial for companies to assign themselves residential IPs?
• Paid volunteers are doing this for AI companies?

Or is there is some other reason for this?

Obviously this is a problem because one can rotate / cycle through residential IPs and if I aggressively block each offender in my logs permanently, then the next person assigned this IP who may be a legitimate user will be unable to access my site.