🔗 linkblog: my thoughts on 'The rise and fall of robots.txt'

14 February 2024 - kudos:

Fascinating read on web crawlers and robots.txt link to “The rise and fall of robots.txt”

🔗 linkblog: my thoughts on 'Major critic of X sues after being banned from platform | Ars Technica'

1 November 2023 - kudos:

The headline obscures something important—that this is about research, access to data, and Terms of Service. Worrying stuff. link to “Major critic of X sues after being banned from platform | Ars Technica”

📝 writeblog: spent 0:47:30 on 'publish scraping library online presence study'

15 February 2023 - kudos:

Spent some time manually reviewing websites today to prepare for later web scraping. I’ll have to figure out how to work with some Wix sites, which are structured oddly behind the scenes.

📝 writeblog: spent 0:16:51 on 'publish digital religion as international religion study'

9 February 2023 - kudos:

Spent a few minutes idly working on a web scraper for this.

📝 writeblog: spent 0:55:21 on 'publish digital religion as international religion study'

6 February 2023 - kudos:

Instead of grading (😬), I spent some time grabbing links and then starting to build a web scraper, though that’s enough of a pain that I might ask a friend to borrow his CrowdTangle access.

7 May 2021 - kudos:

What is the most soothing form of digital data collection, and why is it forum scraping?