Idea: Scrape all the posts from a subreddit as they’re being made, and “archive” them on a lemmy instance, making it very clear it’s being rehosted, and linking back to the original. It would probably have to be a “closed” lemmy instance specifically for this purpose. The tool would run for multiple subreddits, allowing Lemmy users to still be updated about and discuss any potential content that gets left behind.
Thoughts? It’s probably iffy copyright-wise, but I think I can square my conscience with it.
This instance, like most instances, would prefer most content come organically, rather than just being a mirror of whatever reddit has.
I fully understand and respect that, and would never run it on an instance that wasn’t specifically set up for this purpose.
The intention is for it to be something people can OPT IN to, not OPT OUT of.
Anyway some of the content will be repeated because of the waves of information in the internet everyday, so… would be better to gain something of that accumulate of info that reddit has.
I’m not against it as long as I can avoid reddit taking over my feed. I’m happy to get away from reddit, not just because of their business practices, but the culture has changed to something I rather avoid. I’m liking this cozy federated feel so far and would prefer to keep reddit’s culture from taking over my feed again.
Well there is a tool that uses the API for this purpose https://github.com/rileynull/RedditLemmyImporter Maybe one of the little niches carved out for free API use, like for bots or something might work for that, but definitely should work now.
Another interesting idea is for getting historical data, pulling it from the archive.org project from the project r/datahoarders are doing now.
If you are talking about your posts there is no problem, but if you are talking about all the posts and replies of a subreddit, I think it is a bad idea as you are replicating comments and opinions of other people who couldn’t delete them in a future if they wanted.
