I am trying to archive lefty reddit in the likely off chance that these subs get nuked. Any more subs to add to the list? Any advice on going about archiving it? What should I do with in bedded links?
I am trying to archive lefty reddit in the likely off chance that these subs get nuked. Any more subs to add to the list? Any advice on going about archiving it? What should I do with in bedded links?
The main important thing to me is to have comments. Does this store those too?
Also what tool are you using, just a crawler? I could only find bulk-downloader-for-reddit, yet it doesn’t store comments, just links.
Yea that is the exact program I will be using. I think it will be a pain in the ass but I think I will manually do comments, maybe try and get people together and crowd source them
Dang, ya it looks like that tool at least doesn’t support comment downloading. It’d probably be a good idea to try to work that functionality in there somehow.
You could use the Reddit API for that. There should also be an endpoint for the post itself, or maybe thats included if you fetch the subreddit post listing.
https://www.reddit.com/dev/api/#GET_comments_{article}
@Closesniper@lemmygrad.ml I’m sure that bulk-downloader-for-reddit is using PRAW, the python reddit API, so it does already a good API access there, it’d just be adding that functionality to it.
I can use the reddit API however I gotta do it post by post which will be a drag, will look into seeing if I could simplify it in some way
you could loop throw the backed up posts and requests comments for those. that way you can script it at least.
This would be trivial in Python.
Also you got the main ones for me. I really only care about the educational subs, /r/communism101, /r/communism, /r/socialism_101.