Computer Help and Support

Orrex

(64,123 posts) Tue Nov 28, 2017, 09:10 AM Nov 2017

Longshot question about a non-DU discussion forum

I'm the admin a hobbyists' forum on the Hyperboards server, and about a week ago Hyperboards suddenly and summarily announced that they would be shutting down permanently on December 1st, with a total loss of all data.

I can reproduce much of the information because I maintain it in a database on my own system, but a great many posts contain information that isn't similarly backed up.

The format is pretty standard BB-code stuff, and I was wondering if there's some sort of app or utility for downloading or otherwise harvesting/archiving information posted in that format.

Does such a tool exist, or am I out of luck?

Any suggestions you can offer are much appreciated. This is an inconvenience for me, but other forums on Hyperboards stand to lose tens of thousands of memers' posts spanning fifteen years or so.

cross-posting to the Lounge.

9 replies

= new reply since forum marked as read

Highlight:

Longshot question about a non-DU discussion forum (Original Post) Orrex Nov 2017 OP

if you get no answers here or in the Lounge, you might try this forum steve2470 Nov 2017 #1

Thanks for the tip. I'll check it out. Orrex Nov 2017 #2

I know I have downloaded a website scraper mahigan Nov 2017 #3

Time is running out Egnever Nov 2017 #4

That's a good suggestion, but alas I don't appear to have FTP access Orrex Nov 2017 #5

ugh Egnever Nov 2017 #7

hmm another thought Egnever Nov 2017 #8

I'll check on that--thanks again! Orrex Nov 2017 #9

Egnever is correct mahigan Nov 2017 #6

steve2470

(37,468 posts)

1. if you get no answers here or in the Lounge, you might try this forum

Reply to Orrex (Original post)

Tue Nov 28, 2017, 10:00 AM

Nov 2017

https://www.democraticunderground.com/?com=forum&id=1098

Website, DB, & Software Developers

Orrex

(64,123 posts)

2. Thanks for the tip. I'll check it out.

Reply to steve2470 (Reply #1)

Tue Nov 28, 2017, 10:11 AM

Nov 2017

mahigan

(85 posts)

3. I know I have downloaded a website scraper

Reply to Orrex (Original post)

Tue Nov 28, 2017, 01:11 PM

Nov 2017

some time ago but I also know I have never used it. I'll try to find it later today. Meanwhile, you might be able to find something at one of these links. https://www.bigdatanews.datasciencecentral.com/profiles/blogs/top-30-free-web-scraping-software or here https://www.hongkiat.com/blog/web-scraping-tools/

Please let us know if you find something that might work.

Egnever

(21,506 posts)

4. Time is running out

Reply to Orrex (Original post)

Tue Nov 28, 2017, 02:01 PM

Nov 2017

Do you have access to the backend of the site or just the manegment console?

If you can get to the folder structure of the site I would jump on there using FTP if possible and download all of the folders immediately. That at least should give you the ability to cull through the data at your own pace instead of being under the gun.

I am not aware of a specific program to do what you are asking websites aren't really my thing discussion boards even less so. Still they are still setup in a standard folder structure and if you can access the server directly you should be able to copy all of the data.

Orrex

(64,123 posts)

5. That's a good suggestion, but alas I don't appear to have FTP access

Reply to Egnever (Reply #4)

Tue Nov 28, 2017, 02:10 PM

Nov 2017

The admins are, in a word, inaccessible, and they've made no comment beyond the 10-day shutdown warning.

The Wayback Machine happily preserves a lot of what I thought would be lost, so I have some wiggle room there.

Egnever

(21,506 posts)

7. ugh

Reply to Orrex (Reply #5)

Tue Nov 28, 2017, 02:19 PM

Nov 2017

that really sucks I am sorry you are facing this. Make sure you check all the tools theydo make available to see if there is away to access that back end. sometimes there are goofy http programs provided by the host as well.

Best of luck sorry I can't be more help.

Egnever

(21,506 posts)

8. hmm another thought

Reply to Orrex (Reply #5)

Tue Nov 28, 2017, 02:28 PM

Nov 2017

Do you have access to any web design software like Dreamweaver or something similar? You might be able to use one of them to log into the site to gain access to the folder structure..

Orrex

(64,123 posts)

9. I'll check on that--thanks again!

Reply to Egnever (Reply #8)

Tue Nov 28, 2017, 03:19 PM

Nov 2017

mahigan

(85 posts)

6. Egnever is correct

Reply to Egnever (Reply #4)

Tue Nov 28, 2017, 02:17 PM

Nov 2017

If you actually have access to the server, downloading the the folders with ftp is your best bet. Scraping the site is your last alternative. I did a little looking in my download directory and the scraper I downloaded but never used is called NeoDownloader. Another I came across is called Visual Web Ripper. Please bear in mind that I haven't used either of these applications myself - they may do what you need but but try Egnever's solution first. Good luck.

Reply to this discussion