Locking threads

[Alex Libman]

I did it!  I got Gene to lock up his Christian Anarchy thread!  What finally did it was mentioning what a hazard that thread was to scraper scripts that loop through recently active threads.  (Though do think that the Jesushomoerotisation and the Satanist threads might have softened him up... a bit.   )

Then I noticed something interesting - posts on locked threads have a "Delete" button, but no "Modify" button, which means there's no significant reason for a scraper script to download a thread once it's locked!  (Or, even if it follows thread ID enumeration instead of parsing through category indexes, there's no reason to download more than one page - this forum has ~8,000 second or latter pages.)

So - locking big old threads that you don't want necro'd is good for this forum.  Most importantly, it would remove ambiguity about when an attempt to enrich an old thread by appending to it would be unwelcome.


I'm not positive, but I think admins could simply lock all "orphaned" threads from users that have "cycled" or deleted their accounts by running:

Code: [Select] UPDATE smf_topics SET locked=1 WHERE id_member_started=0;

And to lock all threads that haven't been active for a certain number of days (until the thread starter explicitly unlocks them) would be something like:

Code: [Select] UPDATE smf_topics t, smf_messages m SET t.locked=1 WHERE t.id_last_msg = m.id_msg AND DATE(FROM_UNIXTIME(m.posterTime)) < '2011-01-01';

[John Shaw]

I'm not positive, but I think admins could simply lock all "orphaned" threads from users that have "cycled" or deleted their accounts by running:

Seems like a waste of time considering that you're the only person with a history of doing that shit to annoying levels.

[Alex Libman]

That's subjective.

[Amazing Richard]

I don't get it. What's so bad about Gene's thread that he decided to lock it??? Are you saying that spambots are attracted to large threads....so long threads= lots of spam, and Gene does not want to bring spam to the BBS, therefore he locked the thread???

[The ghost of a ghost of a ghost]

I don't get it. What's so bad about Gene's thread that he decided to lock it??? Are you saying that spambots are attracted to large threads....so long threads= lots of spam, and Gene does not want to bring spam to the BBS, therefore he locked the thread???

That's it.....Exactly!

[Alex Libman]

I was just kidding about Gene - the persistence and size of that thread has been joked about for quite some time now.  I'm sure Gene had better reasons to lock it than just spam-bots.

This thread is mostly about preserving information externally, which isn't as simple as just getting all the new posts, because people can edit their posts afterwards and fuck up history entirely.  There's no easy way to find modified posts, only newly posted ones.  So it's reasonable for a scraper script to get a list of updated threads, and then loop through all the pages.  That works perfectly - except for super-long threads, there you have to decide how many of the last pages you want to check for edits.  If the script re-scrapes the whole Christian Anarchy thread, then that's 31 megs of bandwidth (without HTTP compression) per bot per iteration.  An archivist bot might also want to revisit all inactive unlocked threads for gaps and necro-edits after a while...  And good thing this forum has a reasonable robots.txt crawl-delay setting, nhliberty's is 3 minutes between hits!

No big deal really, but locking old threads does help a bit.

[Zhwazi]

When I read the subject my first thought was multiprocessor synchronization. XP

[Alex Libman]

Actually, yes...   See, the Internet is just one ginormous multi-core multi-threaded multi-everything wisdom computation machine, and this forum is its leading CPU.  Someday I'm going to post something so Libmantastic that, without any fuss, the stars will go out...   

[BonerJoe]

You want some locked threads? I'll give you some locked threads.

[anarchir]

I think Gene locked the thread because he was tired of getting stuck in a corner logically with his debates. So, he waited for the first opportunity to lock it that wouldnt make it seem like he was a sore loser, Libman provided the opportunity, and he went for it. When the heat dies down he'll open it back up again.

[Alex Libman]

Bingo.  And...

[Alex Libman]

Other potential solutions to optimize crawling (and possibly some other things):

• Disable post editing (and possibly deleting) after some time period.

• Add a query option to show all thread messages on one page (I don't mean the print mode, which is very lossy of formatting and other stuff).

• Fix the thread HTTP response headers to report the timestamp last message was posted or modified.  (Currently the "Date" and "Last-Modified" headers report the current time, and the "Expires" is fixed to "Mon, 26 Jul 1997 05:00:00 GMT".  If those are needed that way for caching reasons than the actual date could be reported in a new custom header pair.)

• Let me set up something like "bbs-archive.freetalklive.com" containing a static HTML archive of each thread.  (I'll maintain it and pay for CDN hosting, you just provide permission and sub-domain CNAME pointer.)  You can then losslessly delete old threads from the live BBS and set up redirecting to the static archive.  This would reduce your hosting costs and accelerate everything quite a bit.

[Riddler]

bababababababababababababababababababababababa