Welcome to the Utopia Forums! Register a new account
The current time is Sat Nov 23 23:53:11 PST 2024

Utopia Talk / General Talk / UGT Search Function
earthpig
GTFO HOer
Thu Mar 11 19:47:45
so, playing around.

windows users will need to google search "wget for windows" and "grep for windows".

wget -r www.utopiaforums.com

this will begin the process of downloading everything found on the website www.utopiaforums.com to a folder called www.utopiaforums.com. you can play around more if you only want UP or only UGT or only sports.

may take a while. im doing that now, and playing around with what is already downloaded.

then, to search every thread ever for a term:

grep "a term" *

"a term" can be a date, or a poster's name, or a phrase.

if you only want to search for recent threads:

make a folder called 'recent'.

sort the files by name in your favorite file manager. this will sort them by forum, and then by thread number. counting starts at zero, so the larger numbers are the more recent ones. copy the most recent 100 or 200 into the folder 'recent'.

then, grep away in that folder.

example:

[chris: ~/up/www.utopiaforums.com]$ grep "sword dicks" *

boardthread?id=tank&thread=30380&showdeleted=true:it now links to the fabulous new "sword dicks" webcomic.
boardthread?id=tank&thread=30380&time=1267123150776:it now links to the fabulous new "sword dicks" webcomic.
boardthread?id=tank&thread=30817&showdeleted=true:I can totally see this sword dicks shit on a t-shirt some day. I can't tell if that is a good thing or not<BR>
boardthread?id=tank&thread=30817&showdeleted=true:this, but with sword dicks.
boardthread?id=tank&thread=30817&showdeleted=true:Stick figure sword dicks makes me laugh.
boardthread?id=tank&thread=30817&showdeleted=true:I mean, imagine the irony. making sword dicks on an maxIpad.
boardthread?id=tank&thread=30817&time=1268333203480:I can totally see this sword dicks shit on a t-shirt some day. I can't tell if that is a good thing or n<BR>
boardthread?id=tank&thread=30817&time=1268333203480:this, but with sword dicks.
boardthread?id=tank&thread=30817&time=1268333203480:Stick figure sword dicks makes me laugh.
boardthread?id=tank&thread=30817&time=1268333203480:I mean, imagine the irony. making sword dicks on an maxIpad.
boardthread?id=tank&thread=31130&showdeleted=true:(in b4 sword dicks and xkcd)
boardthread?id=tank&thread=31130&time=1268276947393:(in b4 sword dicks and xkcd)

[chris: ~/up/www.utopiaforums.com]$



then you can copy/paste the "boardthread?..." stuff into yer web browser to see that thread.
earthpig
GTFO HOer
Thu Mar 11 19:52:21
grep -li "term" *

will merely point you to threads that mention the term, and be non-case-sensitive.

example:

[chris: ~/up/www.utopiaforums.com]$ grep -lic "sword dicks" *
boardforum?id=tank
boardforum?id=tank&showdeleted=true
boardforum?id=tank&time=1268356658688
boardthread?id=tank&thread=30182&showdeleted=true
boardthread?id=tank&thread=30182&time=1267635237514
boardthread?id=tank&thread=30380&showdeleted=true
boardthread?id=tank&thread=30380&time=1267123150776
boardthread?id=tank&thread=30817&showdeleted=true
boardthread?id=tank&thread=30817&time=1268333203480
boardthread?id=tank&thread=30920&showdeleted=true
boardthread?id=tank&thread=30920&time=1268027868948
boardthread?id=tank&thread=31130&showdeleted=true
boardthread?id=tank&thread=31130&time=1268276947393
[chris: ~/up/www.utopiaforums.com]$
earthpig
GTFO HOer
Thu Mar 11 20:43:18
ok. it would be ridiculously expensive for TC to pay for all of us wgetting the entire website all the time.

ill do it once a week, zip it, and rapidshare it. a much lighter burden on TC. 2 things ill ask:

-let me know if you download it, so i know someone is at least benefiting from my efforts... and so when it gets up to 10 downloads, someone else can see that and re-upload it to rapidshare.
-click an ad when you do.

http://rapidshare.com/files/362199846/11march2010.zip.html

5.4 mb

then you can search through it using whatever you wish.
earthpig
GTFO HOer
Thu Mar 11 20:48:59
done talking to myself. delete this thread plox.
show deleted posts

Your Name:
Your Password:
Your Message:
Bookmark and Share