Utopia Talk / Politics / Software request

Nimatzo iChihuaha	Mon Jun 27 10:38:50 Is there an app or something, that can index or somehow store the information off of UP threads so that it is searchable? If you manually change the thread id, you see there are many pages that do not show up on google ever. We have a person here *cough* NHILL *cough* who has been bragging about his 1337 coding skills. Hmm? Is there such a thing? Could make me one :)
Habebe Member	Mon Jun 27 10:48:56 Have you looked into Maltego? IIRC it compiles OSINT into an easier to search format.
murder Member	Mon Jun 27 10:56:13 Just ask the NSA for a copy. :o)
nhill Member	Mon Jun 27 11:46:31 I mean, sure, it's not a problem, just depends on the scope. Pretty easy to catalog everything, although the post id design is kinda weird (they apply to both UP and UGT), but literally about 15 minutes of coding to scrape all the data. From there, it's a matter of presentation. Easiest route we could just store in a database. I'd probably just make a quick python script for it, like "python3 utopiapolitics.py user=nimatzo text="Software request"" and it'd spit back the results. Should have something ready in a bit to try.
earthpig GTFO HOer	Mon Jun 27 12:04:50 Is there a way to tell google to index everything? Probably not, but throwing it out there to prevent duplicate efforts.
nhill Member	Mon Jun 27 12:14:57 Google crawls the site, so no there isn't a way without a code change. Would need links to previous posts.
Nimatzo iChihuaha	Mon Jun 27 12:14:59 There is a way http://dev.../ask-google-to-recrawl#sitemap but it seems like a lot of work so that maybe google will include it. Which they write, may take days, weeks or never. Brotha Nhill will delivar :)
murder Member	Mon Jun 27 12:15:55 We could start every thread with "Hey Google" just to make sure its paying attention.
Nimatzo iChihuaha	Mon Jun 27 12:15:58 Or maybe I don't understand what that recrawl request does.
nhill Member	Mon Jun 27 13:01:51 That would work, Nim, but would require Turtle Crawler's help to verify ownership of the domain. Or he could make a sitemap to help Google crawl it.
nhill Member	Mon Jun 27 13:27:44 Got all the posts parsing, time to put them into a sqlite db so we can support querying. Have 2 hour meeting w/ a client, may run long, and I'll get back to it. Fun way to sharpen my Python skills. I hate this language but it's convenient for small scripts like this.
Nimatzo iChihuaha	Mon Jun 27 13:29:05 Thank you :)
earthpig GTFO HOer	Mon Jun 27 14:21:53 My website guy was all about making sure google had the full sitemap, and he's the guy behind the top google results for things like "dallas texas breast enhancement surgery" and "Cincinnati car injury lawyer," so he knows his shit. TC, or someone else, would have to write some shit that dynamically updates the sitemap every time a new thread is created (and you'd have to create a historical archive of all existent threads), and updates google periodically. It's a full list of all URLs, from what I saw, so that google's AI/computers have less thinking/crawling to do.
earthpig GTFO HOer	Mon Jun 27 14:22:05 ^ @ nima
earthpig GTFO HOer	Mon Jun 27 14:24:26 "Explore this vaguely defined region and anywhere it happens to take you, reach your own conclusions about what you find" v "here is an exactly detailed map of all the things I'd like you to visit and what you should find/document at each thing, all you are doing is confirming my map, you aren't making your own map of a new and unexplored region."
nhill Member	Mon Jun 27 16:15:33 Ok doing a full scrape. Fingers crossed that I don't take down the site ;)
nhill Member	Mon Jun 27 16:16:36 1000 done, 89145 to go.
nhill Member	Mon Jun 27 16:18:11 This'll be a full backup of UP in a sqlite database, schema: CREATE TABLE IF NOT EXISTS user ( id INTEGER PRIMARY KEY AUTOINCREMENT, created TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, username TEXT NOT NULL UNIQUE ); CREATE TABLE IF NOT EXISTS post ( id INTEGER PRIMARY KEY AUTOINCREMENT, post_id INTEGER NOT NULL, created TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, link TEXT NOT NULL, title TEXT, content TEXT NOT NULL, username TEXT NOT NULL, FOREIGN KEY (username) REFERENCES user (username) ); lemme know if any data model suggestions just doing it quick and dirty
earthpig GTFO HOer	Mon Jun 27 16:20:23 TC: "what the fuck is THAT? A one computer DOS attempt?" <nhill permabanned>
nhill Member	Mon Jun 27 16:22:09 I did take down the server
nhill Member	Mon Jun 27 16:22:25 But it's back up now and wrote some retry/dedupe logic
nhill Member	Mon Jun 27 16:24:15 Was considering parallelizing the requests to all 48 threads on my box, but...pretty sure now that'd melt things.
nhill Member	Mon Jun 27 16:31:13 Slowed down the script but still took down the website again. C'mon
nhill Member	Mon Jun 27 16:36:54 fun list of usernames, most likely in order of posting. there's a bug where the text search gets confused by numbers at the end of usernames it looks like, but for now you can ride down memory lane ;) turtle crawler master bates sam adams cardinal jac murder pillz nekran madc0w mckobb best therad ever earthpig zombie jesus cthulhu neverwoods rugian joe black agri jnh river of blood earthpig camaban garyd juztav roland canadian cloud strife voodooprophet carbonated water hot rod real fred tj bigboy thunderchief crownroyal bushisdabestpresid mexicantornado ample parking nhill sstrickland phunkyphishstyle assclown vega forwyn and justice for all hrothgar crackhead cherub cow miracle jergul freddy patom master p lil davey and justice 4 all fuzzy duck angel xface visibly shaken standard of reason dakyron kreel talon121 nimatzo seb john paul jones bushdabestpresid bushdabestpresld bushdabestprezid bushdabestprezld bushdabetprezid bushdabetprezld bushisdabestprezid milton bradley paramount doctor x jb you dont know me jo bigger mrpresident07 swordtail murder inc balrog borthas corkie williamthebastard27881 hip91803 سيد محمد عنده اصغر القض daemon harof asgard hi dweebs evenflow poison chen ratta tatta rawrpig muslim pissflaps mcgee70850 resolute steve sdf muwahhid monotheist laa ilaaha ila huwa dabba mastur bates god mickster! bidbp chuggs cleaner josh123 guybrush threepwood josh1234 cheese and whine مسلم 123abcz2 44 guns ...and justice 4 all the children xx2 licker eikey1729 barneydude palem hellfire bubba moonsaw toth snuke frebby saiko pierre werewolf dictator the chink yankeessuck123 poisen aspire magicz dreamz faithless tiakatt db bushisdabetspresid chuck eds headrush lqcl aeros omac qwot swirve ironleg animal bushisdabetspresld redblooded bushlsdabetspresld thinking is evil worcester mr.mickster kendal sstandard of reason still well bushlsdabetspresid bushsdabetspresid bushsdabetspresld bushsdabetspres1d bushlsdabetspres1d bush1sdabetspresid bush1sdabetspresld bush1sdabetspres1d bush1sdabestspres1d bush1sdabestspresid bushisdabestspres1d bushlsdabestspres1d bush1sdabestspresld earthpig65398 ehcks hackety anne not funny angel's pet lost ubes jac valishin backie mr america duckpig rock solid mehul patel kill a commie rollo hughes maria henretta dwight fry cow vagina afrogod3 the archivist asdasdfasdfasdfasdfa j.b. (. )( .) hippo wtf wtf73785 real real fred pedo file lets bump heads chinese man www.yeswecansong.com earthpig19767 iii dark waters mavl habebe barda spiridonovich putin enchantress selene farkeld rugian86136 ron paul ( .)(. ) cloud strife58347 s.s. failboat dark morbid stone hip53321 boogie woogie y2a returns y2a ugly donkey tiggle toggle <( '^' )> :) :( ( o ) ( o ) fuck calhoun brisbane omac legal counsel truther mutliproblemsolved ostie tabarnac tumbleweed alcoholic water gottalar jadick ralph dunny cossie cthulu81023 superdude rugian79433 8===d williamthebastard15060 hood peter walsh ( .)( .) kent hovind bloodymary real fred fbi agent dick swinger shannon steel pigiron im better then you muppet say i svog bahahahahaahah samantha justin the dude brunstgnagg nimatzo88174 s.s. failbot baa baa baa sheep rugian98816 river of blood68784 racistpedorod sweden is life jaja huge schlong alexis twaddle twoddle thechildren adolf hitler afro tau su the_children the children kered kargen rugian40481 mickster hexget drillmore wtf wtf63786 peirre kilo merry go round merry go round river of blood91287 the chlldren firestorm phoenix beer in a brown bag antidote miltons bradley poppy an amerikkkan an amerikkan peep goreth kaylana catapult crystal meth cap snuke53785 fag cleaning woman p0ison jolt varakalt tedurur snuke26616 gamma ht? phunkyphishstyle30175 the christus bigboy lives talonswordsofwisdom suckonitowned mpatk charlie paramount54220 skater chick cthulu93760 wayneg26 seed of stalin earthpig60721 carbpnated water omega3 wrath of orion57841 cthulu90119 wrath of orion47637 daemon79310 wrath of orion87495 phunkyphishstyle18985 williamthebastard28736 rugian78067 cloud strife97112 crown royal35288 rugian91563 rugian95536 daemon26070 cloud strife76013 earthpig54516 wrath of orion caps whore snuke61976 phunkyphishstyle40735 alexander hamilton omega beta filthy jew john mcwaterboard cloud strife84914 alexander suvorov sven svenson cthulu35771 hip37273 eikeys ghost richard cranium real milton bradley s.s. steamboat willy chris the penguin ss-nordland s.s failboat ss operation werwolf ignorantasshole i aprove this post i don't fuckshitcuntnigger stone hephaestus smelter milton licker gary cooper sigfreed mark hamill freddie prinze, jr. julia stiles mad doggie harrison ford president medvedev prime minister putin george w bush leon trotsky viktor yushchenko thomas jefferson winnie the pooh abraham john hanson yankee doodle mozes tigger piglet sd anonymous morgan freeman kanye west tom cruise christina ricci christian bale paris hilton william faulkner william shatner cervantes federico fellini gary oldman eminem signorney weaver kate hudson spike lee denzel washington al pacino robert deniro jay z virginia woolf james joyce miles davis john coltrane rainier maria rilke rene descartes claudius augustus elton john eddie murphy paul mccartney ronald paul john travolta george bush john mccain jesse malcolm barack poison ninja viral hemorrhagic fever achmed the terrorist hoer charper multiaccult freak nation isaksson jeddediah wilklns leonardo asshole uper welcome to the welcome to the poop on you elsie ciaran teejay the new guy rlver of blood zavyx sastan pooting williamthebastard94401 maddox borderline fucker stella stalks niggerlips putz just gravy! cloud strife14344 amsec williamthebastard48069 damian db blackfoot liberty ss nordland daemon15851 john stark billah paramount91038 nimatzo83202 daivler mater bate erectiledysfunction hip99054 earthpig47997 night stryker thunderchicken beer baron ronald reagan uncle tom musiim swordtail11002 cthulu96494 hitler was right hh psycho guy fred felcher rugian45570 bad radar ugly radar poizon chuckle paramount50043 earthpig25134 deepthroat wellduhpeople pissflaps mcgee80254 "real fred wrath of orion90246 doctor fsdf dukhat fsb nimatzo98367 spam geto boy wrath of orion73774 www.nowecantsong.com malv fuckwad fuckyoueikey eikeyyoucunt nsfw tmi mastur bastes muhammad mr steve asgard t j earthpig55049 phunkyphishstyle15734 karl rove alex jones sυperdude fockalot wrath of orion36210 allah cloud strife35691 satan jesus pyro squirrel master splinter ronal reagan barack hussein obama poison'svagina go poison muzzle > clone poison . alias williamthebastard nlght stryker ruglan who's my daddy? raphael mexicantornado llcker turltle crawler p0is0n rugiam hot rod corkle rea1 fred anple parklng valishln the powers that be rugian turtle crawler real fred hot rod hot rod real fred real fred real fred eikeys ghost turtle craw1er worscester ample parklng master .p elkeys ghost poisonous substance blackfoot/blllah river of blood polson .. . .. . .. . . . .. . .. .. . .. . . blackfoot/billah blackfootbillah blackfoot-billah ..-. ..- :-) hip81919 .poison michealangelo rugian31769 williamthebastard74385 let the boy try! lingwe ampel parking williamthebastard99084 fυ swordtail31205 pimply teen stfu the legend bitch on my dick the legend of gtfo fuck jac nimatzo52998 honor well! shaken not stirred jesus christ jesus h christ almighty god otto adolf eichmann amanda george skywalker paramoυnt hot rοd wrath of orion81409 kayiana crown royal50312 pοison snuke82193 madc0w  pillz  superdυde real fred  turtle crawler dick head turtle crawler  let the boy try ample.parking crazy poisonisapedo stupidasspoison to and fro poisοn river of blood  sstrickland  nimatzo84855 billah  turtel crawler plato alejandro milton cloud strife17365 the real billah regular poster calhoune asshat rapper man misanthropy reinhard heydrich ruqian george jimmy himmler phunkyphishstyle38453 4 yearold child regular imposter mcculkin earthpig15319 statistician nimatzo36475 cthulu17975 turtle ?rawler turtle crαwler turtle craw?er turtle crawler turd crawler turtle crawler turtle cr&alphawler turtle cr&alphwler turtle cr?wler turtle crawle? turte crawler urtle crawler turtle crawer turtle crxbwler tauurtle crawler turtle rawler crawler turlle crawler turiotale crawler turtle pokingout hotrod pomicronison yankeessuck 1234567890 cute camel turtle crwler hip99809 ya ab ehcks da lose at ass clown snuke28830 semiregular poster pedo rod ss failboat h0t r0d snot nod henry from boston masterbates supsilonperdude rael fred feel fred votemcshame pedomod wrath of orion55976 cowman mr. multi ms. multi joe black in cognito yankeessuckl23 gary johan goering rugian42119 snot rod gayd hip11687 donatello free porn hot rod. poison. asgard. turtle crawler. rugian . .asgard hot.rod otto.adolf.eichmann edgat allen poe edgar allen poe edgar.allen.poe edgar allen poo twitty edgar.allen.poo. .real fred looking4luv prophet bush williamthebastard37193 allahu ackbar hip58254 reel fred ample pakring riverofamericanblood fred molestschildren ampel.pakring hr molests children jfk rugian57147 rip up .rugian .hot rod asscown vega yankeessuck123. mr kkkanada daemon25558 rainer maria rilke thomas aquinas pissflaps mcgee95600 masterp earthpig44224 mr. fuck mrs. fuck johnny fuck the end hither and yon oddity socrates agent smith anonymous poster s.s. failboat. 15821606158616101585 15821606158616201585 brian dr. spivy cindy narrator steve brian. rugian74607 y2a asgard. mrbbonz diplomatic attache iwonder pussy willow brian . assad sarwar tanvir hussain abdulla ali osama bin laden incognito vaginafart fat bloke wtf wtf79359 mehul mehul matel rugian83611 pedo.mod .poison. rugian84429 williamthebastard57880 james t kirk bloodyvaginafart palinduzcrackncockz baconwrappedham chuckle. nightbloom. tom brady rosa parks daemon97384 dr. fsdf richard girnt butler hip11613 jean luc picard get a clue greatful dead someguy ggamerica george.w.bush uncle abe crown royal92842 john paul gaultier cloud strife74030 wtf wtf84909 sham ademsh pissflaps mcgee57473 calhoun. iampoison cloud strife98342 rugian48085 pissflaps mcgee93122 halfturd pissflaps mcgee89537 phunkyphishstyle30584 fsb jac earthpig11161 9iulian1 jesusisdabestgod earthpig16677 swordtail74021 votemccain ahahaaha palinisdabestpig cthulu87043 fuck pillz
nhill Member	Mon Jun 27 16:37:59 Or could be that people just kept making multis with random numbers. Haven't checked yet :p
nhill Member	Mon Jun 27 16:39:53 http://utopiaforums.com/boardthread?id=politics&thread=390 Did a spot check. Check the last post in this thread, Mr. Nimatzo88174. Guess it's multis. :)
nhill Member	Mon Jun 27 16:43:32 Later I'll save a full scrape just in case the site goes down completely. The web scraping logic is going to have bugs, as it's impossible to predict the structure of all 500,000 or so posts, so it's good to have the source data as backup.
nhill Member	Mon Jun 27 16:52:19 There's more posts than I thought. Almost 1 million based on my projections (which will change as I scrape further). I'm 50,000 posts into it with around 5,000 threads
nhill Member	Mon Jun 27 17:04:50 http://i.imgur.com/00qphhH.mp4 ^video showing the scrape. lot of spam on these forums. ignore the code, it's disgustingly inefficient, quick, and dirty. pay me $1000/hr and i'll clean it up and make it object-oriented & maintainable :p
Nimatzo iChihuaha	Mon Jun 27 17:06:43 Woho. No idea how you would structure the data besides username, as long as all as you can search for words it should be good enough. We had bouts of spam attacks many years ago, many times by bots, they are all those username134128 multis. It is such a blast from the past :)
nhill Member	Mon Jun 27 17:06:49 9000/90200 █░░░░░░░░░
Nimatzo iChihuaha	Mon Jun 27 17:07:40 So cool that you are doing this!
Nimatzo iChihuaha	Mon Jun 27 17:09:27 1 million posts, means the average UP thread gets almost 12 posts.
nhill Member	Mon Jun 27 17:10:31 Text search will work. You'll have to do it on the command line though, and install python plus a few packages. Later I can make it a website with advanced search operators (like searching by user, searching by title, searching by body, or just search everything, etc) Another thing is I could write some logic for the time of posts, but that'd require reverse engineering the year based on the day, and may not be foolproof.
Nimatzo iChihuaha	Mon Jun 27 17:16:24 http://utopiaforums.com/boardthread?id=politics&thread=1 The first post ever made is 2008-08-17
nhill Member	Mon Jun 27 17:42:15 That was the first one to scrape :D I could easily add support for UGT, but want to go through a test run here. Did you calculate that the number above applies to both UGT and UP? Unsure how many threads are specific to UGT, looks like ~10%. I'll know once it finishes. But the average thread getting ~12 posts is correct, once you account for the missing 10% of UGT threads.
nhill Member	Mon Jun 27 17:44:56 1301004 posts at thread 15067 ███░░░░░░░░░░░░░░░░ 16%
nhill Member	Mon Jun 27 17:47:11 Sorry, 131044, typo there. Total is 1246451 apparently. Says so on the homepage.
nhill Member	Mon Jun 27 17:59:29 168491 posts at thread 18195 ████░░░░░░░░░░░░░░░░ 20%
nhill Member	Mon Jun 27 18:09:53 rugian74607 y2a asgard. mrbbonz diplomatic attache iwonder pussy willow brian . assad sarwar tanvir hussain abdulla ali osama bin laden incognito vaginafart fat bloke wtf wtf79359 mehul mehul matel rugian83611 pedo.mod .poison. rugian84429 williamthebastard57880 james t kirk bloodyvaginafart palinduzcrackncockz baconwrappedham chuckle. nightbloom. tom brady rosa parks daemon97384 dr. fsdf richard girnt butler hip11613 jean luc picard get a clue greatful dead someguy ggamerica george.w.bush uncle abe crown royal92842 john paul gaultier cloud strife74030 wtf wtf84909 sham ademsh pissflaps mcgee57473 calhoun. iampoison cloud strife98342 rugian48085 pissflaps mcgee93122 halfturd pissflaps mcgee89537 phunkyphishstyle30584 fsb jac earthpig11161 9iulian1 jesusisdabestgod earthpig16677 swordtail74021 votemccain ahahaaha palinisdabestpig cthulu87043 fuck pillz dear governor palin kill yourself master jewhater rockin rhubarb hip45503 screwupiquit wingman mickey mouse fuck a bitch retribution elf boy kazarus cloud strife59910 cthulu53552 polonium nigel tufnel david st hubbins derek smalls cloud strife18532 agri jr from the other thread: phunkyphishstyle40998 cthulu13001 river of shit river of crap karl rov daemon33936 smite me wtf wtf72500 total eclipse masu crown royal23667 williamthebastard39207 pissflaps mcgee24832 bettar then grammer uncle sam poster ro4444 votemcain hahahaaha chris dodd river of blood43429 hot 39buttplug39 rod hot buttplug rod gop rules snuke44371 god like warrior earthpig83497 williamthebastard19261 wrath of orion76422 daemon34704 hot stick senator barack obama hip51616 senator john mccain bristol palin osamaisdaworstpresid wake up dr seuss crown royal82384 sky..net vote mccain november wtf wtf16056 paramount95319 wrath of orion69676 half billah pissflaps mcgee36676 canaban real liar sam adam ruqlan hip71220 shaam fadems flan cladems alcoholic waiter jackrabbit purvis earthpig97629 cthulu11238 turd fergeson levi martin lara dunn funnybone xarzfgfw hot r0d questioner smem wtf wtf78492 benyehuda lil benny paramount27604 barmaglottr farma dupaston farma nirdosh pharme159 pharmb718 pharmd937 pharmc851 farma dana phentermine cialis feannelerom viagra tramadol xanax visably shaken cloud strife64380 nimatzo42063 rugian93333 royce chandler snuke51375 alarmed jason bourne mattdamon johnmccainthewarhero mccainisdabestpresid paranount sapien cloud strife27639 sam adans michael litoris osuma bin laden ringo starr neil peart geddy lee decksimian hussein08 jayelef pissflaps mcgee97758 pissflaps mcgee48152 island in the sun phunkyphishstyle37891 atlgpjqhuxt0nkhibag1 steve jobs swordtail92517 phunkyphishstyle48545 agent orange earthpig63099 hip92035 the antigay wtf wtf98543 turtle cralwler snoopd.o.dubbadizzy love pump wrath of orion73057 islandinthesun amos gov. sarah palin san adams im bett jonathan swift capnplank fas man roswald us senator rugian82024 pissflaps mcgee28065 pygmy southof76 sarahs bloody cunt biden is gay andy hip14822 swordtail75893 j.mccain swordtail60844 para mount sam addams hip93371 stanley josh angry citizen disgruntled turdle crawler m0nkey wrench pissflaps mcgee b. hussein o. jack black real fred dr a. einstein pissflaps mcgee85554 cow cunt wrath of orion35484 icon sarah pallin fuck this shit crab infested ubes fucking liberals phunkyphishstyle79572 hacker4hire ztgotnd4 ztgot ztgotj ztgotjf zdtgotjf zdtgot6 zuitgot6 zuitgotj6 zuitgh6 david letterman phunkyphishstyle53771 hotter rod phunkyphishstyle52879 the truthgiver rod jimmy niggers r our friend politician madcow swordtail56083 earthpig99879 cthulu90770 and justice4 all buzzkill snuke52005 joseph keller obaminated williamthebastard96813 auslim williamthebastard87224 saggy jac earthpig37061 nimatzo10677 sunrisebloom river of blood90112 angry observer keith olbermann snuke60045 atma01 wrath of onion sheman river of blood51029 daemon53764 herbert hoover once again. wtf wtf54663 snuke52218 i39m better then you s.pammer4 s.pammer1 s.pammer111 river fo blood hot rod s.pammer porchmonkey s.pammer11 s.pammer2 s.pammer12 s.pammer3 s.pammer13 s.pammer14 s.pammer5 s.pammer15 s.pammer6 s.pammer16 s.pammer7 s.pammer17 s.pammer31 s.pammer32 s.pammer33 barack hussein obam williamthebastard69210 s.pammer21 s.pammer22 s.pammer23 s.pammer24 s.pammer26 s.pammer27 s.pammer8 s.pammer28 s.pammer18 s.pammer9 jac cardinal hip46465 im better than you hip75613 williamthebastard75390 wrath of orion97068 mushroomclowd rugean stoner nsbr phunkyphishstyle36852 wrath of orion92082 stark raving earthpig80346 mrpresidet07 cthulu12006 wrath of orion37359 swordtail72346 snuke63338 ugtjesus blue nightbloom wrath of orion40831 samantha adams firestorm phoeinx crown royal22074 mirror hot rod gomccaingo www.yeswecansong.net state department lacticacid lol fag river of blood19605 and justice a all rodney hotsie earthpig lipstick phunkyphishstyle88033 earthpig37816 frank sinatra wyclef jean ambassador xui les claypool mccain palin rick james lil john lil wayne mccains penis cookie monster daemon19824 daemon40225 rugian43458 pissflaps mcgee53960 socialism fascism anarchism theocracy rug1an crown royal88791 phunkyphishstyle10125 big bad spammer phunkyphishstyle30247 earthpig41919 snuke72736 paramount92339 elegant little davey crown royal59079 karl marx hanging chad hip78613 ted stevens rivar of blood pancakes neal boortz joetheplumber earthpig77256 asslard cthulu93096 hot mod dies hot rd cthulu97514 assguard hillary clinton cloud strife39932 nimatzo17123 bill gaytes poisons ghost ghostbusters bunnyblueballs koichi fukuda wtf wtf83890 nat queen cole ma1ssssss cthulu80999 nimatzo48764 joker wtf wtf38728 flicker homeland securty to fuck or to not kark rove robert downy jr. phunkyphishstyle29978 carl sagan stun gun ron howard martin scorsese ddddsj david fincher pissflaps mcgee24474 billy crystal woody allen river of blood83999 h0t rod. hotrod0c hotttrod kjakkjkjjkkk kjakkjkjjkkkk kkkkk kkkkkk kkkkkkoooo kkkkkkjooioo kkkkkkoioo kkkkkokoioo kkkkkokojkjkj kkokojkjkj aaaj aaajjj aaajjjjj kk kkk lll crown royal96565 llllu lllluu lllluuu lllluuuu crack monster arma armaa armaakl lekfkl lekfkll lekfklll lekfkllll lekfklllll lekfkllllll lekfklllllll lekfklllllllk jjj jjjj jjjjj jjjjjj jjjjjjj ppp pppl pppll ppplld ppplldd pppllddd ppplldddd pppllddddd ppplldddddg ppplldddddgg pppllddddg ppss williamthebastard43158 ppsss ppssss ppsssss ppssssss ppsssssss ppssssssss ppsssssssss qll qllr qllrr qllrrr wrath of orion92891 qllrrrr qllrrrrr qllrrrrrr qllrrrrrrr qllrrrrrrrr qllrrrrrrrrr qllrrrrrrrrrr qllrrrrrrrrrrj qllrrrrrrrrrrje qllrrrrrrrrrrjee qllrrrrrrrrrrjeee qllrrrrrrrrrrjeeee qllrrrrrrrrrrjeeeee qllrrrrrrrrrrjeeeeee qllrrrrra qllrrrrrad qllrrrrradd qllrrrrraddd qllrrrrradddd daemon91500 qllrrrrraddddd qllrrrrradddddd qllrrrrraddddddd qllrrrrradddddddd qlnd qlndd cthulu93720 qlnddd qlndddd qlnddddd qlndddddd qlnddddddd qlndddddddd pl4 phunkyphishstyle46895 pl4r pl4rr pl4rrr daemon82757 pl4rrrr pl4rrrrr pl4rrrrrr pl4rrrrrrr pl4rrrrrrrr pl4rrrrrrrrr pl4rrrrrrrrrr pl4rrrrrrrrrrj rrjj rrjjf rrjjff rrjjfff rrjjfffd rrjjfffdd ddddd dddddd ddddddf ddddddfs ddddddfss ddddddfsss ddddddfssss ddddn ddddnk ddddnkd lkjlkjlj ddddnkdd ddddnkddjj lkj hellflre hellfrre hellflrekjkj jkj jkjjj jkjjjjkj jkjjjjkjk jkjj jkjjkjjj jkjjkjjjk hotrod4 hotzrod i...sdnfffffj m.a m.aa m.aaa m.aaaa m.aaaaa m.aaaaaa m.aaaaaaa m.aaaaaaaa m.aaaaaaaaa rugian66717 m.aaaaaaaaaa m.aaaaaaaaaaa m.aaaaaaaaaaaa m.s m.ss m.sss m.ssss m.sssss m.ssssss m.sssssss m.ssssssss m.sssssssss m.ssssssssss m.sssssssssss m.ssssssssssss m.d m.dd m.ddd m.dddd m.ddddd m.dddddd m.ddddddd m.dddddddd m.ddddddddd m.dddddddddd m.ddddddddddd m.dddddddddddd ghghg ghghga 33q4 daess 33q4 daes4s 33q4 daes4s5 33q4 3daes4s5 33q34 3daes4s5 .6a .6aa .6aaa .6aaaa .6aaaaa .6aaaaaa .6aaaaaaa .6aaaaaaaa .6b .6ba wrath of orion48556 .6baa .6baaa .6baaaa .6baaaaa .6baaaaaa .6baaaaaaa .6baaaaaaaa .6baaaaaaaaa .6n .6na .6naa .6naaa .6naaaa .6naaaaa .6naaaaaa .6naaaaaaa .6naaaaaaaa .6naaaaaaaaa .6naaaaaaaaaa .bnn .bnna .bnnaa .bnnaaa .bnnaaaa .bnnaaaaa .bnnaaaaaa .bnnaaaaaaa .bnnaaaaaaaa .bnnm .bnnma .bnnmaa .bnnmaaa .bnnmaaaa .bnnmaaaaa 87. 87.a 87.aa 87.aaa 87.aaaa 87.aaaaa .a3 .a3a .a3aa .a3aaa .a3aaaa .a3aaaaa .a3aaaaaa .a3n .a3nn .a3nna .a3nnaa .a3nnaaa .a3nnaaaa .a3nnaaaaa .p9a .p9aa phunkyphishstyle12564 .p9aaa .p9aaaa .p9aaaaa .0a .0aa .0aaa .0aaaa robert schueller oddfish ccc nimatzo29889 crownropyal george hotrod3 bushy boy taughthow2smokerod .hotrod 89. 89.3 89.3a 89.3aa 89.3aaa 89.3aaaa 89.3aaaaaa 89.3aaaaaaa snuke55025 89.3aaaaaaaa 89.3aaaaaaaaa 89.3aaaaaaaaaa 89.3aaaaaaaaaaa 89.3aaaaaaaaaaaa 893a 893aa 893aaa 893aaaa 893aaaaa 893aaaaaa 893aaaaaaa 893aaaaaaaa 893aaaaaaaaaa 56y 56ya 56yaa 56yaaa 56yaaaa 56yaaaaa 56yaaaaaa 56yaaaaaaa 56yaaaaaaaa 56yaaaaaaaaa 56yaaaaaaaaaa 56yaaaaaaaaaaa 56yaaaaaaaaaaaa 56yaaaaaaaaaaaaa 56v 56va 56vaa 56vaaa 56vaaaa 56vaaaaa 56vaaaaaa 56vaaaaaaa 56vaaaaaaaa 56vaaaaaaaaa 56vaaaaaaaaaa 56vaaaaaaaaaaa 56vaaaaaaaaaaaa 56vaaaaaaaaaaaaa 56na 56naa 56naaa hip43916 56naaaa 56naaaaa 56naaaaaa 56naaaaaaa 56naaaaaaaa 56naaaaaaaaa 56naaaaaaaaaa 56naaaaaaaaaaa 56ne 56nea swordtail99432 56neaa 56neaaa 56neaaaa 56neaaaaa 56neaaaaaa 56neaaaaaaa 56neaaaaaaaa 56neaaaaaaaaa 56neaaaaaaaaaa 56neaaaaaaaaaaa 57 57a 57aa 57aaa 57aaaa 57aaaaa 57aaaaaa 57aaaaaaa 57aaaaaaaa 57aaaaaaaaa 57aaaaaaaaaa 57aaaaaaaaaaa 57aaaaaaaaaaaa 57aaaaaaaaaaaaa 5. 5.a 5.aa 5.aaa 5.aaaa 5.aaaaa 5.aaaaaa 5.aaaaaaaaa 5.aaaaaaaaaa 5.aaaaaaaaaaa 5.aaaaaaadaaaa rugian72086 5.aaadaaaadaaaa 9.ed 9.eda 9.edaa 9.edaaa 9.edaaaa williamthebastard87836 9.edaaaaa 9.edaaaaaa 9.edaaaaaaa 9.edaaaaaaaa 9.edaaaaaaaaa 9.edaaaaaaaaaa 9.edaaaaaaaaaaa 9.eds 9.edss 9.edsss 9.edssss 9.edsssss 9.edssssss 9.edssssssffff jssffff jssfffff jssffffff jssfffffff jssffffffff jssfffffffff jssffffffffff jssfffffffffff jssffffffffffff jssfffffffffffff jssffffffffffffff jssfffffffffffffff cthulu30982 jssffffffffffffffff jssfffffffffffffffff jsa jsaa jsaaa cow 3adaaadad dda444 dda4maaae moomoomoomoomoomoomo sheep sheep1 sheep2 sheep3 sheep4 sheep5 mooo sheep6 sheep7 sheep8 mooomoomoomoomoomoom sheep9 oomoomoomoom sheep10 sheep11 sheep12 oomoomoomoomm sheep13 sheep14 sheep15 sheep16 oomoomoomoommwwdwdw oomoomoomooddmm oomoomooddmm mooooooooooooooooooo moo moomoo sheep34 sheep35 moomoomoo sheep36 sheep37 sheep38 sheep39 sheep40 sheep41 roger my buddy arkham 5yjj introduce a little.. introduie a little.. swordtail67076 inintroduca little.. inintintroducittle.. iniintroduducittle.. iniintrodinucittle.. spock spock. spock.a spock.aa spock.aaa spock.aaaa spock.aaaaa spock.aaaaaa spock.aaaaaaa 444s5 444s5a 444s5aa 444s5aaa 5aaaaaaa 8675309 8675309a 8675309aa cloud strife60124 8675309aaa 8675309aaaa 8675309aaaaa baaaaa baaaaaa bbaaaaaa bbaaaaaas bbaaaaaass bbaaaaaasss dddds ddssjaaaa dd45jaaaad dd45jad3ad dd405jad3ad dd405jad3od dd405jad3od4 clydesdale gargantuan girl marcel prousto hot mod dies. kasjdf8a4ja snuke76342 cthulu34181 camaban1 camaban2 camaban3 swordtail39892 mcstain mccstain lickme10 lickme7 mr. murry gayseqs1 gayseqs3 gayseqs4 gayseqs5 gayseqs6 gayseqs7 mr. murry2 mr. murryj mr. murryja mr. murryjaa mr. murryjaaa mr. murryjaaaa mr. murryjaaaaa mr. murryjaaaaaa mr. murryjaaaaaaa swordtail50468 mr. murryjaaaaaaaa mr. root hotrot hotrot1 hotrot12 hotrot13 hotrot14 mr.murryaa mr. rootter mr.murryaaaa mr.murryaaaaa mr.murryz mr. murry000 mr.murryz1 mr.murryz12 mr.murryz121 mr.murryz1212 mr.murryz12121 mr.murryzgwbush mr. murry00 river of blood81106 mr.murryzgwbush1 mr.murryzgwbush2 mr.murryzgwbush3 nimatzo42550 mr.murryzgwbush4 mr.murryzgwbush5 mr. murry00kk mr.murryb mr. mury00kk mr.murrybc mr.murrybcc mr.murrybccc mr.murrybcccc mr. mury00kkjj mr.murrybccccc mr.murrybcnr2 mr.murrybcnr21 mr.murrybcnr21a mr.murrybcnr21aa ..8 mr.murrybcnr21aaa ...8 ...8a ...8aa ...8aaa ...8aaaa ...8aaaaa ...8aaaaaa ...8aaaaaaa ...8aaana5kaj ...8aaa05kaj ...8a0a05kaj ...8a0aad05kaj daspammer crown royal32483 ...8a0a3d05kaj lickerr licker sports mod yankeessuckl13 yankeessuckl33 r0n paul marcel proust 88.a 88.aa 88.aaa 88.aaaa 88.aaaaa 88.aaaaaa 88.aaaaaaa 88.aaaaaaaa 88.aaaaaaaaa 88.aaaaaaaaaa 88.aaaaaaaaaaa 88.aaaaaaaaaaaa 8.8. 8.8.a 8.8.aa 8.8.aaa 8.8.aaaa 8.8.aaaaa 8.8.aaaaaa 8.8.aaaaaaa 8.8.aaaaaaaa 8.8.aaaaaaaaa 8.8.aaaaaaaaaa 8.8.aaaaaaaaaaa 8.8.aaaaaaaaaaaa 8.8.aaaaaaaaaaaaa 8.8.aaaaaaaaaaaaaa 8.8.s 8.8.ss 8.8.sss 8.8.ssss hip39775 8.8.sssss 8.8.ssssss 8.8.sssssss 8.8.ssssssss 8.8.sssssssss 8.8.ssssssssss 8.8.sssssssssss 8.8.ssssssssssss 8.8.sssssssssssss 8.8.ssssssssssssss 8.8.sssssssssssssss rugian10746 8.8.d 8.8.dd 8.8.ddd 8.8.dddd 8.8.ddddd 8.8.dddddd 8.8.ddddddd 8.8.dddddddd 8.8.dddddddddd 8.8.ddddddddddd 8.8.dddddddddddd 8.8.da 8.8.daa 8.8.daaa 8.8.daaaa 8.8.daaaaa 8.8.daaaaaa 8.8.daaaaaaa 8.8.daaaaaaaa 8.8.daaaaaaaaa pissflaps mcgee30169 8.8.daaaaaaaaaa snuke15676 8.8.daaaaaaaaaaa 8.8.daaaaaaaaaaaa 8.8.daaaaaaaaaaaaa 8.9. 8.9.a 8.9.aa 8.9.aaa 8.9.aaaa 8.9.aaaaa 8.9.aaaaaa 8.9.aaaaaaa 8.9.aaaaaaaa 8.9.aaaaaaaaa 8.9.aaaaaaaaaa 8.9.aaaaaaaaaaa 8.9.aaaaaaaaaaaa 8.7. 8.7.a 8.7.aa wtf wtf47496 8.7.aaa 8.7.aaaa 8.7.aaaaa 8.7.aaaaaa 8.7.aaaaaaa 8.7.aaaaaaaa 8.7.aaaaaaaaa 8.7.aaaaaaaaaa 8.7.aaaaaaaaaaa 8.7.aaaaaaaaaaaa 8.7.aaaaaaaaaaaaa 8.7.aaaaaaaaaaaaaa 8.7.aaaaaaaaaaaaaaa 8.88. 8.88.a 8.88.aa 8.88.aaa 8.88.aaaa 8.88.aaaaa 8.88.aaaaaa 8.88.aaaaaaa 8.88.aaaaaaaa 8.88.aaaaaaaaa 8.88.aaaaaaaaaa 8.88.aaaaaaaaaaa 8.80. 8.80.a 8.80.aa 8.80.aaa 8.80.aaaa 8.80.aaaaa 8.80.aaaaaa 8.80.aaaaaaa bigboy bananas 89t55aaaaaaa bnasaaa bnasaaaj banana boat ma1aa ma1 ma1a ma1aaokk ma1aaaokk ma1aaaaokk ma1aaaaaokk ma1aaaaaaokk ma1aaaaaaaokk ma1aaaaaaaaokk ma1aaaaaaaaaokk ma1s ma1ss ma1sss ma1ssss ma1sssss ma1sssfkjsjsjjis1jjj ma1ssskjssis ma1ssskjssis1 ma1ssskjssis1j ma1ssskjssis1jjj ma1sssfkjssjis1jjj ma1sssfkjssjjis1jjj river of blood67374 ma1sssfkjsjjsjs1jjj maa1s1 maa1s1j. murdaaaer adiefs sperimimi ork palins clit mr karl marx more usernames to keep us entertained
nhill Member	Mon Jun 27 18:12:16 There are 41 variants of Rugian. We could have a UP infamy leaderboard based on impersonator lists.
nhill Member	Mon Jun 27 18:19:23 Ok looks like I resolved all the potential crashes based on the server going down (it'll sleep for 3 seconds and try again after a 503 so the server doesn't stay down). Going to let this run, should take about 5 hours to scrape and categorize every post.
murder Member	Mon Jun 27 18:28:38 Wouldn't it have been a good idea to clear this with TC first?
nhill Member	Mon Jun 27 18:31:14 No.
murder Member	Mon Jun 27 18:34:09 "Check the last post in this thread, Mr. Nimatzo88174." It looks like I haven't changed much judging by that thread. ========================================== murder Moderator Sat Aug 23 08:46:08 Why wouldn't they? It's not like Bush is going to do anything to stop them from invading another ally. ========================================== murder Moderator Sat Aug 23 09:06:27 1) Because Georgia was supposedly an ally, and only a nation of punk bitches would allow an ally to get overrun and occupied without responding with force to drive out the invader. 2) I do not accept the premise that tens of thousands of Americans would die ... much less hundreds of thousands. I am endlessly annoyed at the assertion that standing up to the Soviets is equivalent to nuclear war. That is stupid. The Soviets are not suicidal, so any short of Abrams tanks rolling towards Moscow is simply NOT going to result in a Soviet launch no matter how many times that nonsense is repeated. ========================================== murder Moderator Sat Aug 23 10:54:10 I think we should've bombed the ever loving crap out of Soviet forces inside Georgia ... and that most certainly is war. ========================================== lol :o)
nhill Member	Mon Jun 27 18:35:34 The site can handle more than 50 users at once, don't worry ;) I'm scraping public information, is no different than a web browser. If TC didn't want me to do it, too bad. Would just have to use another server to run it with a new IP. Can't ban all 454,400 servers, could scale it out to one thread read per server. :) Once information is public, it is fair game. I did tweak the script to stay under 50 tps, which is easy for the server to handle.
murder Member	Mon Jun 27 18:36:35 He could shut the whole thing down. It's not like he spends a ton of time posting here.
nhill Member	Mon Jun 27 18:37:14 http://utopiaforums.com/boardthread?id=politics&thread=5
nhill Member	Mon Jun 27 18:37:32 http://utopiaforums.com/boardthread?id=politics&thread=22
nhill Member	Mon Jun 27 18:37:48 http://utopiaforums.com/boardthread?id=politics&thread=59
murder Member	Mon Jun 27 18:45:19 Turns out I was too high strung to moderate. I got annoyed at all the "Hot Rod is a pedo" crap and just started deleting posts and threads left and right. The funny thing is that no one even noticed. I had to ask TC to relieve me of my mod powers to end my own tyrannical reign. :o)
nhill Member	Mon Jun 27 18:49:17 http://utopiaforums.com/boardthread?id=politics&thread=197
nhill Member	Mon Jun 27 18:49:26 http://utopiaforums.com/boardthread?id=politics&thread=139
nhill Member	Mon Jun 27 18:49:33 http://utopiaforums.com/boardthread?id=politics&thread=117
nhill Member	Mon Jun 27 18:49:42 http://utopiaforums.com/boardthread?id=politics&thread=215
nhill Member	Mon Jun 27 18:49:48 http://utopiaforums.com/boardthread?id=politics&thread=240
nhill Member	Mon Jun 27 18:50:00 ^some old posts of yours murder
nhill Member	Mon Jun 27 18:55:21 http://utopiaforums.com/boardthread?id=politics&thread=300
nhill Member	Mon Jun 27 18:55:25 http://utopiaforums.com/boardthread?id=politics&thread=370
nhill Member	Mon Jun 27 18:55:33 http://utopiaforums.com/boardthread?id=politics&thread=469
nhill Member	Mon Jun 27 18:55:40 http://utopiaforums.com/boardthread?id=politics&thread=540
nhill Member	Mon Jun 27 18:55:47 http://utopiaforums.com/boardthread?id=politics&thread=939
murder Member	Mon Jun 27 19:47:55 Oil is back at ~ $110 I'm still right about Soviet Russia. Jewmer lol "There's nothing magic about the NATO alliance. It is after all full of Frenchmen, appeasers and us." Oh me ... don't ever change. :o)
Habebe Member	Mon Jun 27 19:49:15 This is awesome.
nhill Member	Mon Jun 27 22:50:21 habebe I'll query your earliest posts
nhill Member	Mon Jun 27 22:52:52 http://utopiaforums.com/boardthread?id=politics&thread=882
nhill Member	Mon Jun 27 22:53:12 http://utopiaforums.com/boardthread?id=politics&thread=1313
nhill Member	Mon Jun 27 22:53:24 http://utopiaforums.com/boardthread?id=politics&thread=1308
nhill Member	Mon Jun 27 22:53:36 http://utopiaforums.com/boardthread?id=politics&thread=2127
nhill Member	Mon Jun 27 22:53:47 http://utopiaforums.com/boardthread?id=politics&thread=2131
nhill Member	Mon Jun 27 22:53:58 http://utopiaforums.com/boardthread?id=politics&thread=2519
nhill Member	Mon Jun 27 22:54:07 http://utopiaforums.com/boardthread?id=politics&thread=2690
nhill Member	Mon Jun 27 22:54:18 http://utopiaforums.com/boardthread?id=politics&thread=2867
nhill Member	Mon Jun 27 22:54:30 http://utopiaforums.com/boardthread?id=politics&thread=2945
nhill Member	Mon Jun 27 22:54:45 http://utopiaforums.com/boardthread?id=politics&thread=3928
nhill Member	Mon Jun 27 22:54:56 http://utopiaforums.com/boardthread?id=politics&thread=4036
nhill Member	Mon Jun 27 22:55:08 http://utopiaforums.com/boardthread?id=politics&thread=4327
nhill Member	Mon Jun 27 22:55:18 http://utopiaforums.com/boardthread?id=politics&thread=4479
nhill Member	Mon Jun 27 22:55:28 http://utopiaforums.com/boardthread?id=politics&thread=4609
nhill Member	Mon Jun 27 22:55:38 http://utopiaforums.com/boardthread?id=politics&thread=5583
nhill Member	Mon Jun 27 22:55:47 http://utopiaforums.com/boardthread?id=politics&thread=5812
nhill Member	Mon Jun 27 22:55:59 http://utopiaforums.com/boardthread?id=politics&thread=6376
nhill Member	Mon Jun 27 22:56:09 http://utopiaforums.com/boardthread?id=politics&thread=6755
nhill Member	Mon Jun 27 22:56:19 http://utopiaforums.com/boardthread?id=politics&thread=8545
nhill Member	Mon Jun 27 22:57:15 http://utopiaforums.com/boardthread?id=politics&thread=15254
nhill Member	Mon Jun 27 22:57:24 http://utopiaforums.com/boardthread?id=politics&thread=16085
nhill Member	Mon Jun 27 22:57:34 http://utopiaforums.com/boardthread?id=politics&thread=16088
nhill Member	Mon Jun 27 22:57:57 http://utopiaforums.com/boardthread?id=politics&thread=17161
nhill Member	Mon Jun 27 22:58:09 http://utopiaforums.com/boardthread?id=politics&thread=18018
nhill Member	Mon Jun 27 22:58:28 http://utopiaforums.com/boardthread?id=politics&thread=18111
nhill Member	Mon Jun 27 22:58:38 http://utopiaforums.com/boardthread?id=politics&thread=18113
nhill Member	Mon Jun 27 22:58:47 http://utopiaforums.com/boardthread?id=politics&thread=18221
nhill Member	Mon Jun 27 22:58:58 http://utopiaforums.com/boardthread?id=politics&thread=18437
nhill Member	Mon Jun 27 22:59:09 http://utopiaforums.com/boardthread?id=politics&thread=18535
nhill Member	Mon Jun 27 22:59:20 http://utopiaforums.com/boardthread?id=politics&thread=19005
nhill Member	Mon Jun 27 23:01:53 http://utopiaforums.com/boardthread?id=politics&thread=31327
nhill Member	Mon Jun 27 23:02:02 http://utopiaforums.com/boardthread?id=politics&thread=33115
nhill Member	Mon Jun 27 23:02:11 http://utopiaforums.com/boardthread?id=politics&thread=33119
nhill Member	Mon Jun 27 23:02:20 http://utopiaforums.com/boardthread?id=politics&thread=33694
nhill Member	Mon Jun 27 23:02:29 http://utopiaforums.com/boardthread?id=politics&thread=33788
	show deleted posts