Nice web crawler type of code I made

Post by **floodhound2** » 25 Jan 2007, 20:57

All right.
I have debugged and developed a piece of code that searched the internet for what ever I want – currently it retrieves the temperature for my home town. Then it sends all the information to my server - that I am having some company host. My web site is automatically up dated and I can adjust the time intervals in which it updates. I think that this rocks because I can get news, data, and other things that I want. If some one wants a copy of the code or just the executable let me know and ill send it. It was written in VB6  but hey the thing works great.

Check out my site and see it for your self.
http://www.floodhound.com

PEACE

Nerdz · Post by **Nerdz** » 25 Jan 2007, 21:09

LoL, at school we had to do something like this... it was a little java program that get all link from a page...

By doing this, some of us made a little c++ appz which steal all drum tabs from website. We had like 10k tabs in like 5minutes!

Good job

p99 · Post by **p99** » 26 Jan 2007, 00:22

I uh heard from a uh little birdie that you can look for file types. Like jpg or mpeg.....

Making things multi-threaded speed things up.... and kill your bandwidth faster. I screwed areound with a python script i found that did this. Let's just say I filled my hardrive with downloaded "samples" in a few hours.

But nice application. I'm sure we'll be doing this in my VB class soon. Nvm the school district decided that we shouldn't have access to the internet with programs in vb and c++. Only those 2 languages.... I can still use a compiled python script and do some socketing. So... that's school security I guess

Post by **pseudo_opcode** » 26 Jan 2007, 03:06

yeah thats cool, and it would be cooler if you can directly fetch readings from a thermometer at your home, that way

p99 · Post by **p99** » 26 Jan 2007, 04:13

You could do that with much less than a webcrawler.

I guess somewhat like you can have a camera that updates it's picture every 5 seconds and the page refreshes.

Just have to get the reading from the thermometer to the computer.

I just remember seeing this guy from microsoft came and showed off his nifty rf-watch. He was having information constantly sent to it over FM radio waves. News weather.

I bet the FM service costs a fortune to use though. But cool nonetheless.

Post by **floodhound2** » 26 Jan 2007, 08:33

Well I basically go to the website and download the html code. Then I do some string manipulation giving me the results I am looking for. I toss pictures and the sort I just wanted the temp.