The full text indexer and searching service are written using python3, web2py and the whoosh framework. The web crawler is written using python3 and beautifulsoup4. There is a lot of work to be done, but I wanted to share a screen shot of it in action. I started on it just yesterday.
Edit * I am requesting suck-o forum users to submit websites you think are worthy of being added. My plan is to index 1000 URLS a day, and in 5 years, I hope for a good amount of searchable data. Please submit your suggestions as a reply to this post. *cheers
*cheers Mabo
