Those of you who have been on the Skype chat know that over the past 6 days, I've been working on a project that I've thought about for at least two or three years, and finally got around to coding. It builds on knowledge I got from a previous project, trying to categorize all the links on the internet via a recursive PHP spider, with a system that instead harvests the keywords on each pages. wordNet is not intended as a search engine in the traditional sense of the word, however. You will be able to give it a url or word, and it can return either urls or words based not on links between the sites but on common concepts. wordNet attempts to figure out the relations between concepts and phrases in human knowledge through a system I have unnecessarily deemed "inductive reasoning harnessing mass consciousness". Anyway, you can play with the budding project here:
http://wordnet.cemetech.net

And here's the obligatory logo:

You need the display a bit more spread out (I searched for cemetech and I can barely see the words because they are on top of each other).

Feareth not, Sir Harq, as verily since this is an alpha releaseth, only the fair backend hath been completed, and nay, the frontend yet shall bear great fruit.

Give me time.

What, if any, practical uses do you have for this? I could see doing this but just for cemetech.net (or maybe just calc websites), but seriously, do you realize how impossible it will be to navigate once its spidered even 1% of the internet?

Seriously, a google result for "request" came up with over 971,000,000 results. Currently wordnet comes up with 19 URLS with 386 instances of "request" Even if you zoom way in, it is still an unmanagable mess.

Cool idea, but I just can't see any practical use for an internet-wide use.

That's the beauty of it - it doesn't need to index the entire internet to work. All it needs is a decent number of sites; the more it indexes, the more accurate it becomes. The visual style will of course be cleaned up a bit, and an xml interface added. It could even be hook into Google so that, for example, you could search for something, and get the combined results of several Google searches based on both the phrase in question and related phrases and terms. It's meant to use the massive size of the internet to extract similar concepts so that some kind of pseudodatabase of human knowledge without any direct human input possible.

How many sites does your spide crawl in an hour?

There should be a way to zoom in. Also if you click repeatedly and fast enough when you are rotating the results, you can get it to continuously rotate around the last axes until you click again.

1. Shift-drag to zoom
2. Just hold, drag, and release to make it keep rotating.

wordNet can crawl about 1400 sites per day.

I must admit that you have made it look a lot nicer =D .

P.S. You should add a small help button to explain it.

Excellent idea. I'm thinking under the logo I should have: Home | Help | About | Statistics. Eh?

I agree. And also, it is extremely slow right now...

That's because I'm running a massive number of spiders, trying to build up the database.

seems like a cool idea, but

I recommend you try again; seems like you just got a bit of a network hiccup.

Kudos for tring this, but I got this error. I tried many times and got the same.

Warning: mysql_connect() [function.mysql-connect]: Too many connections in /home/cemetech0/public_html/wordnet/db.php on line 2
Could not connect: Too many connections

I did too, I think it is because of the huge amount of spiders he is running... (I think he said 16 yesterday?)

I was up to get ready....


1,237


by today. Needless to say, I'm reworking my spider system.

How many pages does that index per day?

Theoretically, over 1 millino domains, but of course my webserver nearly crtashed under the strain.

Wow, just discovered how to zoom and rotate...this is incredible.