I have been working on my own style of a web directory over the past couple of weeks (oolah.net). The goals for my web directory is to create topic areas that are loaded with useful information, not just links to websites. While most directories just list websites who have paid to be there, I want mine to be rich with RSS feeds, podcasts, images, news, and yes, websites. I want mine to be a true resource, not just a short term revenue stream that doesn't add anything of value to the net.
Because of my interest in building my own directory, I have looked at quite a few other, competing websites, which has led to additional thinking on search engines. I have always had some issue with the basic philosophy behind search engines like Google. Basically, websites are returned from a search based on their popularity. Of course, additional criteria influence the ranking as well, but for Google, the key to ranking the websites lies in how many other websites link to a given website (aka, popularity).
Link popularity has strongly contributed to Google being able to return quality websites, at least early on, when people linked to websites that they found interesting or helpful. Nowadays, the popularity game means that anyone can really attain a good seach engine ranking if they have the time, or pay someone, to run around the internet posting links back to their website. A web page may not be nearly as high quality as another web page, but if it has more links pointing to it, it could potentially gain a higher ranking than website of highest quality.
Google has done a good job up until this point, but they created a game that I believe has watered down the internet. We want to see the website of highest quality, not the mediocre website that has thousands of backlinks. At some point in the future, someone will create a search engine that places a higher rank on quality websites, rather than websites that have lots of backlinks, and the net will be better for it. The philosophy that led to the popularity of Google, will eventually lead to their undoing, unless of course they move away from link popularity.