April: An Exploratory Link-based Search Engine

I've been thinking about building tools for interfacing with the web in new and perhaps unusual ways for quite a while. I have yet to really launch into building them, so as with last month I'm using Twelve Websites as an opportunity to begin that exploration.

The idea for this website is a system for finding new links. I'd like to build new ways to explore the web, beyond just using a text-based search engine. This website should serve as a search engine for finding new websites, not found by searching content, but by links. Drop a link into the site, and it should return ten or so new directions; websites that are linked from the original or are perhaps one to two steps removed.

The main idea I'd like to establish here is that crawling a website can lead to new and unexpected links out to other websites, and that collating them provides some kind of value. I have an intuition that there's something there, I'm just not sure yet. The goal of this project will be to develop a system that allows me to see what it's like to interface with a web crawler in this way and decide if it's worth pursuing further with my much larger project Hyperfov.

So, the centerpiece of this website will be a web crawler. This project feels like it has the potential to get quite complicated implementation (and potentially cost) wise, so I'll need to be careful with increasing complexity. The interface should be dead simple at least.

Very exited to get started on this one.