web-ology.com

Over on Google Launches Personal History Feature:

It seems like google keeps releasing new features to their products on a weekly basis now. This week, google announced the release of their “Personal History Feature.” Now you can look through your past searches and see them based on calendar date, etc. It’s kind of interesting to me how such a simple feature was over looked by the MSN search team but I guess MS in general takes the position of playing catchup instead of innovating with new products.

I finally got around to updating my blog to latest / greatest version of WordPress (1.5). WordPress has always seemed to work and I’m not a very active blogger either way. This past weekend I through together a metabase system for creating tools blogs, news systems, etc. All in all, I think the idea is very sound but it’s hard to fight for enough free time to want to pursue such a project. The first cut worked well and more importantly, created a nice static website. Some times the power of design simplicity can create some interesting challenges.

I ran through various code snippets this weekend and I was amazed at home many 70% to 80% complete projects that I have created over the course of the last few years. I literally had hundreds of various systems and I noticed that most were different for the sake of being different. With each high level system that I think up, I’m reminded of how backwards web development is right now. It takes a simple application like Gmail to wake people up to realize that even something as basic as email can be done *better*.

For now, I’m going back to some basics and I’m tired of creating systems that create websites and spinning my wheels at that level. From here on, I’ve decided to work more on the content publication model and create some basic frameworks that work in all html based environments. Any server scripting specific scripting that I do will be handled differently as well. A rich set of admin tools with a publish on demand and no scripting if not needed will be my new mantra. Now maybe I can spend some free time enjoying some of my computer related hobbies without the need to start from scratch each time.

Overall, my new system is starting to use a basic outline model. In the admin, you can create your blog in simple meta data and list the fields along with with what kinds of input is needed. This creates the admin area for the category that you just created. This creates a very easy to publish model and allows more customization for “lists of properties” or “iterations of processes”. Long story short, you can edit the templates in the admin and it allows you to have very specific looking view by id pages and very custom archive pages with a minimal amount of effort.

My news feed system has toppled the 48,000 mark and is still growing strong, but I’ve rewritten the front end to save statically and mentioned above, I’ve decided to rework some pieces of it but I want to leverage this data to help scale the system that I have now. All in all I’ll have closer to 49,000+ pages to compile into which will eat up disk space but save processoring power by 10 fold.

So far, the use of php or asp can still be easily supported but my goal is to first create the shell + content of the site on disk and to focus the admin tools on a better subset of functionality. The goal is great both a better set of admin tools and to build a better presentation system. There’s no reason why some many CMS’s (many that I’ve written as well) have to call “SELECT * FROM page WHERE name=’home’;”. Even with caching aside it seems point less to spend so much time and energy working with complex caching systems that still have to periodically check to see the state of the content. Instead, I’m going back to my roots and going with a complete static exterior and pages will only change when you modify them, when you schedule the system to release a new version, or when you change a layout portion of the website and the templates are updated.

I completed my first build system this weekend. After researching python, I’ve found a few ideas that I liked alot. I read some manuals and I created a shell build system that’s smart enough to transform my 48K+ feeds into static pages in a few seconds. This could easily be done on a seperate machine and pushed to the client node or even have the website rebuild itself entirely from a database content.

Aside from images, I’m using the database to store just about everything else in it. I could push images into the database as well and build one master script to rebuild the entire website, but I simply don’t need to right now. I’m instead leveraging the database to rebuild the content files and using the build system to intelligently manage that content. I’m still playing around with the idea of running either a seperate server thread or just using cron for now to help schedule releases, etc.

So far it looks promising but my goal is to write one system that can run a dozen or so websites that I have and run them with a minimal amount of effort from here on out.

For many years I’ve switched from one computer to the next and I’ve always hated moving emails. After spending quite bit of my time at work upkeeping a qmail server, the last thing that I want to take on is my own personal email server. Instead I’ve theorized about writing a POP3 proxy combo that I can connect my email client to and it in return stores my messages in MySql. Ideally I’d use an IMAP based connection over POP3 to keep the load to a reasonable amount, but for POP3 would work.

Tonight I decided to take on the challenge and Python was my language of choice for the implementation. While I have a ways to go, it took only an hour to have a working POP3 puller and it was able to store messages in to a MySql database. Python is certainly an empowering language but the strange thing was I literally was pulling data from MySql and email from POP3 before I was able to figure out string concatenation. Python is definitely a gem in the rough.

I found a set of modules to support SPAM checking via Bayes filters which should help cut down my extremely high SPAM ratio right now.

We recently worked on a new DNS server based on MySQL. While working on it, it reminded me of how lacking MySQL replication is. Our goal was to have two machines, each with it’s own DNS client/server and each running it’s own MySQL database. Since we only want to make changes to one server, the need to replicate the data came up. Once setup, as long as the MySQL secondary node is not written to, then everything seems to work fine. However, should one wrong insert statement go to it, in virtually any database then the system gets hosed and refuses to accept any other updates from the primary machine. For now it’s simple enough for our needs but it’d be nice to know that something better was on the way.