Archive for the ‘Misc. Programming’ Category

Blog Upgrade and Various Thoughts

Tuesday, April 19th, 2005

I finally got around to updating my blog to latest / greatest version of WordPress (1.5). WordPress has always seemed to work and I’m not a very active blogger either way. This past weekend I through together a metabase system for creating tools blogs, news systems, etc. All in all, I think the idea is very sound but it’s hard to fight for enough free time to want to pursue such a project. The first cut worked well and more importantly, created a nice static website. Some times the power of design simplicity can create some interesting challenges.

I ran through various code snippets this weekend and I was amazed at home many 70% to 80% complete projects that I have created over the course of the last few years. I literally had hundreds of various systems and I noticed that most were different for the sake of being different. With each high level system that I think up, I’m reminded of how backwards web development is right now. It takes a simple application like Gmail to wake people up to realize that even something as basic as email can be done *better*.

For now, I’m going back to some basics and I’m tired of creating systems that create websites and spinning my wheels at that level. From here on, I’ve decided to work more on the content publication model and create some basic frameworks that work in all html based environments. Any server scripting specific scripting that I do will be handled differently as well. A rich set of admin tools with a publish on demand and no scripting if not needed will be my new mantra. Now maybe I can spend some free time enjoying some of my computer related hobbies without the need to start from scratch each time.

Overall, my new system is starting to use a basic outline model. In the admin, you can create your blog in simple meta data and list the fields along with with what kinds of input is needed. This creates the admin area for the category that you just created. This creates a very easy to publish model and allows more customization for “lists of properties” or “iterations of processes”. Long story short, you can edit the templates in the admin and it allows you to have very specific looking view by id pages and very custom archive pages with a minimal amount of effort.

My news feed system has toppled the 48,000 mark and is still growing strong, but I’ve rewritten the front end to save statically and mentioned above, I’ve decided to rework some pieces of it but I want to leverage this data to help scale the system that I have now. All in all I’ll have closer to 49,000+ pages to compile into which will eat up disk space but save processoring power by 10 fold.

So far, the use of php or asp can still be easily supported but my goal is to first create the shell + content of the site on disk and to focus the admin tools on a better subset of functionality. The goal is great both a better set of admin tools and to build a better presentation system. There’s no reason why some many CMS’s (many that I’ve written as well) have to call “SELECT * FROM page WHERE name=’home’;”. Even with caching aside it seems point less to spend so much time and energy working with complex caching systems that still have to periodically check to see the state of the content. Instead, I’m going back to my roots and going with a complete static exterior and pages will only change when you modify them, when you schedule the system to release a new version, or when you change a layout portion of the website and the templates are updated.

I completed my first build system this weekend. After researching python, I’ve found a few ideas that I liked alot. I read some manuals and I created a shell build system that’s smart enough to transform my 48K+ feeds into static pages in a few seconds. This could easily be done on a seperate machine and pushed to the client node or even have the website rebuild itself entirely from a database content.

Aside from images, I’m using the database to store just about everything else in it. I could push images into the database as well and build one master script to rebuild the entire website, but I simply don’t need to right now. I’m instead leveraging the database to rebuild the content files and using the build system to intelligently manage that content. I’m still playing around with the idea of running either a seperate server thread or just using cron for now to help schedule releases, etc.

So far it looks promising but my goal is to write one system that can run a dozen or so websites that I have and run them with a minimal amount of effort from here on out.

Email Backup

Wednesday, April 13th, 2005

For many years I’ve switched from one computer to the next and I’ve always hated moving emails. After spending quite bit of my time at work upkeeping a qmail server, the last thing that I want to take on is my own personal email server. Instead I’ve theorized about writing a POP3 proxy combo that I can connect my email client to and it in return stores my messages in MySql. Ideally I’d use an IMAP based connection over POP3 to keep the load to a reasonable amount, but for POP3 would work.

Tonight I decided to take on the challenge and Python was my language of choice for the implementation. While I have a ways to go, it took only an hour to have a working POP3 puller and it was able to store messages in to a MySql database. Python is certainly an empowering language but the strange thing was I literally was pulling data from MySql and email from POP3 before I was able to figure out string concatenation. Python is definitely a gem in the rough.

I found a set of modules to support SPAM checking via Bayes filters which should help cut down my extremely high SPAM ratio right now.

Agragated Technology News Feeds Now Public

Monday, April 12th, 2004

About two months ago I started a project to pull news feeds from various sources and to organize them by content. I’ve thrown together a very quick and dirty version that serves mainly as a one stop place to see the latest headlines from various technology and programming related (with emphasis on PHP). The current version of the page will only show the current day’s headlines but I should have a more complete rewrite up by the weekend. For now, visit my news feeds page.

The current system will scan various RSS feeds every 15 minutes and will catalog those changes. When a new feed is added, every reference in that feed shows up as a new article, and that’s why there is a long series of goolge news and slashdot articles instead of an even misture. Some features that I’ve been playing with include interfacing with google and amazon.com to find more content on the subjects and various reference sources.

Once the categories are up, my plan is to build several mini-sites to allow for news / content to be funneled in along with product information to make several on-going informational websites. Sites that keep themselves updated and provide links to new / best selling products that relate to that subject. Until that is ready, I will be looking for good news feeds and good sources for syndicated news feeds (RSS).

A Programmer’s Guide for Everyone

Sunday, March 21st, 2004

I’ve spent alot of time reading “Why’s (Poignant) Guide to Ruby” lately, even though I can’t say that I’m interested in learning Ruby. The sheer brilliance of the writing style has lead me to spend alot of time reading and re-reading the chapters of the guide. If all programming references were this good, then there would be a whole lot more programmers in the world. The entertainment value of this guide is great!

SQL Lite

Saturday, August 16th, 2003

After doing a bit of research, I came up with some handy links: