Milk framework example - martialartslife.net


Yesterday the first fully functional site using the new Fliquid Studios Milk framework was lanched, www.martialartslife.net. This site is quite basic as far as the capabilities of the framework go and will no doubt implement some of the more advances features further down the track.

Some of the base features it does make good use of include:

  • A large number of base controls including form controls, layout controls etc.
  • Automatic Javascript & CSS concatenation and compression.
  • Good caching capabilities
  • CSS Sprite capabilities
  • Themes for controls

The site was developed relatively quickly on along with the actual development of the framework. Many new features were added over the last couple of weeks before the launch which is a testiment to the ease of development provided by Milk.

Creating a virtual development server using Virtualbox


For those of you who read Installing Fedora 10 on Windows XP using VirtualBox, this is another post along the same lines as that one, except this one is for configuring your virtual machine as a development server for when you are forced to code on your Windows host.

f10_in_vb

I am a bit of a snob when it comes to development, I simply refuse to develop on a windows host as I just think that the app will end up being deployed on a nix system, so it should be developed on a nix system. I also think using a windows host for development takes away from a lot of the raw development involved with using nix systems. And, with the amount of quality development and virtualization tools available to us these days, we can develop on anything, using any tool.

If you haven’t set up a virtual machine using Virtualbox and your choice of Linux OS on a Windows machine, then read through my other post before continuing with this one. Also note, you will need to have Apache configured and running on your virtual machine.

Now, what I am going to do is configure VirtualBox to accept connections to itself on port 80 and port 22, so we can develop on one machine (Windows) but using a Linux server.

Configure the network

The first thing we need to do is get our machine on our local network and a proper IP address for it. Open up the settings for your virtual machine and click on the ‘network’ tab, then select ‘Adapter 2’ and change ‘Attached To’ to ‘Host Interface’. This will allow your virtual machine to get a DHCP assigned IP address and actually put it on the network.

vm_network

Save your changes and boot your VM.

Configuring Apache / Port 80

Now we need to make sure our Guest Linux OS can accept connections on port 80. Fedora 10 blocks these connections by default, to open them, edit the file /etc/sysconfig/iptables using vim, emacs, gedit, pico, whatever. Find the line accepting port 22 connections that looks like this;

-A INPUT -m state –state NEW -m tcp -p tcp –dport 22 -j ACCEPT

And add a line below it like this;

-A INPUT -m state –state NEW -m tcp -p tcp –dport 80 -j ACCEPT

Now restart iptables;

$ service iptables restart

Forwarding port 80

The next step is to forward your port 80 to your virtual machine. You can forward any port you like, for example, you might have a web server on your Host machine and not want to forward port 80, so you can choose port 8080. I want a seamless virtual server environment, so I chose port 80.

Open up a command prompt (Start > Run > ‘cmd’ + Enter) and change to your Virtual Box directory;

cd C:\Program Files\Sun\xVM VirtualBox

Now we will get aquainted with the command line tool, ‘VBoxManage.exe’.

To forward port 80 on your host to port 80 on your guest, type the following (make sure you substitute “Fedora 10” with your VM’s name);
NOTE: Leo in the comments has recommended changing ‘pcnet’ to ‘e1000′ for Intel network cards!

VBoxManage.exe setextradata “Fedora 10″ VBoxInternal/Devices/pcnet/0/LUN#0/Config/Apache/HostPort 80

VBoxManage.exe setextradata “Fedora 10″ VBoxInternal/Devices/pcnet/0/LUN#0/Config/Apache/GuestPort 80

VBoxManage.exe setextradata “Fedora 10″ VBoxInternal/Devices/pcnet/0/LUN#0/Config/Apache/Protocol TCP

Then we can view our changes using the following command;

VBoxManage.exe getextradata “Fedora 10″ enumerate

vbmoxmana_port80

Browsing to your virtual server

At this stage you should be able to browse to your virtual server using any of the name based or home based virtual hosts on your vm.
ie, browsing to localhost on your host machine should display the default virtual host on your guest.

However, if you have name based virtual hosts on your guest (as I do) then you need to add those names to your windows ‘hosts’ file so it knows to look for them locally and not on the internet. This is very easy to do.

Open the file c:\Windows\System32\drivers\etc\hosts and add the names of all your name based hosts after ‘localhost’ on the line starting with 127.0.0.1, eg;

127.0.0.1         localhost host1 host2 mysite

Then, after saving, these hosts on your guest should load in your browser from the guest machine.

Setup ssh server

If you want to edit the files on your guest from your host using SFTP through an IDE (like Netbeans, Zend Studio, Eclipse) or an app like WinSCP, then keep reading.

Having an SFTP connection to a server is a great way to integrate your remote files into your local development environment and makes editing much easier and faster. SFTP is FTP over SSH and requires only an SSH connection to the server to work. We need to make sure ssh is running on our Guest;

$ service sshd start

Now, we want to make sure it starts when the machine starts;

chkconfig sshd on

We now need to follow the same procedure to forward the host port 22 to the guest port 22 as we did for port 80 above;
NOTE: Leo in the comments has recommended changing ‘pcnet’ to ‘e1000′ for Intel network cards!

VBoxManage.exe setextradata “Fedora 10″ VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/HostPort 22

VBoxManage.exe setextradata “Fedora 10″ VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/GuestPort 80

VBoxManage.exe setextradata “Fedora 10″ VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/Protocol TCP

Now you will be able to ssh to the guest from your host. You can test this in putty (or anything you’d like) by connecting to 127.0.0.1 on port 22 and logging into your guest machine with your normal user details.

Thats it. You now have a fully integrated virtual server for development. If you have any questions, suggestions, problems or concerns, please let me know in the comments.

Javascript Event Library Updated


Just a small post to announce that the Javascript Event Library has been updated with a few new methods and smaller size (reduced variable length, removed semicolons).

The updated code can be viewed and downloaded via our Projects page.

If you encounter any bugs or would like to collaborate in its development, please let me know.

5 development techniques to improve software quality


Most of these things you should really already be doing and if you’re not then I’d suggest starting to do so right away.

1. Step back and plan
It’s often hard not to just jump straight in to coding, especially with a project you are excited about. Try to resist that urge by stepping back and taking a bit of time to think about things before you start typing. Think about the problem are you trying to solve, any difficulties that may arise and come up with a potential solution. Even better than thinking about the problem and solution, write it down somewhere. If you start coding before you’ve completely got your head around the problem there’s a very good chance you will end up with little more than a mess.

After all if you start coding a solution before you know what that solution is how are you going to know when you arrive at that solution?

2. Document before coding
Documentation is another aspect of coding that is too often overlooked. Some developers believe it is not their job to document, others just don’t get around to it because of time constraints. Documentation of functions/methods and how particular sections of code are supposed to work can help with debugging and avoid the old “What was I/he/she thinking with this code?” situation.

I suggest that whenever you create a new class/function/method (or any other relevant piece of code) create the skeleton of it ONLY, then write the documentation. Only once the documentation is done should you think about implementing the guts of the function. This will ensure documentation doesn’t get forgotten and you will have a much clearer picture of what you are trying to achieve before you set out.

3. Adopt a coding standard and stick to it.
Use consistent indentation, layout, naming conventions etc across all of your code. It not only makes it easier for you and other to read and modify but it will also make it much easier to debug when you come back to look at the code in a few months time. This is easily one of the most important things you can do to improve the quality of your code but is also one of the most forgotten about or ignored.

4. Write test plans and make sure they are used
A lot of the time testing is performed very minimally or in a haphazard way. This is bad because leaves you open to miss areas while testing or not discovering obvious problems because the tester does know understand how the software will be used. This is unfortunately always a risk but it can be greatly minimised by writing various test plans and ensuring they are actually used. Test plans can be quite complex and extensive or simple use cases that are followed by the tester. Either way they are an excellent way of showing what has been tested and over time they can be expanded to cover more of your software.

Similarly to the writing of documentation for each class/function/method when they are created you should additionally create a set of tests for every function as or before it is coded. Again the tests can be expanded over time and are a great way of performing quick regression checks.

5. Reviews
This is by no means a new concept but it is definitely one that is under-utilised. Developers are often afraid of peer reviews because they don’t like their code being criticised and other developers can often be quite harsh. Don’t take it personally, it can be very useful to  have someone else’s eyes spot things you missed and after a few reviews your code will more than likely start to improve in general.

In addition to peer reviews, review your own code. Go back and have a quick read of what you’ve done in 1 week, 1 month, 4 months and 12 months later if you can. You’re almost guaranteed to come up with a better solution than you did before. Although you may not be able to implement your new ideas in that old project you may be able to use them in future.

Conclusion
I am a big believer in continually trying to improve my skills, knowledge and most of all coding standards. By using the techniques above (as well as others) the quality of software can be improved and over time so will your skills. Let us know your thoughts on the techniques listed above. Do you use them already? Do you think they will work for you or do you hate the idea altogether?

Also, I would love to hear of any other techniques people use relating to improving software quality. Leave a comment and let us know.

Resizing images in PHP with GD and Imagick


One task that is quite often required in websites and web applications is resizing of images for thumbnails or any other purpose. In PHP there’s a number of different ways you can approach image resizing each of which has it’s own advantages and disadvantages. In this post we will be using the GD and Imagick PHP extensions to resize images and comparing the differences.

When creating thumbnails there’s a few decisions that are required before you start coding. You need to decide on the width and height of your thumbnails, you need to decide whether you want your images to be cropped to fit your specified dimensions or not and you need to decide on an image format. In the examples below we will be showing how to resizing both using the “crop” method and without cropping.

So let’s launch into this article with a very brief look at the GD PHP extension.

GD
GD is the more commonly used extension for PHP. It is generally easy to install (`yum install php-gd` on Fedora, CentOS etc or `sudo apt-get php5-gd` on ubuntu etc). It has quite extensive documentation with examples and as it is more commonly used there’s a lot more web content available for it.

The disadvantages of GD are that it is slower and more memory intensive. Overall it has slightly less functionality and can be more complex to use.
Let’s take a look at a quick resizing example. In this example we have a jpeg image that we are going to resize to be 150×150 pixels. We are not worried about the format of the image so we won’t be changing it.

<?php
function resize_image($file, $w, $h, $crop=FALSE) {
    list($width, $height) = getimagesize($file);
    $r = $width / $height;
    if ($crop) {
        if ($width > $height) {
            $width = ceil($width-($width*($r-$w/$h)));
        } else {
            $height = ceil($height-($height*($r-$w/$h)));
        }
        $newwidth = $w;
        $newheight = $h;
    } else {
        if ($w/$h > $r) {
            $newwidth = $h*$r;
            $newheight = $h;
        } else {
            $newheight = $w/$r;
            $newwidth = $w;
        }
    }
    $src = imagecreatefromjpeg($file);
    $dst = imagecreatetruecolor($newwidth, $newheight);
    imagecopyresampled($dst, $src, 0, 0, 0, 0, $newwidth, $newheight, $width, $height);

    return $dst;
}

$img = resize_image(‘/path/to/some/image.jpg’, 150, 150);

Resizing images with GD requires that you create a new image and copy the re-sampled section to it. This means that any EXIF data contained in the image will be lost. Note that in the code above we are not actually doing anything with the image. You could either output it directly to the browser or save it to a file using the imagejpeg function.

On the other hand, the Imagick PECL extension has it’s own advantages and disadvantages.

Imagick
Imagick is the less frequently used PECL extension for the ImageMagick project. ImageMagick is a free suite of tools used for image creation and manipulation supporting over 100 different image formats. It is typically a command line tool but also has interfaces available for a number of different programming languages (including PHP).

The Imagick extension essentially provides an API for all of the functionality available in the `convert` command line tool. Imagick is fast, uses less memory and has some powerful functionality available. Quite often Imagick is a lot easier to use (once you figure out how), your code may end up smaller and cleaner.

The down side of using this extension is that the documentation is extremely limited and there are next to no examples available on the web. Installation can often be a painful task as well although it should just be a matter of running the command `pecl install imagick`.

So now let’s move on to an example. As above we are resizing a jpeg image to 150×150 pixels.

<?php
function resize_image($file, $w, $h, $crop=FALSE) {
    $img = new Imagick($file);
    if ($crop) {
        $img->cropThumbnailImage($w, $h);
    } else {
        $img->thumbnailImage($w, $h, TRUE);
    }

    return $img;
}
resize_image(‘/path/to/some/image.jpg’, 150, 150);

As you can see the Imagick code is much less complex. It doesn’t duplicate the image so it uses less memory and doesn’t lose the image EXIF data. Again note that in the code we are not actually doing anything with the image. You could save it to a file using the writeImage method or output it directly to the browser.

Conclusion
As mentioned, depending on your purpose there is advantages and disadvantages to using both GD and Imagick. It is really just a matter of deciding what you need to do and determining which will work best for you.

As usual all comments are welcomed. I’d love to hear your thoughts and preferences relating using either GD or Imagick. Additionally if you have any specific image manipulation requests let us know and we will do our best to answer your questions.

That’s all for now.

10 Sites that every web developer should know


This is a simple post listing some sites that I use very regularly. It is not an exhaustive list, by any means, but it might contain some sites that you didn’t know about. If you like any or have any sites to add to the list, please leave a comment.

Whats My Agent String?

whatsyours
Ok, so this is a small plug for our small Agent String displaying site. Its sometimes irritating to try and grab the agent strings from the browsers you are trying to support, especially if a client has a strange agent string and you are trying to get them to tell you what it is. This just takes a step out of the process.

Browser Shots

This is a brilliant site to render your code in many different browsers and then you can see screen shots of how it looks. Its not overly helpful for debugging but if you are confident your code supports the latest browsers, you can double check here.

Xenocode

xenocode
Xenocode is a cool web based virtualization hub. It allows you to run a variety of applications (including all major browsers) from your browser without needing to install anything (oh except their firefox plugin). Its a great service. Its a little hard to find but you can also download the executable files from their site also meaning you can run the virtual browsers locally without needing to be online (as I have).

Alltop

Alltop is an information resource of the current ‘hot topic’s’ online. It has a ‘tech’ category with hundreds of subcategories like ‘php’ and ‘web design’. You can also create your own account and have your own alltop custom page with news from areas that you think are relevant to you (which saves you having to browse around to find it). See my Alltop page here.

Nettuts

Nettuts is a blog for general web-relevant tutorials. From designing Wordpress themes, to building fancy UI features with JQuery. This site is a must-subscribe-to for any self respecting web worker.

Delicious - Programming

A great resource of the latest bookmarks on Delicious tagged with ‘programming’. You can also follow @delicious_prog on twitter for the best bookmarked sites tweeted to you. I am not sure if this is officially a Delicious service or not.

Clean CSS

Clean CSS is simply a site for cleaning up css. It has an options pane that allows you to make changes to the codes layout so that it fits with your existing coding conventions. If you have ever taken over a project with less-than-impressive coders, then this site will save you lots of time while making the project nicer to work on.

Smashing Magazine

smashing-logo

While Smashing Magazine is primarily a design resource, they do have many good development posts that pop up from time to time. This blog is a fantastic way to keep yourself up to date with the latest in web design trends.

Devshed Forums

The Devshed forums are a great resource for all things tech. The users are friendly and due to their reputation system, it makes it fun to provide your help so you can gain reputation. Well worth looking at when you have that niggling problem that will just not go away.

Github

Github is a free (and paid) git repository host with a social networking angle. Public repositories are free but if you want to make them private then you have to pay. A lof of big projects are on github including railsBlueprint, haml, YUI (Yahoo User Interface) and  scriptaculous. If you have a need for a certain script, its worth searching on here, also if you have a simple script or a work in progress, github make it very easy to manage your code. See my Github profile here.

UPDATE: Sorry about the broken links folks, all fixed now.

UPDATE #2:
I have just used JSTidy and I have to say, its very VERY cool. If you have any compressed js that you need to return to a readable / editable format, this is just the tool for you.

Installing Fedora 10 on Windows XP using VirtualBox


For a lot of developers, they like to have multiple operating systems available for development testing or office chores on one and development on the other. Personally, I like Windows for email, organisation, gaming, but hate developing on windows, I prefer to develop in a Linux environment. Mike on the other hand, likes to have his primary OS as Linux but likes the option of a Windows machine every now and then for testing in IE or using Windows-only applications. For a long time the tried and tested method of running multiple OS’ on a single machine was to do the ol’ dual boot action which was ok, but became tedious if you forgot something from one OS and had to reboot a few times to get it. Virtualisation is not new by any means, but what is relatively new is the free availability of an extremely easy to use Virtual machine client such as VirtualBox. It makes setting up VirtualMachines extremely easy and takes the requirement of dual booting away for ever, especially when you can get such amazing performance from your guest operating system.

I will be installing Fedora 10 (guest) on a Windows XP (host).

What’s needed;
Virtualbox (I am using 2.1.4 in this post)
Fedora 10

Firstly, I’ll go step-by-step (oooh baby) to install VirtualBox and get the Live CD version of Fedora up and running under XP.

Once you have downloaded the VirtualBox installed and obtained a copy of the Fedora 10 Live CD iso (A Live CD is a CD that will boot and run a copy of Linux without needing to be installed, it’ll run entirely from memory - we do get an option to install to HDD once its booted though, which we will do), we need to simply run the VirtualBox setup and create a new virtual machine.

Install VirtualBox

Installing VirtualBox is as easy as installing any other application. Just follow the prompts. As it needs to install a bunch of extra network controllers and what not, you might get a bunch of warnings / confirmations from Windows about whether you would like to continue. If you don’t continue you may as well stop reading now. :)

Once VirtualBox is installed, open the app and click on the ‘New’ button.
empty_virtualbox 

Click next to start moving through the wizard.
new_vm1 

Name your Virtual Machine and select its type from the drop downs.
new_vm2

Select the amount of memory to allocate to your Virtual Machine. I usually choose a minimum of 512mb or it gets just a little too slow to use. (You can also increase your video memory alocation at a later stage).
new_vm3

On this next screen we need to create the hard drive for the vm. Whats needed is a file for VirtualBox to cloak as a hard drive for the vm.
Click ‘New’ and then click ‘Next’ to start the virtual disk wizard.
new_vm_new_hdd1

This next screen is up to you. I like to know that if I need the extra space on the vm, the disk will grow as its needed, but if you are short on space or want to restrict it, you can simply have a fixed-size image. Whatever you prefer.
new_vm_new_hdd2

Name the image and select its base size (default of 8.0GB is fine).
new_vm_new_hdd3

Click Finish to confirm your image settings.
new_vm_new_hdd4

You will be sent back to the Virtual Hard Disk screen again, just click next.
new_vm4

Now you’re pretty much done. Just click through to the end of the wizard and you have created your Virtual Machine. What we need to do now is tell our new VM to use the Live CD iso disk image when we boot up. Go into the settings of your VM and click ‘CD/DVD-ROM’ and check the ‘Mount CD/DVD Drive’. Put a check in ‘ISO Image File’ and click the little folder icon on the right hand side.
Once that opens, click the ‘add’ button and select your iso file for Fedora 10.
new_image

Click ‘Select’ and you’ll return to the ‘CD/DVD-ROM’ screen.
mount_cd

Click ‘OK’ to exit the settings.

Now comes the exciting part. Click ‘Start’ above your Fedora 10 VM and watch the magic. You might get a notification about the host key (Right CTRL) or the colour settings (24 instead of 32 bit), just check ‘do not show this message again’ and click ‘ok’.

When Fedora has booted, you’ll notice its just running Fedora without installing anything, this is the Live CD at work. Click the icon on the desktop that says ‘Install to had drive’.

Install Fedora 10

Follow the prompts through the installer. All of these settings can be changed later so don’t worry too much. Make sure you disable ‘System clock uses UTC’ as it will conflict with the system clock and you’ll get all sorts of weird times displaying in your guest.

After inserting your root password, you will get a prompt stating that the partition cannot be read and would you like to format, click ‘Yes’. You will then come to the partitioning screen. Some people like to create separate partitions for /home and /boot and all the others, but personally as this is for development only (ie, nothing really saved on there) I just leave it as default and click next.
install_partitions

Now we just need to sit back and wait for the installer to do its thing.

When the installer is complete, click on ‘System’ and ‘Shut down’ - do not reboot as the Live CD image is still ‘in’ the machine and you will just boot back into that.

Once the machine is shut down, you can go back into the settings of the CD/DVD-ROM and uncheck the ‘Mount CD/DVD Drive’ (or you could leave that checked and select ‘Host CB/DVD Drive’ and check ‘Enable Passthrough’ for your system cd/dvd drive to work in the guest OS). Once that is done, start the VM again.
Fedora will boot into a ‘Welcome’ setup wizard.

Keep clicking forward until you reach the Create User screen. If you are new to Linux, then you should be aware that you need to have at least one other non-root account which you will use on a day to day basis, only switching to root for certain actions. Create your user account, select the date and time and then (if you want) send your hardware profile.

Once complete, you will receive the login screen, login with the user you created.

Installing KDE

If you are new to Linux, you wont know that there are many different ‘desktop environments’ available and the two most popular are Gnome (installed by default) and KDE. I much prefer KDE, so before I do anything in my new Guest OS, I will install that.
To install KDE:
Click ‘Applications’ > ‘System Tools’ > ‘Terminal’.
Then type the following (each new line represents ‘enter’)

$ su -
{Enter your password}
$ yum -y install kdebase kde-workspace

This is about 90MB.

Once installed, we need to log out (’Administration’ > ‘Logout’) and we need to restart the X environment by holding CTRL+ALT and hitting Backspace.
When X comes back up, we can log back in, but we want to go back to the main login screen (not the ‘this screen is locked’ screen) and down the bottom, from the ’sessions’ list, select KDE, then login.
Once you log in, you’ll see that KDE is super sexy!
Now, we need to update our system and install some other bits to make our VirtualBox experience even greater.

Update the system

In our ‘F’ menu, click into ’search’ type ‘Term’ and open the terminal.
Type

$ su -
{enter your password}
$ yum -y update

Once that has downloaded and installed all updates (337 updates for me at 400MB) we’ll start getting the system ready for the really cool VirtualBox features and we’ll also get a better resolution, too.

In the terminal, we need to install a few new items.

Installing VirtualBox Guest Additions

Type

$ yum -y install kernel-headers gcc

Note: thanks to Carlos in the comments, kernel-devel is also needed!
After it has installed, reboot the machine (its not Windows, I know, but we did just change the kernel files).
When it has booted back up and you have logged in, un-capture your mouse with right CTRL, then on the VirtualBox window, click ‘Devices’ > ‘Install Guest Additions’. This mounts a directory within your Guest file system but you wont be able to see it till you open it with a file browser, so open the F menu, go to File Manager (Dolphin) and browse to /media/VBOXADDITIONS* (Whatever version you have). Then in your terminal…

$ su -
{enter your password}
$ cd /media/VBOXADDITIONS*
$ sh VBoxLinuzAdditions-x86.run

Wait till the install has finished and reboot again (I know, I know…)

Now it starts getting sexy, on boot up, you’ll receive a message about mouse pointer integration, this means that you can mouse in and out of your guest without needing to capture and release your mouse from VirtualBox. You’ll also likely have a better resolution and you will also have cut’n'paste between host and guest (although in my experience, this functionality can be a little flakey). Right CTRL + F will make your guest full screen and if you click on ‘Machine’ (in VirtualBox top menu) and select ‘Auto resize guest display’ you can have your display resize when you resize / full screen the window which is cool.

fedora_installed

You can also do a lot of other configuration tweaks to VirtualBox based on what you want out of it, but for me, this is all I (and most other devs) need. Head on over to the VirtualBox site and check out their wiki if you want to explore some of the other features (like port forwarding to the guest, which is also cool but a bugger to get working).

Now you have a fully functioning Fedora 10 install on your windows machine. You can install / remove / develop / whatever you want. If its in full screen, you wont even notice windows is there (which is always nice).

Following these instructions, you can install any OS inside any other OS. Windows in Mac, FreeBSD in Fedora, Centos in Mac. Whatever you want. VirtualBox really does make virtualisation seriously easy and available to anybody with only minimal tech knowledge needed.

For developers, you might want to start by installing a few of the development packages available to you.

fedora_yum

Do you use virtualisation or dual booting? Is there anything you liked or didnt like about this post? Please leave a comment.

Creating a personal aggregation page using RSS, Blueprint and more



If you are like me, you find yourself posting to several blogs and contributing to many sites in total as well as wanting to promote your social network profiles. In this post, I will create a personal aggregation page for myself where I will post the latest 5 posts from my blogs as well as show my latest twitter updates and links to my social network profiles. I wanted to provide one URL to all sites I commented on / posted to for all my web content, instead of trying to use the most relevant site for whatever I am posting to (ie, using my Sustain Myself blog on eco sites, my Fliquid URL for tech sites etc - I now just use the one address for all.)

This post was initially going to be a simple RSS feed reader in php, then I just kept adding to it and now its a bit of a giant.

In this post I will:

1. Demonstrate how easy Blueprint (CSS Framework) is to work with and create a simple 3 column layout.
2. Write a simple RSS reader in PHP
3. Create some simple .htaccess rewrite rules
4. Glue it all together using MySQL and a cron job

The end result will resemble my personal aggregator at www.cb.net.au.

Blueprint CSS

I have never been too strong with my CSS skills. I tend to look over what other people have built in CSS and then adapt it to my needs. Mainly because I don’t think I have ever given myself enough time to learn it properly and things like float and positioning leave me stranded without a clue. When I decided on my topic for this post I thought I’d love to just touch on some CSS for the layout but my knowledge is not sufficient enough for that. I had heard a lot about Blueprint in the past and thought I’d have a quick look (it’d have to be quick as time is of the essence for me). Generally a quick look for me means that I can’t understand it and I give up trying, at least until I can sit down properly and nut it out. There was absolutely no need for any of that with Blueprint. After one simple tutorial, I had built a 3 column layout with really nice fonts, in no time at all.

Blueprint uses a ‘grid’ layout system which I thought seemed a bit odd at first, but makes a lot of sense. You specify a number of ‘cols’ for your grid, in most examples it is 24. Then, you split your ‘cols’ up into actual columns with widths relative to how much of that 24 you wish to assign.

For example:
We create a container div:
<div class=”container”>
We then create a header div (and specify its width in columns - ’span-24′):
<div id=”header” class=”span-24″>
That tells Blueprint to use all 24 cols to display the header, that means there will be nothing on either side of the header, so when it comes to creating our 3 col layout below, its as simple as giving each column a width in cols that wont equal more than 24;
<div id=”content” class=”span-10″>
<div id=”midbar” class=”span-6″>
<div id=”sidebar” class=”span-8 last”>
And thats it. A 3 col layout. The ‘last’ class will make sure that its layed out properly for the last column. Easy eh?

And just like that, literally, you have a 3 column layout. I won’t touch on the typography features or other layout classes, but a quick read of the Blueprint site and a Google will provide more than enough info to get you started.

This is the HTML for my site:

<div class="container">
    <div id="header" class="span-24 first last">
        <h1 class="loud">cb.net.au</h1>
        <h3 class="quiet">The home of Christian Biggins</h3>
    </div>
    <div class="span-10 colborder" id="content">
        <h2 class="loud">Blog Posts</h2>
    </div>
    <div class="span-6 colborder" id="sidebar">
        <div id="twitter_div">
        </div>
    </div>
    <div class="span-6 last" id="sidebar2">
    </div>
</div>

MySQL

Now, the next part was to create a small db to keep the posts and feeds in. This post is not touching on storage, encoding, anything like that. So this is all pretty simple.
SQL:

  DROP TABLE IF EXISTS feeds;
CREATE TABLE feeds (
  id int(10) unsigned auto_increment,
  Title varchar(255) NOT NULL,
  RSS tinytext NOT NULL,
  SiteURL tinytext NOT NULL,
  LastFetched TIMESTAMP NOT NULL,
  PRIMARY KEY(id)
);

  DROP TABLE IF EXISTS feedcontents;
CREATE TABLE feedcontents (
  FeedID int(10) unsigned NOT NULL,
  FeedData blob NOT NULL,
  PRIMARY KEY(FeedID)
);

RSS Reader

I now needed to write a small RSS reader to scrape my feedburner feeds and put them into a user friendly array to serialize and store in the db. I know there are many RSS readers out there, but I love building my own stuff, I get a kick out of it. Why re-invent the wheel? Why not ask every company in the world why they try to compete with others? :) Thats a topic for another post.
So, my RSS reader (still in BETA - its untested in any feed apart from Feedburner feeds, if you come across a bug, please let me know in the comments or twitter, or email, or anything) can be seen here: http://www.fliquidstudios.com/projects/fliquid-rss-library/
Its a very simply script and can be read farly easily. To instigate:

$rss = new FliquidRSS('http://feedurl');
$rss->parseRSS();
var_dump($rss->xmlarray);

Then you can just use the same object for another url, like so;

$rss->newURL('http://feedurl2');

The problem with RSS, is that one post can spread across multiple elements. Its not as easy as just converting your XML to an array, you need to look for specific elements, such as the opening element, then start ’scraping’ the data and then stop when we hit the closing element. So, to do this we simply create a flag ‘initem’ and as long as that flag is set to true, we gather the required data:

foreach ($this->xmlstruct as $element) {
    if ($element['tag'] == 'item' && $element['type'] == 'open') { // We have just opened a new tag
        if ($this->itemcount >= $this->maxitems) break;
            $inItem = TRUE; // Set our 'initem' flag to true
            $this->itemcount++; // increase our item count
        }
        if ($inItem) {
            if ($element['type'] == 'complete') {
                switch ($element['tag']) {
                    case 'title':
                        $this->xmlarray[$this->itemcount]['title'] = $element['value'];
                    break;
                    case 'link':
                        $this->xmlarray[$this->itemcount]['link'] = $element['value'];
                    break;
                    case 'description':
                        $this->xmlarray[$this->itemcount]['description'] = $element['value'];
                    break;
                }
            }
        }
        if ($element['tag'] == 'item' && $element['type'] == 'close') { // We have just closed the tag
            $inItem = FALSE; // No longer in an item, next item will get a new itemcount.
        }
    }
}

So, that creates a multi dimensional array and each post has the following array keys; title, link, description. There are many more attributes to the rss feed but these are the only ones that were relevant to what I was doing. We discard the rest.

Updating our local posts database

We need to make an ‘update’ file to run in a cron job. You can’t have the RSS reader running onload for the display as it takes a while to load (takes a good 8 - 10 seconds to grab my 3 feeds). So we pop the updating code in a file to run every night or every 12 hours or as regular as you want and keep our tables up to date with our latest posts. Keep in mind that you do not want this file in a web accessible directory as malicious users could run the script non stop or do anything else they wanted with it.
The update files code can be seen here: http://www.fliquidstudios.com/projects/christians-aggregator-updatephp/
As we need the array we store to be usable directly from the database, we serialize the array. This means that PHP will convert the array into a storable representation for MySQL. Now, I cheat here and urlencode my serialized array as I was not going to touch on encoding and decoding strings. It does the job but it is not the best solution for this.
As cron’s can be run from anywhere, we can store it anywhere and invoke it like so:
$ php update.php
A note with the updater and the index files, this post is not about databases. I could be creating my database, doing my querying, quoting results etc in a much more involved and pretty fashion, but I am not touching db’s and db objects so I have done the bare minimum for this. We will post on db objects and the best way to query in a later post.

Twitter updates

I could have used the RSS feed from my Twitter account and treated it in a similar fashion to the RSS feeds I was already gathering, but Twitter provide nice widgets in plain text and flash so I decided to use the plain text (javascript powered) version as later on when I apply styling to my page (in another post), I can make it look the way I want it to easily. Grab the widgets here: http://twitter.com/widgets

My display

The PHP for my index can be seen here: http://www.fliquidstudios.com/projects/christians-aggregator-indexphp/
I just have that PHP code nested in my content div in my layout.

My Links

I got so jack of trying to remember my usernames to all the different sites I am a member on when telling people, so I made a bunch of 301 redirects to those pages using Apaches .htaccess file.
redirect 301 /github http://www.github.com/cbiggins
So, when somebody visits cb.net.au/github they will be redirected to my github profile. Also, it means that I can link to that by assigning my href the value of “/github” - too easy.

So, my final result is this; www.cb.net.au

Further Reading:

Blueprint CSS: http://www.blueprintcss.org/
RSS 2.0 Specification: http://cyber.law.harvard.edu/rss/rss.html

Difference form submission in Javascript - follow up


In December 2008 I wrote an article introducing the concept of Difference forum submission in Javascript. We received some positive responses from this article and some requests for examples of exactly how this would work. This article is intended to provide a more practical view of the concept.

What is difference form submission?
The quick explanation of the concept is upon form submission unchanged fields are disabled (using Javascript) to prevent them submitting back to the server. The advantages of doing this can include reduced request sizes and the prevention of “clobbering” data in applications where multiple users are editing the same records.

For the full explanation see the original Difference forum submission in Javascript post.

Example 1
This first example is fairly simple. It includes a single Javascript function that is called when the submit event of the form is fired. The form has an example of every field type (that I could think of) so you can see which fields get submitted and which are disabled. The forms have a method attribute of “get” so for this example you will be able to see the submitted form field values in the URL.

When the submit button is clicked the diff() Javascript function is called and the form object is passed to it. The function then loops over all the elements in the form checking if they have changed by comparing the appropriate properties (value, defaultValue, checked, defaultChecked, selected, defaultSelected). If the elements have not changed they will be disabled.

Note that there are certain field types not listed in the function on purpose including hidden, button and reset. The field types not listed will obviously never be disabled.

View example here.

Example 2 - jQuery plugin
This second example works the same as the first. The difference is that the Javascript has been rewritten so it is now a working jQuery plugin. The plugin currently has an “addFormDiff” method and a “formDiff” function. The “addFormDiff” method is used to bind the formDiff function to the submit event of the form.

View example here.

I will also note that this is the first time I have written a jQuery plugin (or even used jQuery believe it or not) so any feedback relating to the plugin itself is welcomed.

The jQuery plugin will also be stored at http://plugins.jquery.com/project/formdiff

Testing of the code
All of the code above was tested in the latest versions of Internet Explorer 6, Internet Explorer 7, Firefox 3, Konqueror, Safari, Opera and Chrome so it should be compatible with most of the browsers you are likely to need.

What’s next
There are possible scenarios where you would need to group certain fields together (if one changes they both submit). The code does not currently support this as a feature so it is one possible future addition. There may also be other features that could be added to the library or plugin.

That is all for now. If you find this useful or use any of the above code in your projects please let us know by leaving a comment of sending me an email. As usual all thoughts and feedback is appreciated.

Using Yahoo!’s Term Extractor API


Yahoo’s Term Extractor is a application that extracts keywords from a piece of text that you can then use as tags or meta data keywords or anything you please. You can even have it return keywords based on a context, but we’ll stick to the simple implementation today.

To start with, you’ll need a Yahoo account and an App Id. You can grab one of those here; https://developer.yahoo.com/wsregapp/

Now, what we are going to build initially has been built a million times before and can be used at millions of sites across the ‘net. But we’ll tack something onto it which will make it a much more powerful application.
So, lets look at what we need to do for a simple term extraction from Yahoo;

$url = 'http://search.yahooapis.com/ContentAnalysisService/V1/termExtraction';
$appid = 'blahblahblah-getyourown';
$output = 'php';

$context = urlencode('This article is intended to give a basic introduction and a guide to push you in the right direction with your REST server implementation. As such, the examples used in this article are somewhat abstracted from real world uses. If there is demand for it I will write a much more in-depth “ultimate” REST server implementation article at a later date.');

$url = $url . '?appid=' . $appid . '&output=' . $output . '&context=' . $context;

$ch = curl_init();

// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

// grab URL and pass it to the browser
$response = curl_exec($ch);

// close cURL resource, and free up system resources
curl_close($ch);

$response = unserialize($response);

var_dump($response);

So, lets go through the script so far;
The first few variables are simple enough, we have the $url, the $appid and the $output (which in our case is php, so it will return a serialized string).
Next we have the context, this is the string for which to find terms. At the moment it is just a static string, but we will eventually make a form to use this with. We need to make sure we urlencode() the context or we will get errors from Yahoo when it contains spaces.
Now we construct the entire url using the variables we set above.

We will use cURL to make the request, so we need to set that up now. curl_init() will initialise our cURL session and assign it to the variable $ch.
Then we need to set some options within our cURL session using the curl_setopt() function. The options we set are;

  1. CURLOPT_URL - the url of the REST api we are accessing
  2. CURLOPT_HEADER - we set this to 0 so we do not get the HTTP headers included in our results
  3. CURLOPT_POST - So we send the url as POST data, we could also use CURLOPT_HTTPGET to use GET instead.
  4. CURLOPT_RETURNTRANSFER - If we don’t use this, then when we run curl_exec() it would output the results to the browser, this way we can assign the results to a variable for further manipulation

Now we execute the cURL session and assign the results to a variable instead of outputting them to the screen (see #4 CURLOPT_POST above).
Lastly, we can close the curl session and output our unserialized data to the user. If you run this script, you’ll see output similar to below;

array(1) { ["ResultSet"]=> array(1) { ["Result"]=> array(3) { [0]=> string(21) “server implementation” [1]=> string(15) “right direction” [2]=> string(10) “real world” } } }

Great, thats the very basic introductory way of grabbing a bunch of key terms from a static string. Now we’ll make some changes to our script so we can use a URL to parse.

At the top of the script where we have this;
$context = urlencode(’This article is intended to give a basic introduction and a guide to push you in the right direction with your REST server implementation. As such, the examples used in this article are somewhat abstracted from real world uses. If there is demand for it I will write a much more in-depth “ultimate” REST server implementation article at a later date.’);
We are going to change it to this;

$context = urlencode(strip_tags(file_get_contents('http://www.fliquidstudios.com/2009/01/13/introduction-to-writing-a-rest-server-in-php/')));
$context = substr($context, 0, 7000);

What this does is;

  1. uses file_get_contents() to grab all the contents on that page. Text, images, HTML, the lot.
  2. uses strip_tags() to get rid of any HTML from the string.
  3. uses urlencode() to properly format the string for use in the url.
  4. substr() makes sure the string is less than 7000 chars, to keep the size down (I’m not sure what the max is, but 10,000 throws an error).

Then, if you load your page, you’ll see a lot more results in the array returned.

So, now we have parsed a url, grabbed its contents and passed them to Yahoo for term extraction. But its not overly user friendly, who wants to edit code everytime they want to extract terms?

So, now we’ll build a form for it all to give it a bit of a ui and we’ll also display the results in a nicer way;

We’ll wrap our script in an if statement and add our form;

if (array_key_exists('submit', $_REQUEST)) {

    // SCRIPT GOES IN HERE

} else {

    print '
'; print 'url: '; print 'string: '; print '
'; }

We’ll also change the $context variable to use whichever method we have submitted, we want to make sure that even if the variable exists, it also has content, hence the strlen() function use;

    if (array_key_exists('string', $_REQUEST) && strlen($_REQUEST['string']) > 0) {
        $context = urlencode($_POST['string']);
    } else if (array_key_exists('url', $_REQUEST) && strlen($_REQUEST['url']) > 0) {
        $context = urlencode(strip_tags(file_get_contents($_REQUEST['url'])));
    }

Lastly, we’ll rewrite the results in a nice ordered list;

    print '
    '; foreach ($response['ResultSet']['Result'] as $key => $term) { print '
  1. ' . $term . '
  2. '; } print '
';

Beautiful. Now, we have a fully functioning form that we can use Yahoo’s term extraction on for any string or url we want.

As usual, the code for this post can be found here and the demo can be found here.



←Older