Entries Tagged 'Software' ↓

I posted this story on a non-technical message board I frequently visit, as a service for some of the users there who might not have a clue on how easy it is to retrieve data from a supposedly-formatted drive. I decided to pass it along here as well. I think privacy is very important, and with the proliferation of electronic devices that store data, it’s getting easier to retrieve information from others.

Here’s a story of some dumb-ass kids who recorded themselves smoking weed on a digital camcorder, returned the camcorder to the store, forgot to erase their tapings (most likely due to said kids smoking weed previously) and the next person who bought the open-box camera from the story posted the videos all over the Internet for all the world to see:

Remember to delete your stuff from electronic equipment if you’re returning it, selling it or giving it away. And even then, be wary of it.

I was once sold a USB flash drive from a friend, and although my buddy deleted the contents, I was able to see what he had their previously before with a freely-available program off the Internet (don’t remember exactly which one right now), out of curiosity. Dude had some… let’s say ‘interesting’ pics of his then-girlfriend.

His way of deleting the contents was to simply do a quick format on the drive from his Windows machine before giving it to me. To avoid all the technical details, for those who don’t know, what this does is simply destroy the FAT table and/or boot sector, which is a sort of ‘table of contents’ for the drive, thus nothing appears when you try to access the drive. But the data is still in the drive’s sectors, and unless you rewrite those sectors (either by copying something new in the flash drive or using some software, which I’ll get to in a minute), they’re easily accessible.

The program I’ve used for a while with Windows is called Eraser (I carry a portable version of this program on my USB drive, called Eraser Portable), which makes sure your data is clear from your portable device (like flash drives, SD cards for digital cameras, even iPods). In short, what this does is over-write the sectors on the drive multiple times with ‘garbage’, so it’ll be virtually impossible to get that information from freely-available tools. I’m guessing the government has more advanced tools, so for the overtly paranoid, you’d be better off just smashing the drive to pieces and dipping them in a vat of acid.

Just wanted to pass this info along so you people can keep your privacy, and know how easy it is to grab a hold of your data.

Over the past week and a half, I’ve been busting my ass trying to learn the ropes of Behavior-Driven Development with RSpec. I had been interested in learning either Test-Driven Development or Behavior-Driven Development for quite a while, so I just jumped on one of the two and test the waters out for a while.

I’ve been reading a lot on TDD and BDD for the past couple of months, yet never took the time to implement it to my current projects. The main reason is because I thought it would take me too much time to learn and would bog me down too much, when all I wanted was a workable application running as soon as possible. But, an opportunity presented itself (more on that in the future), so I chose the development technique that seemed better for me: BDD.

Reading about RSpec through other blogs on the Internet, it just seemed so intuitive to use that for testing, I kinda thought it was too good to be true. So after installing RSpec and getting it running on my application, I realized it was true after all. It’s ridiculously easy to write your own tests with RSpec, and it’s almost plain English (just like good ol’ Ruby). It includes built-in stubs and mocks to simulate parts of code, so you can even drop fixtures if you want (although admittedly, a lot of the shortcomings fixtures had were fixed with Rails 2.0). In fact, here’s a simple controller test:

describe ProductsController do

  before(:each) do
    @product = mock_model(Product, :to_param => "1")
    Product.stub!(:find).and_return(@product)
  end

  it "should call the action successfully" do
    get :show, :id => "1"
    response.should be_success
  end

  it "should find the product" do
    Product.should_receive(:find).with("1").and_return(@product)
    get :show, :id => "1"
  end

  it "should assign the found product for the view" do
    get :show, :id => "1"
    assigns[:product].should eql(@product)
  end

end

This test actually works for testing the show action in a controller. I wrote this just to show how readable the code is. I never knew testing would be so readable! And like I said, once you wrap your head around mocks and stubs, it’s ridiculously easy to implement. As a great bonus, RSpec can also integrate with rcov, a tool that actually checks what parts of your code are tested. It’s a great tool for making sure you achieve 100% coverage of your code.

Unfortunately, I didn’t actually do any BDD this time around, as I already had a good chunk of my application up and running already. But now that I have discovered the joys of RSpec, all my upcoming products will be fully done using BDD techniques: I won’t write a line of code until I write the behavioural test before.

A couple of ceveats, though. Hey, nothing’s perfect! First off, if you use different types of plugins for your application, you’ll find some parts of it difficult to test in RSpec. Most plugins use the Ruby’s own Test::Unit testing framework to test their own functionality, but parts in your code where you use these plugins can get you the first time around. Also, if you’re like me, and already have written code for your app, you’ll start finding different ways to fix your code to conform to RSpec testing. This isn’t necessarily a bad thing, as this is most likely a warning sign that you need to refactor that piece of your code anyway. But it’s takes a bit of your time away, so be prepared for this.

In short, RSpec is so easy to use, I don’t see why people shouldn’t use it in their own projects. I suggest every Rails developer should look into RSpec (or any other testing method, really) and stop being lazy. I know I was, but never more!

RSpec References

Ryan Bates from Railscasts has a great screencast that can serve as an intro to testing controllers. And the best part is that it’s free!

* Testing Controllers - http://railscasts.com/episodes/71

Geoffrey Grosenbach over at PeepCode has an excellent three-part series on RSpec, with about three hours of RSpec goodness! No, they’re not free, but the $9.00 per episode is truly worth it. This was my secret weapon in learning RSpec in less than two weeks - The best 27 bucks I’ve spent in my software development career!

* RSpec Basics - http://peepcode.com/products/rspec-basics
* RSpec Mocks and Models - http://peepcode.com/products/rspec-mocks-and-models
* RSpec Controllers and Tools - http://peepcode.com/products/rspec-controllers-and-tools

I’ve been meaning to write about the subject of Source Code Management for a while now. The reason for this is that I’m very surprised about the amount of software developers I personally know who don’t use any type of Source Code Management tool at all. The main reason of this is because these people don’t even know such a tool exists. The very few who do know about these tools just know how it works, but never use it themselves. It’s shocking, to say the least.

When I got this VPS, I immediately installed Subversion for my own personal projects, even though I’m the only one working on said projects. This is where those very few people I mentioned above think wrongly about these tools. Their basic response was “If you’re the only one working on the code, why do need to share it?” Very big misconception, to say the least.

The reason I originally started learning about Source Code Management tools (first started with CVS, then move to Subversion, where I have stayed ever since) because there was an additional programmer in my company who was beginning to modify the software where only I had previously worked on before. So yes, I used SCM tools to ’share’ code and make sure our changed didn’t conflict with each other. But then I realized that I was only using probably less than 5% of its true power.

When I did a major screw-up and pushed some bad code into the repository and then into production by mistake (it was actually incomplete code I had mistakenly committed to the repository), I was grateful that I could roll back almost immediately. Then when I needed to have a separate copy of the code, I learned the wonders of branches and tags.

After these experiences, I wondered how I ever did any serious programming work without any Source Code Management. Right now, whenever I start a new project, I immediately create the basic folder structure for the SCM before doing anything else. I think it’s crazy to go forward with a new project without any SCM in place, which makes the fact that a lot of programmers aren’t using any SCM just insane.

For those of you not using any SCM right now, shame on you! Allow me to redirect you to a couple of interesting articles for my preferred SCM tool of choice, Subversion. Please note, there are a lot of different SCM options out there, both commercial and open-source. I would recommend doing some research first on these different tools before deciding to go with one.

Subversion Book - Version Control With Subversion - Excellent resource not only for finding out how Subversion works, but how SCM tools work in general.
Software Branching And Parallel Universes - Great blog post from Coding Horror that explains how branching works visually, along with some reasons when not to use branching.

http://www.codinghorror.com/blog/archives/000968.html

The past two weeks I’ve been totally separated from all of my learning and reading processes I’ve established to myself, and I felt terrible for doing that. It’s not like I’ve been totally disconnected from everything. I’ve still read all my favorite programming-related blogs, as usual. Still, I haven’t just sat down to absorb everything or practice.

These past two days I decided to get back on track. However, I found myself with the same problem I’ve had for a while now. Whenever I sit down to learn something, I want to learn everything. I don’t mean “learn everything of something“. It’s more like “learn something from everything“.

For the past months, I’ve had a hundred different interests. I’m interested in learning Adobe AIR. Microsoft Silverlight sounds like something I could use in the future. I was sold on Test-Driven Development and ever started to adopt its practices to my daily usage, yet recently I’m liking the sound of Behavior-Driven Development more and more and would like to test that road. I want to learn other web frameworks in different programming languages, like Django and CakePHP. And of course, I’m still totally into Ruby and Rails.

I know a lot of people who don’t mind knowing about everything. But the deal is that by learning (or having the desire to learn) about so many things, you don’t fully learn it all. You only learn a bit about each thing, but never a whole lot. There simply isn’t enough time in the world to do so, especially with a full-time schedule. So to say it’s frustrating is an understatement to me.

One technique I’m finding useful to juggle these interests is to create a necessary project in your mind to learn along the way. This should keep your interests level high during the learning process, while making you stay focused with one or two things at a time. For example, my next project is a web application, where I’ll strictly try to use the practices of Behavior-Driven Development while learning to use RSpec. This way, I’ll be learning a lot of different, yet related, subjects at the same time. After this project is done, I’ll see where Adobe AIR has headed, since it’s still in Beta. If it still piques my interest, I’ll create a new project for one of my current needs, and learn from there.

Being in this world of programming and technology, where it seems like time is always on fast-forward, it can be tough to keep up with what you like. But at the same time, it’s just fun. Even though I get frustrated at times, I’m having a ball learning these things. I guess it all boils down to that for now.

One of the reasons I went exploring into Rails was because of Capistrano, a utility that greatly helps deploying Rails sites into production servers, by automating many of the tedious setup steps needed to deploy new changes into production. I know I can’t be the only one who has once or twice pushed some new change into production, only to discover (by myself or an angry user) that I forgot to bring the database schema up-to-date as well. Capistrano (actually, migrations are the key component here) will never allow that to happen again.

In a nutshell, Capistrano does the following:

• Logs into your server via SSH
• It creates a directory structure that’s useful in case you want to rollback some bad code you mistakenly pushed unto the server
• Uses Subversion (or other SCM) to checkout the latest code committed into the repository and downloads it
• Automatically runs all migrations to make sure the database is up-to-date
• Runs other scripts, like making sure your FastCGI processes are restarted and running correctly, or restarting your web server

After nearly finishing my first public Rails site (coming soon, I promise!), I wanted to learn how to use this tool by deploying the first version into my VPN space using Capistrano. I thought this would be a daunting task, and at the beginning it was thanks to some minor errors, but after that, it was total bliss. After a few hours of tweaking my settings, I finally got it to work, and all deployments from here on out should be as simple as writing cap deploy without and remorse.

I’m going to write how I set up my deployment environment, so if anyone has had similar problems to mine, they can hopefully get past them.

First off, let me write about my app and server environment:

• Operating System: CentOS 4.5
• Webserver: Lighttpd 1.4.15
• Rails Version: 1.2.3
• Mongrel: 1.0.1
• Capistrano: 2.0.0

After installing Capistrano on my development computer (simply using gem install capistrano --include-dependencies), I was ready to “capify” my application. To create the files used for deployment, just issue capify . at the root directory of the Rails application. This creates two files: Capify, which points to the second file, config/deploy.rb, which is the actual deployment configuration file.

The configuration is pretty straight-forward. There are some default settings that can e easily changed to reflect your production server setup. I did just that, and that’s where the ‘gotchas’ started pouring in.

After I changed the default values to my own, I wanted to set up the directory structure in my production server. To do that, simply run cap deploy:setup in the root directory of your application. That should prompt you for your SSH password to create all the directories needed in the directory specified in the deployment file (using the :deploy_to variable). However, when I did that, I got a nice error message: no such file to load — openssl.

After searching a few minutes in Google, I found my problem: I had compiled Ruby from source without the openssl-devel libraries installed in my system. Without the header files, Ruby compiled without OpenSSL support. So after installing the OpenSSL header files and recompiling Ruby (don’t forget to run make clean before recompiling), I was faced with another error message: It stated that my server didn’t exist. Then I remembered I’m running SSH in a non-standard port. Capistrano assumes it’s running on the default port, which is 22. After a few more minutes of searching, I found an option that needed to be added to the deployment file: ssh_options[:port] = xx, where xx is your SSH port number. After these changes, I was golden, as Capistrano asked for my SSH password.

After entering it in and seeing some progress in the directory creation process, I was faced with yet another error message, about a user not existing. I was assuming Capistrano was using the user name from my development box to log into the production server. In any case, this was fixed by adding another option in the deployment file: set :user, "production_user", where production_user is the user name with the appropriate permissions to create the directories and files in the production server. I ran cap deploy:setup once more, and all my directories were created. Success! Little did I know that would be only the first steps, and more troubles were looming ahead.

Once I verified the directory structure was created correctly on my production server, I went ahead and ran cap deploy:cold, which deploys my latest working version to the server, runs all migrations, updates all symlinks to the current code, and runs all remaining processes, like respawning all FastCGI processes, for the very first time. I once again ran into a small snag, as I was having permission problems running some scripts on the production server. After some more minutes of searching, I found that there’s a variable that needs to be set to make sure Capistrano runs the scripts as a specific user with adequate permissions. After adding set :runner, "production_user" (once again, where production_user is the user with the correct permissions to run your application scripts) to my deployment file, I was able to pass the permission parts, but then I hit yet another snag: I was missing a file - script/spin.

I found it odd that Capistrano was looking for this file, as it’s not automatically generated either by Rails or Capistrano. But after calmly reading the Capistrano installation instructions (instead of skimming over most of it), I saw that this file is used to recreate (or create) the FastCGI processes in your production server, to ensure that the users will get served the latest version of your app. There are many different ways to set up your FastCGI processes, depending on what the web server you’ll use. Since I use Lighttpd, I’ll be writing about that here. But you can find tons of useful information on the Internet if you use Apache, nginx or any other web server.

To remedy this problem, all I needed to do was to create the script/spin file (with executable permissions - chmod 0755 script/spin) with the following line (where /root_of_app/ is the path you described in the set :deploy_to: variable in the deployment file):

/root_of_app/current/script/process/spawner -a 127.0.0.1 -i 3 -r 5

This script calls another script called spawner (included in current versions of Rails), which verifies if there are FastCGI processes currently running. If the processes exist, they’re recreated to show the new version of the app. If the processes don’t exist, they’re created. The -a switch indicates the IP address used to direct the FastCGI processes. If you don’t use this switch, it will default to 0.0.0.0, which was causing me problems later on. The -i switch tells the script to create three FastCGI processes in sequential ports. Finally, the -r switch tells the script to verify if these scripts are still active every five seconds. This makes sure that all processes are running smoothly. One switch I didn’t use was the -p switch. By default, the spawner script creates all FastCGI processes starting with port 8000. Using the -p switch, you can specify which is the first port. In my case, the three FastCGI processes are creates using ports 8000, 8001 and 8002. You can change that default if you wish.

After you create the spin script, you’ll need to commit it to your SCM so Capistrano can find it in the production server. Once committed, I re-ran the cap deploy:cold command, and I was greeted with success at the end. My latest version of the application code was sent to the server, all migrations ran, and the spin script created three FastCGI processes on my server. Awesome! My work here with Capistrano was done. After hating Capistrano for a good while, I fixed all the kinks and can now never live without it. I love you, Capistrano.

Feeling good for myself, I immediately fired up my browser and entered my site’s URL. Too bad only 500 - Internal Server Error appeared when I went to the site. Curious, I entered the URL once again, appending :8000 at the end of the URL, and lo-and-behold, the site appeared in all its glory. So the FastCGI processes created with Mongrel were working well. But my web server wasn’t transferring the requests to one of the three processes.

After looking around for more information, I saw how FastCGI processes, Mongrel and Lighttpd work together. In a nutshell, the request for the site is sent to Lighttpd, the web server. Lighty then needs to process this request and send it over to one of the FastCGI processes, which then displays the site on the user’s screen. Lighttpd is simply used in this case as a proxy, and lucky for me, it already has some basic proxy functionality built-in. However, it needs to send the the request somewhere. I saw some tutorials online that set this up, but it seemed to always send the request to only one of the three processes, which wasn’t efficient at all.

Here is where Pound comes into play. Pound is reverse-proxy and load balancer for web servers. Basically, it takes all requests from the web servers and passes it along to the processes running the site, making sure that all processes aren’t over-worked by load-balancing all requests. After installing Pound on my production server, I had to create a configuration file, by default stored in /usr/local/etc/pound.cfg (your location may vary, depending if you compiled and installed the program from source, or just installed a package):


ListenHTTP
Address 127.0.0.1
Port 7999
Service
HeadRequire "Host: .*site.com.*"
BackEnd
Address 127.0.0.1
Port 8000
End
BackEnd
Address 127.0.0.1
Port 8001
End
BackEnd
Address 127.0.0.1
Port 8002
End
End
End


This configuration will make Pound listen to the requests on port 7999 in the local machine (my production server), and forward the site’s requests to one of the three FastCGI processes created by the spawner script I talked about previously. I was surprised at how something so powerful could be easily implemented.

Now all I needed to do was to configure my web server to direct all site requests to Pound, which in turn passed them along to one of the three FastCGI processes. Skipping all the other default settings and changing all sensitive info that may compromise my site, here are my current Lighttpd settings for the site in question:


$HTTP["host"] =~ “(^|.)site.com$” {
server.document-root = “/home/production_user/railsapps/app_name/current/public”
server.error-handler-404 = “/dispatch.fcgi”
server.errorlog = “/var/log/lighttpd/site.error”
accesslog.filename = “/var/log/lighttpd/site.access”
proxy.server = ( “” => ( “site” => ( “host” => “127.0.0.1″ , “port” => 7999, “check-local” => “disable” )))
)


Make sure you load the mod_proxy server module so you can use the proxy.server option mentioned above.

Once I restarted Lighttpd, I entered my site’s URL, crossed my fingers, and… success! I finally had a working site, load-balanced and all. What set out to be a learning process in Capistrano in turn made me use load balancing techniques in my site, which was something I planned on doing, but on another day, thinking it was super-complicated.

From here on out, every time I make a change I want to push to my site, all I need to do is run cap deploy on my development box, and that’s it. Everything will be updated with a simple command. It’s truly worth the time I spent getting it to work. Now I will never have an angry user again because I forgot to update the database schema.

In all, I spent a few hours fixing all the small kinks I encountered along the way. But as all good things go, you need to bust your ass to get things working like you want to. I don’t mind at all, as I learned a whole lot in one day. I hope someone finds some solutions in this writeup.

In case you’re curious, here’s my deployment file, with all the sensitive info changed for obvious reasons:


# Application Name - Anything you want to describe your application
set :application, "app_name"
# The URL of your source code repository, pointing to the latest version
set :repository, "http://svn_repo/trunk"
# Set the user name to connect to the server via SSH
set :user, "production_user"
# Set the user name of the user with permissions to run the application scripts
set :runner, "production_user"
# Set the path where you want your application to be stored
set :deploy_to, "/home/production_user/railsapps/#{application}"
# Option to change the SSH port
ssh_options[:port] = xx
# The URL or IP Address where your application will be stored - Multiple sites can be specified
role :app, “xx.xx.xx.xx”
# The URL or IP Address where your application will be served - Multiple sites can be specified
role :web, “xx.xx.xx.xx”
# The URL or IP Address where your database lives - Multiple sites can be specified
role :db, “xx.xx.xx.xx”, :primary => true
# Task to restart the web server
task :restart_web_server, :roles => :web do
sudo “/etc/init.d/lighttpd restart”
end
# Restart the web server once the deployment is finished
after “deploy:start”, :restart_web_server


During these past months, as I’ve been learning Rails and getting my feet more and more wet with all the integrated AJAX goodness that’s inside Rails, I’ve found it increasingly frustrating to debug erroneous AJAX code. With regular web development, most of the times you can plainly see if you have an error in your code. Sometimes the results are incorrect, or some aspect of the page is displayed wrong, and other times there’s a big, fat error page letting you know something is seriously wrong. However, with AJAX, whenever erroneous code is executed, most of the times you get nothing. The page stays the same, nothing changes, except that feeling of doom inside the pit of your stomach. Then it’s time to hit the code again, trying to see if something’s wrong.

Luckily for us, there’s a very awesome tool made to debug these types of errors quickly and efficiently. Firebug is simply a godsend for anyone who works with web development of any kind. It can help you debug your web apps so easily, this is certainly one tool that I don’t know how I lived without these past couple of years. It’s a Firefox extension, so you need the Firebox browser installed to use it, no matter if it’s on Windows, Linux or Mac. But I wonder, why the hell wouldn’t you not have Firefox installed in your computer already? That’s a no-brainer.

Firebug helps you debug HTML, CSS and JavaScript. Since it’s a browser extension, it does all the debugging right in the browser, while you’re executing your page. You don’t need to load any other tool, it’s all at the touch of your fingertips. It can be used with any programming language. I’ve used it while developing in PHP, Python with Django and, of course, Ruby on Rails. Not only can you debug the code, you can even make Firebug do some neat things, like edit the HTML, CSS or Javascript code directly in the browser to see the changes immediately, and it even has a profiler that can tell you where your site is slow.

Oh, I forgot to mention the best part of Firebug: It’s 100% free. You just have to love Open Source projects. It’s still difficult to believe that people out there can put out such a quality and life-saving tool for everyone’s use without asking for a single cent in return. That’s the beauty of Open Source. I don’t know if there are any commercial tools that do what Firebug does, but I wouldn’t trade this tool for anything.

If you’re a web developer, and haven’t heard of this awesome tool, I suggest you click the link I set above for Firebug and get it as soon as possible. Unless you like torturing yourself with debugging the old-fashioned way.

It’s been ages since I last updated this blog. I’ve been pretty busy for the past month or so, ut it’s due to something that I didn’t want to do in the first place. Let me tell you a quick story about it.

Earlier this year, I convinced my boss to rewrite our current (and, admittedly, buggy) software from PHP to Ruby on Rails, which would be better suited for the job at hand. Since the boss wants everything to be done fast, good and cheap, obliterating the Project Triangle, then I thought that Rails would do well. I could complete the rewriting task fast (thanks to Rails and its plugins doing almost anything with ease), good (thanks to me starting to love Test-Driven Development) and cheap (less time coding and fixing bugs = more time to implement new features). It seemed like a good fit, and he agreed.

However, a few months later, he hired someone to help me with the programming and database tasks. I desperately needed help, so I agreed to this. The person he hired, a young woman with a Masters Degree in Computer Science who teaches at a local community college, then proceeded to evaluate the current system. After proposing various changes, the boss decided that since so much work needed to be done, he might as well profit out of it, so he wanted to have the program rewritten not only for our internal use, but for commercializing the program as well. His language of choice, due to recommendations from the new employee? Visual Basic.NET. Ugh.

I pleaded with him to reconsider, but his mind was set. He wanted to transfer a PHP web application into a Windows-only desktop app. And his understanding was that since Visual Basic.NET has a nice little form designer, it would be much faster to rewrite the program. I countered these points, but he didn’t want to listen to it. He wanted a Windows application he could sell, and the only programming language the other employee knew enough was Visual Basic.NET (yep, even with a Masters Degree in Computer Science, that was her strong point). I was, simply put, screwed.

I have absolutely nothing against Microsoft tools. In fact, I think they’re great. I even suggested using C# instead, as it would be a more robust platform to do this type of project. It’s simply that I think Visual Basic is something that I used in college to learn the ropes of programming, and nothing more. I ditched Visual Basic the first chance I got, immediately after I graduated from college. I hadn’t used Visual Basic for anything since late 2004. I was surely going to need some time to get up to speed in mid-2007. It’s been a while, but I’ve gotten to a point where I can program once more using this language, even if I don’t want to.

However, from my last conversations with my boss, it seems that he doesn’t want me to program much, since I had told him I was way too rusty with Visual Basic. He seems to want me to manage the entire project. I accepted, as it’s more of a challenge than to sit down and develop an application with a programming language I simply wouldn’t use if it were my project.

So, for the past couple of weeks, I’ve been getting my head into .NET programming. While I still am against choosing Visual Basic.NET as the programming language of a rather large software development project, I’ve been entertained with some things that I hadn’t used in a large-scale project with PHP. I’ve learned a lot about Continuous Integration for the automatic build and testing process. I’ve learned how to use NAnt enough to build and test my projects automatically. I’ve even learned the finer points of creating installers that will work on all systems, no matter the dependencies needed. It’s been a change of pace for me, but I’m making the best of it.

Still, I’m very disappointed in my job and this decision that was taken. My heart is with open-source and web-development projects. Sadly, I need to do that on my own time now, but most of my spare time is used learning these new tools I need for .NET development. I’m still looking for work outside of Puerto Rico (I’m 0 for 2 with companies that have contacted me after sending in my resume), so hopefully this changes soon. In any case, I’m making the best out of this situation, and at least I’m learning new things once again. But one day, I can achieve my dream and be part of a team where I know my heart is.

A couple of weeks ago, I purchased the book “Dreaming In Code” by Scott Rosenberg, after spotting it in my nearest Borders bookstore. I had read a bit about this book from Joel Spolsky’s site, and I was very interested in the process of building software that would be “revolutionary”. I was also interested in how the group of developers would work.

I finished reading the book this weekend, and I have to say that it was extremely interesting from the beginning. From the initial idea of the software to the formation of the group, to the many setbacks and would be inevitably encountered, it offered me a great inside look at the group effort in producing software. As I’m already accustomed of working on my own, I wanted to see how larger groups of talented people could (and should) work. And surprisingly enough, the same things that happen to me at work happened to the Chandler developers as well.

What surprised me about this book is the fact that it goes back in time, and mentioned many breakthrough moments of software engineering history, like the 1968 NATO Software Engineering Conference, and Douglas Engelbart’s legendary demonstration in San Francisco (called by some as “The Mother Of All Demos”).

These moments, and many others, were non-existant to me before reading this book. Anyone who has a Computer Science or Software Engineering degree should be required to read this. It’s a very important part of the history of what we do. Like any History major who needs to know about Ancient Rome and Greece, people in the software development field need to know about what happened during the infancy of computers and programming.

While the story of Chandler’s development is a great read and what attracted me to this book, it was refreshing to read about the current problems in software development today. I know that this industry is far from perfect. But the constant changes that occur on a frequent basis simply show that we’re trying to make things better. We need to remember that compared to other scientific and engineering professions, computer science / software engineering is still a baby, an extremely-young 60 year old baby.

I hope that someday things will change for the better in our industry, and the software development process can be a painless, quick one. However, I’m afraid that the way the future generation are being taught the profession, most likely the same mistakes will be repeated over and over again. Like I said, the industry - or better yet, we - are not perfect at all. But we need to learn from our past mistakes. Certainly, those who cannot remember the past are condemned to repeat it.

Note: This site has changed a lot over the last year. I’m not using Rails anymore on the site, so there’s no place where I’m currently using the Digg API. However, I’ll leave this post intact, as it might help others with something similar.

If any of you readers have come in through my main site lately, you should notice that I added a little something on the sidebar. One of my favorite time-wasting activities of the day is browsing Digg. I tell you, there should be some sort of “Diggers Anonymous” for us simple-minded folk who can’t stop from visiting the site for stories many times a day!

Anyway, they recently announced the release of the Digg API, to allow any user to access the stories on the site in any way, shape or form we desire. Now, I must say that I have no real use for this. Of course, being the geek I am, I just wanted to try it out and see how I could use a simple API using Ruby, for use on my Rails site (this one). This is also made in case anyone else wants to do the same. One note, however: I’m only a Ruby / Rails beginner. I’m sure there are much better and efficient ways to do this. Since I haven’t dabbled in this too much, and “it works on my machine“, I’ll leave it as it is for now.

Just for the record, the testing and development for this code was made on my computer running Ubuntu 7.04, along with Ruby 1.8.6 (compiled by myself, not downloaded from the Ubuntu repos), RubyGems 0.9.2 and Rails 1.2.3. It should work correctly with any fairly recent version of the software mentioned above.

The Digg API is rather simplistic right now. All you need is a simple request using a URL like this:

http://services.digg.com/stories?appkey=http%3A%2F%2Fdennmart.com&type=xml&count=5

In the example above, the API is called to the ‘http://services.digg.com’ server. The parameters afterwards (in this case, ‘/stories‘) is the request. You can add more parameters for a finer-grained search.

Once the request URL is complete, you need some additional actions to add as well. The ‘appkey‘ is required, but can be anything for now. Digg isn’t generating application keys (a la eBay or other external API’s), but an appkey is required for ’statistical purposes’, according to the documentation. The ‘type‘ is how the data is going to be returned to the app, and it can be either XML, Javascript, JSON or PHP. I use good ol’ XML for now, until I can actually play around further with the other response types. Finally, the ‘count’ action is to limit how many stories are returned. I’m only going over these options quickly, as the API documentation has much more information.

Once you figure out which request you want to make (using Mozilla Firefox can help greatly, as the XML response is nicely formatted), it’s a matter of getting that data into your Ruby or Rails app. Going back to the ol’ trusty Pickaxe book, I found a nice little module integrated in Ruby called open-uri. This module allows the Ruby application to open a URL (either http, https or ftp) and get the returned contents. To use this module, a simple line of code is needed:

require 'open-uri'

That should load your module correctly. Now it’s just a matter of having Ruby open the API connection and store its XML response in a variable:

response = open('http://services.digg.com/stories/popular?appkey=http%3A%2F%2Fdennmart.com&type=xml&count=5').read

The ‘open‘ function, well, opens the connection to the API, while the ‘read‘ method gets the data that’s returned from the URL. Like I said, rather simple.

However, I was getting timeout errors when calling the ‘open’ function. Strangely enough, the function worked fine when using any other URL. Upon further reading of the API documentation, I found this little tidbit:

“All API requests must include a User-Agent HTTP Header. A request without this header will receive no response.”

So, after that small mistake, I edited the request like so:

response = open('http://services.digg.com/stories/popular?appkey=http%3A%2F%2Fdennmart.com&type=xml&count=5', 'User-Agent' => 'Ruby/1.8.6').read

The ‘User-Agent‘ parameter can be anything. I just decided to use the Ruby version I have. Once I added that, I had my nice XML with the five most recent Digg stories that have been promoted to the front page.

After that, I needed to parse that XML. I searched around the Internet, and found a great little module called ‘XmlSimple‘. This module reads and writes XML, and formats it according to whatever’s needed. In my case, I needed to read the XML response. Just like the ‘open-url’ module previously, you need to load the ‘XmlSimple’ module as well, once installed (”gem install xml-simple“):

require 'xmlsimple'

I did run into some minor problems when loading this module. The Ruby interpreter cried out loud, saying it couldn’t load the module. How come? I verified that the gem was installed correctly, and it was. Then I realized that since this is a gem, I need to have ‘RubyGems‘ loaded before loading ‘xmlsimple’:

require 'rubygems'

I believe that Rails already loads the module for you. But if you’re testing this out on Ruby and not on Rails, you’ll need it.

Okay, once I did that, I was able to load the ‘xmlsimple’ module. Now I need it to parse the XML response that I stored in the aptly-named ‘response‘ variable. A simple line of code can convert the XML into a hash:

XmlSimple.xml_in(response)

That takes the entire XML string and puts it into a neatly organized hash. I really don’t need everything in the hash, just the story title, link and how many diggs the story has. So I just access those particular keys:

digg_hash = XmlSimple.xml_in(response)
story_title = digg_hash['story'][0]['title']
story_link = digg_hash['story'][0]['link']
story_diggs = digg_hash['story'][0]['diggs']

In the example above, the ‘story_title‘ variable has the most recent story title, the ‘story_link‘ is the link that takes you directly to the story, and the ‘story_diggs‘ has the current amount of diggs that story has. The number zero used in the array is the story number. If you want to get more than one, you’ll need to loop through the array and get the other stories.

The code I’m using on the main site right now is the following:

<%
  require 'open-uri'
  require 'rubygems'
  require 'xmlsimple'

  response = open('http://services.digg.com/stories/popular?appkey=http%3A%2F%2Fdennmart.com&type=xml&count=5', 'User-Agent' => 'Ruby/1.8.6').read
  articles = 0
  while articles < 5
%>
<%= XmlSimple.xml_in(response)['story'][articles]['title'] %>
(Dugg <%= XmlSimple.xml_in(response)['story'][articles]['diggs'] %> times)


<%
  articles += 1
  end
%>

In all, this was rather simple, and I’m sure there will be a much more creative use for the Digg API soon. But for those who want to learn how to use it, or those who need a very simple approach, this works just fine. I’m interested in listening on how other people have put this to use. Hope this helps someone!

Note: I noticed that this is the most-viewed in all my blog, thanks to this appearing in Google search results when people search for “Puerto Rican Porn”. Sadly to say, there is absolutely no porn on this site, so if that was your reason of coming here, you’ll be disappointed. However, if you do enjoy reading about technology, feel free to browse any of my recent posts.

Wow, I really can’t believe that this past week, my little ol’ blog has been bombarded with tons of spam comments. And this is a blog that barely has any readers at all. I can only imagine the amount of spam the more popular blogs on the Internet. Really, I don’t want to know where I can get she-male porn videos, or female ejaculation pictures. I’m not an avid pornographer, and rarely scour the Web for that stuff (note that I said ‘rarely‘ - at least I’m honest). But if I’m feeling a bit freaky, I’ll know where to go. It’s not like that stuff is totally hidden.

Anyway, I wanted to mention the Akismet Plugin that’s currently included in all Wordpress installations by default. After I got the first couple of spam messages, I decided it was time to activate the plugin to see how it works. After a couple of days, the plugin has captured more than 30 spam comments, and none have gone through. Pretty good job, if you ask me.

So, if you have a Wordpress blog, I really advise on activing the plugin. It only takes a few minutes, and you’ll be grateful once you stop getting offers for bestiality flicks. If you don’t use Wordpress, Akismet makes plugins for more than 20 different platforms, so you can check if your application is supported. It’s really a great plugin that should be activated by default on any blog or CMS app.