Building a Rails Gmail Client Outside-In

New Article: Building a Rails Gmail Client Outside-In/

For all the new projects that I start with my stakeholders I have been pushing Outside–in software development and Specification by example. Specification by Example really improves the collabora…


2012 Whiskey Row Marathon

On May 5th my wife and I did the 2012 Whiskey Row Marathon 10k. It was very challenging and a lot of fun. The Proceeds from this race go directly into the scholarship fund at the Prescott YMCA to assist children and families in financial need to participate in child care, swimming lessons, youth sports and gymnastics classes. Prescott is my home town and most of my family still lives up there so it was great to do the run and spend time with the family. This was my wife’s first race and she did amazing.

After the race we spent some quality time at PBC enjoying some great beer. My dad also hooked me up with a pretty cool solar lite bottle opener from a local Prescott company called Solarfunstuff.

Our next run will be the 2012 Hospice Run For Life in Flagstaff, AZ on June 28. At 7,000 feet (2,130 m) elevation it should prove to be another challenging and exciting run.


Rendering Partials with Sinatra

Simple way to render partials with Sinatra:
https://gist.github.com/119874#gistcomment-238742


Ruby 1.9.3 IMAP segfault with OpenSSL

After upgrading to Ruby 1.9.3 I spent most of the morning trying to figure out why my IMAP code was giving a segfault

..
1.9.3p0 :001 > require 'net/imap'
 => true 
1.9.3p0 :002 > m = Net::IMAP.new('imap.gmail.com', 993, true, nil, false)
/Users/carlos/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/net/imap.rb:1439: [BUG] Segmentation fault
ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-darwin11.3.0]

-- Control frame information -----------------------------------------------
c:0028 p:---- s:0108 b:0108 l:000107 d:000107 CFUNC  :connect
c:0027 p:0198 s:0105 b:0105 l:000104 d:000104 METHOD /Users/carlos/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/net/imap.rb:1439
c:0026 p:0172 s:0100 b:0100 l:000099 d:000099 METHOD /Users/carlos/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/net/imap.rb:1036
c:0025 p:---- s:0091 b:0091 l:000090 d:000090 FINISH
c:0024 p:---- s:0089 b:0089 l:000088 d:000088 CFUNC  :new
c:0023 p:0023 s:0083 b:0083 l:001b28 d:001470 EVAL   (irb):2
c:0022 p:---- s:0080 b:0080 l:000079 d:000079 FINISH
c:0021 p:---- s:0078 b:0078 l:000077 d:000077 CFUNC  :eval
..

Turns out it was related to OpenSSL and the fact that I forgot to install the Ruby OpenSSL extensions. Chris Irish’s post reminded me of the requirement.

$ rvm remove ruby-1.9.3
$ rvm install ruby-1.9.3 --with-openssl-dir=/opt/local --with-iconv-dir=$rvm_path/usr

..

1.9.3-p0 :003 > m = Net::IMAP.new('imap.gmail.com', 993, true, nil, false)
 => #<Net::IMAP:0x007f808500cc30 @mon_owner=nil, @mon_count=0, @mon_mutex=#, @host="imap.gmail.com", @port=993, @tag_prefix="RUBY", @tagno=0, @parser=#, @sock=#, @usessl=true, @responses={}, @tagged_responses={}, @response_handlers=[], @tagged_response_arrival=#<MonitorMixin::ConditionVariable:0x007f808500c280 @monitor=#, @cond=#<ConditionVariable:0x007f808500c258 @waiters=[], @waiters_mutex=#>>, @continuation_request_arrival=#<MonitorMixin::ConditionVariable:0x007f808312c1d0 @monitor=#, @cond=#<ConditionVariable:0x007f808312c1a8 @waiters=[], @waiters_mutex=#>>, @idle_done_cond=nil, @logout_command_tag=nil, @debug_output_bol=true, @exception=nil, @greeting=#<struct Net::IMAP::UntaggedResponse name="OK", data=#, raw_data="* OK Gimap ready for requests from 174.19.150.134 7if4443230pbt.12\r\n">, @client_thread=#, @receiver_thread=#, @receiver_thread_terminating=false> 
1.9.3-p0 :004 > 


Tornado Pretty Error Pages

http://carlosgabaldon.com/articles/tornado-pretty-error-pages/


Finding your inspiration

Three months ago I decided I wanted to start running, but like many people I did not really like running. I figured the best approach was to just jump “feet” first into the idea of becoming a runner. After a few weeks I was making physical progress but, I was still not mentally into the run.

I was doing the run, but not being in the run.

Then in my quest for inspiration I happened to be reading a Runner’s World article about how Flea from the Red Hot Chili Peppers was inspired by the book Born to Run by Christopher McDougall. So I picked up a copy of the book and started reading. I was amazed how it quickly transformed by view point on running. My next run I decided to ditch the iPod and focus on being mindful during my run.

I focused on my breathing, how my legs felt, the temperature outside, how it made me feel, the trees, the wind, essentially the pure feeling of the run.

So my journey went from:

  1. Born to Run

Now with my first 1/2 marathon complete with a pretty good time,

I know now that I am just starting my journey…


Rails Common Commands

A list of frequent Rails command line commands.

  • rvm use ruby-1.9.2-p0 - ruby version manager; switches to Ruby 1.9.2
  • rvm use ruby-1.9.2-p0 --default - ruby version manager; sets 1.9.2 as default
  • rvm use system - ruby version manager; switches to Ruby 1.87
  • rails new - creates a new Rails application
  • rails server [s] - launches WEBrick web server
  • rails generate [g] - lists available generators
  • rails generate controller --help - provides usage documentation
  • rails generate model --help - provides usage documentation
  • rails generate migration --help - provides usage documentation
  • rails destroy controller [Name] - undo generate controller
  • rails destroy model [Name] - undo generate model
  • rails generate scaffold [Name] --skip --no-migration - scaffold skipping existing files
  • rake db:migrate - runs database migrations
  • rake db:test:clone - clones current environment's database schema
  • rake db:test:purge - empties the test database
  • rake routes - list of all of the available routes
  • rake -T - list of rake commands
  • git init - creates git repo
  • git add . - adds all files to working tree
  • git commit -m "Initial commit" - commits files to repo
  • git status - status of the working tree
  • git push origin master - merges local repo with remote
  • git checkout -b new-dev - creates topic branch
  • git checkout master - switches to master branch
  • git merge new-dev - merges new-dev branch into master branch
  • git checkout -f - undo uncommitted changes on working tree
  • git branch - list branches
  • git branch -d modify-README - deletes branch
  • git mv README README.markdown - renames files using move command
  • heroku create - creates app on Heroku servers
  • git push heroku master - pushs app on to Heroku servers
  • heroku rake db:migrate - runs database migrations on Heroku servers
  • heroku pg:reset SHARED_DATABASE --confirm [app name] - deletes database file
  • heroku db:push - transfer an existing database to Heroku.
  • heroku logs - get logs.
  • rails console - command line interface to Rails app
  • rails dbconsole - command line database interface
  • bundle install - installs gems from Gemfile

Rubyist OS X Dev setup

I just bought a shiny new MacBook Pro with a 2.4 GHz Core i5, 8 GB Memory, on OS X 10.63 and thought I would share my setup.

TextMate

Head over to http://macromates.com/ and download TextMate. Once we have installed TextMate we are going to configure it. From the TextMate menu go to:

TextMate >> Preferences >> Fonts & Colors. Then select the “Twilight” theme.

For smaller projects where we do the markup ourselves we are going to use Haml, which we will install shortly, TextMate does not include this bundle by default, so we need to add it. Open up terminal and enter the following:

cd /Applications/TextMate.app/Contents/SharedSupport/Bundles
svn co "http://svn.textmate.org/trunk/Bundles/Ruby%20Haml.tmbundle/"

Back in TextMate go to Bundles >> Bundle Editor >> Reload Bundles.

Terminal

Next on our list is to configure Terminal.

The first thing we want to do is setup the bash prompt for Git. This cool trick will show our checked out topic branch right at the prompt:

Git_topic_branch_at_prompt by Carlos Gabaldon
Git_topic_branch_at_prompt, a photo by Carlos Gabaldon on Flickr.

Open a Terminal session and type the follow:

sudo mate ~/.bash_login 

To configure we will copy the following into our .bash_login file:


   RED="\[33[0;31m\]"
     YELLOW="\[33[0;33m\]"
 	  GREEN="\[33[0;32m\]"
       BLUE="\[33[0;34m\]"
  LIGHT_RED="\[33[1;31m\]"
LIGHT_GREEN="\[33[1;32m\]"
      WHITE="\[33[1;37m\]"
 LIGHT_GRAY="\[33[0;37m\]"
 COLOR_NONE="\[\e[0m\]"

function parse_git_branch {

  git rev-parse --git-dir &> /dev/null
  git_status="$(git status 2> /dev/null)"
  branch_pattern="^# On branch ([^${IFS}]*)"
  remote_pattern="# Your branch is (.*) of"
  diverge_pattern="# Your branch and (.*) have diverged"
  if [[ ! ${git_status}} =~ "working directory clean" ]]; then
    state="${RED}⚡"
  fi
  # add an else if or two here if you want to get more specific
  if [[ ${git_status} =~ ${remote_pattern} ]]; then
    if [[ ${BASH_REMATCH[1]} == "ahead" ]]; then
      remote="${YELLOW}↑"
    else
      remote="${YELLOW}↓"
    fi
  fi
  if [[ ${git_status} =~ ${diverge_pattern} ]]; then
    remote="${YELLOW}↕"
  fi
  if [[ ${git_status} =~ ${branch_pattern} ]]; then
    branch=${BASH_REMATCH[1]}
    echo " (${branch})${remote}${state}"
  fi
}

function prompt_func() {
    previous_return_value=$?;
    # prompt="${TITLEBAR}$BLUE[$RED\w$GREEN$(__git_ps1)$YELLOW$(git_dirty_flag)$BLUE]$COLOR_NONE "
    prompt="${TITLEBAR}${BLUE}[${RED}\w${GREEN}$(parse_git_branch)${BLUE}]${COLOR_NONE} "
    if test $previous_return_value -eq 0
    then
        PS1="${prompt}➔ "
    else
        PS1="${prompt}${RED}➔${COLOR_NONE} "
    fi
}

PROMPT_COMMAND=prompt_func

Next let’s make terminal a little prettier. From the Terminal menu:

Terminal >> Preferences >> Startup; New window settings: "Homebrew".
Terminal >> Preferences >> Settings; Font select "Andale Mono 18pt".

Now that we have TextMate and Terminal configured we are going to setup our development tools.

Development Tools

XCode Tools

Go to http://developer.apple.com/mac/ and download the latest version of XCode. Once it is downloaded simply run the installer.

Git

Next we will download the git-osx-installer, then run it.

RubyGems

From terminal we need to update our RubyGems:

sudo gem install rubygems-update
sudo update_rubygems

SSH

Still at terminal we will run:

ssh-keygen -t rsa -C "cgabaldon@gmail.com"
cat ~/.ssh/id_rsa.pub | pbcopy

GitHub

Then we will navigate over to GitHub.com >> Account Settings to add our RSA key that we copied to the clipboard.

GitHubAccountSettings by Carlos Gabaldon
GitHubAccountSettings, a photo by Carlos Gabaldon on Flickr.

MySQL

We are next heading over to download mysql-5.1.47-osx10.6-x86_64.dmg, and then once downloaded run the installer.

After MySQL is installed we will install the MySQL preference pane which gets added to the OS X System Preferences. To install we simply double clicking on the MySQL.prefPane contained within the MySQL install package. This gives us a convenient way to start and stop our MySQL server from the System Preferences.

Back to terminal to add our newly installed MySQL server to our path:

cd
sudo mate .bash_login # add: export PATH=/usr/local/mysql/bin:$PATH

Then last, but not least we need to install our ruby mysql gem:

sudo gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config

Sinatra

sudo gem install sinatra

Rails

sudo gem install rails

Other Gems

sudo gem install sequel
sudo gem install haml
sudo gem install do_sqlite3
sudo gem install cheat 
sudo gem install cucumber 
sudo gem install dancroak-twitter-search 
sudo gem install faker 
sudo gem install geokit 
sudo gem install giraffesoft-timeline_fu 
sudo gem install github 
sudo gem install heroku 
sudo gem install json 
sudo gem install mislav-hanna 
sudo gem install nokogiri 
sudo gem install passenger 
sudo gem install rack 
sudo gem install railsmachine 
sudo gem install RedCloth 
sudo gem install redgreen 
sudo gem install reek 
sudo gem install rest-client
sudo gem install right_aws
sudo gem install right_http_connection 
sudo gem install rr 
sudo gem install rspec 
sudo gem install technicalpickles-le-git 
sudo gem install thoughtbot-factory_girl
sudo gem install thoughtbot-paperclip 
sudo gem install thoughtbot-shoulda 
sudo gem install webmat-git_remote_branch 
sudo gem install webrat 
sudo gem install wirble 

Capistrano

To simplify deployment we need to install Capistrano:

sudo gem install capistrano
sudo gem install capistrano-ext

FireFox

We of course need to install FireFox, by downloading and installing.

FireBug

Then Firebug and YSlow to help us on our front end engineering work.

Quicksilver

To help us keep our hands off the mouse we will install Quicksliver so we can exercise our keyboard foo. To install simply extract the tarball and drag to Applications.

Update the mappings to be Apple+Space for Quicksilver and Option+Space for Spotlight.

Fluid

Finally, we will install Fluid to help us turn our commonly visited sites into apps. Once downloaded drag to Applications.

Virtual Box

All production applications get deployed to a LAMP server, so we need a place to test out various configurations locally. Say hello to virtualization! We are going to use VirtualBox; download and install.

Evernote

Never forget anything; install Evernote and sign up.

Concentrate

Multitasking does not work when trying program, distractions will kill you. We are going to install Concentrate to apply the Pomodoro technique to keep focused and productive.

Things

We have a lot of Things to get done, so we need a great task management tool.

Growl

Most of the cooler OSX applications support notifications, Growl is the system for doing that.

Pencil

Pencil is one of the coolest UI prototyping tools I have used since a Sharpie.

Hack

We are now ready to do some serious hacking..


Django LAMP

Over the past year and a half I have been using WordPress to publish my web site & blog. Prior to WordPress I was using Calabro which is build with Python on TurboGears. WordPress is great, but does not give me all the control I need. I have recently decided to move back into the world of freelance web development and therefore want to do more with my main web site carlosgabaldon.com. I wanted to build something that showcases my specialties which are Python, Django, JavaScript, semantic markup, and jQuery so this new web site will be built with all of these great tools. I also thought it would be a good opportunity to document the process that I follow when starting a new project.

I like to get integration and deployment tasks figured out early since these tasks contain the most risk. Here is where I usually start:

  1. Setup the the code repository.
  2. Create the initial project structure; so we have something to deploy and iterate over.
  3. Deploy the initial project to the production web server; Apache on a Linode VPS.
  4. Script the the subsequent deployment process for continuous deployment.

Today I am going to go over step number 3; deploying a LAMP server.

Server

The first thing you are going to need is to get a virtual private server (VPS) or dedicated server account at a hosting company such as Linode or SliceHost. I am using Linode running Ubuntu 10.04 LTS (Lucid Lynx).

SSH

Once you have your server you will need to open a console and SSH into your new server:

$ ssh root@{your.ip.address}

Prerequisites

Now that we are logged in we need to install the prerequisites:

$ apt-get update
$ apt-get upgrade
$ apt-get install libapache2-mod-python python-mysqldb
$ apt-get install python-setuptools
$ apt-get install mysql-server
$ apt-get install subversion

Django

We want to be on the bleeding edge so we are going to install the trunk version of Django. First we checkout django-trunk to /usr/local/lib:

$ cd /usr/local/lib/
$ svn co http://code.djangoproject.com/svn/django/trunk/ django-trunk

Then create our symbolic links to point to our new trunk version of Django:

$ ln -s `pwd`/django-trunk/django /usr/lib/python2.6/dist-packages/django
$ ln -s `pwd`/django-trunk/django/bin/django-admin.py /usr/local/bin

Also, while we are in this directory we will create a symbolic link to the Django admin media:

$ ln -s `pwd`/django-trunk/django/contrib/admin/media /var/www/media
$ cd

Git

We need to install Git and then clone the project repository. To keep things simple for this tutorial we are going to put the web site under the Apache root.

$ apt-get install git-core
$ cd /var/www
$ git clone git://github.com/{YourGitHubAccout/your_django_project}.git

Apache

Next we need to configure Apache. I will be using modpython as the python host for Django. Since Django has support for WSGI we could use modwsgi, but I am going to stick with modpython for this tutorial since it has a proven track record in large scale production environments. The only draw back to using modpython over modwsgi is that an Apache restart is required after each deployment. Also, to keep this tutorial simple we are going have Django sever up the static meda files. This is not an optimal setup for a high traffic web site, but for the initial setup it works fine.

To configure Apache will open the default vhost and edit it to look like below:

$ nano /etc/apache2/sites-enabled/000-default
<VirtualHost *:80>
        ServerName {your.domain}.com
        DocumentRoot /var/www
        <Location "/">
                SetHandler python-program
                PythonHandler django.core.handlers.modpython
                SetEnv DJANGO_SETTINGS_MODULE {your_django_project}.settings
                PythonOption django.root /{your_top_level_folder_name
                PythonDebug On
                PythonPath "['/var/www/'] + sys.path"
        </Location>

        <Directory /var/www/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
        ErrorLog /var/log/apache2/error.log
</VirtualHost>

For the changes to take effect we need to restart Apache

$ /etc/init.d/apache2 restart

MySQL

We need configure the database. Connect to the mysql server using your mysql user name and password.

$ mysql -u root -pXXXX
$ create database {your_db_name};
$ nano /var/www/{your_django_project}/settings.py # update database connection info

Finally, we have Django create the tables.

$ cd /var/www/{your_django_project}
$ ./manage.py syncdb

If everything was done right we can open a browser and point it to http://{your.ip.address}/home/ and we should be running. The final step would be to open your DSN manager (i.e. godaddy) where your domain is registered and point the @ record to {your.ip.address}


Best practices for running a software development team

  1. Build solid relationships all of  your stakeholders.
    • Customers
    • Product Management
    • Executives
    • Business Partners
    • Business Units
    • Customer Support
    • Operational Support
  2. Create a single intake process for all product requests.
  3. Partner with stakeholders when prioritizing requests. Make sure they are part of any product steering committee meetings.
  4. Always prioritize severity 1 & 2  bugs before accepting any product feature requests from your stakeholders.
  5. Scalability, availability, and peformance enhancements should be prioritized very high in relation to product feature requests. Categorizing them as bugs also is way to make sure that fall high on the priority list.
  6. Do weekly reviews of all production & QA bugs.
  7. Do monthly reviews of production & QA logs to do trending for security issues, performance issues, and capacity planning. Aggregate this data into big visible charts where everyone on the team can see it daily.
  8. The development team should have real-time visibility to system level stats from the production environment. Use tools like Ganglia and Cacti to give the team a real-time view into production.
  9. Every person on the development team should spend time sitting with the customer support team listening in on support calls to get a deeper view into customer issues. Every 2-3 months for a few hours is a good schedule.
  10. Define a clear escalation plan for production outages/issues. Who is the first line of defense? Who do they escalate to? How do they classify the issue? When should development get involved? How should data be captured for later root cause analysis? Pull data from the HR system for the escalation up the management chain, do not waste time maintaining separate team lists of contact information. Update you HR system to contain all contact info then pull the data!
  11. Hire a technical writer to setup a wiki with a well designed information architecture for all product documentation. Run books, architectural artifacts, knowledge base, etc..
  12. Ensure that there is clear requirements traceability back to the originating stakeholder request.
  13. There should be design traceability from the high level design to the detailed design to the completed code.
  14. Every person on the development team should be trained on how to do root cause analysis.
  15. After any and every production incident there should be a root cause analysis performed with clear action items and associated owners to prevent the issue from re-occurring. This is not a witch hunt to find someone who to blame, but rather a way to identify what process or lack of process created the environment for the issue to have occurred.
  16. Focus on eliminating waste in all SDLC processes. A good place to start is with any process outside of analysis and writing the code, ask the question is the process adding value?
  17. Balance time to market vs. perfect architecture. What is a reasonable architecture to solve the problem today. If cost, resources, or time constrains the architectural design then have a story of how to undo the trade offs made in the design to meet the time to market demands. What is the plan for getting rid of the technical debt?
  18. Make data driven decisions, but do not require that every decision be made with complete data. Identify the experts in the organization and trust them to make the right decisions based on past experiences. For example, picking the right initial architecture should be based expert judgment, past experience, and some amount of prototypes. But scaling that architecture over time should be completely data driven from the metrics collected on the production systems. Always be measuring, analyzing, and optimizing based on the data.
  19. Everyone on the development team should have a professional development plan. This plan is a contract between the employee and management to ensure that the organization will provide all required resources and opportunities for the employee to realize the career goals.
  20. Foster a culture that supports a healthy balance between work and family life for everyone within the organization. Let people go home to their families at reasonable times. Ensure that people are taking vacations on a regular basis. Value results driven work over duration driven work.
  21. Create a respectful work environment where:
    • employees are valued
    • communication is polite and courteous
    • people are treated as they wish to be treated
    • conflict is addressed in a positive and respectful manner
    • disrespectful behavior and harassment are addressed
  22. Every 2-3 months get the entire organization together to discuss the product roadmap, discuss successfully delivered work, answer questions, motivate the teams, and above all to publicly praise and reward the star team members within the team.

Follow

Get every new post delivered to your Inbox.