Many people are familiar with the advantages of using revision control systems like Git. With the wide adoption of GitHub, which provides hosted Git repositories and elegant tools to create a social network for sharing code, we have an excellent platform for maintaining and collaborating on code-related projects. Maintaining an ExpressionEngine installation using Git is relatively straight forward, and many people I know are using this system to collaborate on EE-based websites.

Capistrano+Git+GitHub will save you time and frustration

By following the steps below and using Capistrano to manage your ExpressionEngine installation, it is possible to completely manage the EE deployment process, checking out the latest code, pushing it to the server, and updating uploaded content and caches automatically. This eliminates the need for file transfers, server-side activities, and in-place code editing, using a completely automatic process.

If you’re interested in learning more about Capistrano and what else can be automated with it, the Capistrano FAQ is an excellent starting point.

What’s going to happen

This article details the Capistrano recipe I’ve created specifically for deploying and managing an ExpressionEngine website, with provisions for preventing user-generated content and transient files from cluttering up your project, maintaining uploaded content across deployments, setting the correct permissions for uploading directories with each deployment, and more. The recipe also handles creating the server-side directories you’ll need to accomplish these tasks.

In order to take advantage of this process, you’ll only need to modify your existing ExpressionEngine project slightly. You’ll also need to install Git and Capistrano on your local machine. Finally, you’ll need to install Git on the server and create a new directory structure to contain the deployed ExpressionEngine website.

The process, described in more detail below, goes like this:

1. Install Git and Capistrano locally
2. Get a local copy of the ExpressionEngine project
3. Create a new local Git repository
4. Prevent user-generated and uploaded content from being added to the repository
5. Import the current code into the repository
6. Install Git on the server
7. Create a special deployment user on the server
8. Create SSH keys for the deployment user on the server
9. Add the deployment user’s keys to GitHub
10. Create a new directory structure for the project on the server
11. Copy existing content and configuration files to the new directory structure
12. Create a local Git clone of the project
13. Capify the project
14. Deploy it
15. Verify the deployment
16. Modify your web server to point at the new project
17. Restart the webserver

That list may seem long, but once you’re done, you’ll be setup for deployment long into the future. This article guides you through each step, so don’t worry if it seems complicated now. In reality, you’ll be eliminating a lot of future tedium and frustration. Once you’re done, you’ll only rarely need to login to your server, as everything will happen automatically, right from your local machine.

Note: This recipe will work fine with your own hosted Git repository. Just modify the references to GitHub, replacing them with the URLs for your own repository.

Regarding private repositories.

ExpressionEngine is licensed, commercial software. Storing it in a public repository would be violating the license agreement, so don’t do that. Either use a paid GitHub account with a private repository, or host your own private, secure repository (here are some instructions explaining how to do it).

Assumptions

There are a few things you’ll want to have in place prior to getting started in order for everything to work:

1. A GitHub account with a new private repository
2. An installation of ExpressionEngine (new or existing)
3. You’ll be deploying the new Capistrano-based EE installation to the same server you’re currently using (not mandatory, but it makes copying assets in the latter steps easier)
4. Competency using the UNIX command line, with a local UNIX or UNIX-like system (Mac OS X, Linux, etc.)
5. Superuser or sudo access on a Linux webserver (you can find these inexpensively)

If you’re new to using the UNIX command line, you might want to check out my PeepCode Screencast, Meet the Command Line, which will get you up to speed in about an hour, while paying for my kid’s education at the same time.

If you’re new to Git as well, there are a handful of resources available, like Everyday Git, Git for the Lazy, the Git Tutorial, the PeepCode Git Screencast, and even a crash course for subversion users.

Installing Git locally

Although Git doesn’t yet ship pre-installed on Mac OS X or most Linux distributions, it’s pretty easy to install on either operating system.

Mac OS X

You can install Git on a Mac using these instructions, but these days I just use Macports, like this:

sudo port install git-core

Linux

How you install Git on Linux varies based on the Linux distribution you’re using, as well as the default package management system. Generally, you’ll invoke your package management tool (with apt-get or yum) followed by install package-name, like this:

sudo apt-get install git-core

That’s usually all there is to it.

Quick Git Config

At a minimum, you’ll want to setup a few defaults for Git and GitHub using the following commands:

git config --global user.name "Your Name"
git config --global user.email you@yourdomain.com

Make sure that the email address you use here is the same one that you use on GitHub.

Now you need to add your public key to GitHub. If you don’t have a public key yet (or don’t know if you do), just follow these steps to create an SSH key.

Installing Capistrano locally

Capistrano is a Ruby Gem. This means you need to have both Ruby and RubyGems installed in order to install Capistrano itself.

Mac OS X

I wrote an article explaining how to build Ruby and RubyGems from source (it also explains how to install Capistrano), but nowadays on Mac OS X, I just use Macports to install them all at once:

sudo port install ruby rb-rubygems rb-capistrano

Linux

Again we rely on the system’s package management tool:

sudo apt-get install rubygems
sudo gem install capistrano

You should now have a working Capistrano installation.

Creating a new Git repository

Now, in GitHub, create a new private Git repository. If you already have a repository containing your EE installation, or are self-hosting, you may need to tweak these steps to match your configuration.

If you don’t already have a local copy of your entire ExpressionEngine directory, you should either download a copy from your current webserver (using SFTP or scp, perhaps), or export one from your current revision control system. Converting a Subversion repository to Git is outside the scope of this article, but Google has some good starting points.

Once your entire ExpressionEngine installation is downloaded, you’ll want to cd into the directory containing the top-level files and directories (e.g. system, themes, images, etc.).

cd mywebsite

Now we need to create a .gitignore file to prevent Git from trying to manage user-generated content and uploaded files. We don’t want them cluttering up the Git repository, and we need for them to live permanently on the server so that they can remain in place across deployments.

Using your favorite text editor, create a new file and paste in the following lines:

images/avatars/uploads
images/captchas
images/member_photos
images/pm_attachments
images/signature_attachments
images/uploads
system/config.php
system/cache/db_cache
system/cache/magpie_cache
system/cache/page_cache

If you’ve changed the name of your system folder as recommended in the installation instructions (see Step 2), you’ll want to change the word system to that word in the lines above. If you’ve created additional upload directories not listed above, you should add them to the list so they will be ignored, too.

Save the new file, naming it .gitignore, at the root of your local project directory.

Now we can initialize a new Git repository, add the files, and push everything up to GitHub:

git init
git add .
git commit -m "Initial import."
git remote add origin git@github.com:you/project.git
git push origin master

Make sure that you change the GitHub URL in the 4th line above to match the URL for your own new Git repository, replacing you and project appropriately.

Now your entire EE installation (minus uploaded content and the config.php file) is safely committed in your private GitHub repository.

Capify the project

Capifying the project is actually just a two-step operation within your project directory:

mkdir config
capify .

Capistrano has created two files, Capfile at the root level, and deploy.rb with the config folder we just created. You can safely ignore the Capfile file. The good stuff lives in the config/deploy.rb file.

The easiest way to download the deploy.rb file is with the command prompt, at the root level of the project folder:

curl http://hivelogic.com/files/deploying-expressionengine-github-capistrano/deploy.rb -o config/deploy.rb

With those changes in place, we’ll want to add the new files and push the changes back up to GitHub:

git commit -a -m "Adding capistrano recipe."
git add .

We’re done configuring the project on our local machine. Now we can work on getting the server configured and ready to receive the EE deployment.

Create a deployment user on the server

I recommend adding a special user on the server to both own the files on the server, and be used for all of the deployment activities, such as downloading the repository and other updates. This also separates the application from your own personal user account, making it easier for multiple people to deploy the app if necessary, without being privy to your account and access information. In the rails and Capistrano tradition, I usually call this user deploy.

Open a connection to your server, ideally using SSH. Once logged in, on most Linux systems, you can create the new deploy user with the following command:

sudo adduser deploy
sudo passwd deploy

The first command will create the new user on the system and make its home directory, and the second command will prompt you for its password.

Create shared SSH keys

Once you’ve created this new user, you’ll need to create a SSH public-key. As I mentioned earlier, GitHub uses SSH public-keys to authenticate users during its push and pull commands. In order for it to identify the new deploy user, you’ll need to add the key to your SSH key list in your account.

Essentially, you’ll want to follow the GitHub instructions for providing your SSH key as we did before, but rather than generating and adding your own key, you’ll want to do this as the new deploy user.

To do that, you have to do is become the deploy user on the server, using the su command, like this:

su - deploy

From this point, you can skip to the Linux instructions, create a new key, and add it to your own GitHub account, giving the deploy user access to your GitHub repository.

Installing Git on the server

You can install Git on the server the very same way you would on a local Linux system, using the following command:

sudo apt-get install git-core

You don’t need to install RubyGems or Capistrano on the server, as you’ll usually be deploying from your local machine.

It can be useful to test access to the GitHub repository with the deploy user interactively the first time, which will cause it to cache the server’s RSA fingerprint, and avoid potential issues down the road. This is easy to do using the following command:

git ls-remote git@GitHub.com:you/project.git master

Of course you’ll need to replace you/project in the command above with your own GitHub username and project name. You’ll be prompted to save the RSA key from the server, which should allow future git commands to run without issue.

You can now exit the deploy user’s shell like this:

exit

This will return you to your own account on the server.

Note for UNIX and Rails geeks: There’s no need to add sudo privileges to the deploy user’s account, because we won’t need to run any commands as root, nor restart any processes, like we sometimes need to do in Ruby on Rails.

Setting up the new website folder

I like to keep production websites in a /var/www/sites directory on the server, with a sub-directory for each website. An example directory might be named /var/www/sites/example.com.

You might already have a website running under a path like the one I’ve specified above and if so, you can just pick a different name for the new installation.

We’ll create a new directory on the server to deploy to, and several sub-directories that we’ll use as part of the new Capistrano-managed deployment. The following commands will create the new directory structure and set the correct ownership and permissions, so that we can deploy:

sudo mkdir -p /var/www/sites/example.com
sudo chown -R deploy /var/www/sites/example.com
sudo chmod -R 755 /var/www/sites/example.com

Of course you’ll need to replace example.com with your own domain name, or the new name you’ve chosen for the deployment directory.

Creating the directory structure with Capistrano

When you deploy with Capistrano, it will clone the latest version of your repository into a new sub-directory in a releases directory, and then create a symbolic link to it at the root level, named current.

Using the custom EE-deployment recipe, Capistrano will also create symlinks to the contents of the assets directory into the new current directory, such as the image uploads folder, preserving uploaded and user-created files in exactly the way that ExpressionEngine is expecting.

After your initial deploy, and from then on, you’ll only need to focus your attention on the current directory, which will contain your latest deployment and the additional symlinked folders. It will resemble in every way your currently deployed EE installation.

We need to create these new directories, but now that we’ve capified our EE installation, we can do it all right from the local machine. So back at our own local command prompt (you might want to open a new Terminal window, as we’ll be working with the server again momentarily), we’ll use Capistrano and the recipe I’ve written to create the new directory structure needed by ExpressionEngine. Just type:

cap deploy:setup

Back on the server, from within the /var/www/sites/example.com directory, you’ll see the new directories that have been created:

.
|-- releases
`-- shared
    |-- assets
    |   `-- images
    |       |-- avatars
    |       |   `-- uploads
    |       |-- captchas
    |       |-- member_photos
    |       |-- pm_attachments
    |       |-- signature_attachments
    |       `-- uploads
    |-- config
    |-- log
    |-- pids
    `-- system

Some of those directories may look a bit familiar to you, but they might seem out of place. That’s because we’re going to be relocating our user-generated and uploaded content, along with the config.php file, into the shared folder, in order to preserve them across deployments and prevent them from existing within our lean repository.

Customize the recipe

Now you need to edit the deployment recipe in order to customize it to your environment. In most cases, the defaults should be relatively close to your own configuration, and you may only need to change a few lines. Let’s go through the first block of the file, which contains the only lines you should need to modify.

Using your favorite text editor, open the config/deploy.rb file.

set :application, "example.com"

Replace example.com with your own domain name.

set :ee_system, "system"

If you’ve changed the name of your system folder as recommended in the installation instructions (see Step 2), you’ll want to change the word system to that word here.

set :deploy_to, "/var/www/sites/#{application}"

This is the full path to a directory you created on the server earlier. I recommend matching the application name to the final directory that will contain the deployment (e.g. “example.com”), so you may only need to change the preceding path.

Be sure not to set this to your existing path, or bad things may happen to your data.

set :ee_previous_path, "/var/www/sites/oldsite"

If you’d like to take advantage of the recipe’s capability to automatically copy existing, user-uploaded content from the existing installation into the new directories will be creating, Change this path to point at your existing installation of ExpressionEngine. Your existing files will not be changed in any way, just copied to a new location.

set :repository, "git@github.com:you/project.git"

Change this to the git clone url for your repository, provided to you by GitHub.

set :branch, "master"

Only change this if you don’t plan to deploy the master branch.

set :user, "deploy"

Only change this if you decided to use a different name for the deployment account which we created earlier.

Now you need to push your changes back to GitHub:

git commit -a -m "Customizing the Capistrano recipe."

Now we can get down to business.

Copying content

If you have user-uploaded content, such as avatars, member photos, or images you’ve uploaded as a part of the publishing process, you’ll want to move this existing content into the newly created shared directory structure.

Note: If you don’t make use of ExpressionEngine’s member features, and don’t upload images yourself when posting, you can skip this step. On the other hand, if you have a busy site with lots of user generated content, you may need to select a time during off-hours, or shut down the site for a few minutes while you run through the final steps of the deployment.

Assuming that you’re deploying the new installation to the same server that contains your old one, you can let the Capistrano recipe I’ve created handle this optional step for you. Just run the following command:

cap deploy:copy_content

This will copy all of the uploaded content in the standard EE directories to their counterparts in the newly-created shared/assets directory.

You may want to verify that everything was copied correctly. You may also want to do this by hand on the server, using the cp command, or scp if you’re deploying to a new server.

Copy config.php

The config.php file contains critical system information that usually varies between production and development systems. Rather than edit this file over and over between deploys, it’s a better idea to keep the config.php file out of source control entirely. We’ve already done this using the .gitignore file we created earlier, but now we need to copy the current config.php file into place on the server. The create_symlinks capistrano command will do this for you when you deploy.

If you’re deploying to a different server, or would prefer to perform this task by hand, you’ll need to copy the config/config.php file into the newly-created /var/www/sites/example.com/shared/config directory by hand, and then set its permissions to 666 using the chmod command.

Deploy

This is it — the real test. Let’s deploy! From within your local project directory, type:

cap deploy

You’ll see a lot of text scroll by as Capistrano runs your newly customized recipe, and if all goes well, your app should be deployed.

You can verify that all of the files and directories look correct on the server. The contents of the current directory should now exactly match the contents of your previous EE installation directory. Remember that many of the directories will now actually be symlinks to the real directories in the shared directory, but they should contain all of the content they did before.

You may have noticed that the system/cache directory is empty. Each time you deploy, the cache directory will be created from scratch with the permissions set so that ExpressionEngine will re-create its cache sub-directories as needed. This prevents a stale cache from remaining between deploys.

You can also clear the cache at any time using Capistrano with the following command from within your project directory:

cap deploy:clear_cache

This will safely remove all of the server-side cache files and directories, which will be recreated on demand by ExpressionEngine.

Reconfiguring Apache

If everything looks the way it should, all that’s left to do is to tweak the webserver configuration file to point at the current directory, rather than your old EE installation. Although every Apache installation is different, most use individual virtual-host configuration files. These files usually reside in the /etc/apache2/sites-enabled directory, or something similar.

Search for the file that contains the configuration for your previous EE installation. Using nano or your favorite command line editor, use sudo to edit the file:

sudo nano /etc/apache2/sites-enabled/001-mysite.com

Locate the following line:

DocumentRoot /var/www/sites/example.com

Change it to contain the full path to your new deployment directory, making sure to add current to the end. Without the addition of the current directory, your EE files will not be found. It should look like this when you’re done:

DocumentRoot /var/www/sites/example.com/current

Now find this line:

<Directory /var/www/sites/example.com/>

And do the same, changing it to reflect the full path to your newly deployed directory, with current appended.

<Directory /var/www/sites/example.com/current>

Now save the file and exit the editor.

Restart Apache

It’s time to make the switch. There are several ways to restart the Apache webserver, depending on how it’s been installed. Try this:

sudo apache2ctl restart

If that doesn’t work, you may need to try a few variants, such as sudo apachectl restart or sudo /etc/init.d/apache restart.

Once Apache is restarted, you’re site should be up and running from the new Capistrano managed, GitHub-baed installation.

If nobody is looking, dancing would be appropriate at this time.

The workflow going forward

From now on, any time you want to push changes to the website, your routine will be simple.

Commit your changes:

git commit -a -m "Something wonderful."

Push your changes to GitHub:

git push

Deploy the site:

cap deploy

That’s it! The cache directories will be re-created with the deployment, refreshing them, and all of your uploaded content will be linked in automatically, each time.

Congratulations, you made it. You’re now deploying ExpressionEngine from GitHub with Capistrano.

Filed in Development, Software, Tutorials.

---

Add a Comment

I welcome your comments on this article. Please complete the form below to add to the conversation.

Note: I reserve the right to edit or remove any comment that is deemed inappropriate, off-topic, or offensive.

Name

Email Address

Website (optional)

Comment

Latest Projects

Copyright © 2010 Secret Space Agency. Published using ExpressionEngine on Slicehost.