Codeship: Deployment Pipelines and ParallelCI for Faster Results

What You'll Be Creating

This is the second part of a sponsored series on Codeship, the secure, cloud-based  continuous integration and delivery service. If you missed part one, you can go back and read it here.

What's Codeship?

Codeship is a hosted continuous delivery service that focuses on speed, reliability and simplicity. In this tutorial, I'll review the basics of Codeship from part one and then walk through deploying your code. 

First, I'll show how to use Codeship to deploy to AWS CodeDeploy. Then, we'll explore Codeship's new ParallelCI, which speeds your tests and builds by delegating tasks in parallel to different virtual machines. I'll also introduce the Codeship API, useful if you'd like more automated control over Codeship's services.

Let's begin with a review of what Codeship does.

Background

With Codeship, you configure the service to build, test and deploy your application from your source code manager to the staging and production platforms of your choice. The service offers a variety of powerful setup options. Currently, Codeship is integrated with popular source code managers GitHub and Bitbucket. It supports a variety of programming languages (Ruby on Rails, Node.js, PHP, Java, Go, Dart, etc.) and deployment platforms (Amazon Web Services, Heroku, Google App Engine, etc.)

The practice of continuous integration involves repeated automated testing and code merging with notifications to alert developers to problems in their code in real time. Codeship combines this with the practice of continuous delivery, which provides for regular code deployment once changes have passed automated tests.

Essentially, when you push your code to GitHub, Codeship rebuilds your application on its secure servers and runs your automated tests. If the tests fail, it notifies your development team via email or integration with popular services such as Slack, HipChat, etc. If your tests pass, Codeship follows your defined pipeline to move code to staging or production servers as specified.

Codeship From Your Developers to Your Customers

Working with a service like Codeship can minimize fallout from bugs, speed their detection, and make your team more comfortable and proficient in releasing regular incremental improvements to your platform—hopefully improving your responsiveness to your customers and your market.

Codeship helps your team write and deploy their code with increased confidence.

Recently, Codeship announced an exciting new feature to speed your tests. It's called ParallelCI.

What's ParallelCI?

ParallelCI allows you to split your test commands across multiple build virtual machines (VMs) to speed up your build time. By running your tests in parallel, Codeship increases your team's productivity with faster builds. You can use ParallelCI to configure your project with up to ten parallel pipelines to gain as much as 10x speed increase for your test suite. 

CodeShip Diagram of ParallelCI Parallel Test Suites in Action

You can read case studies from Codeship clients Bannerman and LendingCrowd, who cut test times by 30% and 50% respectively.

CodeShip Lending Crowd Case Study on ParallelCI - Faster Test Times

Just as a reminder, I do participate in the discussions below. If you have a question or topic suggestion, please post a comment below. You can also reach me on Twitter @reifman or email me directly.

Configuring Your Deployment

Now, let's begin to explore Codeship's deployment features in more detail. 

You choose what Codeship does with your code after your tests pass. For example, perhaps you update your staging server for further tests, or perhaps you roll everything out to your production servers. Codeship allows you to define sophisticated pipelines for testing and deployment.

Codeship Example of Deployment Pipelines

By the way, Codeship also offers this free and very informative eBook: Efficiency in Development Workflows. You may wish to check it out along with the Codeship Blog, which keeps you up to date with new developments and best practices.

The Project Build Log

I'm assuming that you followed our first tutorial and have signed up and begun using Codeship.

If you visit your project's build log, you'll see a list of recent results. Click on the green Success button.

Codeship Get a Green Build Project Build Logs

This will bring up the build result details and the Build options sidebar menu.

Codeship Build Results and Build Options

Here's the Build options menu. Click on Review Test Settings:

Codeship Build options menu

Here's you'll see the Project Settings sidebar menu on the left, which is where we'll find the Deployment options we need.

Codeship General Settings

Here's a detailed view of the Project Settings menu. Click on Deployment:

Codeship Project Settings Menu

Configure Your Deployments

Codeship provides ready-built deployment integration with many popular hosting solutions including Google AppEngine, Amazon Web Services CodeDeploy, Heroku, Engine Yard, Nodejitsu and more. Codeship is an official partner of AWS and CodeDeploy

Codeship Configure Your Deployments

Codeship offers a handful of tutorials for various platforms and a wide range of documentation for Continuous Deployment options.

Codeship Continuous Deployment Documentation

I'm going to walk you through two examples. First, I'll deploy my earlier PHP-based Yii2 Framework build to AWS with CodeDeploy. Then, I'll build the Ruby Discourse Project using ParallelCI to speed my build times.

Using Amazon CodeDeploy With Codeship

In part one, I used Codeship to run automated tests on a fork of the PHP-based Yii2 Framework. Now, let's configure Codeship to deploy my project to an AWS server using AWS CodeDeploy.

First, let's take a walk through Amazon AWS CodeDeploy:

I'll provide a basic walkthrough of AWS CodeDeploy, but my goal isn't to detail it here. Instead, my focus is to help you configure the Codeship connection to AWS CodeDeploy.

First, visit the AWS CodeDeploy Getting Started page and choose a Custom Deployment:

Codeship AWS Code Deploy Application Wizard

Then, create a new application. I've already created an EC2 instance and tagged it CodeDeployDemo.

Codeship AWS Code Deploy Create New Application

Choose a Deployment Configuration. I've already set up a Service Role as well:

Codeship AWS Code Deploy Deployment Configuration

Now that your AWS CodeDeploy Application is set up, you need to configure your Deploy group for your instance.

Codeship AWS Code Deployment Group

This will differ depending on your application and S3 bucket location:

Codeship AWS Code Deploy How to Deploy Your Application

Then, let's return to Codeship. Enter your project and click Project Settings in the upper right. Then click Deployment

Codeship Project Settings for Deployment

Then, click the CodeDeploy button:

Codeship Configure Your Deployment

Fill in the AWS Code Deployment property settings with your Amazon access keys, secret key, and the CodeDeploy App Name and Deployment Group. Specify your S3 Bucket as well:

Codeship AWS Code Deploy Deployment

Now, you may want to review the Codeship documentation for using AWS CodeDeploy and its accompanying blog post.

The next time that you push a commit, Codeship will build your app, test it, and initiate an AWS CodeDeploy operation.

Codeship also offers a number of video guides on their Vimeo channel that may interest you. For example:

  • Deploying Ruby on Rails to Heroku
  • Deploying a Node.js Application to Nodejitsu
  • Deploying a Django Application to Google AppEngine

Here they are in order. Then we'll dive into using ParallelCI to speed our tests.

Deploying Ruby on Rails to Heroku

Here's a short guide to deploy a Ruby on Rails app from GitHub to Heroku, and the video is below:

Deploying a Node.js Application to Nodejitsu

Here's a short guide on How to deploy a Node.js app from GitHub to Nodejitsu, and the video is below:

Deploying a Django Application to Google AppEngine

Here's a short guide on How to set up Continuous Integration and Continuous Deployment for a Django Application from GitHub to App Engine, and the video is below:

Using ParallelCI With Codeship

I'm going to create a fork of the popular community discussion platform Discourse for Ruby on Rails. Then, I'll add a project at Codeship for my forked copy of Discourse.

Codeship Add your SCM Project - Discourse

Click on Environment and configure the environment variables for our Rails project:

Codeship Configure your environment variables

Then we'll use the following setup commands to set up the build instructions:

Codeship Project Build Setup Commands

Then I'll push an example change to the ReadMe file to initiate a Codeship build.

Codeship Push and Build Discourse

Here are the results from a successful build:

Codeship Push and Build Discourse Success

Testing With a Single Pipeline

Now let's add the following tests to a single pipeline.

Codeship Push and Build Discourse Single Pipeline

The next time we commit, Codeship builds the code and runs the test. With one pipeline, it takes nearly ten minutes.

Codeship Single Pipeline Finishes in Ten Minutes

Now, let's switch to ParallelCI. 

Testing With Multiple Pipelines

With ParallelCI, each project has multiple test pipelines that are run in parallel. Each of those pipelines runs in a virtual machine independently of the other. Codeship will first run your setup commands and then any arbitrary test commands you defined for this specific pipeline via the interface. To ease distinguishing different pipelines, you can provide a friendly name for each one.

Here's a video walkthrough with a different codebase of how we'll set up ParallelCI to speed these tests:

For my first test pipeline, I'm going to run the following tests:

Codeship Two Pipelines - Configure the first

For my second test pipeline, I'll run the following:

Codeship Two Pipelines - Configure the second

Then I'll push a new commit, and you can see ParallelCI running below with each pipeline's individual status log:

Codeship Two Pipelines Build and Test Running

After only 7 minutes 26 seconds, the two-pipeline build completes:

It's a sample time saving of nearly 30%. It's a good example of how Codeship's ParallelCI can save your team time when running more sophisticated build and test sequences. Also be aware that Codeship allows for up to 20 parallel test pipelines — the speed gain here can be exponential.

Keep in mind, if you have a deployment configured for a specific branch and each test pipeline reports a successful run, your deployment pipeline will be run. You do not need to change anything if you use one of the integrated Codeship deployments, e.g. Heroku, AWS CodeDeploy, Google App Engine, etc.

Also note that as your build and deployment commands are run on multiple virtual machines, artifacts created during the test steps will not be available during the deployment. If you need artifacts from the previous steps, make sure to regenerate them during the deployment using a script deployment added before the actual deployment.

There's one more thing to mention, the Codeship API.

The Codeship API

If you want even more control over Codeship to fully script and automate your continuous delivery services, Codeship offers an API:

Codeship API Documentation Script and Automate the Codeship

You can configure the builds on your project and restart them as needed.

In Closing

I'd like to summarize what we've learned about Codeship:

  • Codeship is a hosted Continuous Delivery service that focuses on speed, reliability and simplicity.
  • It automatically tests and deploys your applications.
  • It's integrated with all the relevant source code management systems, e.g. GitHub and Bitbucket
  • It supports a variety of programming languages, e.g. Ruby on Rails, Node.js, PHP, Python, Go, Dart, etc.
  • It's well integrated to a variety of popular deployment solutions, e.g. Heroku, AWS CodeDeploy, Google App Engine, etc.
  • It provides Notifications via email and chat services like Slack, HipChat, etc.
  • You get 100 builds/month and 5 private projects for free to get started easily and without risk. Open source projects can use Codeship for free. Educational projects receive a 50% discount.
  • Codeship's newest feature ParallelCI lets you run builds in parallel, speeding up the test suite run time tremendously.
  • If you'd like to automate Codeship, you can use its API.

Codeship is able to provide value to small teams with their free or basic plan, as well as very large companies, that need up to 20 parallel test pipelines, with their Enterprise plan. See the Codeship pricing page for more information. There's even a free 14 day trial for anyone to try out their 20 parallel pipeline testing configuration.

I hope you give Codeship a try. Share your experience in the comments. As always, please feel free to post your questions too. You can also reach me on Twitter @reifman or email me directly. Visit my Tuts+ instructor page to see other tutorials I've written. 

Related Links

Tags:

Comments

Related Articles