Getting Started

Requirements

Behind the scenes, when you build a pipeline in LambdaCD, you are actually building a small app in clojure so you’ll need some tools:

• A recent JDK
• Leiningen as a build tool
• A decent editor. To get started, whatever you are comfortable with is fine. If you want to really dive in, consider something with good Clojure support. Cursive and Light Table are popular choices, as are general purpose editors with Clojure support.

Get Started

$ lein new lambdacd my-first-pipeline
Generating fresh 'lein new' lambdacd project.

$ cd my-first-pipeline
$ lein run
[...]
11:51:18.690 [main] INFO  o.e.jetty.server.AbstractConnector - Started SelectChannelConnector@0.0.0.0:8080
11:51:18.690 [main] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED SelectChannelConnector@0.0.0.0:8080
11:51:18.691 [main] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED org.eclipse.jetty.server.Server@cfe881a
Started server on port 8080

That’s it. You just generated a first pipeline that runs on http://localhost:8080 on your machine. You’ll have a choice of two UIs but for now, let’s stick with the reference UI that comes with the core of LambdaCD:

Click the arrow icon on the first step to trigger the pipeline. Now your pipeline runs, and you’ll notice the steps getting green or red one by one. Click on any of the steps the view the steps output.

A quick tour around the code

So let’s take a look what we just created. If you don’t know Clojure yet, don’t be scared if some of the syntax looks unfamiliar. Your first steps won’t require you to be a Clojure ninja.

$ tree
  .
  ├── README.md
  ├── project.clj
  └── src
      └── my_first_pipeline
          ├── core.clj
          ├── pipeline.clj
          ├── steps.clj
          └── ui_selection.clj

project.clj

This is a Leiningen project definition. You declare your dependencies here and other things about your project. We’ll get back to this later to add additional libraries to LambdaCD.

my_first_pipeline

Your pipeline code lives in this namespace. By default, we split up the structure of the pipeline and the build steps but you can organize your code any way you want.

pipeline.clj

This file is contains the initialization and the structure of the pipeline.

Like any other clojure code file, it starts with the namespace declaration and imports of required namespaces:

(ns my-first-pipeline.pipeline
  (:use [lambdacd.steps.control-flow]
        [my-first-pipeline.steps])
  (:require [lambdacd.steps.manualtrigger :as manualtrigger]))

More importantly, we define a constant pipeline-def which holds your pipelines structure:

(def pipeline-def
  `(
    manualtrigger/wait-for-manual-trigger
    some-step-that-does-nothing
    (in-parallel
      some-step-that-echos-foo
      some-step-that-echos-bar)
    manualtrigger/wait-for-manual-trigger
    some-failing-step))

steps.clj

This file contains the custom build steps that were referenced in the pipeline structure you just saw.

Again, we start with the namespace header. Specifically, we import the lambdacd.steps.shell namespace under the name shell. This provides LambdaCD support for executing commands on the shell.

(ns my-first-pipeline.steps
  (:require [lambdacd.steps.shell :as shell]))

Next, we’ll define the simplest possible build step. Every build step is a clojure function (that’s what the defn is all about) with two parameters, args and ctx. args is how build steps exchange information with each other. ctx contains the build steps LambdaCD context. It provides some functionality you’ll need as your build steps get more advanced. For now, all you’ll need to know is that some things (like shell support) require you to pass this on.

Apart from taking in two arguments, each build step is required to return a map that contains at least if it succeeded or not. All in all, this is the simplest successful build step you could write:

(defn some-step-that-does-nothing [args ctx]
  {:status :success})

Most of the time though, library functions will take care of actually assembling all that for you, so for now, we are just doing the most obvious thing to do in a build pipeline. We execute some shell commands:

(defn some-step-that-echos-foo [args ctx]
  (shell/bash ctx "/" "echo foo"))

(defn some-step-that-echos-bar [args ctx]
  (shell/bash ctx "/" "echo bar"))

(defn some-failing-step [args ctx]
  (shell/bash ctx "/"
                  "echo \"i am going to fail now...\""
                  "exit 1"))

core.clj

This namespace includes the -main entrypoint to your application, taking care of wiring everything together, setting up configuration and starting pipeline and server:

(defn -main [& args]
  (let [;; the home dir is where LambdaCD saves all data.
        ;; point this to a particular directory to keep builds around after restarting
        home-dir (util/create-temp-dir)
        config   {:home-dir home-dir
                  :name     "foo"}
        ;; initialize and wire everything together
        pipeline (lambdacd/assemble-pipeline pipeline/pipeline-def config)
        ;; create a Ring handler for the UI
        app      (ui-selection/ui-routes pipeline)]
    (log/info "LambdaCD Home Directory is " home-dir)
    ;; this starts the pipeline and runs one build after the other.
    ;; there are other runners and you can define your own as well.
    (runners/start-one-run-after-another pipeline)
    ;; start the webserver to serve the UI
    (http-kit/run-server app {:open-browser? false
                              :port          8080})))

ui_selection.clj

This namespace contains the routes and HTML to give you a choice between two UIs. If you at some point have a preference for one UI or want to customize it, this is the place to make changes.

For now, don’t worry about it too much.

Checking out a git repo and doing some real work

Most build pipelines need to touch version control at some point. So let’s see how we can wait for commits, check out a repository and run some tests.

So let’s change our pipeline to something that does this:

(def pipeline-def
  `(
     (either
       manualtrigger/wait-for-manual-trigger
       wait-for-repo)
     (with-workspace
       clone
       run-some-tests)))

A couple of things aren’t there yet:

• wait-for-repo to wait for commits on a repository
• clone to check out the repository and run other steps on the checked out repository
• run-some-tests actually run some tests.

Let’s get started!

First, we add the lambdacd-git library to project.clj¹:

; [...]
:dependencies [; [...]
               [lambdacd-git "0.2.0"]
               ; [...]
               ]

We’ll also import the namespace it provides in steps.clj:

(ns my-first-pipeline.steps
  (:require [lambdacd.steps.shell :as shell]
            [lambdacd-git.core :as lambdacd-git]))

Also, let’s put the repository information into a constant so we can use them in both git-related steps we are going to build. We’ll use LambdaCDs own repository here as an example:

(def repo-uri "https://github.com/flosell/lambdacd.git")
(def repo-branch "master")

Now that we have everything we need set up, let’s create the build steps:

(defn wait-for-repo [args ctx]
  (lambdacd-git/wait-for-git ctx repo-uri :ref (str "refs/heads/" repo-branch)))

(defn clone [args ctx]
  (let [revision (:revision args)
        cwd      (:cwd args)
        ref      (or revision repo-branch)]
    (lambdacd-git/clone ctx repo-uri ref cwd)))

Nothing too surprising here: We call a library function to do the heavy git-lifting for us. Two things stand out: How do we know where to clone the repository and what revision to clone?

Well remember I mentioned before that args is how build steps communicate with each other? We are using this here: with-workspace passes in the path of the workspace directory it created as :cwd and wait-for-repo provides us with a :revision value (we fall back to the branch in case the build was triggered manually).

Now all that’s left to do is actually run the tests (and remember to use our workspace (cwd) as the working directory):

(defn run-some-tests [args ctx]
  (shell/bash ctx (:cwd args) "./go test-clj"))

And that’s all you need. lein run your pipeline again and see the it all work.

Where to go from here?

• Find out how to do implement some common, more advanced use cases
• Read the FAQs and the Documentation
• Check out how to refactor a complex pipeline
• Go through a comprehensive walkthrough, implementing a complete delivery pipeline and going deeper into the details of LambdaCD

Footnotes