Interactive Artwork with mondrian

June 25, 2013

mondrian is a framework for building interactive artwork in ClojureScript. Part library, part opinionated Leiningen template, mondrian gives you a way to evolve your code at runtime, without reloading the browser and without recompiling your code. This article demonstrates the mondrian workflow as it takes you through the development of a simple animation.

Beginnings

I didn’t set out to build a framework. Originally I just wanted to draw some pretty pictures with ClojureScript. I accomplished that goal and then realized that the very next project would require a lot of copy-and-paste. I took a step back, looked at the code, and figured out which portions were generic and could be pulled out into their own library. mondrian was the result of that effort.

Expanding mondrian into a framework also gave me a chance to test some of my theories regarding interactive development in ClojureScript. I am tired of the edit-compile-run cycle and it takes more than a REPL to transcend that experience. Forth, Python, Node.js, etc. all have REPLs, but those are really just for experimentation; once you figure out what works you have to go back to your editor and type everything in again.

You could treat the Clojure REPL just like those other REPLs, and perhaps that’s even what most people are doing, but that’s not what I wanted to do. Thankfully my first introduction to Clojure was through Rich Hickey’s video on concurrency, which, around 1:23, has Rich explaining how he interactively built up his application using the REPL and his editor. He then proceeds to demonstrate that combination throughout the rest of the video.

I decide to learn Clojure after watching that video, but only if I could implement that workflow in my own projects. That turned out to be a difficult path, but after building a few tools I had something that was working for me.

ClojureScript brought its own challenges to the scene and most of my work on mondrian was spent evolving the best practices necessary to overcome those challenges. The sections below focus on using mondrian to interactively build up a software application; in a later article I will dive into the various challenges that I ran into while building out the mondrian workflow.

Interactive artwork, developed interactively

The best way to understand mondrian is to build something with mondrian. I am going to take you through the development of a bouncing red ball. When you’re done, you’ll have this:

Exciting, I know, but it’s the journey that matters here.

Mise en place

mondrian provides a Leiningen template that makes it easy to get started with the framework. Let’s create a new project and get ourselves an editor and a REPL:

Open a shell window and type lein new mondrian bounce to create the project.
Fire up a REPL connected to your favorite editor. I use Vim and whimrepl, but maybe your favorite combination is Vim+tmux, Emacs+SLIME, CounterClockwise, etc.
The mondrian template includes Clojure code that starts a combination web server and ClojureScript REPL target. Both services are launched by evaling the (browser-repl) function. That will also put you into the ClojureScript REPL itself.
Type (in-ns 'bounce.bounce) so that the REPL is pointed at the same namespace that you will be working with from your text editor.
Send your web browser to http://localhost:3000 and you should get a more-or-less blank page. Make sure that the web page is small enough that you can A) see the entire canvas and the “Persist Image” control, and B) see both the web browser and your editor (and ideally the REPL).

At this point we will now switch to your editor and interactively build up our animation.

But first: pipelines and stacks

mondrian animations are driven by the mondrian.anim/start function. That function takes your initial state, an update function, a render function, and an optional error function. The template has provided skeleton versions of all of those functions, our job is to fill them out with code that implements the bounce animation.

The pattern that I use for constructing mondrian animations is to base the animation around an update pipeline and a render stack. The update pipeline takes the animation state from the previous frame and runs it through a series of update functions, each which generates new state (or not) and passes that state to the next update function. That state is then given to each of the functions in the render stack, which presumably use the state to draw something on the canvas.

The data flow through our bouncing ball animation will eventually look like this:

The template defines merge-control-values and clear-background; everything else is up to you.

A red ball

Let’s add a new function to the render stack that draws a red ball. Add this code to the bounce.cljs file and put it immediately after the clear-background function:

(defn draw-ball
  [state]
  (m/fill-style ctx "red")
  (m/circle ctx {:x 100 :y 100 :r 5}))

Send that form to your REPL using whatever form of editor-REPL integration you have selected.

Nothing is happening yet because no one is calling that function. We can fix that by adding the function to the render stack; edit the render-stack function so that it looks like this:

(defn render-stack
  [state]
  (clear-background state)
  (draw-ball state))

Send that new definition of render-stack to your REPL (which I will now stop repeating).

Something is happening now, but it’s not what we wanted. Apparently draw-ball has a bug in it and so our render stack is crashing. This used to require a reload in order to fix, but now mondrian is smart enough to catch these kinds errors and keep the animation loop going. The animation loop is still crashing sixty times a second, but in a minute we will fix that from the REPL.

The bug here is that I am using ctx in draw-ball but didn’t pull that variable out of the state map first. You might think that this failure is contrived, but I actually did make that mistake when writing this article and it pushed me to improve the error-handling workflow. Fixing the bug is easy:

(defn draw-ball
  [{:keys [ctx]}]
  (m/fill-style ctx "red")
  (m/circle ctx {:x 100 :y 100 :r 5}))

Okay, now we have a red ball. The next step is to make it bounce around the canvas. Before we do that programmatically, it’s worth noting that you can move the ball by redefining the function:

(defn draw-ball
  [{:keys [ctx]}]
  (m/fill-style ctx "red")
  (m/circle ctx {:x 100 :y 200 :r 5}))

I changed :y 100 to :y 200 and the ball instantly moves down on the screen. This is fun (?), but it’s not what I had in mind.

State

The location of the ball is currently fixed in the draw-ball function, but that’s not going to work if we want to move the ball around on the screen. Instead we need to get the position of the ball out of our state map:

(defn draw-ball
  [{:keys [ctx x y]}]
  (m/fill-style ctx "red")
  (m/circle ctx {:x x :y y :r 5}))

We don’t actually define x and y in our state yet, so those values are nil and the ball has now wedged itself in the upper-left corner of the canvas. What we want to do is change those values over time in our update pipeline. For now, let’s just stuff some values into the map:

(defn update-pipeline
  [state]
  (-> state
      merge-control-values
      (assoc :x 50 :y 100)))

Just like before, you can redefine update-pipeline over and over again in order to move the ball. Unlike before, this trick turns out to be pretty useful. While writing this article I would frequently corrupt the state in such a way as to make the ball completely vanish. The easiest way to get it back on the screen was to put that assoc call into update-pipeline, send the form to the REPL, and then take the assoc call right back out again. I did this so often that I actually just left the assoc in there, although commented out:

(defn update-pipeline
  [state]
  (-> state
      merge-control-values
      #_(assoc :x 50 :y 100)
      move
      update))

We’re getting ahead of ourselves with those move and update calls though. Now that we’re consuming state we need to start generating some interesting state.

Moving the ball

We need a function that moves the ball around. That function should consider the amount of time that has passed since the last frame and update the ball’s X and Y position based on the speed at which the ball is moving. That function is called move and it looks like this:

(defn move
  [{:keys [delta-t-ms x y] :as state}]
  (let [speed-pps 27
        pixels-per-millisecond (/ speed-pps 1000)
        delta-pixels (* delta-t-ms pixels-per-millisecond)
        direction (math/radians 45)
        dx (math/circle-x delta-pixels direction)
        dy (math/circle-y delta-pixels direction)]
    (assoc state :x (+ x dx) :y (+ y dy))))

move calculates the number of pixels that the ball should be moved based on the amount of time that has passed and the speed at which the ball is moving (in pixels per second). That length can be thought of as the radius of a circle whose origin is the current location of the ball. We move outward from the origin along the direction that the ball is moving in order to find the X and Y locations on the circumference of the circle. That gives us the delta-x and delta-y values, which we add to the ball’s current position.

Link that method into the update pipeline and the ball should start moving:

(defn update-pipeline
  [state]
  (-> state
      merge-control-values
      move))

And move it does! Right off the screen.

Bouncing the ball

The ball needs to bounce off the edges of the canvas, which among other things means that we will have to change the ball’s direction every now and then. Let’s put the direction (in degrees) into state and then use that value (in radians) in move:

(defn move
  [{:keys [delta-t-ms x y direction] :as state}]
  (let [speed-pps 27
        pixels-per-millisecond (/ speed-pps 1000)
        delta-pixels (* delta-t-ms pixels-per-millisecond)
        direction (math/radians direction)
        dx (math/circle-x delta-pixels direction)
        dy (math/circle-y delta-pixels direction)]
    (assoc state :x (+ x dx) :y (+ y dy))))

I could make move smart enough to change the direction on its own, but I like the fact that move is simple and only has to deal with one thing. Let’s add a bounce function that takes care of flipping the direction around when the ball goes outside the canvas:

(defn bounce
  [{:keys [w h x y] :as state}]
  (cond
    (not (< 0 x w)) (update-in state [:direction] #(- 180 %))
    (not (< 0 y h)) (update-in state [:direction] #(- 360 %))
    :else state))

bounce is pretty simple. It checks to see if the ball’s X or Y value has gone out of range and redirects the ball back into the canvas if so. Add it to the update pipeline:

(defn update-pipeline
  [state]
  (-> state
      merge-control-values
      move
      bounce))

Unfortunately the ball is probably still off the screen, which means that bounce is just sitting there spinning the ball around and not getting it back onto the canvas. This is where we employ that trick of temporarily forcing some state into the pipeline and then taking it away:

(defn update-pipeline
  [state]
  (-> state
      merge-control-values
      (assoc :x 100 :y 100 :direction (rand-int 360))
      move
      bounce))

Eval that and then comment out the assoc and eval it again. The ball will now head off in a random direction and start bouncing off the walls.

Also, don’t lose the ball

mondrian uses requestAnimationFrame to drive the animation loop – we call requestAnimationFrame with a callback and the browser invokes that callback as soon as it is time for the next frame to be drawn. Normally that happens sixty times a second, but the browser might delay the call for a variety of reasons, including if the animation is off the screen for a long time.

move doesn’t handle that situation very well. It blindly applies delta-t-ms to the ball’s position and, if we just experienced a large delay between frames, the ball is now far, far off the canvas. bounce can’t fix the problem either, because all bounce will do is turn the ball around on the theory that the next move operation will get the ball back in range.

The easiest way to solve this problem is to clamp the x and y values to the bounds of the canvas:

(defn move
  [{:keys [delta-t-ms x y w h direction] :as state}]
  (let [speed-pps 27
        direction (math/radians direction)
        pixels-per-millisecond (/ speed-pps 1000)
        delta-pixels (* delta-t-ms pixels-per-millisecond)
        dx (math/circle-x delta-pixels direction)
        dy (math/circle-y delta-pixels direction)
        clamped-x (-> x (+ dx) (max 0) (min w))
        clamped-y (-> y (+ dy) (max 0) (min h))]
    (assoc state :x clamped-x :y clamped-y)))

One last thing

We have achieved our original goal – a bouncing red ball. The ball is a bit slow though; ideally we would have a slider that controls the speed of the ball. We could add that to the HTML and refresh the page, but mondrian is all about interactive development, so let’s do something more interesting.

First, let’s modify move to pull speed-pps out of the state map instead of using a hardcoded value:

(defn move
  [{:keys [delta-t-ms speed-pps x y w h direction] :as state}]
  (let [direction (math/radians direction)
        pixels-per-millisecond (/ speed-pps 1000)
        delta-pixels (* delta-t-ms pixels-per-millisecond)
        dx (math/circle-x delta-pixels direction)
        dy (math/circle-y delta-pixels direction)
        clamped-x (-> x (+ dx) (max 0) (min w))
        clamped-y (-> y (+ dy) (max 0) (min h))]
    (assoc state :x clamped-x :y clamped-y)))

The animation freezes at this point because speed-pps is not yet in the state map and so we get a bunch of math-related exceptions while trying to operate on nil. The next step is to get the slider on the page and mondrian provides its own microformats for defining controls. Here is the definition of our slider control:

<div class="slider control"
     data-name="speed-pps"
     data-label="Speed:"
     data-min="10"
     data-max="200"
     data-step="10">
</div>

The easiest way to get that element into the existing web page is to use the dommy library, which mondrian already depends on and is therefore already available from the REPL. Let’s switch over to the REPL and add that slider element to the DOM:

bounce.bounce=> (dommy.core/append!
           #_=>   (dommy.macros/sel1 :.bounce)
           #_=>   [:.slider.control
           #_=>    {:data-name "speed-pps"
           #_=>     :data-label "Speed:"
           #_=>     :data-min 10
           #_=>     :data-max 200
           #_=>     :data-step 10}])
#<[object HTMLDivElement]>
nil
bounce.bounce=>

The slider appears on the web page immediately after you hit return. And, thanks to the merge-control-values call in your update pipeline, the value of the slider is immediately added to your state map. That same merge-control-values is also the thing that detected the new control element and used the div element’s microformat definition to turn it into a jQuery UI slider.

Unfortunately this is a runtime-only mutation which means that it will be lost as soon as you reload the page. Everything else that we have done started from your source file and will persist across reloads. In general, I tend to add mondrian controls to my HTML file and then refresh page in order to get those controls to show up. I care enough about being able to reproduce my runtime state that I am willing to force a reload in those situations.

The bouncing ball animation on this page also includes a “Ball Size” slider. You can implement that slider using the same approach that we just took for the “Speed” slider and I would encourage you to do so.

If you need help, all of the code for this animation is available on GitHub, as is the HTML file that goes along with the code.

We’re done!

Hopefully this article has given you a good feel for mondrian and its interactive workflow. I love the ability to work with animations in real time and add bits of code here and there in order to try out different ideas. That is a huge win for this type of project, but I also feel that the general concepts can work well for other types of applications.