Introduction
Doodle is a compositional graphics library for generative art in Scala that enables users to declaratively create art pieces and other visualizations. A Doodle graphic is generally parameterized by one or more variables such as stroke width and background color, or sometimes something more domain-specific like the magnitude of gravity or iterations of a fractal. In any of these cases, the creator of a graphic may want to use a GUI for changing the parameters to fine tune the art piece or provide a means to make the graphic interactive. Doodle Explore is a specification for an eDSL that enables artists to easily create GUIs that explore their parameter space.
You can find the code here, with the work I did for GSoC ending at the v0.1 tag: github.com/creativescala/doodle-explore
For more usage info, visit the documentation micro-site here: https://creativescala.github.io/doodle-explore/
In the initial Google Summer of Code proposal for Explore, there were three core deliverables:
- the Explore eDSL framework
- the Java2D backend
- documentation
Additionally, there were two optional deliverables:
- the HTML backend
- type-directed construction
Over the summer, I successfully accomplished the three main deliverables, as well as the optional HTML backend using LaminarJS.
Examples / Usage
Doodle has an example function generating a Tree image under doodle.image.examples. Using Explore, it is simple to create and a GUI for it with the following code:
def treeExplorer(using
intGui: ExploreInt[Component],
layout: Layout[Component]
) = {
import intGui._
// creates a GUI with two integer sliders
int("Depth").within(1 to 12).withDefault(3).above(
int("Length").within(1 to 2500).withDefault(500)
)
treeExplorer.explore(
frame,
{ case (depth, length) =>
Image.compile {
Tree.branch(depth, 0.degrees, length / 10.0)
}
}
) def treeExplorer(using
intGui: ExploreInt[Component],
layout: Layout[Component]
) = {
import intGui._
// creates a GUI with two integer sliders
int("Depth").within(1 to 12).withDefault(3).above(
int("Length").within(1 to 2500).withDefault(500)
)
treeExplorer.explore(
frame,
{ case (depth, length) =>
Image.compile {
Tree.branch(depth, 0.degrees, length / 10.0)
}
}
)
A more featureful example is the gravity simulation. Given a update: GravityState => GravityState function, an initial: GravityState, and a render: GravityState => Image, it is simple to create a GUI for the interactive simulation using exploreWithState:
def gravityExplorer(using
intGui: ExploreInt[Component],
choiceGui: ExploreChoice[Component],
booleanGui: ExploreBoolean[Component],
layout: Layout[Component]
) = {
import ...
int("G").within(0 to 10).withDefault(1).above(
int("DT").within(1 to 100).withDefault(16)
).above(
int("Start Velocity").within(0 to 100).withDefault(30)
).above(
labeledChoice(
"Sun Color",
Seq(
("Yellow" -> Color.yellow),
("Red" -> Color.red),
("Blue" -> Color.blue)
)
)
).above(button("Reset"))
}
gravityExplorer.exploreWithState(initial, update)(
frame,
s => Image.compile(render(s))
) def gravityExplorer(using
intGui: ExploreInt[Component],
choiceGui: ExploreChoice[Component],
booleanGui: ExploreBoolean[Component],
layout: Layout[Component]
) = {
import ...
int("G").within(0 to 10).withDefault(1).above(
int("DT").within(1 to 100).withDefault(16)
).above(
int("Start Velocity").within(0 to 100).withDefault(30)
).above(
labeledChoice(
"Sun Color",
Seq(
("Yellow" -> Color.yellow),
("Red" -> Color.red),
("Blue" -> Color.blue)
)
)
).above(button("Reset"))
}
gravityExplorer.exploreWithState(initial, update)(
frame,
s => Image.compile(render(s))
)
Past Work
The most relevant prior work is Doodle's original Explore module, Explore 1.0. Explore 1.0 implemented type driven construction of GUIs; given just an input type, it could construct a GUI outputting the given type. This was convenient, but heavily limiting; without a complex type system and features such as refinement types it is often impossible to strictly constrain GUI outputs to the input domain of a graphic. For example, fractals such as the Sierpinski triangle can be parameterized by their number of iterations, which must be a natural number. Since Scala's Int type includes negative numbers, type driven construction based on it could cause errors. Types also make it difficult to implement features such as default values, or different GUI components for the same type. For example, while checkboxes and buttons both intuitively output a Boolean, their behavior and usecases are different.
Design Constraints
Any visualization algorithm that could be implemented in Doodle is a map from a parameter space to an image. The parameter space may be constrained by types, bounds, or something more domain-specific; there are infinite possible constraints. This motivates Explore to be as flexibile as possible such that users can implement constraints as needed. Additionally, Doodle is cross platform and can easily be extended to support a given backend, so Explore must be easily extensible to as many UI toolkits as possible.
Implementation
The design constraints above and the rest of Doodle's implementation motivate Explore's implementation as a tagless-final eDSL. The core of the eDSL is the Explorer trait. Explorer describes a backend-specific GUI, and is parameterized by A, its output type, as well as several other types specifying its backend. The Explorer instance can be run for a different backend just by updating these types.
An implementation of Explorer requires just one method: def run: Stream[Pure, A]. run should instantiate the GUI and return a lazy stream of its output values. Given this implementation, Explorer provides, among others, the explore(frame, render) method, which will run the GUI given a backend specific frame and a render: A => Picture function which specifies how to draw an image from the values provided by the GUI output.
Once there exists an Explorer type, there must be an implementation of a Explore* trait for each supported component. Each Explore* trait describes a DSL for creating components of the desired type, and generally will supply helpful extension methods for ergonomics. Implemented functions from the Explore* traits generate an intermediate representation for a component, which handles the backend-specific work itself.
Explore itself has a pretty simple implementation; every convenience method is a simplified version of exploreTransformed, which accepts a generic transformer: Stream[A] => Stream[B], a Frame, and render. Using these, it transforms the output of the UI, lazily converts them to images with render, and then uses Doodle's animation functions to render the animation.
Next Steps
There are many possible improvements to Explore. While the core eDSL is very convenient, a type directed approach has benefits in cases where the flexibility isn't needed. Because of Explore's architecture, a type directed version could potentially be added without changing too much. Additionally, composing multiple components creates a deeply nested tuple, which makes writing the render function much less ergonomic. This could possibly be improved on using the Shapeless library.
Aside from improvements for users, there is a lot of room for improvement for developers of new backends or components. Explore's current design is flexible in allowing large differences between backend implementations, but in practice the intermediate representation for each backend is likely to be similar. There is a lot of repeated code between the Java2D and LaminarJS backends, and it might be possible to optimize away, making new backend development easier.
Conclusion
Explore successfully implements a flexible, backend-agnostic DSL for easily attaching GUIs to an existing Doodle program. The library includes implementations for Java2D and browsers using LaminarJS, but because of the tagless final implementation, more backends can easily be added. Additionally, while there are already specifications for the most common types of components including Int, Boolean, and Color, users can easily implement more themselves without modifying library code.