A few weeks ago, I decided to dedicate some time to explore the dagger.io project. The dagger.io project defines itself as a “programmable CI/CD engine that runs your pipelines in containers” and I believe that Dagger is worth considering for at least two reasons:

Firstly, cloud-based CI systems such as Github Actions, Gitlab CI, etc., often employ YAML for describing pipelines. However, YAML is not a programming language. When projects become more complex, having a “real” programming language to write CI/CD pipelines can significantly save time and prevent complications.

Secondly, running a dagger pipeline on your local machine is straightforward and this feature, which is often absent in other CI/CD environments, has the potential to greatly enhance the inner development loop.

No Java SDK 😢

Here is a diagram (I took it from the Dagger website) that illustrates how it works. If you are not familiar with Dagger, I strongly encourage you to read at least the Dagger Quickstart guide to better understand the rest of this post.

How Dagger works (from https://docs.dagger.io/319191/quickstart-basics)

At a low level, the Dagger client communicates with the Dagger engine through a GraphQL over HTTP API.

Regarding client libraries, Dagger provides an SDK library for the following languages: Go, Python, TypeScript, and JavaScript. Additionally, experimental SDKs for Rust and Elixir are also available in the Dagger GitHub repository. However, there is currently no Java client library for Dagger.

Even if a Java SDK is not available, it is still possible to directly use the GraphQL API to send queries to the Dagger engine. To do that I followed this guide: Build a Custom API Client.

There are multiple GraphQL client libraries for Java, I decided to use the Smallrye GraphQL client. It has two advantages: it can be used as a standalone library, and it offers a DSL (Domain-Specific Language) that makes building queries very straightforward.

To send a GraphQL request to the Dagger engine, I will start by using the Dagger CLI to initiate a session with the engine. The CLI injects 2 environment variables, DAGGER_SESSION_PORT and DAGGER_SESSION_TOKEN, to establish the connection. Then, all that is needed is to configure the GraphQL client with these connection parameters and execute the request.

And the execution output of this short program is:

~ % dagger run mvnw exec:java
█ [7.11s] mvn exec:java
┃ [INFO] Scanning for projects...
┃ [INFO]
┃ [INFO] ----------------< org.chelonix:dagger-java >-----------------
┃ [INFO] Building dagger-java 1.0-SNAPSHOT
┃ [INFO]   from pom.xml
┃ [INFO] --------------------------------[ jar ]---------------------------------
┃ [INFO]
┃ [INFO] --- exec:3.1.0:java (default-cli) @ dagger-java ---
┃ SLF4J: No SLF4J providers were found.
┃ SLF4J: Defaulting to no-operation (NOP) logger implementation
┃ SLF4J: See http://www.slf4j.org/codes.html#noProviders for further details.
┃ Linux K4SHC18JH5Q9O 5.15.49-linuxkit #1 SMP Tue Sep 13 07:51:46 UTC 2022 x86_64 Linux
┣─╮
│ ▼ from alpine
│ █ [0.82s] resolve image config for docker.io/library/alpine:latest
│ █ [0.03s] pull docker.io/library/alpine:latest
│ ┣ [0.03s] resolve docker.io/library/alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1
│ ┣─╮ pull docker.io/library/alpine:latest
│ ┻ │
█◀──╯ CACHED exec uname -a
┻

Let’s make a Java SDK 😃

Okay, it works. But I would like to go further and eventually have a real SDK in Java. There is an SDK Contributing guide in the Dagger GitHub repository listing the steps to follow.

1. POC the SDK

After thinking about what the API of a Java Dagger fluent API could look like, drawing inspiration from available SDKs, I have written a proof-of-concept API with minimal features. The primary focus during this phase was to construct the query builder, which refers to the mechanism that transforms the chain of API calls into a GraphQL query. The Smallrye client DSL proved to be extremely helpful as it eliminated the need for manual string formatting to build the GraphQL query.

So far so good, I was happy with my POC, so I continued to the next step.

2. Codegen

One thing I greatly appreciated about Dagger SDKs is that a significant portion of the API code is automatically generated from the GraphQL schema. The advantage is that updating from one version to another becomes much simpler, as it only requires generating the new API based on the updated schema.

For generating the Java code, I initially experimented with Mustache templates, but ultimately, I preferred generating the code programmatically using the JavaPoet library. I found that compared to templates (which often lack readability and may contain complex logic), using JavaPoet simplifies the maintenance of the code generator.

Below is an example using JavaPoet and the corresponding generated Java source file.

I also developed a Maven plugin around the codegen library. When the plugin is executed, it downloads the GraphQL introspection query from the Dagger Git repository (based on the API version to be generated) and sends it to the Dagger CLI. The CLI then returns the GraphQL API schema, which is used to generate the code.

3. Engine connection and automatic provisioning

In the end, I created the entry point for the API, the connect method, which returns a Client object from which pipelines can be assembled. The Client implements the AutoCloseable interface, so it can be used within a “try-with-resources” structure.

If the environment variables DAGGER_SESSION_PORT and DAGGER_SESSION_TOKEN are defined, the SDK will use these variables to initialize a connection with the Dagger engine. However, if they are not set, the SDK should automatically download the corresponding Dagger CLI for the SDK version and the host machine’s architecture. The CLI is then executed by the SDK in a subprocess to open the session with the engine, and the connection parameters are read from the CLI process standard output.

Final result: A Dagger fluent API in Java

The SDK (for Dagger v0.6.2) is available in this GitHub repository.

Below is the “Get Dagger Website” sample pipeline, written using the Java SDK, as featured on the Dagger homepage.

This code sample shows that the API is pretty straightforward to use and understand. There are plenty of other code samples in the code sample module in the SDK repository.

Now what’s next?

As I mentioned earlier, the main goal was to create a POC to better understand how to use Dagger. There is still work to be done to make it production-ready. At least I have already identified the following points:

  • Better error handling: currently, GraphQL errors are returned as they are, leaving it up to the caller to investigate their contents.
  • Improve code generation: Significant refactoring is likely necessary, along with adding more Javadoc comments in the generated code.
  • Review the query builder: I would like to reduce the coupling with the Smallrye GraphQL library and make it more generic, ensuring it is more resilient to potential future changes in the Dagger API.
  • Consider using a different GraphQL client. While Smallrye’s client works well, it is primarily oriented toward server usage and has many dependencies.

And of course, questions and comments are welcome.

A Java SDK Proof of Concept for Dagger.io was originally published in Better Programming on Medium, where people are continuing the conversation by highlighting and responding to this story.

Previous Post
Next Post