Maintaining a medium-sized Java library in 2022 and beyond

Preface and introduction

This is a small ebook about maintaining a medium-sized Java project in 2022. The project in question Neo4j-Migrations available at github.com/michael-simons/neo4j-migrations. Things addressed in here are among others the choice of Java 8 as baseline, the build systems, build-plugins considered to be useful, recommendations about structure but also concepts of certain libraries, GraalVM native and how to make things work with each other.

Neo4j-Migrations has been conceived in early 2020 and has developed from basically one single Spring InitializingBean with a couple of logic attached to it to a multi-module project with a core API integrated into several ecosystems. The domain of that project is not too complex to understand: It’s goal is to run one or more migrations or database refactoring against a Neo4j graph database instance in a reliable fashion, making sure each refactoring is applied only once. Those migrations can come in to flavors: Either as script (a textfile containing Cypher code) or as a class runnable on the JVM.

This will give a couple of interesting things to discuss:

Which Java version to pick (taking into account it was 2020 when that decision was made)?
- How to benefit from modern Java nevertheless?
- How to prepare for or even embrace the Java module system?
Which build tool did I chose and why?
- What are my approaches to make the build as readable as possible?
- How do I tackle programmatic tasks in the build in contrast to declarative ones
- How to automate testing?
- How to automate releasing and where to release?
Can Java be good choice for CLI tooling?
- What is possible with GraalVM native image and what isn’t?
How to provide uniform access to this library in peoples favorite application frameworks?
- Revisiting Spring Boot starter
- Learning about Quarkus Extensions

This publication is of course highly subjective. Actually, like most books, posts and what have you out there: It works for me and this project. It doesn’t work necessarily for your setup. Some things might even be harmful for your cases, probably you find some of them utterly and outright stupid.

What’s the size of the project?

"Sloc Cloc and Code" [scc] gives me the following output for version 1.4.0, along with some cost estimation based on the COCOMO [COCOMO] model:

scc --exclude-dir docs/book
───────────────────────────────────────────────────────────────────────────────
Language                 Files     Lines   Blanks  Comments     Code Complexity
───────────────────────────────────────────────────────────────────────────────
Java                       153     14206     2186      4184     7836        330
XML                         29      3584      154       134     3296          0
AsciiDoc                     8      1562      344         0     1218          0
Properties File              7        37        3        13       21          0
Shell                        6       392       49        65      278         38
YAML                         5       294       28         0      266          0
Smarty Template              3        44        7         0       37          2
Groovy                       2        71        7        32       32          1
HTML                         2        71        5         0       66          0
JSON                         2        23        0         0       23          0
Markdown                     2       755      238         0      517          0
Plain Text                   2       219       33         0      186          0
Batch                        1       182       35         0      147         30
gitignore                    1        33        5         4       24          0
───────────────────────────────────────────────────────────────────────────────
Total                      223     21473     3094      4432    13947        401
───────────────────────────────────────────────────────────────────────────────
Estimated Cost to Develop (organic) $429,833
Estimated Schedule Effort (organic) 9.978566 months
Estimated People Required (organic) 3.826907
───────────────────────────────────────────────────────────────────────────────
Processed 732729 bytes, 0.733 megabytes (SI)
───────────────────────────────────────────────────────────────────────────────

The estimated schedule effort is actually not completely wrong.

1. Which Java version to pick in early 2020?

Shiny new things are always neat, but not every target environment supports the latest bits and pieces While an application that bring its own runtime, deployed in a container of its own or similar has a lot of freedom to just pick the latest and greatest, a library may have not.

1.1. Use the current LTS version or the most widely adopted one?

JDK 11 had been published as LTS-Version in September 2018. 11 incorporates most of the features from 9 and 10, with a couple of things I think would have been useful to have in the core API of this project such as:

actually and without joking, the module system
a way nicer Optional with methods such as or, flatmap and ifPresentOrElse
finally, factory methods for collections
effective-final variables as resources in the try-with-resource statement (var resource = new CypherResource(); try(resource) {} or coming in as a method argument)
some nice additions to the stream API (dropWhile and takeWhile)
the reserved type name var defined by [jep286] in a couple of places
the new not-predicate method
the Flight Recorder from [jep328]

As this project is intended as a library and incorporated into other peoples programs I am not that much interested in the addition of new garbage collectors. If this had been an application, those additions would have had a much bigger influence.

Neo4j-Migrations is a library, however. I want as many people as possible to be able to use it in their applications if they have the need for such a functionality. In 2020 JDK 8 was still the correct choice: It’s a given fact that enterprise users are slow to migrate their runtime. Of course, it is valid to argue that a nudge might help them to migrate, but the reality seems to be different. And as a matter of fact, it’s not only enterprises. Or at least not only enterprises directly. One of the biggest drivers of the Java ecosystem, the Spring Framework, still targets JDK 8. And so do value adding libraries on top of it, like Spring Data, JHipster and many more: I wanted Neo4j-Migrations to benefit from that ecosystem without forcing users to bump their JDK if they use my library.

So looking at the above list and seeing a couple of things that would have been nice to have in contrast to actual value, the decision was made for JDK 8, and I am happy with it.

At the time of writing JDK 17, the current LTS, has been out for 4 months already. JDK 17 brings many more things to the table. The most prominent features being text blocks, records, sealed classes and pattern matching. Those, together with the fact that some runtimes like [Helidon] and [Quarkus] already made the jump to 11 since 2020 and Spring Framework 6 will even have JDK 17 as baseline and has just dropped the first beta^[1] would have probably led to a different decision.

1.2. Caveats and how to address them

The caveats with rooting for JDK 8 come in the form of a couple of "But what if…".

What if it isn’t the worst idea to actually be prepared for [jep261], the Java module system delivered in JDK 9 and used in JDK 17 via [jep396] to encapsulate JDK internals by default
What if some features are so useful that I still want them? Like at least being able to put up bigger sign than just naming a package internal to prevent people from using things they should not use because I plan to change them as I see fit?
What if I myself want to use a library that already made the jump beyond 8?

Those can be addressed by a couple of things:

1.2.1. Don’t rely on a derived automatic module name

What is an automatic Java module name? An automatic module name basically indicates the absence of a module descriptor (module-info.java). If the JAR file containing the classes in question has the attribute "Automatic-Module-Name" in its main manifest then its value is the module name. The module name is otherwise derived from the name of the JAR file.

First and foremost, be a good citizen and define an automatic module name right from the start and don’t rely on a name derived from your libraries JAR file. This prevents that users of your library that are running on JDK 9+ on the module path have to deal with a derived name in their module-info.java when referring your library. Why would that be an issue? Because you will break them as soon as you want to benefit from modules yourself in which you need to declare a non-automatic name which will most likely be different from the one derived from the JAR file. Christian Stein has a couple of suggestions in [stein-maven-coordinates-and-module-names] I followed loosely when choosing the module names of Neo4j-Migrations and other current projects.

1.2.2. Use the latest and greatest JDK

Use the latest and greatest JDK to compile and know about the --release flag available for javac^[2] since JDK 9. The --release flag is different from --source and --target: In contrast to the combination of the latter, --release will not only make sure that the --target matches the --source but also that only API is being used that is available in the targeted release. In other words: With --source and --target alone you would be able to compile Java 8 syntax into Java 8 byte code but still using API only available in higher JDKs when not using JDK 8 itself. The release flag prevents this.

1.2.3. Learn about Multi-Release Jars

Multi-Release Jars, defined in [jep238], extend the JAR file format to allow multiple, Java-release-specific versions of class files to coexist in a single archive. This allows for a couple of things:

You are not restricted to an automatic module name for your library targeting primarily Java 8. A valid module-info.java can be part of the Jar for JDK 9+
In case you would benefit so much from new API that it is worth the effort of maintaining two versions of class, you can do that
One thing I did for JDK 17 was making use of sealed classes so that I can provide more guidance in the API which interfaces I introduced only for the API itself and which one and through what hierarchy people are supposed to implement.

The downside to it is IDE support in my experience. I haven’t found a good way to work with the source structure of a Multi-Release Jar, but I will write more about it in Chapter 2.

1.2.4. What about JDK 9+ dependencies?

This is a problem if you need such dependencies and still want to support JDK 8. The only solution here is not to have them.

2. Building things

In this part the choice of the build tool, plugins for it and externalized programmatic additions will be discussed

2.1. Welcome to Burning Geek Con: The choice of a build tool

We have suggested the "Insult Con" back in 2016 already. Choosing a build tool is probably worth a whole track alone. Let’s discuss the options:

We might add ant and a couple of others to it as well, plus every homegrown solution out there. Of course, a homegrown solution can be totally fine as well. Neo4j-Migrations can be seen as homegrown, too of course. It solved one issue and evolved later. Just because a thing is on GitHub it doesn’t mean it’s automatically better than what you would write inside your company.

To cut it short: I opted for good, old, boring Maven. Why? Because I am not only familiar with it, but it’s boring in a positive way: If you don’t escalate in a pom file, most projects look more or less the same and most of the time, they behave the same.

2.1.1. Why not Gradle?

I never felt like I need the promoted flexibility it brings. I don’t need the Groovy (or Kotlin) these days for the mast majority of tasks I have in my builds. Also, I did observe several times that using newer versions of Java than the ones that have been latest when a specific version of Gradle has been released, causes problems. This is because Gradle depends on [ASM]. ASM is an all-purpose Java bytecode manipulation and analysis framework. It can be used to modify existing classes or to dynamically generate classes, directly in binary form. As such it is tailored to specific Java versions.

While this is not inherently a bad thing, I prefer not having to deal with it, especially as there wasn’t anything on the table I couldn’t get from Maven.

Of course there is the valid argument of incremental builds: Gradle seems to be a lot better in that regard in contrast to Maven. Also, there is the build cache who has done great things for Spring Boot as reported back in 2020^[3]:

The Spring Boot team’s primary reason for considering a switch to Gradle was to reduce the time that it takes to build the project. We were becoming frustrated with the length of the feedback loop when making and testing changes.

— Andy Wilkinson in "Migrating Spring Boot’s Build to Gradle"

In this project here I would benefit much more in shaving more than just seconds from the lengthy restart time of the database containers being used to verify migrations against.

2.1.2. Why not Bazel?

Bazels focus seems to be a lot on incremental build, local or distributed caching. While it was intriguing to try out something completely new, it would have added a steep learning curve for me and I was more focussed on solving a business usecase.

2.1.3. Why not Bach?

Bach is an interesting contender in that regard that it uses Java and Java Modules to build Java Modules. I did work a bit with the author, Christian Stein, on it, and we migrated one other pet project of mine to use it. I like the "Java only" approach a lot, and I think it has potential. However, it is Java Modules only. While I am gonna speak about modules in this publication, Neo4j-Migrations is not only Java modules.

In addition, Bach is a lot of work in progress. Of course, this is not bad and someone has to solve the egg-and-hen-problem here, but I am just not up for that call.

2.1.4. Maven it is

Before I jump into how I configure my projects these days with Maven, just a quick recap: The word Maven is a Yiddish word for "expert", derived from the Hebrew word mayvin (מבין), which means "who understands." A maven is an expert who understands the skill or subject at hand:

Maven approaches a couple of things with "convention over configuration": It tries to tackle dependency management, the build process and the deployment. It all evolves around a standard cycle of validate, compile, test, packaging, integration testing, installation and deploymenet.

One of the biggest critics here is having the "Project Object Model descriptor" - POM for short - materialized as pom.xml doing both dependency management, build description and eventual being a representation of a deployed artifact.

I have decided that I can happily live with that.

Read the results from the Maven Dependencies Pop Quiz^[4] by Andres Almiray. Especially before you post stuff like this thing here to the internet. When in doubt, Andres is right.

2.2. My Maven best practices

Whatever you take away from this section, please consider always having the following properties defined:

First, specify the projects encoding with

<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

or with whatever you prefer. Otherwise, Maven - and Java - will default to the OS default encoding which is not always what you expect so that your build is suddenly OS dependent.

Second, on Java versions prior to 11 and / or older versions of the Maven compiler plugin always define both

<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>

With recent JDK versions and Maven compiler plugin you would define this:

<maven.compiler.release>11</maven.compiler.release>

In the above example, you could safely use Java 17 and compile for 11 (or for 8): The system will guarantee that your source, the generated bytecode and the API used are appropriate for the chosen target. Prior to that you might have created a library that while being technically compatible with older versions but accidentally used newer API not available in older runtimes.

2.2.1. Use properties and dependency management for versions

Even in single-module project (Neo4j-Migrations is a multi-module project), I put every version of every dependency and every plugin I use into a property in the <properties /> element. For dependencies, I use the <dependencyManagement /> element. The later might seem like overkill in a single module, but it will be a lifesaver when you encounter non-converging dependencies: Those are dependencies that are transitive dependencies of your direct dependencies with mismatching versions: Your direct dependency A depends on X:1, your direct dependency B on X:2.

When you declare a dependency on A before B, Maven will resolve X:1. When B is declared before A, it will resolve X:2. WHy? Maven does build a graph of dependencies. For transitive dependencies, it will pick out the nearest dependency. That is, the one that has the fewest number of hops from the direct dependency. This severely threatens the stability of your build.

Instead of excluding the transitive dependency from one direct dependency, you should use dependency management to exactly specify which version you want (and from which you hope that is compatible with both direct dependencies).

The list of properties containing the versions helps me to keep an overview of what I have. The dependency management allows me in submodules to omit the version altogether. Have a look at my pom.xml, in the properties section^[5] and the dependency management^[6].

If you don’t want to read all the following, you might consider having a look at the OSS Quickstart archetype [oss-quickstart] from Gunnar Morling.

2.2.2. Make use of the enforcer plugin

The self-titled "The Loving Iron Fist of Maven™", the [maven-enforcer-plugin]. It helps you to keep things straight. I used it to

enforce the Java version required to build this project
the Maven version being used
that all dependencies converge

See my configuration in Listing 1.

Listing 1. My enforcer config

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-enforcer-plugin</artifactId>
    <version>${maven-enforcer-plugin.version}</version>
    <executions>
        <execution>
            <id>enforce</id>
            <goals>
                <goal>enforce</goal>
            </goals>
            <phase>validate</phase>
            <configuration>
                <rules>
                    <requireJavaVersion>
                        <version>17</version>
                    </requireJavaVersion>
                    <DependencyConvergence />
                    <requireMavenVersion>
                        <version>${maven.version}</version>
                    </requireMavenVersion>
                </rules>
            </configuration>
        </execution>
    </executions>
</plugin>

It is so good, it brings tears to peoples faces^[7].

2.2.3. More conventions

What is better than conventions? Well, more conventions^[8]. The first one who brought the order of elements in a POM-file to attention was longtime Java User Group contributor and friend Franz van Betteraey. The second one came from the Spring Boot team. I remember in one PR we discussed the order of dependencies.

All of that got me thinking: I don’t want to think too much, and just get it done. Ever since then, I add the "Sortpom Maven Plugin" [sortpom] to new projects like that:

Listing 2. Apply automatic sorting to pom files

<plugin>
    <groupId>com.github.ekryd.sortpom</groupId>
    <artifactId>sortpom-maven-plugin</artifactId>
    <version>${sortpom-maven-plugin.version}</version>
    <configuration>
        <encoding>${project.build.sourceEncoding}</encoding>
        <keepBlankLines>true</keepBlankLines>
        <nrOfIndentSpace>-1</nrOfIndentSpace>
        <sortProperties>true</sortProperties>
        <sortDependencies>scope,groupId,artifactId</sortDependencies>
        <createBackupFile>false</createBackupFile>
        <expandEmptyElements>false</expandEmptyElements>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>sort</goal>
            </goals>
            <phase>verify</phase>
        </execution>
    </executions>
</plugin>

Either it automatically sorts the poms in the verify phase or call if independent: ./mvnw sortpom:verify@sort. It orders the elements of my pom in a stable, consistent order and also the dependencies, grouped by scope. Remember what has been written about resolving transitive versions? When you just add a dependency on top of the dependency list, it might have the same dependency as another direct dependency has below. Now suddenly the new transitive dependency will be picked up. Sorting does not completely solve this problem, especially not without dependency management, but in many cases results might be less surprising.

2.2.4. Escalate away from conventions

The moment I would have to jump through so many loops to get a thing done via Maven configuration, I escalate in two steps. Both are done via the Exec-Maven-Plugin [exec-maven-plugin]. The plugin is able to execute Java in the same JVM or alternatively, external processes, either other programms or Java based calls. The latter can be configured to use the same class path like the build.

So here are my 3 approaches to go from conventions to a programmatic approach: The farer away it get’s from Maven and convention, the more issues might arise on "other peoples" machines:

Execute a Java program with your current classpath

Maybe everything you need for a task is already inside your project. First step of escalating. For the CLI module I use [picocli] and I added the AutoComplete.GenerateCompletion subcommand in MigrationsCli ^[9]. Thus, my CLI learned how to generate a shell completion script. There’s a lot about that topic in the "Autocomplete for Java Command Line Applications"^[10] manual. I want to distribute the script as a resource, too. I don’t want to manually add it as a resource. Therefore, my build should call the classes being build. This is an excellent use case to for [exec-maven-plugin] calling Java with the current classpath as shown in Listing 3:

Listing 3. Using Maven exec plugin to call single program (neo4j-migrations-cli/pom.xml)

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <executions>
        <execution>
            <id>generate-cli-completion</id>
            <goals>
                <goal>exec</goal>
            </goals>
            <phase>package</phase>
            <configuration combine.self="override">
                <executable>java</executable>
                <arguments>
                    <argument>-classpath</argument>
                    <classpath />
                    <argument>${name-of-main-class}</argument>
                    <argument>generate-completion</argument>
                </arguments>
                <outputFile>target/neo4j-migrations_completion</outputFile>
            </configuration>
        </execution>
    </executions>
</plugin>

Note that we can use the <classpath /> element here that passes the whole classpath to the java executable
Of course there is a property in the pom.xml holding the name of my main class, which is used in several places
The rest are a couple of args to the actual class being called

Call a single program

This is the next step of escalation. If I am just dealing with a single thing I need todo outside Maven or a plugin, I check if it is a single call, simple to parameterize. Compressing my GraalVM native binaries with UPX is such a thing as shown in Listing 4:

Listing 4. Using Maven exec plugin to call s single program (neo4j-migrations-cli/pom.xml)

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <executions>
        <execution>
            <id>compress-binary</id>
            <goals>
                <goal>exec</goal>
            </goals>
            <phase>package</phase>
            <configuration combine.self="override">
                <executable>upx</executable>
                <skip>${skipCompress}</skip>
                <arguments>
                    <argument>${project.build.directory}/neo4j-migrations${executable-suffix}</argument>
                </arguments>
            </configuration>
        </execution>
    </executions>
</plugin>

Scripting things

If there is neither a class in my project solving my needs nor a single executable, I resort to scripting. A [bash] script seems reasonable these days, as bash or something compatible should be available in most places. If you want to stick to what you most like have in your project, use a Java script with [JBang].

I use [japicmd] to compare previous and current versions of the project checking for semantic version or rather for changes that would break semantic versioning. For this I need the current version number (given of course via the <version> tag) and the previous version (given via a property named neo4j-migrations.previous.version).

The previous version number needs to be updated when a release has been prepared, just before the release plugin will commit and tag the changes. I added a corresponding exec-maven configuration in the parent pom.xml

Listing 5. Using Maven exec plugin to call a script (pom.xml)

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>${exec-maven-plugin.version}</version>
    <configuration>
        <skip>true</skip>
    </configuration>
    <executions>
        <execution>
            <id>release-prepared</id>
            <goals>
                <goal>exec</goal>
            </goals>
            <configuration>
                <executable>bin/update-previous-version.sh</executable>
                <skip>true</skip>
            </configuration>
        </execution>
    </executions>
</plugin>

You notice the execution has an identifier. It is referred in the release-plugin section:

Listing 6. Calling a release completion goal (pom.xml)

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-release-plugin</artifactId>
    <version>${maven-release-plugin.version}</version>
    <configuration>
        <autoVersionSubmodules>true</autoVersionSubmodules>
        <useReleaseProfile>false</useReleaseProfile>
        <releaseProfiles>release</releaseProfiles>
        <tagNameFormat>@{project.version}</tagNameFormat>
        <goals>deploy</goals>
        <pushChanges>false</pushChanges>
        <localCheckout>true</localCheckout>
        <arguments>-Drelease -DisDryRun=${dryRun}</arguments>
        <preparationGoals>clean exec:exec@prepare-release verify</preparationGoals>
        <completionGoals>compile exec:exec@release-prepared</completionGoals>
    </configuration>
</plugin>

The script is rather simple (you’ll find it in the bin folder):

Listing 7. Extracting a prepared version from Maven release files and updating a property (update-previous-version.sh)

#!/usr/bin/env bash

set -euo pipefail
DIR="$(dirname "$(realpath "$0")")"

NEW_OLD_VERSION=$(sed -n 's/project\.rel\.eu\.michael-simons\.neo4j\\:neo4j-migrations-parent=\(.*\)/\1/p' $DIR/../release.properties)
$DIR/../mvnw versions:set-property -DgenerateBackupPoms=false -Dproperty=neo4j-migrations.previous.version -DnewVersion=$NEW_OLD_VERSION -pl :neo4j-migrations-parent

While a Maven maven would maybe have solved this with pure XML declaration, I couldn’t and would go as so far. This is easier to read.

These days, you are not restricted to shell-scripts. You can just stick to Java if you want. Have a look at my test_native_cli.java script in the same folder. It is a Java program, orchestrating a [Testcontainer] and testing the native build. Right now, it is called from a GitHub action, but it would be easy enough to call it from Maven. All the power of Java right at your hands, as a script via [JBang].

3. Frameworks

This part will speak about the "big" frameworks and what possibilities they offer and how you can hook up your library into them. This is not about "low level" frameworks, such as a class scanning library, a command line tooling, but more outwards or up the stack.

I am talking about Spring Framework, Spring Boot, Quarkus, Helidon, Micronaut or similar. I will cover Spring Boot and Quarkus here as those are the ones Neo4j-Migrations offers direct support for.

3.1. General considerations

All the big frameworks come with a lot of dependencies already. If your library is written in such a way that it uses a ton of dependencies as well, you or your users most likely end up in dependency issues pretty quickly. I honestly do think that the JDK offers so many great APIs, being it for all the datastructures you need, sorting algorithms, String and general I/O operations and even upto HTTP clients (just checkout out java.net.http.HttpClient for that matter), that there is often no good reason at all to add a heavyweight dependency to your project.

Things I have in mind here are commons-lang, guava and the like: First, look at what the JDK offers for a task at hand or what is needed to create something decent yourself before you get yourself a couple of megabytes of dependencies.

On the other hand, specialized needs such as connecting to a database with a JDBC implementation or a specialized driver such as in Neo4js case with the Bolt driver can’t be solved within a reasonable amount of work. The same applies to [ClassGraph] for example. ClassGraph is a library I use for scanning the class path for resources and for classes implementing a given interface. I theoretically could do this all by myself but hardly as good (or as complete) as dedicated library.

However, you should never expose the API of such a library directly on your API. Why? Because if you do, you will be inevitably glued to the evolution of a third party tool and that is probably not what you want if you like to follow semantic versioning or things like that. Another reason: Sometimes such an API might not work in a target environment like one of the frameworks to follow, and you might want to plug-in their solution for the same problem.

It’s a kind of magic!

The above headline is not only the name of a great Queen song from 1986 accompanying the movie Highlander in which the same statement is used to "explain" the immortals fighting for "the price", but it can also be used as a question. "Is this a kind of magic?":

Is @springboot still considered as magic by anyone or has it already become a default and understood level of abstraction?

— Maciej Walkowiak
https://twitter.com/maciejwalkowiak/status/1495528915535814660

The discussion following was good and interesting. I have that approach in a talk for a while now. As a user of a car or even of a computer itself: I don’t need to know how the single parts work or work together. To be honest, while I do know the basic principle of how a combustion engine or an electric engine works, I couldn’t care less most of the time. There are better trained specialists out there who can help me.

It gets more complicated when looking at computer: As a software engineer I better know the basic concepts of I/O, processor, slow and fast memory and what have you. Should I be able to pick a computer apart and reassemble it? Does it make me a worse software engineer when I never built my own PC? Should I even be aware how the electronics works? Questionable.

Things a are a bit different for a developer when looking at software: While some bootcamps are probably trying to cookie-cut as many developers as possible and telling people that at least some computer science knowledge is overrated and frameworks do all the heavy lifting, this is just not true. You should have a general ideas about complexity, memory usage, runtime behaviour of algorithms and the like. And you can’t get around that you need to know in the end how a target framework of choice behaves.

I firmly believe understanding the mechanics of something like Spring Boot, Micronaut, Quarkus or even Helidon SE (which as a much more visible and explicit "functional" approach) is paramount, for both a developer using such tech for creating applications and business value as well as for developers wanting to contribute their thing to a framework.

So here we are, let’s break the magician’s code once again:

3.2. Spring Boot

A lot has been said already about "the magic" about Spring Boot. Actually, I am one of the people who did start talking about it early on, see [a-kind-of-magic-2016]. As this is known now by a lot of people and the concept of Spring Boot starter and their autoconfiguration is along now for more than 6 years actually, I am going to refer to one of my talks from 2016 until the time being: "It’s a kind of magic? - Custom Spring Boot Starter"^[11]

3.3. Quarkus

Quarkus is a framework primarily developed by Red Hat. It is a Java framework tailored for deployment on Kubernetes. Key technology components surrounding it are OpenJDK HotSpot and GraalVM. The goal of Quarkus is to make Java a leading platform in Kubernetes and serverless environments while offering developers a unified reactive and imperative programming model to optimally address a wider range of distributed application architectures. Its first version was published in March 2019.

It has a couple of design principles such as

Container first
Developer productivity

Wikipedia lists unifying imperative and reactive as well, but for the purpose of this chapter, the two above will do. Container first with an emphasis on low memory usage and fast startup times go hand in hand with developer productivity:

While I get back to speed of compiling, startup or restart in a moment, other things important for Quarkus are

unified configuration (Spring Boot has this of course too, and I personally like Springs configuration support better than SmallRye’s one (SmallRye config is library use by Quarkus))
"Batteries included" approach: Quarkus uses [Testcontainer] for bringing up any downstream service dependencies such as databases, message brokers and the like. In case there is explicit configuration for such a service, the so called "Dev services" will kick in and bring up a Neo4j database for you for example
Live coding: Quarkus undertakes some effort to determine whether classes or resources have changed when a request reaches an application started in development mode. If so, that class is transparently recompiled and all parts of the application affected will restart (that works actually much better than Spring Boots restart class loader)

The fast startup times, the unified configuration and the dev services are all relevant when you want to provide a Quarkus extension for your library or service. These three points are a good chunk of why the Quarkus extension are designed the way they are.

3.3.1. Creating a closed world at build time

The main "trick" inside Quarkus' sleeves is doing as much work and processing during build time to create a closed world. Closed world meaning that "everything" (i.e. configuration, CDI beans, resources, entities and such) is known before the startup of an application. That closed world is actually written to bytecode in many cases. Vice versa, stuff outside that assumption for which can be guaranteed that there’s no execution path touching it, will not be loaded into a JVM.

All that goes together with avoiding reflection as much as possible. That would start by finding entities, JAX-RS resources, over to (re)configuring things during runtime and wiring up collaborators (aka autowiring or injecting things) and many more stuff we as developers using any framework with dependency injection take for granted.

If the assumption of a closed world sounds familiar: You might have heard in the context of the [GraalVM]. More specifically together with its native image compilation: Turning a proper JVM program into a native machine binary. GraalVM itself removes any unreachable code found within the application as well as any of its dependencies for the image to be as small as possible. A native executable starts usually much faster and uses far less memory than a traditional JVM.

There are a couple of different approaches to that topic. [Micronaut] for examples goes "all in" in terms of annotation processing. Or - in the words of their architect Graeme Rocher - "Micronaut is an implementation of an annotation-based programming model."^[12] More specifically most if not all the annotations used in Micronaut applications are processed at compile time, not at runtime as other frameworks do. "The AnnotationMetadata API is a construct that is used both a compilation time and at runtime by framework components. AnnotationMetadata represents the computed fusion of annotation information for a particular type, field, constructor, method or bean property and may include both annotations declared in the source code, but also synthetic meta-annotations that can be used at runtime to implement framework logic."
I am pretty sure that Spring Framework 6 and the efforts undertaken at VMWare will move into a similar direction. I have written a small annotation processor^[13] for Neo4js [Cypher-DSL] and the general Java Api for that is well-designed.

Build steps

Much of the following is taking straight from "Writing your own extension"^[14], a page probably not many casual users of Quarkus will ever open. It does however contain a lot of valuable resources. Most important to understand the behaviour of your extension and an application eventually using it are the following three distinct phases of bootstrapping a Quarkus app:

Augmentation: This initial phase is done via Build Step Processors, having access to an index containing annotations found and ways of access to further descriptors, properties and build items created by previous processors. The outcome of those build steps can have many forms: It ranges from additional resources put into the applications, hints which classes to include in a GraalVM native image or (CDI) items available in your final application. Either way, the outcome is recorded into actual bytecode. Furthermore, all build step processors can interact with Recorders. Those records are a way of blue printing what should be eventually executed, so that it can be recorded into bytecode.

Bytecode recorded goes into one of the following phases:
Static init: The framework is going to create a static method to be called from a main class that is going to initialize as much of the application as possible.

The outcome of the static init phase can be recorded as is during augmentation / build time: Any library needed to produce it can be dropped from the final class path so that it will never be loaded into the JVM running the application.

The static init is even more important for GraalVM native image: The static init is executed on a JVM during ahead-of-time compilation and any objects retained there are directly serialized into the native executable. This means that if a framework can boot in this phase then it will have its booted state directly written to the image. Of course this will be then executed but the computation has already be done at this point.
Runtime init: Steps recorded here are executed from an applications main method respectively during native executable boot. There are no restrictions what can be done here. Ports and such can be opened.

It is hopefully apparent that as much as possible should be pushed into static init to achieve the quickest startup possible.

3.3.2. Project setup for an extension

Quarkus extensions are usually setup as multi-module projects containing the following modules:

neo4j-migrations-quarkus-parent
├── deployment
│   └── src
├── integration-tests
│   └── src
└── runtime
    └── src

The parent project is usually responsible for dependency, plugin and build management, while the deployment module contains all the necessary augmentation code and the runtime module the actual behaviour to be recorded. Integration tests are well, integration tests.

Those are not your usual integration test! To test your extension the best way possible you will have eventually to create a real application like module and subject to augmentation. Just like a user would when using your extension. I found this to be one of the hardest thing to understand and learn.

The flow of dependencies is crucial to understand: A deployment module actually must depend on the runtime module as the runtime module will contain the actual recorders and their dependencies. The reason: The recorded bytecode can of course only be executed when it has access to the runtime classes. A runtime module on the other hand must not dependent on the deployment module as that would pull in all Quarkus augmentation and deployment code into the runtime, quite the contrary of what should be achieved.

On the other hand, deployment modules are of course allowed to depend on other deployment modules so that module B can consume any build items from A and use them. The same applies for runtime modules: They are free to depend on other runtime modules.

The Quarkus people use Maven a lot. The make use of artifacts and extensions in a couple of places. Basically, using de-jure or de-facto standards for getting you up to speed and running. You can use

mvn io.quarkus:quarkus-maven-plugin:create-extension \
  -DextensionId=my-ext \
  -Dname="My extension"

to create a new extension. The process will ask you for a group id and depending on that, you will end up with a "standalone" layout, the "Quarkiverse"^[15] layout (in case your group id starts with io.quarkiverse.) or from within the Quarkus source tree itself, the "Quarkus core" layout.

3.3.3. The effect on Neo4j-Migrations

How does the above restrictions respectively the philosophy affect Neo4j-Migrations and what can I enlist to demonstrate the possibilities of a good deployment module? A couple of things, as it turns out:

First, Neo4j-Migrations depends on another deployment module, namely the Quarkus-Neo4j extension. How to access its primary build item.

Neo4j-Migrations searches for resources (Cypher-Scripts) in both the class path as well as the file system. The latter is less than a problem then the former: The file system is of course always accessible (well, mostly always), regardless if things are run on the JVM or native image. The class path is certainly accessible as well, but as we learned before, stuff that is not accessed at build time or explicitly declared to be included in the image just won’t be there.

The same rules apply for Java (or Kotlin for that matter) implementations of JavaBasedMigration (an interface that allows for programmatic migrations).

While resources and classes will always appear in the Quarkus JVM based output, they won’t appear in a GraalVM native image without additional hints (compare the tooling available for creating native GraalVM images). We are lucky, Quarkus offers a couple of hints that will make sure that classes and resources are both read during startup and appear in a native image.

Furthermore, I wanted to make sure that a Migrations instance is available as CDI bean, so that a user can access it and call for info, clean and other operations.

And last but not least, can we add some decent information to the developer console? Let’s have a look at each of the above points.

The relevant Quarkus code of Neo4j-Migrations is in this module and its submodules: neo4j-migrations-quarkus-parent, The main processor is of course in the deployment module ac.simons.neo4j.migrations.quarkus.deployment.MigrationsProcessor and the main recorder is ac.simons.neo4j.migrations.quarkus.runtime.MigrationsRecorder.
Every method annotated with @BuildStep lives in a processor!

Declaring the feature

This is the easiest step of them all:

Listing 8. MigrationsProcessor#createFeature

public class MigrationsProcessor {

        static final String FEATURE_NAME = "neo4j-migrations";

        @BuildStep
        @SuppressWarnings("unused")
        FeatureBuildItem createFeature() {

                return new FeatureBuildItem(FEATURE_NAME);
        }
}

It is a FeatureBuildItem which just indicates a name for the feature provided by the extensions.

What’s with the @SuppressWarnings("unused"): That’s the downside of "magic" code: The @BuildStep gets invoked by the Maven plugins, so a static analyzer like Sonar (and most IDEs) have a hard time figuring out that this method is actually used. I spent enough time on the quadruple A-Rating in sonarcloud which I don’t want to be spoiled with false positive of unused code.

Depending on another deployment module

Neo4j-Migrations needs a connection to the Neo4j database it should be used with. This connection is provided via an instance of Neo4js Driver. That driver comes from Quarkus Neo4j. As the driver instance is of course a runtime dependency - it’s a bit hard to create a persistent database connection during build time and then serialize it to byte code - we need to depend on the corresponding deployment module from our own deployment module of course:

Listing 9. pom.xml (in deployment)

<dependency>
        <groupId>io.quarkiverse.neo4j</groupId>
        <artifactId>quarkus-neo4j-deployment</artifactId>
</dependency>

And the for full measurement, the runtime dependency in the runtime module:

Listing 10. pom.xml (in runtime)

<dependency>
        <groupId>io.quarkiverse.neo4j</groupId>
        <artifactId>quarkus-neo4j</artifactId>
</dependency>

Now, how to use it? We must step ahead to the build step that puts everything together. It looks like this:

Listing 11. MigrationsProcessor#createMigrations

@BuildStep
@Record(ExecutionTime.RUNTIME_INIT) (1)
@SuppressWarnings("unused")
MigrationsBuildItem createMigrations(
        MigrationsBuildTimeProperties buildTimeProperties,
        MigrationsProperties runtimeProperties,
        DiscovererBuildItem discovererBuildItem,
        ClasspathResourceScannerBuildItem classpathResourceScannerBuildItem,
        MigrationsRecorder migrationsRecorder,
        Neo4jDriverBuildItem driverBuildItem, (2)
        BuildProducer<SyntheticBeanBuildItem> syntheticBeans
) {
        var configRv = migrationsRecorder
                .recordConfig(
                        buildTimeProperties, runtimeProperties,
                        discovererBuildItem.getDiscoverer(),
                        classpathResourceScannerBuildItem.getScanner()
                );
        var configBean = SyntheticBeanBuildItem
                .configure(MigrationsConfig.class)
                .runtimeValue(configRv).setRuntimeInit()
                .done();
        syntheticBeans.produce(configBean);

        var migrationsRv = migrationsRecorder
                .recordMigrations(configRv, driverBuildItem.getValue()); (3)
        var migrationsBean = SyntheticBeanBuildItem
                .configure(Migrations.class)
                .runtimeValue(migrationsRv)
                .setRuntimeInit()
                .done();
        syntheticBeans.produce(migrationsBean);

        return new MigrationsBuildItem(migrationsRv);
}

1	Remember what has been said in Section 3.3.1.1? This annotation moves the bunch of code eventually into runtime. The main reason here being the fact that we cannot longer postpone the access to a valid physical database connection
2	This is a build item produced by a different extension. By declaring it as a method parameter, the build-plugins will inject those items during build time for you
3	Here we retrieve the value from the build item. Be aware, it’s return type is actually a `RuntimeValue<Driver>`. The name is true to its nature: It is a value only accessible during runtime. It cannot be used here.

The usage of the runtime value of another extension must happen in a recorder in the runtime module. Listing 12 shows the relevant piece of the recorder and actually the smallest one in terms of LoC: All the heavy lifting has been done while creating the config:

Listing 12. MigrationsRecorder#recordMigrations

@Recorder (1)
public class MigrationsRecorder {

        public RuntimeValue<Migrations> recordMigrations(
                RuntimeValue<MigrationsConfig> migrationsConfig,
                RuntimeValue<Driver> driver
        ) {
                return new RuntimeValue<>(
                        new Migrations(migrationsConfig.getValue(), driver.getValue()) (2)
                );
        }
}

1	Marks the class as `@Recorder`. Any processor method annotated with `@Record(ExecutionTime.RUNTIME_INIT)` must declare an argument being a recorder.
2	Both parameters passed to `recordMigrations` are runtime values. Only here (and actually, only when the byte code created with this method) is executed, access to the runtime values value is possible.

Properties and configuration

To quote the Quarkus manual:

Some Quarkus configurations only take effect during build time, meaning is not possible to change them at runtime. These configurations are still available at runtime but as read-only […]. A change to any of these configurations requires a rebuild of the application itself to reflect changes of such properties.

Which makes perfectly sense in terms of Section 3.3.1.1: The more things are settled during build, the more things can be written in ~~stone~~ bytecode.

So there 4 different configuration phases:

build time: Values are read and available for usage at build time.
build and runtime fixed: Values are read and available for usage at build time, and available on a read-only basis at run time.
bootstrap: Values are read and available for usage at run time and are re-read on each program execution.
run time: Values are read and available for usage at run time and are re-read on each program execution.

Phases that are relevant for Neo4j-Migrations are build and runtime fixed as well as runtime. In therms of numbers, most options for Neo4j-Migrations are actually runtime options (compare the list here), such as the schema database or the target database (they might change depending on the URL (which is also runtime)) as well as the name of the user who executed the migration and which user to impersonate. It’s just fair to assume that when an application is connected to a different database, those change.

They go in a properties class annotated with @ConfigRoot indicating its phase as ConfigPhase.RUN_TIME:

Listing 13. MigrationsProperties (Excerpt)

@ConfigRoot(prefix = "org.neo4j", name = "migrations", phase = ConfigPhase.RUN_TIME)
public class MigrationsProperties {

        @ConfigItem
        public Optional<List<String>> externalLocations; (1)

        @ConfigItem
        public Optional<String> database;
}

1	A way out of the build time properties later on.

Our build time properties is a lot smaller and the whole class fits in the book:

Listing 14. MigrationsBuildTimeProperties.java

@ConfigRoot(prefix = "org.neo4j", name = "migrations", phase = ConfigPhase.BUILD_AND_RUN_TIME_FIXED)
public class MigrationsBuildTimeProperties {

        @ConfigItem
        public Optional<List<String>> packagesToScan;

        @ConfigItem(defaultValue = Defaults.LOCATIONS_TO_SCAN_VALUE)
        public List<String> locationsToScan;
}

Both these classes live in the runtime module and the only technical difference is the phase. In the Listing 14 it is ConfigPhase.BUILD_AND_RUN_TIME_FIXED: The source of those properties is consulted during build time, passed to the system, evaluated in a sensible way and eventually written to byte code.

And what should be pushed into build time? Discovering of classes and resources! Both of which Neo4j-Migrations need to do its refactoring work.

Why the externalLocations property during runtime? This is an additional feature that allows to configure filesystem only locations for those cases in which they aren’t any migrations to discover during build but instead are stored completely independent of the application.

How to access these properties? They must be injected into @BuildSteps, Listing 11 being one example. It receives both build- and runtime properties. Listing 15 is one more example coming up:

Discovering and registering resources and classes during deployment

Quarkus uses [Jandex] for discovering and indexing classes (of a given type, with certain annotations or listed in configuration files). The documentation is a bit sparse around that topic, but Baeldung^[16] has you covered. The index can be accessed through the core build items BeanArchiveIndexBuildItem and CombinedIndexBuildItem. The latter allows programmatically adding classes to the index via a computing view on the index: When a class is asked for by name, it will be added automatically.

Listing 15. MigrationsProcessor#createDiscoverer

@BuildStep
@SuppressWarnings("unused")
DiscovererBuildItem createDiscoverer(
        CombinedIndexBuildItem combinedIndexBuildItem,
        MigrationsBuildTimeProperties buildTimeProperties
) {

        var packagesToScan = buildTimeProperties.packagesToScan
                .orElseGet(List::of);
        var index = combinedIndexBuildItem.getIndex();
        var classesFoundDuringBuild = findClassBasedMigrations(
                packagesToScan, index); (1)
        return new DiscovererBuildItem(StaticJavaBasedMigrationDiscoverer
                .of(classesFoundDuringBuild));
}

Remember what I said in Section 3.1 about abstracting away any third-party API you might be using? This is the part where it becomes relevant. While I do love [ClassGraph], it is not the perfect candidate for the task at hand: Discovering resources and classes in a Quarkus build environment. While it worked in Quarkus 2.6, things broke in 2.7. Together @lh I was able to fix this, but I was convinced to try Quarkus way. In case you’re interested in the abstraction itself: It’s called Discoverer<JavaBasedMigration> and it lives in neo4j-migrations-core inside the neo4j-migrations-core package. In most scenarios it uses the ClassGraph implementation, in Quarkus it is a StaticJavaBasedMigrationDiscoverer that is preloaded during build time with a fixed set of classes.

I spare you the gory details of findClassBasedMigrations and just pick the essential part:

Listing 16. Asking the Jandex for all implementors of a given interface:

var dotName = DotName.createSimple(JavaBasedMigration.class.getName());
indexView
    .getAllKnownImplementors(dotName)
    .forEach(classInfo -> {
        // Do things with the classInfo found
        // For example loading it:
        try {
            Thread.currentThread().getContextClassLoader()
                .loadClass(cf.name().toString())
                .asSubclass(JavaBasedMigration.class);
        } catch (ClassNotFoundException e) {
            // Whatever
        }
    });

If you need access to class during build time, it must be loaded with the current threads classloader. Now what to do with those classes? I first create a DiscovererBuildItem in Listing 11. DiscovererBuildItem is a custom class, shown in the following listing:

Listing 17. DiscovererBuildItem.java

final class DiscovererBuildItem extends SimpleBuildItem {

        private final StaticJavaBasedMigrationDiscoverer discoverer;

        DiscovererBuildItem(StaticJavaBasedMigrationDiscoverer discoverer) {
                this.discoverer = discoverer;
        }

        StaticJavaBasedMigrationDiscoverer getDiscoverer() {
                return discoverer;
        }
}

This build item can be passed along as needed. And it is needed indeed: It is not enough to tell Neo4j-Migrations core "here are a set of classes", but these classes need to be registered for reflections as they are - most likely - never called from application code and therefore removed.

A ReflectiveClassBuildItem to the rescue. This is a build in class that is used like this:

Listing 18. MigrationsProcessor#registerMigrationsForReflections

@BuildStep
@SuppressWarnings("unused")
ReflectiveClassBuildItem registerMigrationsForReflections(DiscovererBuildItem discovererBuildItem) {

        var classes = discovererBuildItem.getDiscoverer()
                .getMigrationClasses().toArray(new Class<?>[0]);
        return new ReflectiveClassBuildItem(true, true, true, classes);
}

The boolean parameters corresponds which parts are eligible for reflection (constructor, methods, fields).

Last but not least: Why are resources (like in Cypher-script files) also build time fixed? Because they must be explicitly included in the native image when compiled via GraalVM, otherwise they are kept out. The abstraction of discovering is similar to searching for classes, you can find it in the processor, too.

They key to have resources in the native image is the NativeImageResourceBuildItem:

Listing 19. MigrationsProcessor#addCypherResources

@BuildStep
@SuppressWarnings("unused")
NativeImageResourceBuildItem addCypherResources(
        ClasspathResourceScannerBuildItem classpathResourceScannerBuildItem
) {

        var resources = classpathResourceScannerBuildItem.getScanner()
                .getResources();
        return new NativeImageResourceBuildItem(resources.stream()
                .map(ResourceWrapper::getPath)
                .collect(Collectors.toList()));
}

Quarkus offers some ways to create additional resources during build. These most be added explicitly to native as well, this is not done automatically.

Adding synthetic beans

As a final step, I wanted the user to be able to access the Migrations object in their (CDI) code and if enabled, execute migrations as soon as the Quarkus application starts.

The former is done via a BuildProducer<SyntheticBeanBuildItem> dependency, seen in Listing 11. Such a producer is a way to return multiple (different) build items from a build step. Here, I add the configuration and migrations bean as SyntheticBeanBuildItem.

Last but not least, things happening on startup is done via a ServiceStartBuildItem. What is passed to a startup build item needs to be recorded in the build step producing the item as shown in Listing 20:

Listing 20. MigrationsProcessor#applyMigrations

@BuildStep
@Record(ExecutionTime.RUNTIME_INIT)
@SuppressWarnings("unused")
ServiceStartBuildItem applyMigrations(
        MigrationsProperties migrationsProperties,
        MigrationsRecorder migrationsRecorder,
        MigrationsBuildItem migrationsBuildItem
) {

        migrationsRecorder.applyMigrations(migrationsBuildItem.getValue(),
                migrationsRecorder.isEnabled(migrationsProperties));
        return new ServiceStartBuildItem(FEATURE_NAME);
}

Contributing to the developer console

I tried to cover what’s possible in a small video here. These things are usually done via an additional processor if they have some dynamic nature or just with "plain" Qute template.

In terms of dynamic features (like in our case, accessing the Migrations bean and calling methods on it), there’s an SPI for that allowing to register VertX-HTTP based handlers.

In terms of build time and runtime behaviour they are similar to what has been discussed above, in terms of SPI I did read some Quarkus source code.

3.3.4. Quintessence

Without deep understanding how a framework - in this case Quarkus and the underlying target deployment of GraalVM native - works, it is not really possible to integrate an like Neo4j-Migrations in an meaningful way.

But I’d go a step further: Without understanding the implications of deployment augmentation and runtime behaviour: The startup time, the quick turnaround, it’s nice and shiny, but it’s too easy to miss on something when things go wrong.

My recommendation: For a "day 2" scenario, make sure what kind of target environment you have.

4. Releasing things

In this chapter, we will discuss such things as semantic versioning, indicating breaking changes early and of course, release tooling.

4.1. Semantic versioning

4.2. Release tooling

Resources

[ASM] https://asm.ow2.io
[a-kind-of-magic-2016] https://info.michael-simons.eu/2016/06/01/talking-about-custom-spring-boot-starter-at-spring-io/
[bash] https://www.gnu.org/software/bash/
[ClassGraph] https://github.com/classgraph/classgraph
[COCOMO] https://en.wikipedia.org/wiki/COCOMO
[Cypher-DSL] https://github.com/neo4j-contrib/cypher-dsl
[exec-maven-plugin] https://www.mojohaus.org/exec-maven-plugin
[GraalVM] https://www.graalvm.org
[GraalVMTooling] https://info.michael-simons.eu/2020/09/15/about-the-tooling-available-to-create-native-graalvm-images/
[Helidon] https://helidon.io
[JBang] https://www.jbang.dev
[Jandex] https://github.com/wildfly/jandex
[japicmd] https://github.com/siom79/japicmp
[jep238] https://openjdk.java.net/jeps/238
[jep261] https://openjdk.java.net/jeps/261
[jep286] https://openjdk.java.net/jeps/286
[jep328] https://openjdk.java.net/jeps/328
[jep396] https://openjdk.java.net/jeps/396
[maven-enforcer-plugin] https://maven.apache.org/enforcer/maven-enforcer-plugin/
[Micronaut] https://micronaut.io
[oss-quickstart] https://github.com/moditect/oss-quickstart
[picocli] https://picocli.info
[Quarkus] https://quarkus.io
[scc] https://github.com/boyter/scc
[sortpom] https://github.com/Ekryd/sortpom
[SpringBoot] https://spring.io/projects/spring-boot
[SpringFramework] https://spring.io/projects/spring-framework
[stein-maven-coordinates-and-module-names] https://sormuras.github.io/blog/2019-08-04-maven-coordinates-and-java-module-names.html
[Testcontainer] https://www.testcontainers.org