Mono-repo build with Gradle

Sometimes, we are faced with a project that is part open source and part proprietary. Here’s how I address the challenges of synchronisation of the builds.

By Bonitasoft R&D Engineer Baptiste Mesta

When faced with a partly open source and partly close source project, it is common to use a git sub-tree to synchronise them. The open source project is added in a folder of the closed source project, and all development happens in that root project (alias the mono-repo). On top on that we want our build tool to handle the sub-tree project as it is a part of the mono-repo.

This is the tricky part, as the sub-tree must be buildable and must be also buildable from the mono-repo. In cases of multi-module projects, we also want to express dependencies between modules of those projects and have our IDE detect those dependencies.

One solution is provided by Gradle: it is called includeBuild. This solution works great but has a few drawbacks for this use case.

The problem with includeBuild

When using the includeBuild method, if you run gradle test at the root of the project, it will run tests only on projects “natively” in the root project. This can be ok when you only need to include libraries with their own life cycle, but in our use case we want to have a single task to build and test everything.

With includebuild we would have

$ gradle run
> Task :other-module:run
com project: Hello World

Even if we have a task named run in one module of the sub-tree, only the task from module “natively” in the root is ran.

With our approach all tasks with the given name will be launched.

$ gradle run
> Task :other-module:run
com project: Hello World
> Task :my-project:module-b:run
oss project: Hello World

Sources of the example

You take a look at the example project here:

https://github.com/baptistemesta/gradle-sub-tree-example.

It will be helpful to follow what I have done in this post.

How it works

The main idea is to include the settings.gradle of the sub-tree in the mono-repo project. However in order to keep the name constant when declaring dependencies, we must play a little bit with the paths of the projects.

Setting up the sub-tree

The sub-tree do not declare its projects in settings.gradle but in an other file, here oss-settings.gradle.

All declared projects are children of a new root project, here :my-project and this project is also included.

The settings.gradle import this file and change the paths:

apply from: ‘oss-settings.gradle’
def fixPath
fixPath = { project ->
  String relativeProjectPath = project.projectDir.path.replace(settingsDir.path, “”)
  project.projectDir = new File(relativeProjectPath.replace(“/my-project/”, ‘’))
  project.children.each fixPath
}
rootProject.children.each fixPath

Dependencies between modules can be expressed as usual but it must include this new root project.

dependencies {
  compile project(‘:my-project:module-a’)
}

Setting up the mono-repo

Once the sub-tree itself is setup, few tweaks must be made to the mono-repo project.

settings.gradle must apply the oss-settings.gradle, set the path correctly, include the root project of the sub-tree then add its own projects as usual.

apply from: ‘sub-tree/oss-settings.gradle’
def fixPath
fixPath = { project ->
  String relativeProjectPath = project.projectDir.path.replace(settingsDir.path, “”)
  project.projectDir = new File(relativeProjectPath.replace(“/my-project/”, ‘sub-tree/’))
  project.children.each fixPath
}
rootProject.children.each fixPath
include ‘:my-project’
project(‘:my-project’).projectDir = “$rootDir/sub-tree” as File
include ‘:other-module’

Results

In this example project, running the run task in the sub-tree directory gives:

$ gradle run
> Task :my-project:run
oss project: Hello World

and running the same task in the mono-repo:

$ gradle run
> Task :other-module:run
com project: Hello World
> Task :my-project:module-b:run
oss project: Hello World

Plugins sharing

We often use custom plugins directly in the buildSrc to share build logic. This behavior can be kept using the following method.

In the sub-tree project add you buildSrc directory and add an extra build file to declare the plugin:

apply plugin: ‘groovy’
apply plugin: ‘java-gradle-plugin’
gradlePlugin {
  plugins {
    ossPlugin {
      id = “oss-plugin”
      implementationClass = “OSSPlugin”
    }
  }
}

In the mono-repo, the buildSrc project must include this project as a runtimedependency

build.gradle

dependencies {
  runtime subprojects
}

settings.gradle

include ‘:oss-buildSrc’
project(‘:oss-buildSrc’).projectDir = “$rootDir/../sub-tree/buildSrc” as File

With this technique, plugins from the plugins of the sub-tree project can be used in the mono-repo projects and the mono-repo can declare its own plugins.

When running the tasks from the sub-tree:

> Task :my-project:module-b:customOSSTask
OSSPlugin is applied

When running the tasks from mono-repo:

> Task :other-module:customComTask
ComPlugin is applied
> Task :other-module:customOSSTask
OSSPlugin is applied
> Task :my-project:module-b:customOSSTask
OSSPlugin is applied

Conclusion

This is the solution I found to handle this kind of use case. If you have encountered similar situations, I’d be very interested to know how you handled these kinds of issues.

This post was originally published on 24 July 2018 by Baptiste Mesta.