Generation Gap Pattern

Heiko Behrens — Thu, 23 Apr 2009 19:35:51 +0000

The Generation Gap Pattern is a common way to weave manually written code into generated software artifacts. While this pattern describes how to structure your classes logically it does not teach you where to place these classes physically as files into the filesystem.

Figure 1: The Generation Gap Pattern

Separating generated files from manually written code is best practice for many reasons. It enables you to use version control wisely and makes the distinction between generated and manually written code easy. The Eclipse EMF Modeling Workflow Engine together with M2T Xpand allows you to accomplish this goal in different ways.

In this post, I will present three different approaches that share some common ideas. Each strategy uses a dedicated directory for generated class files whereas manually written subclasses are stored for themselves in a different directory.

General Directory Layout

Both sets of classes belong to the same package but reside in different source folders. Framework classes should be kept in yet another directory and might be placed in a package specific for the framework as seen in Figure 2.

Figure 2: Schematic directory layout of an Eclipse Java project

In Eclipse you can configure such a project structure by adding new source folders (New → Source Folder…) or by declaring ordinary folders as source folders in the project properties (Properties → Java Build Path → Source).

Please note that folder names such as src-generatedClasses are meant to be schematically and stand for a category of folders each. In real world projects you might have several source folders that contain generated artifacts and you might deal with many folders or even projects that contain manually written classes. In the following sections you will see how these categories of folders can be used in practice.

Generate Once

Peter describes a directory layout many Xtext users are familiar with. Generated artifacts go into a folder src-gen whereas manually written classes will be stored in a folder src-once. Each time you run the generator the content of folder src-gen will be purged and generated again whereas the the content of src-once will remain untouched.

Figure 3: Generate Once collects one-time generated artifacts for future editing

Since each model element A leads to two classes AbstractA and ConcreteA (Generation Gap Pattern) you can reduce the amount of manual work by generating the class ConcreteA if it does not exist, yet. During the next run of the generator the class ConcreteA won’t be generated again, hence the name “once”.

Configure an additional outlet in your EMF MWE file:

Using M2T Expand you can than refer to the outlet MANUAL where appropriate

«DEFINE ConcreteClass FOR Element»
   «FILE this.fileName() MANUAL»
   ...
«ENDDEFINE»

This approach works well in many scenarios but has its limits. As seen in Figure 3 problems arise whenever you delete or rename a model element. In these cases each concrete artifact at src-once that does not correspond to a model element anymore is abandoned and will remain with compile errors. This is desired for classes that hold manual code but is cumbersome for those that do not.

Figure 4: Generate Once in package explorer

For scenarios where a huge amount of generated classes is customized only in a few cases the “Generate Once” approach leads to unwanted manual refactoring effort including file movement. Life is getting worse when you are working with a complex folder structure instead of a single flat directory. Figure 4 might give you an idea of how things will look for growing models.

Conditional Generation 1 (with one gen-folder)

The following approach plays well when only a small subset of classes has to be extended with manual code. According to the Generation Gap Pattern for each model element A there is a generated class AbstractA as well as an concrete class ConcreteA. But in contrast to the “Generate Once” approach both files will be placed in the very same folder src-gen and therefore deleted and regenerated for each generator run.
There is one exception to this behavior. Whenever the generator finds a file ConcreteA in the folder src-man it skips the generation process for this particular file. This is how you introduce manual code.

Figure 5: Conditional Generation 1 reduces manual refactoring effort

At first glance this approach seems to be similar to the solution above. But when deleting or moving model elements this approach really pays off. Each concrete class that does not contain manual code will be deleted and regenerated. Therefore, when moving or deleting the model elements no additional manual work has to be done. Also, a quick look at src-man reveals any code that has been written manually.

Figure 6: Conditional Generation 1 in package explorer

To implement the desired behavior of the generator users of MWE can configure an outlet with a special veto strategy. Please note that the outlet MANUAL does not need to keep files (overwrite=”true”), they will be purged by the directory cleaner anyway.

Such a veto strategy would read like:

public class CondGenVetoStrategy implements VetoStrategy, VetoStrategy2 {

	private String path;

	public void setPath(String value) {
		path = value;
	}

	public String getPath() {
		return path;
	}

	public boolean hasVeto(FileHandle fileHandle) {
		URI fileToBeGenerated = fileHandle.getTargetFile().toURI();
		URI pathToGenerateInto = new File(fileHandle.getOutlet().getPath()).getAbsoluteFile().toURI();
		URI relativeFileName = pathToGenerateInto.relativize(fileToBeGenerated);
		File potentiallyExistingFile = new File(this.getPath(), relativeFileName.toString());

		return potentiallyExistingFile.exists();
	}

	public boolean hasVetoBeforeOpen(FileHandle handle) {
		return hasVeto(handle);
	}
}

oAW 4.x: VetoStrategy2 has been introduced with oAW 4.3.1.

Unfortunately, the advantages of this approach do not come for free: Compared to “Generate Once” it is more difficult to introduce manual code since it is harder to understand the needed steps. One has to create a concrete class in the src-man folder manually (or move it from src-gen and remove all comments that mark them as generated) where no class exists, yet. This might seem to be a no-brainer but can lead into trouble when defining a process for large projects. This is especially true for the unlikely case that you check-in generated code of src-gen.

Conditional Generation 2 (with two gen-folders)

A slighty different way that basically follows the same idea introduces yet another folder src-gencond.

Figure 7: Conditional Genration 2 lists classes that can implemented manually

As you can see in Figure 7 the generated artifacts are split into the folder src-gen where the abstract classes reside and the newly introduced folder src-gencond. The latter contains only the concrete classes that could be replaced by manually implemented classes. In contrast to the “Conditional Generation 1” approach the users are now able to find classes that can be augmented be manual implementations without any challenge. Users still have to move classes from src-gencond to src-man by hand but it is simpler to communicate the necessary steps.

Figure 8: Conditional Generation 2 in package explorer

As you can see in Figure 8 one can easily identify that there are two classes that could be implemented by hand. Such a folder structure can pay off when different groups of developers are dealing with a fairly large and scattered set of classes.

In order to realize “Conditional Generation 2″ you simply reconfigure the outlet MANUAL to write its output to src-gencond. Also, a second directory cleaner removes all files from this directory before code generation starts.

Comparison

Each of these solutions do their job in real world MDSD projects. Where projects with rather stable interfaces go with the “Generate Once” approach larger projects address growing maintenance effort with the “Conditional Generatation 1/2” solution. The move from “Conditional Generatation 1” towards “Conditional Generatation 2” is rather trival and can smoothly be taken within a running project.