Pluggable Annotation Processing API

GWT Con - 2015 - LTE Consulting

Arnaud Tournier

Passionnate developper, trainer and architect at LTE Consulting.

Speaker at Devoxx, GWT.create, Paris/Toulouse JUG, etc...

Email : ltearno@gmail.com

Twitter : @ltearno

Website : www.lteconsulting.fr

Full stack (x86_64 to JavaScript)

Presentation available on

lteconsulting.fr/annotation-processing

And the demo project is available at github.com/ltearno/gwtcon-jsr269

GWT 3 will drop generators !

JSR 269 to the rescue???

Pluggable Annotation Processing API

Code generation in Java (source, byte-code and resource).

Integrated with the Java compiler.

Based on annotations. The developer's annotation processor gets most of the program's AST.

Used for ?

  • RPC stubs,
  • Reflection stubs,
  • UI generation,
  • Configuration file generation,
  • Code checkers, Build breakers,
  • Dependency injection,
  • Glue code generation,
  • Your own needs !

In the GWT context

GWT 3 will abandon generators because the functionality exists in standard Java : JSR 269.

Even with GWT 2.8 it makes sense to use it.

Causes migration problems, with most of the time quick resolution.

Good points

API is easy to use.

Generated code is visible and debuggable,

Generated code is known before compilation so you can reference it directly (no GWT.create).

No overhead at runtime.

User experience (refresh time in IDE).

Does not depend on byte code : GWT compatible

Bad points

Only annotated elements trigger processing. API makes it difficult to coordinate processing of multiple classes over multiple rounds (bad for incremental compilation).

Dependency to external resource is not managed either.

A Brief history

Javadoc comments

XDoclet (2002)

/****
* Account entity bean
*
* @ejb.bean
*     name="bank/Account"
*     jndi-name="ejb/bank/Account"
*     primkey-field="id"
*     schema = "Customers"
* ...
*/
public class MonBean { ... }

APT

Introduced in JDK 5, Annotation Processing Tool was removed with Java 7 because it can't support new language elements.

Runs outside of javac.

API includes com.sun.mirror packages.

Pluggable Annotation Processing API

Fixes the sins of the past.

JSR-269 has been included since Java 6 (2006).

Runs inside of javac.

API is able to welcome new language features.

How it works

Annotation processors must be registered.

Java source files are compiled during rounds.

Each round, processors are activated and receive the program's AST.

They can then generate files which will be part of the next round.

When no file is generated during a round, real compilation happens.

A sample

Let's say we want to develop a tool that automatically generates UIs from any POJO...

Demo's plan

We will have to

Write an annotation,

Write an annotation processor,

Register our processor through SPI,

And package our library.

Annotation creation

This is the annotation we use to trigger the custom annotation processing :

import java.lang.annotation.*;

@Target( { ElementType.METHOD } )
@Retention( RetentionPolicy.SOURCE )
public @interface AutoUi
{
}

Annotation processor implementation

@SupportedAnnotationTypes({"fr.lteconsulting.AutoUi"})
@SupportedSourceVersion(SourceVersion.RELEASE_8)
public class AutoUiProcessor extends AbstractProcessor  {
    @Override
    public boolean process(
            Set<TypeElement> annotations,
            RoundEnvironment round) {
        for(TypeElement element : 
            round.getElementsAnnotatedWith(AutoUi.class)) {
            ...
            JavaFileObject javaFile = filer.createSourceFile(classFqn);
            Writer writer = javaFile.openWriter();
            ...
        }
        return true;
    }
}

Registering through SPI

Java compiler searches annotation processors through SPI.

Add a file named META-INF/services/javax.annotation.processing.Processor containing the annotation processors' fqn list :

fr.lteconsulting.AutoUiAnnotationProcessor

Other ways to register : javac has special flags. The CompilationTask also has methods to set the processors to be used.

Packaging

The simplest way is to have the annotation and its processor in the same jar package.

Maven tip: dont forget to use the<compilerArgument>-proc:none</compilerArgument> options

Using the processor

In a project with the processor's jar in the classpath, we can use the annotation...

Eclipse tips :

  • Eclipse uses its own java compiler, JDT. Use m2e-apt to configure your project if you work with maven.
  • Don't forget to close the processor project to have it activated.

The Pojo class

@AutoUi
public class Person {
    @Label("Name")
    private String firstName;
    
    private String lastName;

    private int age;

    // getters and setters
}

The Generated class

public class PersonAutoUi extends Composite {
    private final TextBox firstNameTextBox = new TextBox();
    ...

    public void setPerson(Person pojo) { ... }

    public void updatePerson(Person pojo) { ... }

    ...
}

Using the generated class

public class Application implements EntryPoint {
    public void onModuleLoad() {
        Person person = new Person(...);

        // Use of the generated UI code
        PersonAutoUi editor = new PersonAutoUi();

        // Feed the ui
        editor.setPerson(person);

        // Update the pojo with the ui values
        updateButton.addClickHandler((e)->{ editor.updatePerson(person); });

        RootPanel.get().add(editor);
    }
}

How is it possible ?

Using the not yet generated file is possible because the java compiler deffers processing of the NotFoundSymbolException.

The error is raised at the end of the parsing and annotation processing process if the symbol has not been generated.

The API : a brief introduction

API Overview

Filer class : generate files (source, byte-code, resource)

Language Model classes : browse the program's structure,

Messager class : to communicate with the user,

Other tools : element and type tools

see javadoc of the javax.annotation.processing and javax.lang.model packages.

API : Java source representation

Element : representation of a language construct (class declarations, methods, ...). Ex: getSimpleName(), ...

  • Supports all the language structures through the accept(visitor) and getKind() methods.
  • Hierarchical structure : getEnclosedElements(), getEnclosingElements().

TypeMirror : Type representation, almost like Class<?>

API : Accessing elements

Elements are given as parameters in the process(...) method of the generator. Annotated elements are retrieved like this :

  • round.getElementsAnnotatedWith(AutoUi.class).

All the classes parsed during the current round can be obtained with :

  • round.getRootElements().

Elements can also be retrieved with the utility methods :

  • elementsUtils.getTypeElement(fqn),
  • and elementsUtils.getPackageElement(fqn).

API : Filer

javax.annotation.processing.Filer

Creating a new Java source

// obtains Filer from the abstract super class
Filer filer = processingEnv.getFiler();

// create only create new files
JavaFileObject jfo = filer.createSourceFile(classFqn);

// compose your java fi
PrintWriter pw = new PrintWriter( jfo );

API : Messager

Outputs messages to the user.

Can also generate errors and break the build. Very handy to assert things on the code.

messager.printMessage(Kind.ERROR, "Cannot find an ID field !");

IDE integration : hints on the API for the user.

API : Tools

Many tools can be retrieved from the processingEnvironment field of the AbstractProcessor

Filer getFiler();
Messager getMessager();

Elements getElementUtils();
Types getTypeUtils();

Map<String, String> getOptions();
SourceVersion getSourceVersion();
Locale getLocale();

Other static methods and classes are helpful :

ElementFilter, AbstractVisitors...

Unit tests

Testing a compilation

JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();

// Oracle JDK: task can be cast into
// com.sun.source.util.JavacTask
CompilationTask task = compiler.getTask(...);

// forces the processors
task.setProcessors(processors);

boolean successful = task.call();
diagnosticCollector.getDiagnostics();  // structured logs
fileManager.getOutputFiles();

Compile-Testing

github.com/google/compile-testing

Annotation Processor testing library developped by Google to help developping of the Auto and Dagger projects.

Positive tests

assert_().about(javaSource())
   .that(forResource("PojoTest.java"))
   .processedWith(new AutoUiProcessor())
   .compilesWithoutError()
   .and()
   .generatesSources(forResource("PojoTestAutoUi.java"));

And negative ones

JavaFileObject fileObject = forResource("PojoErrorTest.java");

assert_().about(javaSource())
    .that(fileObject)
    .processedWith(new AutoUiProcessor())
    .failsToCompile()
    .withErrorContaining("No getter found")
    .in(fileObject)
    .onLine(23)
    .atColumn(5);

Miscellanous

Limitations

Not a full access to the code's AST (instructions).

Processors cannot depend one on the other.

Incremental compilation is difficult when having dependencies to more than one element or to external files.

on Eclipse : Alt+F5, on maven : have to disable incremental compilation.

Most of the time those limitations are not embarassing.

Hacking

  • Lombok : based on JSR 269 and hacking both javac and jdt in order to acces to internal implementations and mutate the class AST.
  • Technical explanations in The Hacker's guide to JavaC

Note on using templates

Try to generate the minimal amount of code, and base it on generic implementations. This will ease debugging.

Tools :

  • Velocity, ...
  • Java Poet, ...
  • String.replaceAll()

Libraries known using JSR-269

  • JPA meta-model generation (JSR-317),
  • Dagger,
  • Google Auto,
  • Immutables,
  • Lombok,
  • GWT (RequestFactory),
  • Hexa Binding...

That's all, thanks !

See you !

Slides : lteconsulting.fr/annotation-processing

Demo project : github.com/ltearno/gwtcon-jsr269

Twitter : @ltearno

LTE Consulting : lteconsulting.fr

LinkedIn : fr.linkedin.com/in/lteconsulting