Use standard techniques

In cooking, working with traditional techniques often provides the best results. The recipes have had the most testing and have been optimized over time. Compare to cooking with water. We know the properties of cooking in water well: The maximum temperature is more or less self regulating (as long as there is water in the cauldron) and under otherwise similar conditions such as pressure, the result is pretty much predictable and almost risk-free.

The same with programming. The techniques shown in this part about reflection should NOT be used on a daily basis but be reserved for special cases, such as making a tool set (maybe called a framework), that can then make working on standard recipes a bit more efficient for the programmer. Frameworks typically pay their keep with a little performance degradation, because they need some extra work either at application startup, in preparation of the executable, when used or all of the former.

If the use of a framework makes the normal programming more complex, you might want to consider not using it.
In all cases, you should be aware of the consequences.

Reflection or class introspection is used to leverage the information that is available in classes. Using the information that is available about a class and by implication about the instances of the class can help to avoid copy-and-waste programming. It helps keeping the code DRY

1. Reflection

Reflection is a multi-edged tool.

  • Reflection can be used to access parts of instances that would otherwise not be available.

  • Reflection can be used to list information about fields, methods, and constructors.

  • Access via reflection is slower than regular access, because of the safety/security checks that are made on each access

  • It is also slow for the extra indirection needed to do the work when compared to the optimized instructions for say access a field.

  • It is less type-safe, so you loose much of the comfort you have in the IDE, such as code-completion or intellisense(tm).

    • For instance you lookup a method by name (a String) and that makes you deal with at least one exception. This still does not produce the actual method, but instead a Method object which you must give the reference to the object on which you want to apply the method and the parameters to that method in an Object[] array.

Some of the problems can be mitigated a little with the proper amount of smartness, in particular caching previous results, and also by using MethodHandle in the java.lang.invoke package introduced in Java 7.

Things that can be done using reflection is building access templates to construct or read and write the fields of plain old java objects, typically entity classes, without having to copy and paste a lot of similar code. This, and code generation, is the approach that some frameworks use to reduce the amount of boilerplate or copy and paste code that needs to be written.

We will be doing all of that in this part.

2. Reflection and correcting student exams.

Not unit testing, but assessing the grades student get for their solutions.

We teachers use reflection quite a bit when correcting the students' solutions to tasks in performance assessments. This is also the way that the MOOC in PRC1 does parts of it’s testing. This is a special kind of unit testing, that is NOT the norm, since it is utterly unsuited for TDD, but serves as an illustration on what you can do with reflection.

As teacher / examiners

  • We cannot expect that a class that the candidate should write is present.

  • At the time of writing the tests the class certainly is not available. We therefore have to instantiate the objects using reflection.

  • Sometimes the task states the requirement of only having a specific set of field types with specific visibility or other properties.

  • We want the candidates to stick to the naming conventions.

  • When you run the jacoco coverage plugin, jacoco itself adds so called synthetic members, which are not found in the source code of the business class, but are added by jacoco by way of instrumentation, and we need to strip those out in our assessment of the students' code.

  • We want to check that a method or field has the required visibility, static-ness, finality, or abstractness.

As an example verifying the visibility and other modifiers such as static and final may be subject of the correction work we need to do.

In the java .class file, the Modifier of fields, methods, and of types such as classes and interfaces are simply defined as a bit set packed into an int, and stored with the byte code in the class file.

The following modifiers are defined as follows:

Modifier

keyword

int value

Applies to

PUBLIC

public

1

C, F, M

PRIVATE

private

2

C, F, M

PROTECTED

protected

4

C, F, M

STATIC

static

8

C, F, M

FINAL

final

16

C, F, M

SYNCHRONIZED

synchronized

32

M

VOLATILE

volatile

64

F

TRANSIENT

transient

128

F

NATIVE

native

256

M

INTERFACE

interface

512

C

ABSTRACT

abstract

1024

C, M

STRICT

strict

2048

C, M

Applies to means Class, Field or Method.

Note that default or package private has no modifier bit of its own. If none of public, protected, or private is set, then that is when you get the default.

As an example, a public final method has modifier 1+16+1024 = 1041, or 0x411.

Below you see a selection of the helper methods we use to correct performance assessments.

Does the class' field name comply with the standard naming conventions?
/**
     * Check the field definition of a class, including naming conventions.
     *
     * @param targetClass to check
     * @param modifiers visibility, static, final
     * @param type of the field
     * @param fieldName of the field
     * @throws SecurityException when field is not accessible.
     */
    public static void checkFieldAndNaming( Class<?> targetClass,
                                            int modifierMask,
                                            int modifiers,
                                            Class<?> type, String fieldName )
            throws SecurityException {
        if ( ( PUBLIC | STATIC | FINAL ) == modifiers ) {  (1)
            assertAllUpper( fieldName );
        } else {                                           (2)
            char firstChar = fieldName.charAt( 0 );
            assertEquals( "first char not lower case", "" + (char) Character.
                          toLowerCase(
                                  firstChar ), "" + (char) firstChar );
        }
        checkField( targetClass, modifierMask, modifiers, type, fieldName );
    }
1 Needs to be all UPPER CASE
2 Needs to start with a lower case character.
Check the modifiers on a field.
/**
 * Check the field definition of a class.
 *
 * This method tests if the required modifiers are set. Example: to check
 * private, but not require final, specify both modifierMask and Modifier.PUBLIC |
 * Modifier.PRIVATE | Modifier.PROTECTED and as modsRequired, thereby accvepting any
 * value of the final modifier.
 *
 * @param targetClass to check
 * @param modifierMask visibility, static, final
 * @param modsRequired required modifiers
 * @param fieldType of the field
 * @param fieldName of the field
 * @throws SecurityException when field is not accessible
 */
public static void checkField( Class<?> targetClass,
                               int modifierMask,
                               int modsRequired,
                               Class<?> fieldType,
                               String fieldName )
        throws SecurityException {
    Field f = null;
    try {
        f = targetClass.getDeclaredField( fieldName );
        assertEquals( "field " + fieldName + " should be of type "
                + fieldType, fieldType, f.getType() );
        int fieldModifiers = f.getModifiers();
        if ( ( modifierMask & fieldModifiers ) != modsRequired ) {
            fail( "field '" + f.getName()
                    + "' should be declared '"
                    + Modifier.toString( modsRequired )
                    + "', you declared it '"
                    + Modifier.toString( fieldModifiers ) + '\'' );
        }
    } catch ( NoSuchFieldException ex ) {
        fail( "your class '" + targetClass
                + "' does not contain the required field '"
                + Modifier.toString( modifierMask )
                + " "
                + fieldType.getSimpleName()
                + " " + fieldName + "'" );
    }
}

Class Genealogy

In the following exercise you will use the information that is available in the Class object. You will find the interfaces, super classes, and interfaces implemented by the super classe and the class as well as the fields that are defined in each of the classes in the field hierarchy.

Class Genealogy
jbuttontree
Figure 1. inspiration for this task

Your task is to create a program that lists the class hierarchy of class names given on the command line.

The output should be a tree-like structure with Object at the top and the named class at the bottom and all intermediate super classes in between in proper order. For every level in the hierarchy, add two spaces for indentation. At the end the program of the hierarchy it should show the non-static fields that are defined in all classes in the hierarchy with the modifiers in the order of definition.

To get a class object you can use Class.forName(String name) which, if the class is loadable by the JVM, is loaded.

As usual, start with writing the tests. The test class has two tests:

  1. One to show the genealogy of the Genealogy class itself.

  2. One to show the class hierarchy of javax.swing.JButton.

The picture only shows the direct lines of ancestry, not the interfaces. In this task you should show the implemented interfaces plus the fields.

The elements (or supertypes, i.e. super classes and interfaces) that should be contained in the javax.swing.JButton are:

java.lang.Object

java.awt.Component

java.awt.image.ImageObserver

java.awt.MenuContainer

java.io.Serializable

java.awt.Container

javax.swing.JComponent

javax.swing.TransferHandler$HasGetTransferHandler

javax.swing.AbstractButton

java.awt.ItemSelectable

javax.swing.SwingConstants

javax.swing.JButton

javax.accessibility.Accessible

At the bottom of the hierarchy we want all non-static Declared fields in the class hierarchy in declaration order, that is in order from the top of the hierarchy to the bottom and within the classes in field order.

To determine if a field is non-statid use
Predicate<Field> nonStatic = ( Field f ) -> !Modifier.isStatic( f.getModifiers() );

When you ask an interface for its modifier, it will say INTERFACE ABSTRACT, which is not very informative. So it would be better to only show the visibility of a type and field. Visibility is encoded in the lower three bits of the type or member Modifier, meaning that String visibility = Modifier.tostring( m.getModifiers() & ( 7+16 )); produces the visibility and finality in string form.

In the project you will find some sample classes to test your application with.

Testing can be done by using assertThat(String).contains(String…​.). Doing so will get a quick full coverage, but testing the exact result may be a bit tricky.

Formatting test can be done with your eye-balls.

You do not have to test the sample classes.

Example output for java.lang.StringBuilder
$ ./run.sh java.lang.StringBuilder
class hierarchy of [java.lang.StringBuilder]
 public java.lang.Object
   abstract java.lang.AbstractStringBuilder  implements public java.lang.Appendable, public java.lang.CharSequence
     public final java.lang.StringBuilder  implements public java.io.Serializable, public java.lang.Comparable
//declared fields:
{
     byte[] value  // declared in: AbstractStringBuilder
   , byte coder  // declared in: AbstractStringBuilder
   , int count  // declared in: AbstractStringBuilder
}

Note that the fields are package private, and declared in a package private class, the AbstractStringBuilder. If you run the program on java.lang.StringBuffer and see almost the same. This is because these two classes are siblings sharing the same immediate parent class, the package private AbstractStringBuilder, which does most of the actual work.

2.1. Generating code using Templates

In the Java versions before Java 16, generating source code could be a bit tedious, because in Java a String can’t contain line breaks, that is a string cannot contain multiple lines with only one set of quotes. The Java 14+ text block feature solves that, and is final in Java 16, but we will deal with it here anyway.

If you know what your code looks like, your can put it in a file, maybe even starting by copying from an existing java class. The template below is created that way and has our advised code style.

Assume your template text looks like
package %1$s;

import deconstructorregistry.Deconstructor;
/**
 * This is generated code. Do not edit, your changes will be lost.
 */
public class %2$sDeconstructor {

    /**
     * The purpose of self registration is not being able to
     * create new instances, other then by the class loader.
     */
    private %2$sDeconstructor() {
    }

    static {
        Mapper.register( %2$s.class, new %2$sDeconstructor() );
    }

    /**
     * Deconstruct an entity into an array.
     * @param %2$s the victim
     */
    @Override
    public Object[] deconstruct(  %2$s %3$s ) {
       return new Object[]{
            %4$s
       };
    }
}

In the template you see special 'tokens' like %2$s, which means: use the second parameter, and interpret as string. In this case the template specifies 4 parameters:

  1. is the package name,

  2. is the entity type name,

  3. is the parameter name of the deconstructor method

  4. is the the place where the list of getters should land.

Reading the template from a file and put it in a String constant.
    private static String templateText( String templateName ) {
        String text = "";
        Class clz = Constants.class;

        try ( InputStream in = clz.getResourceAsStream( templateName ) ) {
            text = new String( in.readAllBytes​() );
        } catch ( IOException ex ) {
            Logger.getLogger( Constants.class.getName() )
                    .log( Level.SEVERE, ex.getMessage() );
        }
        return text;
    }
    public static String CODE_TEMPLATE = templateText(
            "CodeTemplate-java.txt" );
Use the template with the positional parameters.
    String classText = String.format( CODE_TEMPLATE,
            GENERATED_PACKAGE, (1)
            typeName, (2)
            paramName, (3)
            getters( entityType ) (4)
    );
1 Parameters to
2 the
3 template
4 as explained above.

Entity Deconstructor Generator

Some of the operations you want to do on entities do not belong to the responsibility of the entity. As an example: providing various external representations, such as a list of entities as a csv (comma separated value) file. It is not the entities responsibility to format its information in every possible format. Often the toString() method is not really fit for business but mainly meant for debugging. You use an external class to achieve a specific format. The EntityDeconstructor is such a helper for the simplest cases, such a simple csv. Writing such helper class can be largely automated, taking most of the boilerplate coding away from the easily bored but 😎 programmer.

Entity Deconstructor

This exercise has two parts, the generator and the registry, so the generator generates code that fits the registry exercise

To ease the handling of handling entities in a business application, a deconstructor method is a nice to have.

A deconstructor takes an entity and returns an array of Objects containing all fields of the entity.

This allows things such as getting the data to create a csv representation without having to do that inside the entity class. In the last exercises in this part you will use even more information, so you can create things such as json format, yaml, or fill a prepared statement when you want to put entities into a database using jdbc (which we will do next week).

example usage
  for( Object fieldValue: deconstruct( student )) {
     // do something with the field value
  }

This deconstructor is not automatically provided by the IDE, and it takes a bit of wrestling with the editor. Why not generate a deconstructor from the information that is in the class-object of the entity?

A deconstructor would have the following API, with the Student as example:

Given a student entity:
class Student{
    private final Integer snummer;
    private final String lastname;
    //...
    private final Boolean active;
    // rest left out

    // methods left out
}
the StudentDeconstructor could look like this (abbreviated).
public class StudentDeconstructor {

  public static Object[] deconstruct( Student s ) {
    return new Object[]{
        s.getSnummer(),
        s.getLastname(),
       // some left out for brevity
       s.getActive()
    };
  }
}

The generated deconstructor is specialized for the Student class, and it is also really fast, because it uses no reflection by itself.

In this exercise you will create the deconstructor java code given the entity class name on the command line.

    generateDeconstructor sampleentities.Student > path/to/StudentDeconstructor.java

The fine print

  • The generated code must be a valid class, and be acceptable by the java compiler.

  • The parameter on the command line is the fully qualified entity name such as sampleentities.Student.

    • Use Class.forName(String) to try to load the class.

  • The package declaration should be the same as that of the entity.

  • The entity classes should be available in binary form so we can reflect on them.

  • The name of the Deconstructor type should be the name of the entity type with "Deconstructor" appended. E.g. StudentDeconstructor.

  • The type and signature of the deconstructor method should be public static Object[] deconstructor( EntityType ), like public static Object[] deconstructor( Student ).

  • The field values should be obtained using the getter for the field. Assume 'get' as prefix for all methods, unless the field is of type boolean or Boolean.

  • The getter should be constructed according to the convention, get+<fieldname with first letter capitalized>. E.g the getter for field firstname is getFirstname.

  • The values obtained by the getters should be placed in the order of field declaration of the entity type.

  • The generated code should be fit for human consumption, with reasonable indentation so that eye-ball inspection of the generated code is meaningful.

In the project you will find a pre-made set of tests in which you have to add some test data and details of the tests.

In the tests:

  • Remove all _unneeded_[1] white space. This can be done easily with:
    stream().map (line → line.trim()).filter(l → !l.isEpmty()). This trick has been packed in a method called cleanCode that takes the whole generated text, cleans it and returns it as a list of Strings.

  • Test each aspect with a separate test data line. Use the Student class, which is given in the sampleentities package.

Windows run script to run the entitydeconstructor from the command line. Called run.cmd
@echo off
rem @author David Greven - https://github.com/grevend
set jar=target/entitydeconstructor-2.0-SNAPSHOT.jar
if not exist %jar% cls & echo Maven... & call mvn package
echo.
java -jar %jar% %*
bash script to do the same as the above. Will work in git-bash too.
#!/bin/bash
jar=target/sqltablegenerator-1.0-SNAPSHOT.jar
if [ ! -e ${jar} ]; then mvn package; fi
java -jar ${jar} "$@"

3. Service class Self registration.

Some services provided by classes are greatly helped if the using class does not have to instantiate a class, but can simple look it up.

Looking up an object instead of creating your own instance can also help performance, because an next lookup can return the same object, so the expense of the instantiation is paid only once, when creating the first instance. This can make an expensive construct still usable, once the instance is used often enough. This trades startup cost for flexibility and less repetitive programmer’s (you) work.

This looking up is done in something called a registry.

How cool would it be that a service like a Deconstructor could register itself, by supplying the key (the class of the entity it can deconstruct) and as value itself. Then the client code (the method that wants to do the deconstucting) can simply look up the deconstructor

  void useEntity( E e ) {

    for (Object f : Deconstructor.forType( e.getClass() ).deconstruct( e ) ) {
      // use fields.
    }
  }

To make this work, we need to do a bit of designing, a class diagram will help.

selfregister
Figure 2. A subclass registering it self in a superclass.

The self registration works by the fact that the class loader initializes the class, which include executing the static blocks in the code.

The registry can now be implemented as follows:

Loading a class that then self registers.
public abstract class Deconstructor<E> {

    private static final ConcurrentMap<Class<?>, Deconstructor<?>> register
         = new ConcurrentHashMap<>();

    public static <E> Deconstructor<E> forType( Class<E> et ) {
        if ( !register.containsKey( et ) ) {
            loadDeconstructorClass( et );
        } // assume loading is successful
        return (Deconstructor<E>) register.get( et );
    }

    private static <E> void loadDeconstructorClass( Class<E> forEntity ) {
        String deconstructorName = forEntity.getName() + "Deconstructor";
        try {
            Class.forName( deconstructorName, true, forEntity.getClassLoader() );
        } catch ( ClassNotFoundException ex ) {
            Logger.getLogger( Deconstructor.class.getName() ).log( Level.SEVERE,
                    ex.getMessage() );
        }
    }

    protected static void register( Class<?> et, Deconstructor<?> dec ) {
        register.put( et, dec );
    }

    /**
     * Method to be implemented by (potentially generated) leaf deconstructors.
     *
     * @param entity to deconstruct
     * @return field values in array
     */
    public abstract Object[] deconstruct( E entity );
}
Self registration in a subclass
class StudentDeconstructor extends Deconstructor<Student> {

  private StudentDeconstructor(){
    // do what is needed to make this a valid object
    // or leave empty to suppress a default constructor.
  }

  /**
   * Static block to self register.
   */
  static {
    Registry.register( Student.class, new StudentDeconstructor() );
  }

  // other details left out
}

Note that the self registering class does not have to be public, so you can keep it nicely tucked away as a package private XXXDeconstructor, which can be kept in sync with the entities it supports in the same package.

Whenever you modify any of the types that are processed by say a Deconstructor, regenerate the Deconstructors for those types in that same package.

Self Registering Deconstructor.

The problem with the Deconstructor of the previous exercise is that the user class must know the entity and the Deconstructor, to find a matching pair.
Finding the entity type is easy, just ask the (non-null) entity for its type by using Class<?> entity.getClass().
Finding the Deconstructor is not as easy, certainly for a utility class that wants to turn any list of any kind of entity into a csv file.
The idea is then to use the entity to lookup the matching Deconstructor. This will loosen up the coupling between classes and its users.

Self-registering Deconstructor

In this project you can use some of the code of the previous exercise, with a few tweaks.

Enhance the generated Deconstructor in the previous exercise to make it self registering. To do that, make a static block that registers the generated Deconstructor, for instance the StudentDeconstructor.

  • Make the generated Deconstructor extend the Deconstructor<E>, where E is the entityType of the requested Deconstronstructor.
    E.g. StudentDeconstructor extends Deconstructor<Student>.

    • This make the generated Deconstructor a leaf class.

  • Make the constructor or the generated Deconstructor private.

  • Add the self registering static block.

  • The deconstruct method can be the same as that from the previous exercise.

  • and lastly, make the deconstruct method non-static, because you cannot overwrite static methods, but need a normal method in the leaf class.

While developing, use the new generator to generate a few deconstuctors, and see if the deconstructors work, by enabling the DeconstructorTest in the sampleentities test package.

4. Optional demystified

Many terminal stream operations return an Optional. Rationale: the required element may not be present, or the Stream is empty to start with. Or the filter throws everything out.

Optional is an interesting concept all by itself.

  • An Optional can be considered as a very short stream of at most one element.

  • You can apply a map operation to an Optional, which transforms the content of the optional (if any) and produces an optional containing the result of the mapping.

  • You can even stream() the optional, which allows you to apply all operation that a stream provides (since 9).

All this can be used in an algorithm, in which you want to chain a bunch of methods, each of which requiring that the parameter is non-null.

Traditional returning null
  SomeType result;
  var a = m1();
  var b=null;
  var c=null;
  if ( null != a ) {
    b = m2( a );
  }
  if ( null != b ) {
    c = m3( b );
  }

  if ( null != c ) {
    result = m4( c );
  }
  return result;
  // do something with result
Using Optional. Returning Optional<SomeType>
  Optional<SomeType> result
     = m1ReturningOptional() (1)
      .map( a-> m2( a ) )
      .map( b-> m3( b ) )
      .map( c-> m4( c ) );
  return result;
  // do something with Optional result
1 The chain starts with a method returning an optional. The remaining methods m2(…​), m3(…​), and m4(…​) are unchanged.

In the last example, an Optional<SomeType> is returned, that can either be unwrapped of passed on as such for further processing.

TL;DR: If you apply map as a chained operation to an Optional, the function supplied as parameter to the map(..) method is applied to the _content _of the optional and returns the result in another Optional.

Generic Mapper.

Disclaimer: No students were hurt or will be hurt by developing a student mapper.

A mapper is a technical term that describes that something is turned into something else. That still sounds dangerous like turning naughty kids into frogs, but in this case it is simply turning an object into something the other side can deal with. Like turning a student object into an array of its constituting parts, so it can be put into a database or sent across a wire. Mwah, still sounds scary…​

The mapper below can turn an entity into an array of Object, and vise-versa, can Stream the object as Field-value pairs or as name-value pairs, and can provide other (meta) information about the fields as a list of Fields of the entity class when needed.

The deconstructor part is the same as the deconstructor in the previous exercises.

A Mapper takes the responsibility of reflecting on an entity type, and be able to extract information from the class, to be able to manipulate the instances of the entity type. To services it provides are:

  • Constructing and deconstructing entities of the type.

  • Providing and caching the meta information of the type.

  • Providing the identity-type of the entity. In many uses of mappers, such as databases or mapping,
    the key type of the identity field is important, so that is kept to.

  • Do expensive and difficult reflection operations once and cache the collected information.

The Mapper you will create in this exercise can be a building block in the 'Plumbing' of your application, and can be extended with functionality as needed.

GenericMapper

To get to this exercise, we started with copy and pasting, the deconstructor bits and combined them into this new and final project and functionality. You do not have to do the copying, that has already been done in the new project.

generatedmapper
Figure 3. The Generic Mapper class diagram

The mapper we create is a continuation of the Deconstructor and DeconstructorGenerator and brings it all together as a library/API and the accompanying Generator in one package/module.

We started with the deconstructorregistry project. Simply copy the classes over to the new project and refactor the name of the Deconstructor to Mapper, because it will become one quite soon. Also refactor/rename the Leaf mappers generated by your deconstructorgenerator to XXXMapper, and while you are at it, rename the DeconstructorGenerator to MapperGenerator, because that is its final role.

As far as the deconstruction part of the mapper is concerned, we are done. We can now also lookup mappers using the mechanism explored in the the DeconstructorRegistry exercise.

The missing bits are the meta data we want to keep (cache) for the entity class, so we have the meta data, such as field name and type handy.

  • Reflect on the type (Class<E>.class) to find the declared fields, and save this info in an array for later use. (Cache it).

  • Use the types of the declared fields to create a MethodType Object matching a constructor of the entity that takes all fields.

    • If the programmer has his constructor created by the IDE, and selected all fields, then you should be fine.

  • unreflect the constructor to get a MethodHandle and use that to create Function<Object[],E>, which is equivalent to a factory method that calls a constructor to create an entity. Save this function object as field in the mapper.

  • Keep the entityType for later reference in a field and provide an abstract Class<?> keyType() method, so we can ask the subclass for the type of <K>.

Creating the method type for a Constructor that takes all fields
    Class[] fieldTypes = ...               (1)
    MethodType ctorType = MethodType.methodType( void.class, fieldTypes ); (2)
1 Use the saved field types.
2 Get the method handle.

With the above, our mapper is almost done.

You can now implement the public Stream<FieldPair> stream(E e)

  • Start by getting the fields via the deconstruct method.

  • Use an IntStream to generate the indices to visit all elements of the field array: IntStream.range( 0, fields.length ).

  • Map (with mapToObj) each index to a new FieldPair, whereby you take the key from the name of the fields array, and the value from the deconstructed Object[].

  • Return the resulting array.

Stream an entity as Stream of FieldPairs
    public Stream<FieldPair> stream( E entity ) {
        Object[] fieldValues = deconstruct( entity ); (1)
        return IntStream
                .range( 0, entityFields.length )      (2)
                .mapToObj( i -> new FieldPair( entityFields[i].getName(),
                fieldValues[i] )
                );
    }
1 cached array fieldValues and
2 deconstuctor result array have the same length by definition.

Done.

The mapper can now be used as follows:

Student mapper in action one a Student.
    static Object[] studentArgs = new Object[]{
        snummer, lastName, tussenVoegsel, firstName, dob, cohort, email, gender,
        group, true
    };

    static Student jan = new Student(
            snummer, lastName, tussenVoegsel, firstName, dob, cohort, email,
            gender, group, true
    );

    @Test
    public void tStudentMapperConstructs() {
         Mapper<Student,Integer> mapper = Mapper.mapperFor( Student.class );
         Student xJan = mapper.construct( studentArgs );
         assertThat( xJan ).usingRecursiveComparison().isEqualTo( jan );
     }

     //@Disabled("Think TDD")
     @Test
     public void tStudentMapperDeconstructs() {
         Mapper<Student,Integer> mapper = Mapper.mapperFor( Student.class );
         Object[] deconstruct = mapper.deconstruct( jan );
         assertThat( deconstruct ).isEqualTo( studentArgs );
     }

We also expanded on the idea of generating the leaf deconstructors. In this case we have a MapperGenerator that uses a template to generate mappers. The generating part is almost exactly the same as in the deconstructor generator exercise. Only the test data is a bit longer.

Here is the Template
package %1$s;

import %2$s;
import genericmapper.Mapper;
import java.util.function.Function;

/**
 * Generated code. Do not edit, your changes will be lost.
 */
public class %3$sMapper extends Mapper<%3$s, %4$s> {

    // No public ctor
    private %3$sMapper() {
        super( %3$s.class );
    }

    // self register
    static {
        Mapper.register( new %3$sMapper() );
    }

    // the method that it is all about
    @Override
    public Object[] deconstruct(  %3$s %5$s ) {
           return new Object[]{
              %6$s
           };
    }

    @Override
    public Function<%3$s, %4$s> keyExtractor() {
        return ( %3$s %5$s ) -> %5$s.%7$s;
    }

    @Override
    public Class<%4$s> keyType() {
        return %4$s.class;

    }
}

When you have a class hierarchy, make sure that the fields are in the order you want. At the moment of writing, NetBeans IDE creates the parameters for the constructor in the order sub class fields first, then super class. You may either have to

  • Refrain from using class hierarchies in your entities (probably wisest).
    This is the advise Java 14+ implicitly gives with the introduced record feature which is final in Java 16.

  • Take extra care when creating sub-entities

  • Modify the mapper, that tries to resolve this potential problem be selecting the proper field order.

The later can be done by trying the top-down field order first and try the bottom-up order when that fails and then give up. DO NOT try all possible mutations of the field order because that is a real big problem.

When you use the Mapper as above, do not forget to Generate the Mappers, otherwise your program will almost certainly fail.

5. Use a Maven Plugin to generate mappers on the fly

To make the generic mapper even more useful, in particular for project 2, we have added a maven plugin. Add the maven plugin as shown below to the maven project that contains the entities.

Automagically generate mapper with mvn compile in the entities project. Pom from AIS example PRJ2 revised-implementaion
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <repositories>
        <repository>
            <id>fontysvenlo.org</id>
            <url>https://www.fontysvenlo.org/repository</url>
        </repository>
    </repositories>
    <pluginRepositories>  (1)
        <pluginRepository>
            <id>sebiplugins</id>
            <url>https://www.fontysvenlo.org/repository/</url>
        </pluginRepository>
    </pluginRepositories>
    <parent>
        <groupId>fontys</groupId>
        <artifactId>AirlineInformationSystem</artifactId>
        <version>1.1-SNAPSHOT</version>
        <relativePath>..</relativePath>
    </parent>
    <artifactId>BusinessEntities</artifactId>
    <packaging>jar</packaging>
    <version>1.1-SNAPSHOT</version>
    <dependencies>
        <dependency>
            <groupId>io.github.sebivenlo</groupId>
            <artifactId>sebiannotations</artifactId>
            <version>1.0-SNAPSHOT</version>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>io.github.sebivenlo</groupId>
            <artifactId>genericmapper</artifactId>
            <version>[2.2.0,)</version>  (2)
            <type>jar</type>
            <scope>provided</scope>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>io.github.sebivenlo</groupId>
                <artifactId>mapperplugin</artifactId> (3)
                <version>1.0</version>
                <executions>
                    <execution>
                        <id>gen-src</id>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>sebimappergenerator</goal>
                        </goals>
                        <configuration>
                            <entityPackages>
                                <entityPackage>businessentities</entityPackage> (4)
                            </entityPackages>
                            <classesDir>${basedir}/target/classes</classesDir> (5)
                            <outDir>${basedir}/src/main/java</outDir>(6)
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>
1 do not forget to declare an extra pluginRepository.
2 You need to have (on your pc) version 2.2.0 or higher of the genericmapper. Change your version accordingly in the pom of project genericmapper22.
3 The mapper plugin has groupId io.github.sebivenlo, which should be no surprise, and the artifactId is mapperplugin.
4 Specify which package is processed by the plugin.
5 Tell the mapper generator plugin where to look for entities .class files.
6 Here you can set where the generated files land. This setting makes them land in the same package as the entities themselves. (That package is businessentities in the example).

Do not forget to add the pluginRepository like in the example.

You can also generate mapper in the test package. For that add an execution with a different id and with appropriate values for classesDir and outDir.

To make this work, you need an additional class MapperGeneratorRunner, given below, in the genericmapper project (for you in 2021 genericmapper22).

You may have to tweak your MapperGenerator a bit. In theory you can drop all static methods in that class, since those have been moved to the runner. Call it a belated separation of concerns.

The code is also directly available as raw

The exerise projects (without solution, so the original exercises) are also on github, under mapper plugin .

GenericMapperRunner, code to add
MapperGeneratorRunner. There, I said it.
package genericmapper;

import static genericmapper.Constants.generatedJavaFileName;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.PrintStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Collection;
import java.util.Collections;
import java.util.Set;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.stream.Collectors;
import java.util.stream.Stream;

/**
 * Helper for mapper plugin.
 * This class accepts the parameters of the plugin and passes it to
 * individual MapperGenarator instances.
 *
 * @author Pieter van den Hombergh {@code Pieter.van.den.Hombergh@gmail.com}
 */
public class MapperGeneratorRunner {

    public static void main( String[] args ) {
        String pOutDir = System.getProperty( "mapper.generator.outDir", "out" );
        String pClassesDir = System.getProperty( "mapper.generator.classesDir",
                "target/classes" );
        String[] packNames;
        if ( args.length > 0 ) {
            packNames = args;
        } else {
            packNames = new String[]{ "entities" };
        }
        new MapperGeneratorRunner( pClassesDir, pOutDir, packNames ).run();
    }

    final String classesDir;
    private final String outDir;
    final String[] packNames;

    public MapperGeneratorRunner( String baseDir, String outDir, String[] packNames ) {
        this.classesDir = baseDir;
        this.outDir = outDir;
        this.packNames = packNames;

    }

    public void run() {
        try {
            for ( String packName : packNames ) {
                generateMappers( outDir, classesDir,
                        getCanditateEntityNames(
                                classesDir, packName ) );
            }
        } catch ( IOException | ClassNotFoundException ex ) {
            Logger.getLogger( MapperGenerator.class.getName() )
                    .log( Level.SEVERE, null, ex );
        }
    }

    /**
     * Get the list of candidate entities from the compiled classes
     * directory.The method removes the following file name patterns from the
     * available files below the start directory.
     * <ul>
     * <li>any filename containing a dash, such as in doc-info.class or
     * module-info.class</li>
     * <li>Any class name ending in Mapper.class</li>
     * </ul>
     *
     * @param startDir   to start
     * @param entPackage package for entities
     *
     * @return list of possible entity classes.
     *
     * @throws IOException dir
     */
    public Set<String> getCanditateEntityNames( String startDir,
                                                String entPackage )
            throws IOException {
        Path startPath = Path.of( startDir );
        Path root = Path.of(
                startDir + fileSep + entPackage.replaceAll( "\\.", fileSep ) );
        if ( Files.exists( root ) ) {
            try ( Stream<Path> stream = Files.walk( root,
                    Integer.MAX_VALUE ) ) {
                return stream
                        .filter( file -> !Files.isDirectory( file ) )
                        .filter( f -> !fileNameContains( f, "-" ) ) // avoid info files
                        .filter( f -> !fileNameEndsWith( f, "Mapper.class" ) )
                        .filter( file -> fileNameEndsWith( file, ".class" ) )
                        .map( p -> startPath.relativize( p ) )
                        .map( Path::toString )
                        .map( s -> s.substring( 0,
                        s.length() - ".class".length() ) )
                        .map( s -> s.replaceAll( "/", "." ) )
                        .collect( Collectors.toSet() );
            }
        } else {
            return Collections.emptySet();
        }
    }

    static boolean fileNameEndsWith( Path file, String end ) {
        return file.getFileName().toString().endsWith( end );
    }

    static boolean fileNameContains( Path file, String needle ) {
        return file.getFileName().toString().contains( needle );
    }

    static String pathSep = System.getProperty( "path.separator" );
    static String fileSep = System.getProperty( "file.separator" );

    void generateMappers( String outDir, String classPathElement,
                          Collection<String> entityNames ) throws
            ClassNotFoundException, FileNotFoundException, MalformedURLException {
        URLClassLoader cl = new URLClassLoader( new URL[]{
            Path.of( classesDir ).toUri().toURL()
        } );
        for ( String entityName : entityNames ) {
            Class<?> clz = Class.forName( entityName, true, cl );
            String fileName = generatedJavaFileName( outDir, clz );
            File dir = new File( fileName );
            dir.getParentFile().mkdirs();
            String javaSource = new MapperGenerator( clz ).javaSource();
            if ( !javaSource.isBlank() ) {
                try ( PrintStream out = new PrintStream( fileName ) ) {
                    out.print( javaSource );
                    out.flush();
                }
            }
        }
    }
}

Once you have added the class and updated your entities (sub) project, simply build the entities project (mvn package or mvn install), and the entities will find their mappers alongside in the same package.

You may have to do a few tweaks in the Constants class, but they should be self evident. When you are done, bump the version of the genericmapper to 2.2.0 instead of 2.2-SNAPSHOT, so the mapperplugin can find it.

It is best to keep the entities plus their mappers in a separate package and maybe also in a (maven) project of their own. This loosens coupling and allows both client and server project use the same dependency for the entities and friends.

6. Improvements on the Mapper.

In the weeks after publications of the genericmapper exercise we have invested further time into the mapper. You have noticed one of the improvements through the fact that we replaced the original exercise with genericmapper22 which is in fact nothing else then version 2.2.0 of said mapper.

We found some additional improvements and do not want to deprive you from the ideas in the improvements.

Improvements

  1. Better error handling when a mapper can’t find a constructor it expects. Remember that the mapper top class tries to infer the constructor from the fields, including the order in the fields types. The improved log message contains the expected signature of the constructor, like public Tutor( String firstname, String lastname, String tussenvoegsel, LocalDate dob, String gender, Integer id, String academicTitle, String teaches, String email );

  2. The mapper filters out the transient fields. A transient field is a field that should NOT be serialized. We think it is proper to not include the transient fields to be part of the persisted information.

  3. Removed unwanted code. In particular remove the obsolete internal StringMapper. it is NOT wanted and only causes problems.
    The string mapper seemed necessary at a time during development of the mapper, but is is not and should be removed. The StringMapper was introduced when we added the recursion through the class hierarchy from an entity the bottom, up to but excluding Object. This allows entities that have super classes like Student extends Person.

To introduce the improvements in your project you need to do the following edits:

If you already completed the exercise, you do not have to commit these improvements.
If you did not, the improvements will be considered (but not tested) in the tests.

Remove

  • Remove the file StringMapperTest.java from the test packages, or disable it with a @Disabled annotation at the class level.

  • Remove the entirety of the inner class StringMapper from Mapper.java or comment it out. Best is to remove it.

  • Remove the static block in Mapper.java that loads the StringMapper.class into the mapper registry.

static block to remove. Block loads StringMapper, which is now obsolete
  static {
        System.out.println( "loading  StringMapper" );
        try {
            Class.forName( "genericmapper.Mapper$StringMapper" );
        } catch ( ClassNotFoundException neverhappens ) {
        }
  }

Add

  • Add the following method to the Mapper class. The produces the improved error handling when a method cannot be found.

improved log about missing constructor.
    /**
     * Better info when a constructor with a specific signature can't be
     * found.
     *
     * The method attempts to produce the missing constructor signature.
     *
     * @param ex     exception that triggered this method
     * @param fields to compute the detail info.
     */
    void logCannotFind( Throwable ex, final Field[] fields ) {
        ex.setStackTrace( Stream.of( ex.getStackTrace() )
                .filter( steFilter ).toArray( StackTraceElement[]::new ) );
        Supplier<String> msg = () -> String.format(
                "failed to find constructor for class '%s'\n\twith exception type '%s' \n\tand signature\n %3s\n",
                entityType.getName(), ex.getClass().getName(),
                "\tpublic " + entityType.getSimpleName() + "( " + Stream
                .of( fields )
                .map( f -> f.getType().getSimpleName() + " " + f.getName() )
                .collect( joining( ",\n\t\t\t" ) ) + "\n\t);" );
        logger.log( Level.SEVERE, ex, msg );
    }

Replace

  • Replace the code in the catch block of the method final Function<Object[], E> findConstructor( final Field[] fields ) with a call to the above method.

previous code in catch Block
    catch ( IllegalAccessException | NoSuchMethodException ex ) {
            Supplier<String> puk = () -> String.format(
                    "failed to find constructor for class %s with exception type %s and message %3s",
                    entityType.getClass().getName(), ex.getClass().getName(),
                    ex.getMessage() );
            Logger.getLogger( Mapper.class.getName() )
                    .log( Level.SEVERE, puk );
    }
new code in catch block.
     catch ( IllegalAccessException | NoSuchMethodException ex ) {
            logCannotFind( ex, fields );
     }

Exercise SQL table sqlgenerator

You cannot only generate Java code, but also code in another programming domain, such as SQL. In the exercise below we will create a table definition (DDL) for a postgresql database using the information from the entity classes

SQL table generator

Write an application that interprets the class definition of an entity class via reflection and spits out an SQL table definition. This definition can be saved or be fed to a database server to create a table.

The following java types and their SQL counterparts should be supported. PostgreSQL types are used.

The name of the generated table should be the name of the entity class sans package name and in simple plural. Simple plural means append an 's' character to the name. Rationale: A table contains Students, plural, not one student. That is because SQL defines both the row definition and the table at the same time.

The relationship between the table and the entity is the definition of columns in the table versus the fields in the entity.

Java

SQL

java.lang.String

TEXT

java.lang.Character

CHAR(1)

java.lang.Integer

INTEGER

int

INTEGER

java.lang.Short

SMALLINT

short

SMALLINT

java.lang.Long

BIGINT

long

BIGINT

java.math.BigDecimal

DECIMAL

java.math.BigInteger

NUMERIC

java.lang.Float

REAL

float

REAL

java.lang.Double

DOUBLE PRECISION

double

DOUBLE PRECISION

java.time.LocalDate

DATE

java.time.LocalDateTime

TIMESTAMP

The generator should also support the following annotations on the fields of the entities.

@Id should generate a SERIAL or BIGSERIAL dependent on the field being a Integer or a Long and have the PRIMARY KEY attribute.
@NotNull should result in a NOT NULL constraint.
@Check should copy the value (text) of the annotation as parameter to the CHECK constraint.
@Default should copy the value as the DEFAULT value. Quote the value where needed.

Beware of primitives with table definitions
If your entity has fields of primitive types, make sure that you give the table definition a NON NULL constraint for the corresponding column, otherwise a query might result in a null value, which can’t be cast to a primitive type. If you forget this you will be bitten by unexpected NullPointerExceptions. By implication, having a primitive field implies non-null, with or without annotation.

The resulting table definition should be acceptable by a PostgreSQL database. Test that manually.

Example Module class.
@Data                (1)
@AllArgsConstructor  (2)
public class CourseModule {

    @ID
    private Integer moduleid;

    @NotNull
    String name;

    @NotNull @Default( value = "5" ) @Check(credits > 0)
    Integer credits;
    // extra ctor to make the default values take effect.
    public CourseModule( Integer moduleid, String name ) {
        this( moduleid, name, 5 );
    }
}
1 Lombok framework Data annotation, that add getters (for all fields) and setters (for non-final fields), equals and hashcode and a readable toString.
2 Makes sure there is a constructor that takes values for all fields.

The ID field is a bit special, because when creating an entity before putting it in the database will NOT have an ID yet, because the ID value is typically assigned by the database by means of a sequence or other identity generating mechanism.

Example Output.
CREATE TABLE coursemodules (
  moduleid SERIAL PRIMARY KEY,
  name TEXT NOT NULL,
  credits integer NOT NULL DEFAULT (5) CHECK (credits > 0)
);
Windows run script to run the sqlgenerator from the command line.
@echo off
rem @author David Greven - https://github.com/grevend
set jar=target/sqltablegenerator-1.0-SNAPSHOT.jar
if not exist %jar% cls & echo Maven... & call mvn package
echo.
java -jar %jar% %*
bash script to do the same as the above. Will work in git-bash too.
#!/bin/bash
jar=target/sqltablegenerator-1.0-SNAPSHOT.jar
if [ ! -e ${jar} ]; then mvn package; fi
java -jar ${jar} "$@"

Reading

Horstmann V1 Ch 5.7

Exercises in this part.


1. the white space is there to improve redability, but is the first that can be discarded, which is what a compiler does as one of the first steps