We will start with a short recap and some refinements on λ-expressions, followed by application in streams.

1. Functional Interface rationale.

  • Java: backwards compatible: do not break existing code with new features.

  • Way forward: enhancements to the collection framework.

    • Extending interfaces without breaking implementations

      • add default functions

      • add static helper functions

Example: compare java 7 and java 8 Comparator:

  • The Java 7 version has two abstract methods, which were the only kind in Java before 8.

  • The Java 8 version has one abstract method, making it into a functional interface,
    7 default methods and
    8 static methods whose functionality is now available to every class implementing Comparator.

Have a look yourselves: Java 7 Comparator and Java 8 Comparator

The concept of functional interfaces or single abstract method is the secret ingredient to let the compiler guess the appropriate shape of the function that matches a lambda expression.

1.1. <S.H.A.P.E.> of java.util.functions.

The java.util.functions package, which was introduced in Java 8, contains only functional interfaces and nothing else. The package contains the most common shapes of functions that may occur in lambda expressions. You can see that all reference type functional interfaces heavily rely on generics. The use of the proper generics definition is essential for functional interfaces to function properly.

We will use the notation <input type> → <result type> for the shape of a Functional interface. Note that the shape is determined by the one (abstract) method of the functional interface. The style used in java.util.function is somewhat mnemonic in choosing the generic type parameters in the type interface type definition.

The Functional Interface is your handle to the shape of the allowed lambda expressions, input, output and optional throws.

  • T as only parameter, either a input or output e.g.

    • Consumer<T>: shape <T> → {} black hole? Eats Ts.

    • Supplier<T> : shape () → <T> , source of Ts.

  • Function<T,R>: shape <T> → <R> also known as mapping function, from one type to another.

  • BiFunction<T,U,R>: shape <T,U> → <R>.

There are also a few specialized interfaces that deal with the primitive types int, long and double. Their rationale is performance and memory footprint, in short: efficiency. Dealing with e.g. arrays of primitive types is much more efficient than using the associated wrapper types, int vs Integer. You save the object allocation memory-wise and the dereferencing and unwrapping execution time-wise.

  • ToDoubleBiFunction<T,U>: shape <T,U> → double producing a double from two arguments.

  • LongConsumer: shape long →() munges longs to make them disappear. :-)

The package documentation makes an interesting read.

Exceptions in Lambda expressions need special consideration. In particular checked exceptions actually counteract the conciseness of lambda expression. In those cases, where the lambda expression or reference method throws an exception, that exceptions is commonly wrapped in an unchecked exception, e.g. UncheckedIOException to wrap an IOException. UncheckedIOException has been introduced in Java 8 for just that purpose.

You can also roll your own.
If you need to declare a functional interface that potentially throws an exception, the following is a way to do that:

throwing functional interface
interface ThrowingBiFunction<T,U, R,<X extends Throwable>> {
    R apply(T t, U u) throws X;
}

Shape wise it looks like <T,U> → R | X pronounced as from T and U to either, R the result, or X, the thrown exception.

2. Where is toString?

You might miss toString() on lambda expressions, and you would be right: Lambda expressions lack a useful toString. What you get by calling toString() on a lambda expression is something rather useless, concocted by the JVM.

toString() is useless with lambda expressions
  Comparator<Student> gradeComp = (a,b) -> Double.compare(a.getGrade(),b.getGrade());

When you invoke toString() in say jshell, you get.

jshell> class Student{ double grade; double getGrade(){return grade;}}
|  created class Student

jshell> Comparator<Student> gradeComp = (a,b) -> Double.compare(a.getGrade(),b.getGrade());
gradeComp ==> $Lambda$16/0x00000008000b3040@1e88b3c

jshell> gradeComp
jshell> gradeComp.toString()
$3 ==> "$Lambda$16/0x00000008000b3040@1e88b3c"

2.1. Method References

Since the shape of a functional interface has everything a method definition has, bar its name, it should be no surprise that this can be used to advantage. Everywhere where a functional interface is allowed you can also provide a so called Method reference. Quite often, but not always this can be a shorthand and thereby improving readability.

Method references have two parts, separated by two colons: ::.

  • The left hand side of the :: defines the method context, either instance of class (for method references to static methods)

  • The right hand side is the method name. The compiler will do the matching in case of overloaded methods, but sometimes it may give up and will let you know by using the word ambiguous in its complaint.

There are four kinds of references:

Table 1. Source Orace tuturial.
Kind Example

Reference to static method

ContainingClass::staticMethodName

Reference to an instance method of a particular object

containingObject::instanceMethodName

Reference to an instance method of an arbitrary object of a particular type

ContainingType::methodName

Reference to a constructor

ClassName::new

Reference to a method of this instance

this::instanceMethodName
You are not allowed to leave out the left hand side nor the ::, not even if the reference context is this.

3. Streams, SQL like operations expressed in Java 8

Here too, the package documentation is an interesting read.

A stream can be considered a conveyor belt on which objects experience operations. The belt starts at a source (for instance a Supplier) and can on or more intermediate operations and end with a Terminal operation.

streamops

from Lucas Jellema of amis blogs

Here your see a physical stream and a collect operation.

Conveyor and collecting

3.1. SQL like functions

assume you want to know the sum of the wages for women.
select sum(salary)
from   people
where  gender = 'FEMALE'
same in Java, get the sum of payments to women.
    int womensPay =
        people.stream()                                         (1)
            .filter(p -> p.getGender() == Person.Gender.FEMALE) (2)
            .mapToInt(p -> p.getSalary())                       (3)
            .sum();                                             (4)
1 use people collection as source and start stream
2 where …​ where gender = female
3 mapping is often retrieving the value of a member, get salary, note that it is a Map ToIntFunction<Person> specialised function
4 aggregation, like sum() here is a terminal operation
variant using a method reference.
    int womensPay =
        people.stream()
            .filter(p -> p.getGender() == Person.Gender.FEMALE)
            .mapToInt( Person::getSalary )                      (1)
            .sum();
1 This alternative uses a method reference, which can be substituted for a lambda expression when their shapes match.

Exercise simple stream

Simple Stream

Make sure you use at least NetBeans IDE version 12.2, which has nice stream support.

In this exercise you work with the simplestream project.

Starting with streams can be a bit confusing. You will have the best experience if you let NetBeans IDE and the compiler do some of the thinking for you. And then throw in some tricks.

You can consider a pipeline to have two ends: the start, which is the method that typically is called stream(), and the end, which is your terminal operation. Often you can begin coding by adding the terminal operation immediately and insert any intermediate operation between the start and this terminal operation of the stream.

count all students.
  int studentcount = students.stream()
  .count();

There will be some example code in the simplestream project in your repository.

We have the following facts on the students.

There are 50 students in total. The number of male students is 16. The youngest student is Shirleen Simpson, student number 3134539. The Female students whose first name starts with an 'A' are Arlinda, Ariel, Aida, Annette, and Anitra.

Since you are experimenting with streams, do not bother to write business methods. There are four test methods in total, which you need to implement. Inside such method, write the stream code and the assertJ assertions. The javadoc above the tests give some extra hints.

Write tests for operations to compute from the given data set:

  • The total number of students.

  • The number of male students.

  • The youngest student. The stream method to use returns Optional<Student>. Think of the proper terminal operation.

  • The list of Female students whose name start with 'A'. Use Collectors.toList() to collect the matching students.
    You can either combine two predicates with the proper logical method or concatenate the strings, before you do the test.

  • The list of student than do not have an a or A in their name.
    Try with concatenating first and last name and also with an or in one predicate, using separate method names.

  • The list of students that match a certain criterium, expressed as a Predicate<Student>.
    Construct the predicate and invoke the method with this predicate as parameter. Be imaginative with the functionality.
example predicate
    Predicate<Student> redHaired s -> s.getHairColor() == HairColor.RED;

In all cases use streams and lambdas.

(1) Add a field to the Student class holding a Map of study-topics to grade: Map<String,Double>.
(2) Stream the students and flatMap the students to a stream of grades and compute the overall average of the grades.
(3) Stream the students and flatMap to Map.Entry<String,Double> and collect into a Map of study-topics to average grade.

Exercise CSV to Objects

In the video we explain streams and the next exercise, design or idea wise. The coding is left as an exercise.

Stream explained with paper
CSV to Objects

In this task you will write a small utility class that will help you to test things and is also useful to load data from files and e.g. put them in the application or mock data source. The use case of this utility class is to read data from files and turn them into real (Java) objects.

The class is a CSVObjectStream and has three constructors and two methods, stream(…​) and asList(…​).

Of course, you start with tests. To do that, you will find a students.csv file in the NetBeans-IDE project.

The test should:

  • Create a CSVObjectStream

    • Create a Path to "students.csv" using the Paths (java.nio.file.Path) utility class in the same package.

    • Optionally provide the seperator

    • Optionally provide a Predicate<String> lineFilter that filters out lines from the csv source based on the predicate e.g. the header line

  • Both stream and asList methods take two parameters:

    1. A function Function<? super String[], ? extends T> creator that transforms an array of strings into an object of type T.
      For the constructor call use Factories::createStudent as creator. This is a method reference. See lambda theory if you forgot what that means.

    2. A Predicate<? super String[]> called rowFilter that ensures that the array is fit for purpose.
      Example: the first string in the array of strings contains only digits and is the student number.
      The asList method uses the stream method, and collects the stream into a list.

The testing part should do the following:

  • Collect the resulting stream in a list and assert that

    • The list is not null.

    • The list is not empty.

    • The list has 50 elements.

    • The list contains a student with the name "Olympia Oliphant" at index three.
      Use the assert that extracting method with a fieldname as parameter. See the assertJ API and doc for examples.

  • In all the asserts, use a message and make sure you put the arguments to the assertThat method chained in the proper order.

You can use it as is shown in the listing below

Show that it works
To show the use of this class, create a csv file, or take it from an earlier project, and define a type, say student with studentnumber, name, birthdata, and gender.

To top the demonstration off, add a filter function that accepts a list of students, which streams said list and filters out students by some rule, e.g. by gender and or age.

A Helper class called Factories might be useful in your demo. Factories in our solution has two methods, one to create a student from an array of String, as is required by the CSVObjectStream class and one method to turn object back into csv records. We call the last one studentAsCSVLine. For the remaining few details look at the class diagram below.

You may want to keep this project and reuse it in other exercises.

Note that the Student test class is a static inner class in the factories, which is only available in the test source tree.

To complete the exercise, add a separate Main file that demonstrates the working of your module.

Using the csvobject utility class.
CSVObjectStream<Student> os
            = new CSVObjectStream<Student>(Paths.get( "students.csv" ) );

        os.stream( Factories::createStudent,
                    r -> r[ 0 ].matches( "\\d+" ) )
            .forEach( System.out::println );
csvobjectclasses
Figure 1. class diagram csvobjectstream

javadoc of csvobjectstream generated from the solution

Exercise LambdaLibrary

Lambda Library

Within this assignment you will get more familiar with the syntax of lambda expressions, as well as how to apply them. You will use the streams to get some work with collections done in a smart way.

Note: to run the FX application, don’t click "run" in Netbeans. Assuming you are using a testeasypom based project, then first enable the fx profile , then right click the project and either

  • select 'Run maven' and select exec or

  • in the Navigator, right-click the 'javafx run' entry and select 'execute goal'.
Screenshot of what the Lambda Library FX application looks like
Figure 2. The FX application works out of the box and you should not touch any of the FX parts. You will only work on the library itself.

Your task is to implement the business logic for a library. This library provides some methods to get access to the books and to retrieve several views on that books. A simple swing-GUI which displays the books of the library is given. You can use that to see parts of your implementation in action.

As usually, you will find an NetBeans project for this assignment inserted in you subversion repository. Note that the project will run, but not all functionality is implemented, yet. Proceed in the following steps:

  1. Start working on this assignment by studying the Javadoc which is given. Take especially care of the class LibraryModel since this will form the requirements you will have to implement.

  2. Implement one of the test cases in the test class LibraryModelTest. A good starting point is the testGetBooks() method.

  3. Implement the method for which you wrote the test(s) in the previous step.

    • It may need more tests if a method has more aspects to fulfill. Each aspect could have a different test.
      It is quite common to have more test methods per business method.

  4. Repeat the steps 2 and 3 until all functionality is implemented.

Note that this task has a dependence on the previous task cvsobjects, to read a csv file as book objects.

Additional information on lambda expressions and the java 8 features can be found at the following locations:

javadoc of the application. This provides the requirements of all the methods to be tested and implemented. You can derive your test data from the `library.csv} file.

Update The provided test class has two methods called book(int) and books(int …​), where the parameter is called id or ids. This could lead to confusion, when you interpret that as the id of the book in the csv file. It is not that number but simply the index in the List of books, so starts at 0, not 1.

Windows run script
@echo off
rem @author David Greven - https://github.com/grevend
mvn -P fx javafx:run

4. Testing Lambda expressions needed to implement an API interface.

Exercise MinMaxCollector

MinMaxCollector

The java Stream class provides multiple reduce operations, such as findFirst(), findAny(), sum(), min(), max(), and average() (for number values). In case you want to have the extreme values of a collection, and try to do that with a streaming operation, you need both min and max. Streaming the collection twice would be a sub-optimal solution, because when streaming the collection once, both the minimum and maximum value have been seen, and it would be smarter to catch both in one go.

You can solve this problem using the reduce method <U> U reduce​(U identity, BiFunction<U,​? super T,​U> accumulator, BinaryOperator<U> combiner), but that requires you to come up with three parts, identity (simple) accumulator and combiner every time you need min and max. From the programmers perspective, streaming twice will then become attractive, because it is minimal programming. Or the programmer might fall back to old-skool for-loops (or even use forEach(…​)), which will do the trick but is not the way to use streams.
And simple for-loops can’t be parallelized.

In this exercise you will be implementing the Collector<T,A,R> interface, such that the resulting class can add functionality to already existing classes. Special with this interface is that is returns functions, implemented as lambda expressions that do the required work, which are the used to to build the required Collector object.

If the stream is simply a stream of numbers such as a stream of int, long, or double, do not bother to create this class. Instead use summarizingDouble and its siblings defined in Collectors . You should always first look for an available solution in some library or in the JDK, and adapt your problem a bit to minimize the amount of new code your write. See Horstmann Vol II chapter 1.
Unless it is an exercise.

When developing this exercise, I learned the hard way that theory and practice do not always meet head-on. In this case, I hoped that using a parallel stream would improve the performance gain, and it does, but not as spectacular as hoped.

Table 2. test with 20,000,000 elements. Times in nano seconds
mode i7 4 core 2012 AMD Ryzen 9 3900X 12-Core Processor

standard

699,012,000 ns

423,917,000 ns

min-max time

579,755,000 ns

423,917,000 ns

standard parallel

211,857,000 ns

104,720,000 ns

min-max parallel

177,224,000 ns

76,433,000 ns

Standard is stream(…​).min() followed by stream().max();

Simple API, that can be used in the following way:
   List<Student> students=...;

   Optional<MinMax<Student>> extremesByAge =
       student.stream()
               .collect( minmax ( comparing(Student::getDob ) ) ); (1)
1 The resulting Optional will contain a MinMax with two students, the min and max aged student. Explain why the youngest is at position max.

You will implement your own reducer, which actually is also a kind of collector, because it will produce a MinMax Pair. The class is called MinMaxCollector, which has an static inner class MinMax.

To be able to determine min and max, you must construct the MinMaxCollector with a Comparator<T>

The design is already done, it is part of the stream API and is captured in the Collector interface, which prescribes the following methods: (from the API doc).

  • Supplier<A> supplier() A function that creates and returns a new mutable result container.
    In your solution this simply returns a new MinMax with e.g. () → new MinMax(comparator)

  • BiConsumer<A,​T> accumulator() A function that folds a value into a mutable result container.
    A will be our MinMax, T the next element, so that the lambda shape will be (a, t) → a.accept( t ).

  • Set<Collector.Characteristics> characteristics() Returns a Set of Collector.Characteristics indicating the characteristics of this Collector.
    This tells the stream framework what the characteristics of our collector component. Simply return Set.of( UNORDERED, CONCURRENT ), meaning that your solution can be used concurrently, as in with many threads in parallel. We will test that it can.

  • BinaryOperator<A> combiner(). A BiOperator takes two elements of the same time and combines them into a third element of the same time. The same as '+' combines two ints into a third int. In our case it will combine two MinMax objects into one. You can use one of the two to accept() the minimum and maximum of the other, so that the first will then contain min and max of both.

  • Function<A,​R> finisher() Perform the final transformation from the intermediate accumulation type A to the final result type R.
    This turns the result into an Optional if the stream produced data (one element is sufficient, will be min and max at the same time). If no element was in the stream, an empty Optional has to be returned.

minmaxcollector
Figure 3. class diagram

The class diagram shows all members of the two classes involved, the MinMaxCollector and the inner MinMax class.

TDD

Since we implement an interface, the IDE will want to implement all methods in one fell swoop. That is okay, because it will make implementations that will for certain fail the test, and that is good because it will have the red we need at the start.

We will start the tests in the same package as the collector, allowing access to the package private members and constructor, such as that of the MinMax inner class and its package private methods.

We also want an inner class MinMax, which will carry the result and will also play the role of accumulating object. Because in unit testing it is always smart to start from the bottom up, we will start at MinMax. Since we do not want to tie the MinMax inner class to the MinMaxAccumulator, we make it a static inner class. To make that work properly, you should choose a different generic type token for the inner and outer class. If the outer uses <T> then the inner could used <E>.

  1. Since there is no real test to assert the effects of the constructor, we just implement it. The test will follow.
    The minmax object should have three fields, E min, E max and a Comparator<E> to determine who is who of the former.

  2. TEST There are getters for min and max. Implement them and assert (test) that they both return null on a fresh minmax object.

The MinMax class has one accept method which it inherits by being a Consumer<E>. On each invocation of accept a MinMax object will consider if the newly accepted e is a better candidate for either min or max.

The following 3 tests can be combined into a parameterized test. Use the split trick from part 1 The test data for the tests can look like this:

@ParameterizedTest
   @CsvSource(value={
       // input, min, max
       "P,P,P",
       "A|B,A,B",
       "B|C|A,A,C",

   })

  1. TEST The accept(E) should feed the min max with elements. To test, create a MinMax<String> with a Comparator<String>: (a,b)→ a.compareTo(b).
    Feed it one String and ensure that both min and max are no longer null, and have the same value as the accepted string.

  2. TEST Add a String distinct of the other and test that it lands at the proper spot, min or max.

  3. Add a third distinct string.

An aspect that you cannot really test for, but can help performance a bit is to return immediately after you assigned both min and max on the first accept (when either was still null). Note that if you do well you only need to check one field, not both.

You are done with the inner class.

Let us go to the outer class MinMaxCollector.

  1. TEST that supplier() returns an MinMax object with the proper comparator.
    then implement.

  2. TEST that the characteristics return the correct set: UNORDERED and CONCURRENT. Then implement.

  3. TEST that the combiner method works. Give it two MinMax-es one with A,C as strings, one with B,D as strings. The result of the lambda should return a MinMax(A,D) that you should test for proper content. Then implement.

  4. TEST that finisher returns an empty optional when min or max (either test will do) is null, and a optional containing the minmax otherwise. Then implement.

  5. TEST that accumulator accepts a MinMax object and a new T, such that if T is less than min, it will replace min and if it is greater that max, it will replace that.
    start with a MinMax that already accepted one value, between min and max. Here too, you can use a tabular test. Just invoke the accumulator() method and do the testrecipes with the returned object.

The collector is now complete.

Integration test from different package, client.

Create a test in a different package, so you are sure that you will only use the public API (public and protected members of public types).

  1. TEST Use the collector as reduction step at the end of a stream with some unique strings. Assert that the result is as expected.

There is a demonstration program in the client package, ready for use, that demonstrates the use of the collector.

5. Collection Enhancements and general tips

It never hurts to browse through an API once in a while. You might get new ideas or find a cool feature.

5.1. Improved Map in Java 8

I personally find the java.util.Map quit interesting because it got a number of very cool features in Java 7 and 8 that are really very helpful if you want to implement things like caches. If you are into caching and expect many entries, consider a WeakHashMap, that also helps to solve the problem of keeping the cache small and auto-evacuate when entries are no longer used.

In order of appearance in the api doc:

  • compute Attempts to compute a mapping for the specified key and its current mapped value (or null if there is no current mapping).

  • computeIfAbsent If the specified key is not already associated with a value (or is mapped to null), attempts to compute its value using the given mapping function and enters it into this map unless null.

  • computeIfPresent If the value for the specified key is present and non-null, attempts to compute a new mapping given the key and its current mapped value.

  • getOrDefault Returns the value to which the specified key is mapped, or defaultValue if this map contains no mapping for the key.

  • putIfAbsent If the specified key is not already associated with a value (or is mapped to null) associates it with the given value and returns null, else returns the current value. and of course

  • forEach Performs the given action for each entry in this map until all entries have been processed or the action throws an exception.

which allows you to quickly look at the contents of a map with a one-liner
    map.forEach((k, v) -> System.out.println(k + "=" + v));

The fun thing is that all these methods are default methods, so they apply to all classes that implement the java.util.Map interface, including the ones you roll yourselves, might you feel the inclination. To implement a special map that has specific extra properties, you only need to extend AbstractMap which needs the implementation of one or two methods only, depending if you want an unmodifiable of modifiable one.

5.2. Use specialized collections

Java provides some specialized collections that can improve performance quite a while still adhering to the general contract.

  • EnumSet and EnumMap use the fact that the values of an enum are consecutive, and that each value is associated with its ordinal value, which is an int. In many case the number of values is overseable, making a EnumSet efficiently implementable using the techniques used in a BitSet. The EnumMap can simply use an array or references as backing store. The enum value then maps a value if and only if the array contains a reference at the ordinal index of the array. These are one of the fastest maps in Java.

5.3. BitSet

Sometimes you want to quickly check the membership of an element to some set. If the set is an oversee-able number of int or int like values, such as characters, a BitSet might be useful. It uses longs or arrays of longs as backing store, dependent on the number of values that can be in the set. This means that a bitset with 64 possible values takes only one long to hold all relevant information. It has many set operations.

For instance you could use a bitset to classify characters in a special way (if not already done in the general Character API).

Vowels
    private static BitSet vowels = new BitSet(128);

    static { // done when the class is loaded.
        for ( char c : "aeiouAEIOU".toCharArray() ) {
           vowels.set(c);
        }
    }

    boolean isVowel(char c) {
       return vowels.get(z);
    }

    public static void main(String... args){
       System.out.println("z is vowel: "+isVowel('z'));
    }

6. Optional demystified

Many terminal stream operations return an Optional. Rationale: the required element may not be present, or the Stream is empty to start with. Or the filter throws everything out.

Optional is an interesting concept all by itself.

  • An Optional can be considered as a very short stream of at most one element.

  • You can apply a map operation to an Optional, which transforms the content of the optional (if any) and produces an optional containing the result of the mapping.

  • You can even stream() the optional, which allows you to apply all operation that a stream provides (since 9).

All this can be used in an algorithm, in which you want to chain a bunch of methods, each of which requiring that the parameter is non-null.

Traditional returning null
  SomeType result;
  var a = m1();
  var b=null;
  var c=null;
  if ( null != a ) {
    b = m2( a );
  }
  if ( null != b ) {
    c = m3( b );
  }

  if ( null != c ) {
    result = m4( c );
  }
  return result;
  // do something with result
Using Optional. Returning Optional<SomeType>
  Optional<SomeType> result
     = m1ReturningOptional() (1)
      .map( a-> m2( a ) )
      .map( b-> m3( b ) )
      .map( c-> m4( c ) );
  return result;
  // do something with Optional result
1 The chain starts with a method returning an optional. The remaining methods m2(…​), m3(…​), and m4(…​) are unchanged.

In the last example, an Optional<SomeType> is returned, that can either be unwrapped of passed on as such for further processing.

TL;DR: If you apply map as a chained operation to an Optional, the function supplied as parameter to the map(..) method is applied to the _content _of the optional and returns the result in another Optional.

Exercises in this part

Reading

  • Horstmann Core Java, Ed 11, Vol I CH 6.2 Lambda Expressions

  • Horstmann Core Java, Ed 11, Vol I CH 9 Collections

  • Horstmann Core Java, Ed 11, Vol II CH 1 Streams

Slides