Backward compatible configuration key values for MicroProfile Config

Introduction

With MicroProfile config, you can define the application configuration using key-value pairs which can be retrieved from various resources.
You can use it for defining the configuration in a very flexible way and this for is useful for your applications but also for frameworks which need some config.

But one day, you like to change the key for whatever good reason. But can you do this easily? If you have written the application, it probably is. But what if you have written a little framework. Do the developers read the release notes where you have stated the changes?

The backward compatibility struggle

When your configuration parameter is required, the change will quickly be detected by the developer. They upgrade to your new version and get an exception that the key is not defined. Annoying maybe but not that dramatic.
The scenario where the parameter is optional is a much greater threat. The developer has defined a custom value, overriding your default but by changing the key, the default value is picked up again. Unless the developer has written your release not notice that a change of the key name is required.

So we need a way to define the fact that the key config.key.old is now config.key.new and ideally the value for the old parameter should be picked up.

The Alias config ConfigSource

The solution for the above-described problem can be solved with the tools we have at our disposal within MicroProfile Config itself.
We can define a Config Source which will be consulted at the end of the chain. As you probably know, you can define multiple Config Sources. Each will be asked to provide a value for the key. If a source cannot supply the value, the next source is contacted.
When our ConfigSource is contacted at the end of the chain, we can see if the developer (of the framework in this case) has defined an alias for this parameter key. In this case, we define that the search for a value for config.key.new, should also be tried with key-value config.key.old. So our special ConfigSource just asks for the value of the config parameter with the old key. If there is a value found with this key value, it is returned. If nothing comes up, it returns null as required to have the default value then be selected.

The Atbash Alias config ConfigSource

The Alias config ConfigSource concept is thus fairly simple. The Atbash config extension contains this feature since his latest release (version 0.9.3)

The configuration is also fairly simple. We only need to configure the mapping from the old to the new key value. This can be done by adding a properties file on the classpath. The file much have the structure: Alias.<something>.properties and must be located within the config path. This file needs to be created by the framework developer in case he is changing one of the configuration key values.

In our example here, the contents of the config/alias.demo.properties should be

config.key.new=config.key.old

Do you want some more information and an example? Have a look at the demo in the Atbash demo repository.

And another nice thing, it works with Java 8 and Java 11 classpath.

Conclusion

By adding a ConfigSource at the end of the chain, we can make the key values of our configuration parameters backward compatible. In case the developer still uses the old values of the key, we can look up the new key value and put a warning in the log. This makes sure that the application still works and informs the developer of the changed name.

Have fun.

Extensible Resource API

Introduction

There are various scenarios where you like to use a resource, like a classpath resource, file or URL, and want to make it configurable for the developer. If you are creating a little framework for example which needs some data that needs to be adjustable depending on the application you are using it in, the resource should be easily configurable.
This is where this Atbash Resource API can be very handy.

Reading an InputStream

There are various sources which can give you an InputStream to the resource you are pointing to. File and URL are the two well-known classes for this. But getting them is different if you are using FileInputStream for example. And it is again different when you want to read a resource from the classpath.

The Resource API wants to uniform the way on how you can obtain an InputStream. The be.atbash.util.resource.ResourceUtil#getStream(java.lang.String) method takes a String, the Resource Reference, pointing to the resource you want to open and it will find out how it should retrieve the InputStream.
The prefix is the most important indicator of how the resource should be approached. By default the prefixes http:, classpath: and file: are supported. But other types can be implemented by the developer if needed.

ResourceReader

The Resource API is extensible so that other types of resources can be accessed. To do this, implement the be.atbash.util.resource.ResourceReader interface. The load() method tries to open the resource and returns the InputStream. The method is allowed to return null when the type of resource can’t be handled by this ResourceReader or when the resource doesn’t exist.
Each ResourceReader implementation should have the be.atbash.util.ordered.Order annotation on the implementation class so that the implementations can be tried in a certain order. Your custom implementation will be picked up by the Service Loader mechanism.
The implementations will then be consulted based upon the order, from low to high, to see if it can handle the resource reference. The method canRead() from ResourceReader is used for the verification of the resource reference existence.

With MicroProfile Config

The Resource API can be used with MicroProfile config. You can define a configuration parameter pointing to the default resource (like classpath) And the developer can then overwrite these values by specifying another resource using one of the supported MicroProfile methods.
Since you use the ResourceUtil#getStream(), any resource like file and URL can be supported.

Extending

As mentioned above, the ResourceReader can be used to create a custom implementation to read from specific resource. But it can also be very handy during testing. You can define a custom ResourceReader which reads some data from a Map for instance. That way you can easily point to different resources during testing.

You can have a look at the Atbash demos where a little demo is prepared. The class be.atbash.demo.utils.resource_api.spi.MapBasedResourceReader implements the ResourceReader interface.

Conclusion

This Resource API is a small and simple extensible API to get an InputStream from a resource like a file, URL or a resource on the classpath. It saves the developer to verify where the resource is located and calling the correct code. You can easily extend it by implementing the ResourceReader interface which can be very handy during testing.
And the last nice quality it has, it runs on Java 7, 8 and 11 (classpath option)

You can also have a look at some documentation here.

Have fun.

Atbash Configuration Server

Introduction

MicroProfile Config specification defines how configuration values can be picked up by the application. It has the concept of a Configuration source and converters to supply the values you need.

By default, only a ConfigSource for a classpath defined file and environment and system variables are defined.

But since we have the concepts in place, we can create our custom ConfigSource to retrieve the configuration from a central server.

Configuration Server

Instead of putting the configuration of each application within the application itself, we can store it in a central location.

It is very convenient to have an overview of the configuration of all your application at one place. Which makes updating it also very simple.

The first version of the Atbash Configuration Server exposes the configuration of an application through a JAX-RS endpoint.

The URL

/config/{application}

will give you the configuration values as a JSON structure for the specified application.

{
   "pets":"dog,cat",
   "dateValue":"2017-11-15",
   "value2":"500",
   "classname":"be.atbash.config.examples.se.testclasses.ConvTestTypeWStringCt",
   "value1":"Classpath Value",
   "value3":"true",
   "dateValueFmt":"16-11-2017,dd-MM-yyyy"
}

Creating the Configuration server

The configuration server is available within the directory  <server> of the repository https://github.com/atbashEE/atbash-config-server

You can build the server in 3 flavors

1. Java EE war.

With the maven command, mvn package -Pee7-servera WAR file is created which can be run by any Java EE 7 compliant server (Java 8 required)

2. WildFly Swarm executable JAR

The Configuration server can also be wrapped within a WildFly swarm application with the mvn package -Pwildfly-swarmcommand.

3. WebSphere Liberty executable JAR

There is also a profile defined to create a WebSphere Liberty JAR file, use the mvn package -Plibertycommand.

More servers will be supported when they have support for Config Spec 1.2.

Configuration of Server

Define within a config-server.properties or config-sever.yaml file on the classpath the following configuration information.

– rootDirectory: Root directory containing the configuration files for the application. See further for a directory structure.

– applications: List of known applications.

example (as yaml)

rootDirectory : /Users/rubus/atbash/config-server/server/demo
applications : ["app1", "app2"]

Possible directory structure

/demo
├── /app1
│  ├── app1.properties
│  └── app1-test.properties
└── /app2
   └── app2.properties

A subdirectory for each application so that the configuration values are nicely separated for each application.

All the features of Atbash config (like stage, yaml support, date support, etc) are available.

Client configuration

A ConfigSource implementation is available which integrates the result of the call to the JAX-RS endpoint into the configuration of the application.

In this first version, Atbash configuration extension is required (which makes it a bit more work) but this requirement will be removed in the next version.

Add the Maven artifact to your dependencies

    <dependency>
        <groupId>be.atbash.config</groupId>
        <artifactId>atbash-config-client</artifactId>
        <version>${atbash-config.version}</version>
    </dependency>

Define the base name of the configuration file which contains the configuration of the Config server.

Create a class implementing the be.atbash.config.spi.BaseConfigurationName interface and defining this class for usage with the ServiceLoader mechanism of Java.

Define the class name within src/main/resources/META-INF/services/be.atbash.config.spi.BaseConfigurationName

be.atbash.config.examples.DemoBaseName

Define the configuration within the specified properties file.

config.server.url=http://localhost:8181
config.server.app=app1
config.server.stage=test

* config.server.url: The _root_ of the Config server endpoint.

* config.server.app: The name of your application

* config.server.stage: (optional) Indication of the stage of the application to retrieve specific values.

We can use all the regular features of Configuration or Atbash config to specify the stage value, like environment or system properties.

The client can be used in a Java SE, Java EE or a MicroProfile environment.

Conclusion

With the Atbash Configuration server, you can store the configuration values for all your applications at a central location. The values can be picked up by a JAX-RS call through the Client ConfigSource.

The server can be run by WildFly Swarm, Liberty or a regular Java EE 7 server. The client can be used for all environments capable of using the MicroProfile Configuration spec (including Java SE and Java EE)

A future version will bring you some more features and makes it easier to set things up.

Have fun.

Implicit converters for MicroProfile Config

Introduction

Last month (January 2018), an update of the MicroProfile Config specification, version 1.2, was released which makes it easier to convert the configuration values to class instances.
It adds the concept of implicit/automatic converters and removes the requirement for creating specific converters in many cases.
The first implementations are already available and Atbash Configuration is now also updated.
This blog describes what the implicit/automatic converters are and how you can use them.

Converters

The configuration values we define are Strings because they are defined as a set of Characters in the configuration files. Using the configuration could be limited to a system which is capable of reading those Strings and that the developers need to convert these values to the proper instance (like Integer, Boolean or application specific classes) themselves.
But in that case, each developer will write his own set of converters and in the case of Integers and Booleans, for example, they are all the same.
So the framework itself can take care of these converters. And since we have then already the concept of a convert, it is easy to foresee an SPI which allows the creation of custom Converters by the developer for their application specific classes.
These basic converters for Integer and Boolean for example where present since the first release of the spec. Just as the SPI to add your own converters.

Implicit Converters

But the creation of all those custom converters can be minimized further. Since our application specific classes can be created from a String, they probably have some means of instantiating them through a method which takes a String parameter
– The constructor
– A static method with names like valueOf or parse.
So with the new implicit/automatic converters feature, when we have some class like this
public class ClassWithStringConstructor {
    private String value;

    public ClassWithStringConstructor (String value) {
        this.value = value;
    }

    public String getValue() {
        return value;
    }
}
and have a parameter
some.key = Atbash

We can retrieve an instance of the class, with the value using

config.getValue("some.key", ClassWithStringConstructor.class)

or when we are in a CDI enabled context, we can also use injection

@Inject
@ConfigProperty("some.key")
private ClassWithStringConstructor parameterValue;

Collection type parameters

Another nice addition to this 1.2 release, is the use of collection type parameters. The ability to have multiple parameter values, like a list of values which is converted to an Array, List or Set.
It is quite common that a configuration value is, in fact, a list of value.
pets = dog,cat
previously, the developer needed to split the String and use the individual parts separately (although this is a very simple task with String.split() ), can be done automatically.
config.getValue("pets", String[].class)

With injection, we can automatically convert it to a List or a Set.

@Inject
@ConfigProperty(name = "pets")
private List<String> pets;

The List and Set variants can’t be used in a programmatic way (using config.getValue) since the second parameter doesn’t accept parameterized types (like List<String>).

When you have a value which contains a comma it in, you need to escape it
a,b,c\\,d

which results in 3 configuration values, where the last one is c,d.

Implicit combined with Collection.

We can even combine the collection type feature with the implicit/automatic converter feature. If we instead specify the class name like ClassWithStringConstructor instead of String, the configuration framework will use the appropriate way of converting.
config.getValue("pets", ClassWithStringConstructor[].class)

or

@Inject
@ConfigProperty(name = "pets")
private List<ClassWithStringConstructor> pets;

Octopus Config

As you could read in a previous blog, Octopus Config is a Java 7 port from Geronimo config. The support for the implicit/automatic converters and collections (like the arrays, List, and set) is taken from them.
Octopus Config itself is also updated so that the YAML usage supports the Collection type parameters. The example above with the pets can be written in the YAML equivalent format as
pets: [dog , cat]

The release 0.9.1 contains also some small other features are listed here. Most notable features are

– Improved Class converters based on different classloaders.
– Test artefact.
– Logging in Java SE environment
– Improvement information in logging with @ModuleConfigName
– Compatibility with another MicroProfile Config implementation on Java 8.

Conclusion

With the addition of the implicit/automatic converters, the need for writing custom converters can be dropped and conversion will take place by convention.
The Octopus Config version v0.9.1 is updated to incorporate these features together with some small other improvements and fixes.
Have fun.

First release of Atbash Configuration project

Introduction

Although configuration is important and essential in any application, there is no real standardization available today.

There were several attempts to create a standard around it so that each application could read his parameters in a consistent way. But as of today, there is no final standardization, nor within Java SE, not within Java EE.

Since it is so important, The Eclipse MicroProfile project created a specification for reading parameters in their Micro-services architecture. And it was almost the first thing they did (which highlights the importance they gave it). That work is now continued under the wings of the JCP (as JSR-382) so that it can become a real standard within Java SE and Java EE.

Instead of using other frameworks and libraries which have support for reading configuration, I wanted to switch to the MicroProfile Config project. Because I wanted to be prepared for the feature but encountered a few issues.

Short version

Adapted the code of MicroProfile Config API and Apache Geronimo config to Java 7 and added my own extensions which compose Atbash Config.

The Maven artefacts are available on Maven Central and the user manual is here.

My issues with MicroProfile Config project

Issue is a heavy term, my requirements are not (yet) available within the Config project but the good thing is that they are planned within the JCP specification work.

I want to read configuration parameters

  1. With Java 7
  2. For any Java EE project, not limited to Eclipse MicroProfile applications.
  3. Use single artefact which contains the configuration parameters for different environments embedded in it (within different files)
  4. Use YAML structured configuration files.

And that is not possible for the moment because

  • The MicroProfile Config project is build using Java 8 code structures. So it is not possible to run it in a Java 7 environment.
  • The Apache Geronimo Configuration project implements the API and is a very good candidate to use in my Java EE programs. Because it is created as a standalone project (not as part of a server like other implementation such as Payara, KumuluzEE, WebSphere Liberty or others) and thus you can add this dependency to any Java EE application. But configuration file name is still related to MicroProfile (/META-INF/microprofile-config.properties).
  • MicroProfile Config project has support for overruling properties by specifying them in the environment or as JVM system parameters. But there is no feature to define multiple files nor the ability to define multiple files within the artefact for each environment.
  • Only properties files are supported. And when you have larger projects, it can be handy to use a YAML structure so that it is more clear which properties are closely related.

Solution

Adapt MicroProfile Config API and Apache Geronimo Configuration.

Except for the first issue, support for Java 7, my own configuration extension could be a solution for my requirements. So I had no other choice than to copy the code and adapt it. And although the code is licensed under the Apache OpenSource license (which allows me to do this) it doesn’t feel right. So I want to thank explicitly all those committers that created the code for their wonderful work and I also made explicitly a reference within the notice file and documentation.

Atbash configuration extension

The extension consist of a custom implementation of the ConfigSource interface. That way I’m able to register another source for reading configuration parameters and fulfill my other 3 requirements.

– Implement the BaseConfigurationName interface and define the class also for the ServiceLoader mechanism. Now I’m able to use any filename like demo.properties or demo.yaml when the value demo is specified as the base name.
– By specifying the environment with the JVM system parameter -S, I can add a prefix to the filename and supply additional or override parameters for a certain environment
– YAML file support is implemented with the use of Snakeyaml where the file is converted to key-value pairs.

For more information, you can read the manual.

Getting started

There are 3 ways how you can use the Maven artefacts

1. Use plain MicroProfile config with Java 7.

Add the adapted Geronimo config code to your project

<dependency>
   <groupId>be.atbash.config</groupId>
   <artifactId>geronimo-config</artifactId>
   <version>0.9</version>
</dependency>

2. Use the Atbash configuration extension with Java 7

You need the Geronimo config and atbash config project

<dependency>
   <groupId>be.atbash.config</groupId>
   <artifactId>geronimo-config</artifactId>
   <version>0.9</version>
</dependency>

<dependency>
   <groupId>be.atbash.config</groupId>
   <artifactId>atbash-config</artifactId>
   <version>0.9</version>
</dependency>

3. Use the Atbash configuration extension with Java 8

In this scenario, you don’t need the adapted code and can use the regular implementations

<dependency>
   <groupId>org.apache.geronimo.config</groupId>
   <artifactId>geronimo-config-impl</artifactId>
   <version>1.0</version>
</dependency>

<dependency>
   <groupId>be.atbash.config</groupId>
   <artifactId>atbash-config</artifactId>
   <version>0.9</version>
   <exclusions>
      <exclusion>
         <groupId>be.atbash.config</groupId>
         <artifactId>microprofile-config-api</artifactId>
      </exclusion>
   </exclusions>
</dependency>

Conclusion

MicroProfile Config is the first real standardization around reading parameters for configuration. They have built their implementations on top of Java 8.

So when you need a general usable library with Java EE 7 (which is using Java 7 as a base version), you can use the Atbash config code (adapted from the MicroProfile Config API and Apache Geronimo config as implementation) which has a few handy extensions.

This makes you future proof at the time JSR-382 will be adopted by the different application servers.

Thank you for reading until the bottom of this blog entry and have ave fun.