HOCON 3.0 Specification

[Aaronontheweb]

HOCON 3.0 Specification

In the previous releases of HOCON, we let the OSS project do its own thing without any plan for integrating it into Akka.NET and replacing the stand-alone HOCON engine built into the Akka.Configuration.Config class. This was a mistake and led to us having to drop stand-alone HOCON integration as part of the Akka.NET v1.4.1 milestone.

Motivation for Moving to Stand-alone HOCON

Why bother moving from Akka.Configuration.Config? Isn't it good enough as is? Why support an entire configuration library?

The motivations for separating HOCON from Akka.NET are the following:

1. Akka.NET is a massive library with 5,000+ tests that have to be run on at least 3 runtimes whenever a change is made to the core Akka package, which is where Akka.Configuration.Config is housed. If we want to innovate around configuration, it's very expensive for us to do this without separating the libraries into their own projects.
2. There has been a lot of innovation around configuration since the release of .NET Core, especially with the new Microsoft.Extensions.Configuration namespace. We're opposed to taking a dependency on such a library directly inside Akka.NET itself (OSS hygiene) - but if we could consume Microsoft.Extensions.Configuration and use that to generate a HOCON Config object prior to starting an ActorSystem, that could be done easily in a separate library.
3. Support for stand-alone HOCON tools, such as linters and Intellisense, could be developed more easily by keeping the HOCON tooling separated from Akka.NET.
4. Integration with other runtimes, such as ASP.NET Core, can only feasibly by done if HOCON is separated into a stand-alone library.
5. Quality of life improvements, such as clearer HOCON validation errors, performance improvements, and so on can be more cheaply and quickly developed inside a library with a smaller testing footprint.

HOCON 2.0 and Earlier

HOCON 2 and earlier releases of HOCON were developed without a formal plan for introducing them in a non-breaking way into Akka.NET, the biggest consumer of HOCON currently. This will be remedied in HOCON 3.0.

However, our plan is to deprecate HOCON 2 as quickly as possible and move onto HOCON 3 using the spec contained herein.

HOCON 3.0 Development Plan

The HOCON 3.0 development will comport to the following rules and requirements:

Functional Requirements

1. All HOCON Config behaviors must support the previous behavior contract of Akka.Configuration.Config - if Config.GetString(string hoconPath) didn't throw an exception when hoconPath was not found inside the current Config object, the new HOCON Config shouldn't either.
2. All public HOCON method calls, anything that would be reasonably consumed inside Akka.NET or any other downstream consumer, must be implemented on a public interface - call it IHoconConfig, and that interface will be implemented on both HOCON.Config and Akka.Configuration.Config with matching behaviors.
3. All Config objects must be immutable and read-only once they are returned from the ConfigurationFactory or equivalent parser / loader class.
4. Reading a HOCON value from a Config class should produce as few object allocations as possible - preferably zero.
5. HOCON must follow the formal HOCON syntax specifications during parsing - any parsing exception that occurs must include a detailed error message showing the HOCON key that caused the parse exception to occur.
6. Fallbacks must not be merged unless a public Config Config.Merge() call or equivalent is made explicitly by the caller.
7. The read and write models must be kept separate - i.e. using a mutable builder pattern to create the initial HOCON object graph during parsing is fine, but the output of that parse operation should be housed in a separate read model.
8. HOCON and Akka.Configuration.Config namespaces should be able to exist side by side inside the same application without creating massive conflicts or breaks.
9. Applications that use Akka.Configuration.Config should be able to switch to a new version of Akka.NET that uses the stand-alone HOCON NuGet package without modifying any code or configuration (except for invalid HOCON that the previous parser didn't enforce.)

Testing Requirements

1. All HOCON tests from the current Akka.Configuration.Config test suite must pass, unless that feature is explicitly being deprecated or is already obsolete.
2. All changes to the Config class must be benchmarked on each pull request using an NBench or equivalent test suite. Specifically we must measure throughput, memory allocation, and garbage colletion:
   1. Parsing performance
   2. Fallback chaining performance WithFallback(Config c)
   3. Lookup performance with variable layers of complexity (i.e. 1x nested key, 2x nested key, 10x nested key)
   4. Lookup performance with variable layers of complexity and fallbacks (i.e. 1x nested key, 2x nested key, 10x nested key)
3. All IHoconConfig must be tested using identical behavior specifications to ensure consistent behavior across implementations.
4. Need to integration test HOCON:
   1. Independently, using complex configurations (many fallbacks with overlapping values)
   2. Inside a full-fledged Akka.NET application (https:/akkadotnet/akka.net-integration-tests) once Akka.NET IHoconConfig compatibility has been added.
5. Need to concurrently test:
   1. HOCON reads / writes on a Config object while new fallbacks are being added to it (guarantee immutability.)

IHoconConfig interface

To give us some idea of what this aforementioned IHoconConfig interface might look like, here's an example based on what we can extract from the Akka.Configuration.Config class in Akka.NET v1.3.17:

public interface IHoconConfig
{
    /// <summary>
    /// Determines if this root node contains any values
    /// </summary>
    bool IsEmpty { get; }

    /// <summary>
    /// Retrieves a boolean value from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the value to retrieve.</param>
    /// <param name="default">The default value to return if the value doesn't exist.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The boolean value defined in the specified path.</returns>
    bool GetBoolean(string path, bool @default = false);

    /// <summary>
    /// Retrieves a long value, optionally suffixed with a 'b', from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the value to retrieve.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The long value defined in the specified path.</returns>
    long? GetByteSize(string path);

    /// <summary>
    /// Retrieves a long value, optionally suffixed with a 'b', from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the value to retrieve.</param>
    /// <param name="def">Default return value if none provided.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The long value defined in the specified path.</returns>
    long? GetByteSize(string path, long? def = null);

    /// <summary>
    /// Retrieves an integer value from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the value to retrieve.</param>
    /// <param name="default">The default value to return if the value doesn't exist.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The integer value defined in the specified path.</returns>
    int GetInt(string path, int @default = 0);

    /// <summary>
    /// Retrieves a long value from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the value to retrieve.</param>
    /// <param name="default">The default value to return if the value doesn't exist.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The long value defined in the specified path.</returns>
    long GetLong(string path, long @default = 0);

    /// <summary>
    /// Retrieves a string value from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the value to retrieve.</param>
    /// <param name="default">The default value to return if the value doesn't exist.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The string value defined in the specified path.</returns>
    string GetString(string path, string @default = null);

    /// <summary>
    /// Retrieves a float value from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the value to retrieve.</param>
    /// <param name="default">The default value to return if the value doesn't exist.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The float value defined in the specified path.</returns>
    float GetFloat(string path, float @default = 0);

    /// <summary>
    /// Retrieves a decimal value from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the value to retrieve.</param>
    /// <param name="default">The default value to return if the value doesn't exist.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The decimal value defined in the specified path.</returns>
    decimal GetDecimal(string path, decimal @default = 0);

    /// <summary>
    /// Retrieves a double value from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the value to retrieve.</param>
    /// <param name="default">The default value to return if the value doesn't exist.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The double value defined in the specified path.</returns>
    double GetDouble(string path, double @default = 0);

    /// <summary>
    /// Retrieves a list of boolean values from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the values to retrieve.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The list of boolean values defined in the specified path.</returns>
    IList<Boolean> GetBooleanList(string path);

    /// <summary>
    /// Retrieves a list of decimal values from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the values to retrieve.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The list of decimal values defined in the specified path.</returns>
    IList<decimal> GetDecimalList(string path);

    /// <summary>
    /// Retrieves a list of float values from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the values to retrieve.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The list of float values defined in the specified path.</returns>
    IList<float> GetFloatList(string path);

    /// <summary>
    /// Retrieves a list of double values from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the values to retrieve.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The list of double values defined in the specified path.</returns>
    IList<double> GetDoubleList(string path);

    /// <summary>
    /// Retrieves a list of int values from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the values to retrieve.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The list of int values defined in the specified path.</returns>
    IList<int> GetIntList(string path);

    /// <summary>
    /// Retrieves a list of long values from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the values to retrieve.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The list of long values defined in the specified path.</returns>
    IList<long> GetLongList(string path);

    /// <summary>
    /// Retrieves a list of byte values from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the values to retrieve.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The list of byte values defined in the specified path.</returns>
    IList<byte> GetByteList(string path);

    /// <summary>
    /// Retrieves a list of string values from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the values to retrieve.</param>
    /// <param name="strings"></param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The list of string values defined in the specified path.</returns>
    IList<string> GetStringList(string path);

    /// <summary>
    /// Retrieves a list of string values from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the values to retrieve.</param>
    /// <param name="defaultPaths">Default paths that will be returned to the user.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The list of string values defined in the specified path.</returns>
    IList<string> GetStringList(string path, string[] defaultPaths);

    /// <summary>
    /// Retrieves a new configuration from the current configuration
    /// with the root node being the supplied path.
    /// </summary>
    /// <param name="path">The path that contains the configuration to retrieve.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>A new configuration with the root node being the supplied path.</returns>
    IHoconConfig GetConfig(string path);

    /// <summary>
    /// Retrieves a <see cref="TimeSpan"/> value from the specified path in the configuration.
    /// </summary>
    /// <param name="path">The path that contains the value to retrieve.</param>
    /// <param name="default">The default value to return if the value doesn't exist.</param>
    /// <param name="allowInfinite"><c>true</c> if infinite timespans are allowed; otherwise <c>false</c>.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns>The <see cref="TimeSpan"/> value defined in the specified path.</returns>
    TimeSpan GetTimeSpan(string path, TimeSpan? @default = null, bool allowInfinite = true);

    /// <summary>
    /// Converts the current configuration to a string.
    /// </summary>
    /// <returns>A string containing the current configuration.</returns>
    string ToString();

    /// <summary>
    /// Converts the current configuration to a string 
    /// </summary>
    /// <param name="includeFallback">if true returns string with current config combined with fallback key-values else only current config key-values</param>
    /// <returns>TBD</returns>
    string ToString(bool includeFallback);

    /// <summary>
    /// Configure the current configuration with a secondary source.
    /// </summary>
    /// <param name="fallback">The configuration to use as a secondary source.</param>
    /// <exception cref="ArgumentException">This exception is thrown if the given <paramref name="fallback"/> is a reference to this instance.</exception>
    /// <returns>The current configuration configured with the specified fallback.</returns>
    IHoconConfig WithFallback(IHoconConfig fallback);

    /// <summary>
    /// Determine if a HOCON configuration element exists at the specified location
    /// </summary>
    /// <param name="path">The location to check for a configuration value.</param>
    /// <exception cref="InvalidOperationException">This exception is thrown if the current node is undefined.</exception>
    /// <returns><c>true</c> if a value was found, <c>false</c> otherwise.</returns>
    bool HasPath(string path);
}

This prototype is incomplete as it doesn't offer types of Config access that is commonly used inside Akka.NET, such as iterating over the Dictionary<string, HoconValue> content of an unwrapped HoconObject - that type of functionality will need to be incorporated into this interface too.

Requirement: 100% of HOCON access that occurs inside Akka.NET should be doable through methods that have access to the IHoconConfig interface alone. In other words, we shouldn't have to cast IHoconConfg into an Akka.Configuration.Config or other type of interface

Using IHoconConfig in Practice

Here's an example of how Akka.NET would eventually change to begin consuming stand-alone HOCON. The big sea change here is Akka.NET will be programmed to consume just the IHoconConfig interface, not any specific implementation of it. That's what will make the migration from one version to another feasible.

First, we'd add a method overload that supports IHoconConfig:

public class Settings{
	public Settings(ActorSystem system, Akka.Configuration.Config config) 
	: this(system, (IHoconConfig)config){ } // call down to next constructor
	
	public Settings(ActorSystem system, IHoconConfig config)
	{
	    _userConfig = c;
	    _fallbackConfig = (IHoconConfig)ConfigurationFactory.Default();
	    RebuildConfig();
	
	    System = system;
		
		// read from IHoconConfig, instead of Akka.Configuration.Config
	    ConfigVersion = config.GetString("akka.version", null);
	    ProviderClass = GetProviderClass(config.GetString("akka.actor.provider", null));
	  
		// rest of constructor
	}
}

Once these overloads have been added, Akka.NET will be reading against the IHoconConfig interface instead of the concrete Akka.Configuration.Config class. This will make it much easier to introduce stand-alone HOCON in the future, since it's just another implementation of the same interface with the same API. Only the underlying implementation is different, not the experience of consuming it.

Development Plan

Here is how we intend to develop HOCON and integrate it back into Akka.NET over the course of the Akka.NET v1.4 lifecycle:

1. Develop standardized IHoconConfig interface based on Akka.Configuration.Config inside its own library, Hocon.Configuration.Abstractions.
2. Release HOCON.Configuration.Abstractions NuGet package with only the IHoconConfig interface coded at first (no parser, no implementation code,) reference it from Akka.NET, and have Akka.Configuration.Config implement it. Develop comprehensive behavioral test suite for IHoconConfig it inside Akka.Configuration.Tests. Ship that version of Akka.NET v1.4.
3. Begin adding IHoconConfig overloads to common Akka.NET methods that take HOCON configuration objects: ActorSystem.Create, Settings, RouterConfig, and others. Add overloads that accept an IHoconConfig object and have the Akka.Configuration.Config methods call down to the IHoconConfig implementation. Existing test suite shouldn't return different results in the event that the IHoconConfig implementation is complete and accurate. Release Akka.NET v1.4 version with these changes.
4. In parallel with item 3, develop new stand-alone HOCON client API per the requirements above and have it comply with the testing plan and compliance plan for IHoconConfig interface.
5. Swap the last explicit Akka.Configuration calls inside Akka.NET with equivalent stand-alone HOCON library calls, but only after 100% of HOCON-consuming methods inside Akka.NET are programmed against IHoconConfig - i.e. Akka.NET should call HOCON.HoconConfigurationFactory.Default() inside of Akka.Config.ConfigurationFactory.Load().
6. Mark methods that take Akka.Configuration.Config objects as arguments as [Obsolete] and warn that these methods will be removed in a future major release of Akka.NET (1.5, 1.6, etc.)
7. Remove the Akka.Configuration.Config methods once the next major release of Akka.NET is shipped.

HOCON Features

So what are the new features that should be added to stand-alone HOCON?

1. Environment variable substitution;
2. Substitution from Microsoft.Extensions.Configuration sources;
3. Consuming HOCON Config objects inside a Microsoft.Extensions.Configuration source object;
4. Detailed parsing error messages; and
5. Significantly improved performance.

[Aaronontheweb]

Separation of classes - Akka.Configuration.Config never gets modified or updated during this process aside from being changed to implement the IHoconConfig1 interface, which Akka.Configuration.Config should already support since the interface is derived from it.