Choosing to optimise for dynamic configuration
In the last few years, dynamic configuration has become more important in software development architectural decisions. The reason for this is the need for business and product owners to be able to change the behaviour of a system without delay or deployment.
In many scenarios, it is not economically viable or actionable to stop a live system to make changes. With deployment cycles sometimes lasting months, the need for important and urgent changes needed can be a huge headache in a company.
It is for this reason that the architectural decision is taken to make functionality dynamically configurable.
Complex problems needing configuration
Understanding dynamic configurable systems is best explained at the hand of examples.
I have previously worked on a rules engine where all rules were saved in a SQL Database as strings. C# would evaluate all the rules and return the result.
Another example is companies requiring dynamic forms, depending on a large list of variables. This happens quite often in the insurance and financial services industry – the business rules are complex.
In both these scenarios, the data can change quickly – and the system needs to allow for this.
Software configuration examples
With the extreme cases above, let us look at other examples where dynamic configuration can help software be more flexible:
- Software developers can add code to toggle functionality on or off
- Security, roles and permissions can be handled by a user interface. This will cut the developer out of the process
- Dynamic forms that will allow the system to display what is necessary with a large set of variables can be added.
Why would configuration be a bad idea?
Writing highly configurable code can allow for fluid and dynamic systems yet is not necessarily fit for every product:
- The overhead and time spent on the initial solution can cause delays in getting the product to market
- Unless the coding stories are broken down and monitored, it can allow for scope creep and over-engineering.
- The benefit does not outweigh the effort
- Depending on the implementation, there could be a performance impact that would be challenging to debug.
- Configuration can in some circumstances make code more complex and provide job security to your developers
- For complex systems such as rules engines, the business logic is moved into the database. This will lower the unit test coverage, as changes can be made to the rules on the fly.
Planning for configuration
It is generally easier to update database entries, compared to the process of fixing code and deploying the solution. This is why dynamic configuration is often moved into the database.
The Database can be updated in a variety of ways:
- File drops – updating settings and the database through uploading files
- HTTP endpoints – these endpoints can be called by another system or by developers to change the behaviour of the system.
- Remote app configuration – allow certain settings to read from a remote source such as Azure App Configuration
- Creating SQL lookups and/or configurable fields that can be updated directly
It is often the case that a large amount of code infrastructure will be required to make items configurable. It, therefore, makes sense to choose wisely which settings, rules and/or fields need to be changed on the fly.
- Shortlist and prioritise the settings that will be changed often
- Plan a mechanism so that it can be reused for other settings
- Estimate the work involved to make the code configurable and allow code reuse of the configuration mechanism
- Implement a solution that will allow for dynamic configuration
When it comes to coding, we love over-engineering. We also love making systems dynamically configurable.
This allows systems to stay online while we change settings and rules.
It makes sense if the trade-off with the time of implementation is justified.