Whether you are creating an application for users, a library for devs, or a microservice, how your software handles configuration is a critical concern and needs to cater to your users’ needs. Approaching configuration management as a part of the user interface will lead to better, user-friendly design decisions.
Configuration is part of a broader category of user-changeable inputs to software.
Configuration is user-controllable settings which change the behavior of the software at runtime. This includes things like logging levels, feature flags, and user preferences like dark/light mode. Configuration is provided either directly by the user (for example, CLI arguments) or from stored values, such as environment variables or config files.
Other kinds of user inputs are beyond the scope of configuration, such as plugins, data which the software will process, and complex user-defined logic like workflows, schemas, kubernetes manifests, or logstash pipelines.
llm_model="gpt-oss" is a config setting, the model file itself is not.There is no hard and fast rule to distinguish configuration from other kinds of inputs. What starts as a simple configuration can quickly grow into large complex collections of settings. It is helpful to keep GUI settings in mind when thinking about what belongs in the scope of configuration. If it is easy to include on a settings page (such as a dark mode toggle), it’s probably a good config option. Often config options are short and simple enough for a user to type into a CLI, compared to more complex entries that you may want an IDE and syntax highlighting to manage. Consider all of your’ users,
That’s the critical question because the answer depends on who your users are and their needs. In a GUI or web-app it is common practice to include a ‘settings’ menu. Config files are a common solution for many applications. There are many other ways to configure software like:
Often, but not always, software libraries receive their config settings from the code which utilizes them.
When someone edits your config file, they’re not “tinkering with internals”. They’re using an interface you designed — whether you meant to or not. Since configuration deserves the same care as any other UI element, it should:
Configuration values enter the system as strings. Your application turns them into the typed values that it needs. Don’t make your users debug type mechanics. Handle parsing and validation in code.
This helps you craft helpful error messages like: “TIMEOUT must look like 30s, 5m, or 250ms” “THRESHOLD must be between 0 and 1”
This supports configuration using Environment variables, and is important for users running software in containers, on cloud infrastructure, and other contexts ENV variables are preferred.
Lists and key:value maps require special consideration. It often makes sense to include lists of options in a configuration. A short list in a CLI arg, or .yaml file can be simple and useful… But lists are difficult to represent as strings, in Environment variables, and require extra care when converting them to types. Long lists (more than you want to type out by hand) probably represent Data not configuration.
Interpolation and templating in config starts innocently: “Let me reuse a base URL” “Let me reference DATA_DIR” “Let me compute a path” Then it grows to include conditionals, defaults, string functions, environment lookups, precedence rules, edge cases around escaping… It becomes its own language (DSL)
Configuration should declare inputs. If values need to be derived, do the derivation in code where it can be tested.
If you require advanced user-defined logic, keep it separate from your runtime config. Build the appropriate testing and documentation around it, and avoid conflating simple settings like dark mode and quiet output with programmatic logic.
keep the standard config small and boring (good!) move complex, user-defined behavior into a separate lane:
This separation protects your config UI from becoming a fragile all-purpose control panel.
This is one of the most expensive mistakes in “research tools that became real systems.” Configuration answers: How should the tool run? Data answers: What should the tool process? If your config file starts containing lots of records, big lists, or evolving rule sets, it’s probably not config anymore—it’s data.
Config is not a substitute database.
Configuration is part of your user interface, treat it like you would any other UI: keep it small, understandable, and hard to misuse.
In practice that means designing for the real world: multiple config sources (file, CLI, env vars), users copying examples, and “future you” debugging a run at 2am. Favor simple, string-shaped inputs at the boundaries, validate and parse inside the program, and resist the temptation to smuggle computation into configuration through interpolation.
And when your users truly need complex logic don’t force it into the same settings system. Give it a separate lane with the right tooling, validation, and tests.
Boring configuration is a feature. It makes your tool easier to run, easier to deploy, and easier to trust.
]]>skunkworks quarantine module to isolate it
is a useful strategy when it is necessary.
“How will I know the good [code] from the bad?” In the skunkworks module it will be
We Should™ only merge code which meets our quality and test coverage standards…
Sadly, this is not always practical or possible. There are times when a quick fix or a new feature needs to be merged ASAP. Even after citing the Big Ball of Mud pattern as a dire warning, it may be necessary to merge a big chunk of messy untested jank and hope for the best.
I don’t want to come across as too negative here, because there are very good reasons to be flexible when it comes to code quality. If there is an outage, getting back up and running is the top priority. We do not want to block teammates or other teams by waiting on a perfect implementation. There are many, many, real-world forces that legitimately drive us to cut corners.
There are many strategies for dealing with this scenario, but in a crunch where minutes matter, a few simple and ineffective strategies tend to dominate.
These two quick and easy strategies work well together: code comments can reference tickets, tickets support thorough explanations and discussion about HOW TO “FIXME”.
My preferred solution is to create a new submodule named skunkworks
to contain any code which doesn’t meet quality and testing standards.
Every time a developer sees from sunkworks import ... they know to be cautious.
When that code breaks, the traceback will clearly show that the exception bubbled up from skunkworks.
No one enjoys, or benefits from, the finger-pointing accusations and defenses that can arise when code fails catastrophically. Using an explicit quarantine module can help.
When your choose to create a skunkworks module,
you’re explicitly recognizing the need to take on tech-debt.
All of your stake-holders should be made aware, and know to expect problems.
It says “We need to use some substandard code. We acknowledge and accept the risks”
Everyone involved gets to share the responsibility, not just the author or approver of a specific pull request. It is an architectural and managerial decision that the whole team makes together.
Putting all the untrusted code in one place makes finding what to clean up easy. Moving code out of skunkworks can take precedence over all other cleanup and refactoring work.
It’s a great place for junior devs to start, tell them to “go clean out skunkworks”.
They can git grep skunkworks to see where its used.
You’re giving them working prototype-quality code,
which they will need to write tests for,
refactor and clean up until it meets standards, and then merge into code base.
It’s easier to prioritize for cleanup. Instead of arguing that “the {some subsystem} code is a mess, and we need to clean it up.” Try “The code which {big important think people care about} relies on is still in skunkworks and we need to get it out of there before something breaks”
]]>TLDR: BatConf is thread-safe, well tested, and ready for your free-threaded applications
Fortunately for us, configuration management is conceptually thread-safe. We want to load configuration values from their source(s) once, and read them many times. We do not expect configuration values to be written or updated at runtime.
BatConf’s architecture is essentially thread-safe:
“You’re far too trusting,” -Grand Moff Tarkin
While BatConf is conceptually and architecturally thread-safe, it is not aggressively thread-safe. Users are prohibited from creating attributes on a Configuration object, updating Environment variables at runtime, or creating new config sources that are unsafe.
I don’t think we need to be overly concerned with users creating their own concurrency problems. There is no hidden concurrency inside our package, so any thread safety issues should be apparent in user code.
But what if we missed something in the design, and there’s some unexpected concurrency issue?
Simply saying “It Should™ be fine” is not very satisfying. I would much rather put some tests in place.
Since we already have an excellent set of test suites, running them with the GIL disabled gives us reasonable certainty that there are no major issues.
I was able to create a new Conda environment, install python-freethreading,
and run our test suite without any errors!
Then we added 3.14t to our CI testing matrix on github, everything passes. Nice!
Next we added a new suite of free-threading test cases. These tests utilize BatConf Configuration objects in multiple threads.
They demonstrate the expected and officially supported use-cases, provide examples, and guarantee that these uses-cases work both with and without the GIL.
Digging into the Python Free-Threading Guide, I was further assured that our pure-python package should be relatively safe, as much more care needs to be taken with packages that include extensions in other languages.
There are several plugins for PyTest which allow us to run test cases in parallel, including pytest-freethreaded and pytest-run-parallel.
These failed spectacularly! Generating some surprising errors, and even locking up the pytest process!
FAILED tests/integration/configuration_test.py::FreeFormConfigTreeTests::test_sub_configs_respect_environment_variables - KeyError: 'SHELL'
Wait… why is it looking for SHELL? git grep SHELL confirms that doesn’t even appear in our code base…
FAILED tests/example/example_test.py::CLITests::test_configuration_override_from_cli_args - AttributeError: '_patch' object has no attribute 'temp_original'
Errors bubbling up from the _patch object… uh oh.
While the package its self is thread-safe, the test suites are not.
We make extensive use of unittest.mock.patch to isolate objects under test from side effects,
and it is not thread-safe.
Different threads compete to patch/unpatch the same objects.
Tests can run with patches from other tests in place.
Rewriting all our tests to be thread-safe is not on the roadmap. Running all of our test suites together takes about 1.06s, so performance is not a concern. Isolating test cases with mutex locks, and refactoring the code to use more dependency injection for the sake of testing, does not provide much value, and is unlikely to reveal thread-safety issues in the code itself.
BatConf is ready for use in your free-threaded Python applications!
Free-threading in Python is the best thing since f-strings. I can’t wait to build more truly parallel Python code, and develop new testing techniques to guarantee the safety and reliability of that code.
~ℒ
]]>TLDR: Assigning descriptors to class attributes allows those attributes to behave like Properties. The values can be computed lazily, only when needed, and referenced
.directlyinstead of requiring a.method_call()
While implementing type-checking for Pandas dataframes using Pandera,
I wanted to create re-usable Field definitions.
Repeating the definition of a date field in every DataFrameModel subclass
or using inheritance for individual fields felt clunky and was difficult to read.
It turns out each field in a DataFrameModel definition needs to be a unique instance. You can read about the details in this github issue.
With a little help from Niels Bantilan, the maintainer of Pandera, we came up with this solution using partials to define reusable Field definitions
from functools import partial
from pandera import DataFrameModel, Field
NormalizedField = partial(Field, ge=0, le=1)
class GoodModelDF(DataFrameModel):
xnorm: float = NormalizedField()
ynorm: float = NormalizedField()
class AlsoGoodModelDF(DataFrameModel):
xnorm: float = NormalizedField()
While debugging the problems with reusable fields, I discovered the unexpected behavior that motivated this change.
My first attempt to reuse a Field, resulted in a perplexing error:
NormalizedField: float = Field(ge=0, le=1)
class BadModelDF(DataFrameModel):
field_0: float = GenericField
field_1: float = GenericField # Bug: this breaks the model
class Config:
strict = True
Calling BadModelDF.validate(some_dataframe) raised the exception:
SchemaError: column 'field_0' not in DataFrameSchema {'field_1': <Schema Column(name=field_1, type=DataType(float64))>
The root cause of that error is due to the behavior of DataFrameModel._build_columns_index
, but is not relevant to this story.
Digging into the DataFrameModel source code
I saw that the .__fields__ attribute should™ contain the data I’m looking for…
but when I viewed BadModelDF.__fields__, it was an empty dict {}!
And the .__schema__ value was None.
This was frustrating… I defined a valid DataFrameModel, why would its attributes all be empty, uninitialized?
The DataFrameModel class had a sort of secret __init__ method.
You needed to call the to_schema
method (or any other method which calls it) first,
because .to_schema() populates those data attributes.
Being deeply unsatisfied with this “spooky action at a distance”, I knew how it should™ work. I expect the value of the attributes to always be correct.
We also don’t want to initialize these values when the class is interpreted,
because it is computationally expensive.
There are cases where we will never use the computed values,
for example when we set PANDERA_VALIDATION_ENABLED=False.
In fact, I don’t want to calculate the __schema__ and all the other data attributes at all,
just to read the __fields__ data.
IMHO: All™ data attributes on an object should behave like attributes, not methods.
I want to reference a value like person.age >= 21 not person.get_current_age() >= 21.
Python Properties provide this behavior for computed values on class instances.
Sadly, properties do not work on classes, they work on instances of the class. The interpreter reads a property definition like this:
def MyClass:
@property
def value(self):
return compute_the_value()
something like “when I create an instance of this class, make its value attribute a property”.
Accessing a property of a class returns the property object <property at 0x###>,
not a computed value as we may have hoped.
Fortunately, Descriptors allow us to write class attributes which behave just like properties!
A descriptor is a class, which has a __get__ method with the appropriate signature.
We can assign an instance of the descriptor to a class attribute,
and it will behave just like a property, returning the computed value.
In [1]: class ValDesc:
...: def __get__(self, obj, objtype=None):
...: return 42
...:
In [2]: class SomeClass:
...: value = ValDesc()
...:
In [3]: SomeClass.value
Out[3]: 42
I had several goals for improving the Pandera Model interface:
.to_schema() to populate data attribues.__fields__ should™ be idempotent, and always provide valid values.To accomplish this,
I added several Descriptor classes to the DataClassModel,
updated the old get_ and to_ class methods, and updated other class methods
to access the data attributes directly, while preserving the behavior of the API.
It worked! I improved the interface for Pandera’s Model class, eliminating a confusing behavior. This change to a core component of the package was not disruptive, and the full test suite passed without modification. We also achieved some marginal performance improvements.
You can read the full commit in the PR. This is a brief illustration of some of the changes:
class _ClassDescriptor:
def __init__(self):
self.cache = {}
class _FieldsDescriptor(_ClassDescriptor):
"""Descriptor which allows __fields__ to act as a class property."""
def __get__(self, obj, cls) -> TFields:
if self.cache.get(cls) is None:
self.cache[cls] = cls._collect_fields()
for field, (annot_info, _) in self.cache[cls].items():
if isinstance(annot_info.arg, TypeVar):
raise SchemaInitError(
f"Field {field} has a generic data type"
)
return self.cache[cls]
class _SchemaDescriptor:
def __get__(self, obj, cls):
...
class DataFrameModel(Generic[TDataFrame, TSchema], BaseModel):
...
__fields__: ClassVar[TFields] = cast(TFields, _FieldsDescriptor())
__schema__ = _SchemaDescriptor()
...
# Update class methods to use the new descriptors, instead of calling class methods
@classmethod
def to_schema(cls) -> TSchema:
"""Create :class:`~pandera.DataFrameSchema` from the :class:`.DataFrameModel`."""
return cls.__schema__
@classmethod
def to_yaml(cls, stream: Optional[os.PathLike] = None):
"""
Convert `Schema` to yaml using `io.to_yaml`.
"""
return cls.__schema__.to_yaml(stream)
...
Thanks for reading,
Lundy
]]>