Controls

#11: Toward real-time interaction

Across many previous entries, I complained many times that my agent rarely asks questions. It is easy to blame your tools and easy to ignore the possibility that you’re using them wrong. In the quest to deliver features quickly, the incentive is often to forge ahead without reflecting on tour process.

This is what the plan agent is for. In Opencode, the instruction to ask questions is in the session prompt. I had just been ignoring a core feature. In the next feature, I would remedy that. The earliest iterations of Fugue focused on an imperative API. Shortly after starting, the focus shifted toward ingesting a declarative document. This was the right choice, but we lost real-time manipulation of the audio graph along the way. As the library prepares for use within a real application, it needs that functionality back. It needs a way to model knobs, sliders, and buttons.

I want to add the concept of "controls" to this library. A control is like an input but is changed through user interaction. This could be a knob or button in a user interface. A control may or may not also map to a signal input. While it is up to the individual module to control, generally a signal input overwrites a value set by a control. Many of these correspond to entries in the patch document. For example, the adsr module has `attack`, `decay`, `sustain` and `release` in its config. These should also be exposed as controls. Another example is the `oscillator` module with its `frequency` and `oscillator_type` config values. The `frequency` is an example of a control that can then be overridden my a signal input.

Again, the purpose of these controls is to allow real-time interaction. So, they will ultimately be accessible from a visual interface, REPL, or something similar.

Please develop a plan to implement this concept in the engine and all existing modules.

...

>> How should controls interact with signal inputs?:
The signal should overwrite the control

>> What granularity should controls have for enum types like OscillatorType
>> or FilterType?
Use index-based f32

>> Should controls be grouped into a single handle per module or individual
>> handles per parameter?
Use a single controls struct per module

>> For tracking which inputs are connected to signals (vs using control 
>> defaults), when should this be determined?
It will be very important to eventually support adding and removing connections between modules in real time. I assume that means we need per-sample detection. If there are other options, feel free to explore them.

>> Should we rename existing handles (Tempo, MelodyParams) to follow the new 
>> *Controls naming convention?
Yes, rename them for consistency

>> For Mixer's array controls, should we implement both the typed MixerControls 
>> methods AND the indexed get_control/set_control on the Module trait?
I think we need to stick with the general get_control/set_control paradigm with hierarchical keys like 'level.1' (or 'step.0.gate' for a more complex control). Otherwise, future applications will need too much knowledge of the underlying modules. This become problematic when we eventually support dynamically imported modules, for example. If you want to explore other options, feel free to.

After that, we had a clear plan, and I handed it off to the build agent for the implementation. It neglected to update the examples, as it often does, but the generated code was otherwise functional and cleaner than previous iterations that excluded the plan agent.

The implications of adopting plan and build agents extend beyond just getting the model to produce better code. This step is analogous to a startup evolving from two devs in a garage to a company trying to produce real value. An older process of bouncing ideas off each other and hacking your way to production start falling short. It may be too early for rigorous process, code coverage requirements, ops teams, and the other trappings of big companies, but thinking things through is overdue.

This where we cross from vibe coding to something else. Something better.

#11: Controls

2026.2.5