Knowledge-Centric Agents
A framework for workflow generation in visual creation systems that emphasizes modeling and reasoning with knowledge across multiple abstraction levels, rather than direct text-to-JSON generation.
When to use
When generating complex, structured outputs like visual creation workflows (e.g., ComfyUI) that require expert-level reasoning and hierarchical understanding, and where direct text-to-code generation is insufficient.
How to apply
Distill hierarchical knowledge (pseudo-codes, skeletons, strategies) from existing workflows.
Inject this knowledge into an LLM via supervised fine-tuning.
During inference, the model uses reversible reasoning to synthesize executable workflows, optionally with self-reflection.
Glossary
- hierarchical knowledge
- Expert know-how layered from high-level strategies down to concrete steps — not a flat dump of instructions.
- pseudo-codes
- Informal step outlines that capture workflow logic without being fully runnable code yet.
- supervised fine-tuning
- Training an LLM on labeled examples (input → desired output) so it internalizes a skill beyond one prompt.
- reversible reasoning
- Working forward to a plan and backward to check it still matches the goal — useful when workflows branch.
- self-reflection
- A second pass where the model critiques or revises its own draft before committing to the final workflow.