Hand-rolled Claw (3): All-around Perception — Multi-modal & Dynamic Skill Tree
[!TIP] Objective: Enable multi-modal (vision/files) perception via CLI parameters and build a dynamically-loaded plugin-based skill system. 1. Letting the CLI “See” the World Multi-modal capabilities don’t necessarily require complex SDKs. In VISAGENT, we leverage the native support for file paths in the gemini CLI (@path) to achieve sensory integration at the RoleEngine layer: def _do_raw_invoke(self, message, files=None): # Construct multi-modal suffix mm_suffix = "" if files: mm_suffix = "\n" + "\n".join([f"@{f}" for f in files]) # Append to final Prompt full_input = f"{message}{mm_suffix}" # ... execute subprocess Field Experience: To handle complex visual tasks, we encapsulated a dedicated vision_expert skill. By using the DEEP reasoning mode, we guide the AI through Chain-of-Thought thinking, enabling precise identification of screenshots and UI components. ...