arxiv_ai 90% Match Research Paper AI Researchers,LLM Developers,Agent Developers,ML Engineers,Robotics Engineers 1 week ago

OrchDAG: Complex Tool Orchestration in Multi-Turn Interactions with Plan DAGs

large-language-models › training-methods

📄 Abstract

Abstract: Agentic tool use has gained traction with the rise of agentic tool calling, yet most existing work overlooks the complexity of multi-turn tool interactions. We introduce OrchDAG, a synthetic data generation pipeline that models tool execution as directed acyclic graphs (DAGs) with controllable complexity. Using this dataset, we benchmark model performance and propose a graph-based reward to enhance RLVR training. Experiments show that the dataset presents a challenging but solvable benchmark, and the proposed reward is effective when combined with GRPO-style algorithms, highlighting the importance of leveraging topological structure and data complexity in multi-turn tool use.

Authors (3)

Yifu Lu

Shengjie Liu

Li Dong

Submitted

October 28, 2025

arXiv Category

cs.AI

arXiv PDF

Key Contributions

OrchDAG introduces a synthetic data generation pipeline that models tool execution as directed acyclic graphs (DAGs) to address the complexity of multi-turn tool interactions. It provides a challenging benchmark dataset and proposes a graph-based reward to enhance RLVR training, demonstrating effectiveness with GRPO-style algorithms.

Business Value

Enables the creation of more robust and capable AI agents that can handle complex, multi-step tasks involving various tools, leading to advanced automation and problem-solving capabilities.

Paper Metadata

Innovation Type

Novel Data Generation Pipeline and Reward Mechanism

Deployment Feasibility

The pipeline is for generating training data, which then improves LLM agents. The pipeline itself requires implementation and validation.

Limitations Addressed

Underestimation of complexity in multi-turn tool interactions,Lack of suitable benchmarks for complex tool orchestration,Ineffectiveness of standard RL training for intricate tool use scenarios

Performance Gains

Proposed reward is effective when combined with GRPO-style algorithms

Technical Tags

Tool OrchestrationMulti-Turn InteractionsDirected Acyclic Graphs (DAGs)Synthetic Data GenerationReinforcement Learning from Human Feedback (RLHF)Graph-Based RewardAgentic Tool UseComplex Tool InteractionsData ComplexityGRPO Algorithms

Research Topics

LLM AgentsTool UseReinforcement LearningData GenerationAI PlanningAutonomous Systems

Methods & Architectures

Synthetic Data Generation Pipeline (OrchDAG)Modeling tool execution as DAGsGraph-based reward functionReinforcement Learning (RL)GRPO-style algorithms Large Language Models (LLMs)

Applications & Tasks

AI Agents Tool Integration Complex Task Automation Robotics Software Development Overlooking complexity of multi-turn tool interactions in agentic tool callingNeed for challenging but solvable benchmarks for multi-turn tool useImproving RL training for complex tool interactions Generating synthetic data for complex tool orchestrationBenchmarking model performance on multi-turn tool useEnhancing RLVR training with graph-based rewardsLeveraging topological structure and data complexity

Datasets & Benchmarks

Benchmarks

OrchDAG dataset for benchmarking model performance

Performance on multi-turn tool use tasksEffectiveness of graph-based reward

Related Fields

Artificial IntelligenceMachine LearningReinforcement LearningSoftware EngineeringGraph TheoryAgent Systems

Keywords

Tool OrchestrationLLM AgentsMulti-Turn InteractionsDAGsSynthetic Data GenerationReinforcement LearningGraph RewardAgentic Tool UseOrchDAGComplex TasksGRPO

Academic Context

#LLM Agents#Tool Use#Reinforcement Learning#Data Generation#AI Planning#Autonomous Systems

Commercial Potential

Potential Products

Advanced AI agents for complex workflow automationPlatforms for training and evaluating multi-tool LLM agentsSynthetic data generation services for agentic tool use

Target Industries

TechnologySoftware DevelopmentAutomationRoboticsResearch & Development

Use Case Examples

An agent that can plan and execute a complex software development task involving multiple coding tools and APIsA research assistant that orchestrates database queries, simulation runs, and data analysis pipelinesAn AI system that manages complex multi-step processes in manufacturing or logistics

Competitive Edge

Addresses the critical challenge of multi-turn tool interaction complexity by providing a structured data generation approach and a specialized reward mechanism for RL, going beyond simpler tool-use scenarios.

Market Opportunity

Growing market for LLM development tools and agent frameworks.

Revenue Models

Licensing of the data generation pipelineproviding synthetic datasetsconsulting services.

Resource Requirements

Compute Needs

Requires significant compute for generating complex DAG structures and for RL training.

Data Requirements

Requires definitions of tools, their functionalities, and potential interaction sequences to generate DAGs.

Deployment Constraints

Complexity of modeling real-world tool interactions as DAGs,Ensuring the quality and diversity of generated DAGs,Effectiveness of the graph-based reward in diverse scenarios

Scalability

The DAG-based generation approach is designed for scalability by allowing the creation of increasingly complex interaction graphs.

Regulatory Considerations

None directlybut the generated data could be used in applications subject to regulations.

Production Readiness

Maturity Level

Research Advancement

Time to Market

1-2 years for adoption in LLM agent development.

Patent Potential

Moderate, related to the DAG-based data generation pipeline and graph reward mechanisms for tool orchestration.

View Full Paper Back to Papers