arxiv_ai 95% Match Research Paper ML Researchers,GNN Developers,AutoML Practitioners,Data Scientists 1 week ago

Graph Neural Architecture Search with GPT-4

graph-neural-networks › graph-learning

📄 Abstract

Abstract: Graph Neural Architecture Search (GNAS) has shown promising results in finding the best graph neural network architecture on a given graph dataset. However, existing GNAS methods still require intensive human labor and rich domain knowledge when designing the search space and search strategy. To this end, we integrate Large Language Models (LLMs) into GNAS and present a new GNAS model based on LLMs (GNAS-LLM for short). The basic idea of GNAS-LLM is to design a new class of GNAS prompts for LLMs to guide LLMs towards understanding the generative task of graph neural architectures. The prompts consist of descriptions of the search space, search strategy, and search feedback of GNAS. By iteratively running LLMs with the prompts, GNAS-LLM generates more accurate graph neural network architectures with fast convergence. Experimental results show that GNAS-LLM outperforms the state-of-the-art GNAS methods on four benchmark graph datasets, with an average improvement of 0.7% on the validation sets and 0.3% on the test sets. Besides, GNAS-LLM achieves an average improvement of 1.0% on the test sets based on the search space from AutoGEL.

Authors (6)

Haishuai Wang

Yang Gao

Xin Zheng

Peng Zhang

Jiajun Bu

Philip S. Yu

Submitted

September 30, 2023

arXiv Category

cs.LG

Science China Information Sciences 2025

arXiv PDF

Key Contributions

Introduces GNAS-LLM, a novel approach that integrates LLMs (specifically GPT-4) into Graph Neural Architecture Search. By designing specialized prompts, it guides LLMs to understand the GNAS task, enabling them to generate more accurate GNN architectures with fast convergence, significantly reducing human effort and domain knowledge requirements.

Business Value

Automates the complex and time-consuming process of designing effective Graph Neural Networks, accelerating the development of AI solutions for graph-structured data in areas like social networks, molecular modeling, and recommendation systems.

Paper Metadata

Innovation Type

Integration of LLMs into NAS

Deployment Feasibility

High. Leverages existing LLM technology (GPT-4) and focuses on automating a complex ML task.

Limitations Addressed

Intensive human labor in GNAS,Requirement for rich domain knowledge in search space and strategy design,Inefficiency of traditional GNAS methods

Performance Gains

Average improvement of 0.7% on benchmark graph datasets compared to state-of-the-art GNAS methods.

Technical Tags

Graph Neural Architecture Search (GNAS)Large Language Models (LLMs)GPT-4Search Space DesignSearch StrategyPrompt EngineeringGraph Neural Networks (GNNs)Benchmark Datasets

Research Topics

Neural Architecture Search (NAS)Graph Neural NetworksLarge Language ModelsAutomated Machine Learning (AutoML)Graph Representation Learning

Methods & Architectures

LLM-based GNAS (GNAS-LLM)Prompt engineeringIterative LLM generation Graph Neural Networks (GNNs)Large Language Models (LLMs)GPT-4

Applications & Tasks

Machine Learning Graph Analysis Data Mining Automating GNN architecture designReducing human labor in NASImproving GNAS efficiency and effectiveness Finding optimal GNN architecturesGenerating GNNs for specific graph datasetsAccelerating GNAS convergence

Datasets & Benchmarks

Datasets

Four benchmark graph datasets

Benchmarks

Outperforms state-of-the-art GNAS methods on four benchmark graph datasets

Accuracy of generated GNN architecturesConvergence speed

Related Fields

Machine LearningGraph Neural NetworksNeural Architecture SearchNatural Language ProcessingArtificial Intelligence

Keywords

GNASGNNLLMGPT-4neural architecture searchprompt engineeringgraph learningautomated machine learningsearch spacesearch strategy

Academic Context

#Neural Architecture Search (NAS)#Graph Neural Networks#Large Language Models#Automated Machine Learning (AutoML)#Graph Representation Learning

Technology Stack

Frameworks & Libraries

GPT-4

Commercial Potential

Potential Products

Automated GNN design platformsGNN model optimization servicesAutoML tools for graph data

Target Industries

Social MediaBiotechnologyE-commerceCybersecurityTelecommunications

Use Case Examples

Designing GNNs for social network analysisOptimizing GNNs for drug discoveryAutomating GNN development for recommendation systems

Competitive Edge

Revolutionizes GNAS by using LLMs to automate search space and strategy design, overcoming the manual effort and domain expertise required by traditional methods.

Market Opportunity

Growing market for AutoML and specialized ML solutions.

Revenue Models

SaaS for automated GNN designconsulting services.

Resource Requirements

Compute Needs

Moderate to High (depends on LLM inference and GNAS search)

Data Requirements

Requires access to various benchmark graph datasets for evaluation.

Deployment Constraints

Reliance on powerful LLMs like GPT-4 can incur API costs and latency.

Scalability

Scalability depends on the efficiency of the LLM and the GNAS search process.

Production Readiness

Maturity Level

Research

Time to Market

Medium

Patent Potential

High (novel integration of LLMs and GNAS)

View Full Paper Back to Papers