Back to Research
Current Research · The University of Hong Kong
GRAPH OPTIMIZATION
FOUNDATION MODEL
HKU & UC Berkeley · Supervised by Prof. Zuo-Jun Max Shen
Under Review · Management Science
Under Review · Management Science
Graph Optimization
Foundation Model
Operations Research
Transformer
1,945Max network nodes
24.5%Distance reduction
5Task families
100×Faster than LKH3 / OR-Tools
PAPER
Graph Foundation Models: Bridging Language Model Paradigms and Graph Optimization
Yunhao Liang, Pujun Zhang, Yuan Qu, Shaochong Lin, Zuo-Jun Max Shen
Under Review at Management Science
OVERVIEW
Graph optimization is foundational to modern operations — routing vehicles through road networks, detecting communities in social networks, and influence-based targeting. Yet prevailing approaches treat each task instance in isolation, discarding persistent topological knowledge embedded in the underlying network.
GOFM shifts the focus from task-specific solvers to a task-agnostic structural prior. It internalizes network topology and distance geometry by encoding structure-aware random walks into a Transformer through progressive masked reconstruction. At inference, this learned prior facilitates diverse optimization tasks via lightweight constrained decoding, ensuring strict feasibility without retraining.
FRAMEWORK ARCHITECTURE
The GOFM framework consists of three stages:
Stage I: Structure-aware random path generation via Multinomial Logit sampling ·
Stage II: Transformer pretraining with multi-target masked reconstruction ·
Stage III: Task-based guided decoding to feasible solutions
SUPPORTED GRAPH OPTIMIZATION TASKS
GOFM handles five classical problem classes with a single pretrained backbone:
Shortest Path (SP) · Traveling Salesman Problem (TSP) ·
Capacitated Vehicle Routing Problem (CVRP) ·
Community Detection (CD) · Influence Maximization (IM)
KEY CONTRIBUTIONS
- Foundation-model framework for graph optimization: A single pretrained model performs well across multiple NP-hard graph tasks on real sparse networks, without any architectural changes.
- Self-supervised pretraining via structure-aware walks: Structure-aware random walks combined with progressive masked reconstruction equip the model with a holistic structural prior over the target graph.
- Cross-task reuse on real networks: Validated on networks up to 1,945 nodes across five routing families; generalizes beyond routing to community detection and influence maximization.
KEY RESULTS
GOFM achieves competitive solution quality compared to widely used OR solvers such as LKH3 and Google OR-Tools, with inference one to two orders of magnitude faster on city-scale graphs. Unlike neural baselines trained on complete-graph distributions, GOFM maintains near-perfect feasibility across all settings.
On the Amazon Last Mile dataset, GOFM delivers a 24.5% reduction in total route distance — a direct impact on real-world logistics operations.