A Proximal Policy Optimization Approach with Spatio-Temporal Graph Neural Networks for DAG Task Scheduling in Cloud-Edge-End Computing
Yangshuo Qi · Chenwei Wang · Zihan Shen · Songlin Sun
ICSINC 2026 · Accepted manuscript
Paper · Overview · System Model · Method · Results · Author Contributions · Quick Start · Citation
Scheduling a workflow across cloud servers, edge nodes, and end devices requires decisions that account for both task dependencies and changing resource availability. A locally fast assignment can still create a downstream bottleneck or concentrate work on a small set of machines.
PPO-STGNN combines spatio-temporal graph representation learning with proximal policy optimization for heterogeneous DAG scheduling. The policy jointly reasons about the resource topology, the task graph, feasible task-node assignments, and multiple scheduling objectives; multi-teacher behavior cloning provides a stronger initialization than learning the joint action space from scratch.
| Topology-aware state | Constrained policy | Multi-objective learning |
|---|---|---|
| Encodes the physical resource graph and logical DAG dependencies. | Masks infeasible task-node assignments before PPO action selection. | Optimizes makespan, SLR, and CPU/memory load balance. |
Artifact scope. The repository contains the paper-aligned source release, deterministic CPU checks, and engineering documentation. Processed workloads, checkpoints, and historical experiment logs are not redistributed. The figures below are the original figures from the arXiv paper, not newly reproduced plots. See the reproducibility notes for implementation and artifact boundaries.
Figure 1. Cloud-edge-end hierarchical computing scenario with heterogeneous cloud, edge, and end resources.
End devices generate DAG workflows, edge servers provide nearby moderate-capacity execution, and cloud servers supply high-capacity processing over higher-latency links. Scheduling decisions must respect task precedence, node capacity, and communication cost across this hierarchy.
Figure 2. Overall PPO-STGNN workflow: graph-based state encoding, PPO action selection, action refinement, environment execution, and multi-objective reward feedback.
- State representation. Build task-DAG and resource-graph observations from ready tasks, dependencies, node utilization, and resource history.
- Spatio-temporal encoding. Produce node-level and graph-level embeddings that preserve structural and temporal context.
- Constrained joint action. Select a ready task and execution node while masking infeasible assignments and refining the selected action.
- Multi-objective optimization. Train the actor-critic policy with rewards derived from makespan, scheduling length ratio, and CPU/memory balance.
- Behavior-cloning initialization. Pretrain from multiple heuristic teachers to reduce policy cold-start in the joint scheduling action space.
Figure 3. Comparison with FCFS, LeastLoad, HEFT, and Greedy on makespan, SLR, and load balance; lower is better for all three metrics.
The paper reports the lowest makespan (124.8 s) and load-balance score (127.9) for PPO-STGNN. HEFT records the lowest SLR (7.8), making the trade-off visible rather than collapsing all objectives into a single ranking.
Figure 4. Validation trends for MLP-PPO, PPO-StaticGNN, and PPO-STGNN across makespan, SLR, and load balance.
The comparison isolates the value of graph structure and temporal context: MLP-PPO omits topology, PPO-StaticGNN introduces structural information, and PPO-STGNN additionally models temporal resource dynamics. These are results reported in the paper; the public CI validates software behavior rather than reproducing the benchmark curves.
This was a collaborative research project, and all paper authors are credited above. Zihan Shen contributed to the scheduling system, objective design, policy initialization, and experimental evaluation, and maintains this public code release.
| Contribution area | Work completed | Repository entry points |
|---|---|---|
| Scheduling system | Developed task-dependency and heterogeneous-resource modeling for constrained DAG scheduling. | Simulator, graph encoders, PPO agent |
| Objective design | Formulated rewards for completion time, scheduling efficiency, CPU/memory balance, and utilization. | Environment and rewards, configuration |
| Policy initialization | Applied multi-teacher behavior cloning to mitigate cold-start in the joint action space. | Teacher collection and training, training entry point |
| Experimental evaluation | Benchmarked against FCFS, LeastLoad, HEFT, and Greedy and compared policy encoders. | Baseline policies, baseline runner, ablations |
The release checks use Python 3.12 and the dependency versions recorded in requirements.txt. This is a newly tested environment, not a claim about the original training environment.
git clone https://github.com/Ailta666508/PPO-STGNN.git
cd PPO-STGNN
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-dev.txtFor Windows, activate with .venv\Scripts\Activate.ps1 in PowerShell. GPU training requires a compatible PyTorch/CUDA installation; the automated checks exercise CPU execution only.
python scripts/verify_source.py
python -m pytest -qWithout external data, the suite runs 14 deterministic tests covering experiment-configuration snapshots, reproducible RNG and kernel configuration, objective normalization, action masking, small PPO updates, and checkpoint round trips across all three encoders. Five dataset-dependent checks are skipped. With a local dataset, the suite also checks DAG validity, split separation, and baseline environment interaction. Passing these checks validates software behavior; it does not reproduce the paper's benchmark results.
Release preparation also included 8 CPU tests with the local dataset and a small end-to-end training/evaluation run on macOS arm64. See the validation record for the tested versions and scope. GitHub Actions checks source integrity and the data-free CPU suite on every push and pull request.
Training requires a separately obtained copy of the processed workload; no public download is provided in this release. If you have authorized access, place it in ./new500DAG or pass its location with --data-dir. See Data Access.
python train_ppo.py \
--data-dir ./new500DAG \
--project-root ./runs/quick \
--quick --device cpu --seed 42--quick reduces the workload and training budget for a functional check. It is not a paper evaluation setting. New outputs go to runs/quick/. For baseline evaluation, full training, and controlled architecture comparisons, see Experiments.
cecoppo/
env_cec_dag.py # Cloud-edge-end simulator, observations, masks, rewards
graph_encoder.py # Temporal graph, static graph, and MLP actor-critic models
ppo_agent.py # Rollouts, GAE, clipped PPO updates, checkpoint I/O
baselines.py # Non-learning scheduling policies
experiment_utils.py # Behavior cloning, training, evaluation, plotting helpers
config.py # Environment and PPO configuration dataclasses
config_io.py # Validated configuration snapshots and fingerprints
train_ppo.py # Main policy training and evaluation entry point
export_experiment_config.py # Persist a resolved configuration before a run
run_baselines.py # FCFS / LeastLoad / HEFT / Greedy experiments
run_ablations.py # Architecture comparisons and component ablations
plot_results.py # Plotting entry point
docs/ # Paper, provenance, implementation and run notes
tests/ # Deterministic CPU software checks
scripts/ # Source and paper-asset integrity verification
The locally supplied workload contains 500 DAGs split into 398 training / 48 validation / 54 test workflows. Its resource pool contains 90 nodes; the default experiment configuration selects 38. The processed dataset and the simulator's runtime configuration are distinct. See Data and Implementation.
- Reproducibility notes separate paper claims, released code, and historical local artifacts.
- Experiment guide documents baseline, training, and ablation entry points.
- Data and implementation records workload access and simulator assumptions.
- Release validation records the tested environment and verification scope.
@misc{qi2026ppostgnn,
title = {PPO-STGNN: A Proximal Policy Optimization Approach with
Spatio-Temporal Graph Neural Networks for DAG Task Scheduling
in Cloud-Edge-End Computing},
author = {Qi, Yangshuo and Wang, Chenwei and Shen, Zihan and Sun, Songlin},
year = {2026},
eprint = {2609.03503},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2609.03503},
note = {Accepted at ICSINC 2026}
}No repository-wide license was included with the supplied package. This release does not assign a new license to the paper, code, or processed data. Please contact the authors about reuse and redistribution, and check the rights of any upstream data separately.
For questions about this release or to report a reproducibility issue, please open an issue.
Note: This project was initially developed locally. The Git repository was created when the codebase was prepared for publication, so the early development history is unavailable. Subsequent updates are tracked in this repository.