Chrome Extension
WeChat Mini Program
Use on ChatGLM

Towards Efficient Graph Processing in Geo-Distributed Data Centers

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS(2024)

Northeastern Univ | Alibaba Grp

Cited 0|Views34
Abstract
Iterative graph processing is widely used as a significant paradigm for large-scale data analysis. In many global businesses of multinational enterprises, graph-structure data is usually geographically distributed in different regions to support low-latency services. Geo-distributed graph processing suffers from the Wide Area Networks (WANs) with scarce and heterogeneous bandwidth, thus essentially differs from traditional distributed graph processing. In this paper, we propose RAGraph, a Region-Aware framework for geo-distributed graph processing. At the core of RAGraph, we design a region-aware graph processing framework that allows advancing inefficient global updates locally and enables sensible coordination-free message interactions and flexible replaceable communication module. In terms of graph data preprocessing, RAGraph introduces a contribution-driven edge migration algorithm to effectively utilize network resources. RAGraph also contains an adaptive hierarchical message interaction engine to switch interaction modes adaptively based on network heterogeneity and fluctuation, and a discrepancy-aware message filtering strategy to filter important messages. Experimental results show that RAGraph can achieve an average speedup of 9.7x (up to 98x) and an average WAN cost reduction of 78.5$% (up to 97.3%) compared with state-of-the-art systems.
More
Translated text
Key words
Data centers,Wide area networks,Bandwidth,Computational modeling,Iterative algorithms,Fluctuations,Measurement,Graph processing,geo-distributed data centers,heterogeneous network
求助PDF
上传PDF
Bibtex
AI Read Science
AI Summary
AI Summary is the key point extracted automatically understanding the full text of the paper, including the background, methods, results, conclusions, icons and other key content, so that you can get the outline of the paper at a glance.
Example
Background
Key content
Introduction
Methods
Results
Related work
Fund
Key content
  • Pretraining has recently greatly promoted the development of natural language processing (NLP)
  • We show that M6 outperforms the baselines in multimodal downstream tasks, and the large M6 with 10 parameters can reach a better performance
  • We propose a method called M6 that is able to process information of multiple modalities and perform both single-modal and cross-modal understanding and generation
  • The model is scaled to large model with 10 billion parameters with sophisticated deployment, and the 10 -parameter M6-large is the largest pretrained model in Chinese
  • Experimental results show that our proposed M6 outperforms the baseline in a number of downstream tasks concerning both single modality and multiple modalities We will continue the pretraining of extremely large models by increasing data to explore the limit of its performance
Upload PDF to Generate Summary
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Data Disclaimer
The page data are from open Internet sources, cooperative publishers and automatic analysis results through AI technology. We do not make any commitments and guarantees for the validity, accuracy, correctness, reliability, completeness and timeliness of the page data. If you have any questions, please contact us by email: report@aminer.cn
Chat Paper

要点】:本文提出了RAGraph,一个面向地理分布式数据中心的区域感知图处理框架,以优化迭代图处理效率并降低广域网络成本。

方法】:RAGraph框架通过区域感知的图处理、贡献驱动的边迁移算法、自适应层次消息交互引擎和差异感知的消息过滤策略来提升处理效率和网络资源利用率。

实验】:在多个数据集上进行的实验表明,RAGraph相比于现有先进系统,实现了平均9.7倍(最高98倍)的速度提升和78.5%(最高97.3%)的广域网络成本降低。具体的数据集名称在论文中未明确提及。