Résultats de recherche pour « Dynamic graph processing »

Graph as one of the most fundamental and representative data structures has found a wide spectrum of emerging application domains such as social media, financial transactions, biology science, and road networks. Recently, with the proliferation of graph applications, graph processing has attracted much attention in both industry and academia. Among them, most existing works focus on the static graphs in which the vertices and edges are immutable. However, in the real world, graphs are constantly and dynamically changing, bringing tricky challenges to process such dynamic graphs. This paper surveys the recent advances in dynamic graph processing, including centrality, graph coloring, cohesive subgraph, path traversal, and graph separation. We summarize the computational complexity models for dynamic algorithm analysis, theoretically compare the efficiency of algorithms among different research topics. Moreover, we also explore the research opportunities for the future.

An unprecedented growth in data generation is taking place. Data about larger dynamic systems is being accumulated, capturing finer granularity events, and thus processing requirements are increasingly approaching real-time. To keep up, data-analytics pipelines need to be viable at massive scale, and switch away from static, offline scenarios to support fully online analysis of dynamic systems. This paper uses a challenge problem, graph colouring, to explore massive-scale analytics for dynamic graph processing. We present an event-based infrastructure, and a novel, online, distributed graph colouring algorithm. Our implementation for colouring static graphs, used as a performance baseline, is up to an order of magnitude faster than previous results and handles massive graphs with over 257 billion edges. Our framework supports dynamic graph colouring with performance at large scale better than GraphLab's static analysis. Our experience indicates that online solutions are feasible, and can be more efficient than those based on snapshotting.

With the rapid growth in data volume, workloads from various domains have undergone drastic changes in recent years. Today, streaming workloads are commonplace. This generates the need for systems and algorithms that can receive and process streams of data with high throughput. Various graph applications are shifting away from the static graph model and incorporating a dynamic model, where updates to the graph can be received. In a dynamic setting, solutions to algorithms need to be updated alongside the graph. However, re-processing the whole graph every time can be infeasible given the size of current graphs. This raises a series of questions regarding how to process dynamic graph algorithms in a reasonable amount of time. In this paper, we explore the existing methods in the literature used to achieve scalable dynamic graph processing. We define different aspects and abstractions used for dynamic graph processing and categorize all approaches on the basis of their scalability.

Many real-world systems, such as social networks, rely on mining efficiently large graphs, with hundreds of millions of vertices and edges. This volume of information requires partitioning the graph across multiple nodes in a distributed system. This has a deep effect on performance, as traversing edges cut between partitions incurs a significant performance penalty due to the cost of communication. Thus, several systems in the literature have attempted to improve computational performance by enhancing graph partitioning, but they do not support another characteristic of real-world graphs: graphs are inherently dynamic, their topology evolves continuously, and subsequently the optimum partitioning also changes over time. In this work, we present the first system that dynamically repartitions massive graphs to adapt to structural changes. The system optimises graph partitioning to prevent performance degradation without using data replication. The system adopts an iterative vertex migration algorithm that relies on local information only, making complex coordination unnecessary. We show how the improvement in graph partitioning reduces execution time by over 50%, while adapting the partitioning to a large number of changes to the graph in three real-world scenarios.

An efficient data structure is fundamental to meeting the growing demands in dynamic graph processing. However, the dual requirements for graph computation efficiency (with contiguous structures) and graph update efficiency (with linked list-like structures) present a conflict in the design principles of graph structures. After experimental studies of state-of-the-art dynamic graph structures, we observe that the overhead of cache misses accounts for a major portion of the graph computation time. This paper presents GastCoCo, a system with graph storage and coroutine-based prefetch co-design. By employing software prefetching via stackless coroutines and designing a prefetch-friendly data structure CBList, GastCoCo significantly alleviates the performance degradation caused by cache misses. Our results show that GastCoCo outperforms state-of-the-art graph storage systems by 1.3× - 180× in graph updates and 1.4× - 41.1× in graph computation.

The rapid growth of social networks has led to the generation of massive, dynamic, and interconnected datasets, necessitating efficient computational frameworks for real-time analysis. Parallel dynamic graph processing has emerged as a critical tool for addressing these challenges, enabling scalable and efficient analysis of evolving social networks. This paper explores the applications of parallel dynamic graph processing in social network analysis, focusing on its role in community detection, influence maximization, anomaly detection, and real-time recommendation systems. By leveraging parallel computing architectures, dynamic graph algorithms can process large-scale, continuously changing network data with reduced latency and improved accuracy. We discuss key techniques, such as incremental computation, load balancing, and distributed memory management, that enhance the performance of dynamic graph processing in social network contexts. Furthermore, we highlight case studies and emerging trends that demonstrate the transformative potential of parallel dynamic graph processing in extracting actionable insights from complex social network data. This work underscores the importance of advancing parallel dynamic graph processing techniques to meet the demands of modern social network analysis.

The paper presents structures and techniques aimed towards co-designing scalable asynchronous and decentralized dynamic graph processing for fine-grain memory-driven architectures. It uses asynchronous active messages, in the form of actions that send “work to data”, with a programming and execution model that allows spawning tasks from within the data-parallelism combined with a data-structure that parallelizes vertex object across many scratchpad memory-coupled cores and yet provides a single programming abstraction to the data object.

Graph processing on GPUs is gaining momentum due to the high throughputs observed compared to traditional CPUs, attributed to the vast number of processing cores on GPUs that can exploit parallelism in graph analytics. This paper discusses a graph data structure for dynamic graph processing on GPUs. Unlike static graphs, dynamic graphs mutate over their lifetime through vertex and/or edge batch updates. The proposed work aims to provide fast batch updates and graph querying without consuming too much GPU memory. Experimental results show improved initialization timings by 1968-1269024%, improved batch edge insert timings by 30-30047%, and improved batch edge delete timings by 50-25262% while consuming less memory when the batch size is large.

With the continuous growth of user scale and application data, the demand for large-scale concurrent graph processing is increasing. Typically, large-scale concurrent graph processing jobs need to process corresponding snapshots of dynamically changing graph data to obtain information at different time points. To enhance the throughput of such applications, current solutions concurrently process multiple graph snapshots on the GPU. However, when dealing with rapidly changing graph data, transferring multiple snapshots of concurrent jobs to the GPU results in high data transfer overhead between CPU and GPU. Additionally, the execution mode of existing work suffers from underutilization of GPU computational resources. In this work, we introduce CGCGraph, which can be integrated into existing GPU graph processing systems like Subway, to enable efficient concurrent graph snapshot processing jobs and enhance overall system resource utilization. The key idea is to offload unshared graph data of multiple concurrent snapshots to the CPU, reducing CPU-GPU transfer overhead. By implementing CPU-GPU co-execution, there is potential for enhanced utilization of GPU computing resources. Specifically, CGCGraph leverages kernel fusion to process shared graph data concurrently on the GPU, while executing all snapshots in parallel on the CPU, with each snapshot assigned a dedicated thread. This approach enables efficient concurrent processing within a novel CPU-GPU co-execution model, incorporating three optimization strategies targeting storage, computation, and synchronization. We integrate CGCGraph with Subway, an existing system designed for out-of-GPU-memory static graph processing. Experimental results show that the integration of CGCGraph with current GPU-based systems obtains performance improvements ranging from 1.7 to 4.5 times.

Control parallelism and data parallelism is mostly reasoned and optimized as separate functions. Because of this, workloads that are irregular, fine-grain and dynamic such as dynamic graph processing become very hard to scale. An experimental research approach to computer architecture that synthesizes prior techniques of parallel computing along with new innovations is proposed in this paper. We establish the background and motivation of the research undertaking and provide a detailed description of the proposed omputing system that is highly parallel non-von Neumann, memory-centric and memory-driven. We also present a message-driven (or even-driven) programming model called "diffusive computation" and provide insights into its properties using SSSP and Triangle Counting problems as examples.