Résultats de recherche pour « Graph Transformers »

Attributed graphs have recently emerged as a powerful tool for representing diverse data in numerous real-world sensors. Among various applications, unsupervised graph anomaly detection (UGAD) aims to identify abnormal data that significantly deviate from the majority of normal nodes without label annotations. Hence, UGAD can provide crucial assistance in enhancing the reliability of IoT, intelligent sensors and so on. Under the class-imbalanced reality caused by anomaly scarcity, the common paradigm of UGAD focuses on learning a model that primarily captures normal patterns. However, the traditional Graph Neural Network (GNN) paradigm suffers from local-aggregation limitations and over-smoothing, constraining their discrimination capacity. To address these issues, we introduce Graph Transformers (GTs) into UGAD task, termed as unsupervised attributed graph Anomaly detectioN wIth Masked grAph TransformErs (ANIMATE). Leveraging the global receptive field of Transformers, we can capture graph information that preserves the distinguishable characteristics of abnormalities from a global perspective. Furthermore, we employ masked auto-encoders to reconstruct node features and prompt our model to focus more on learning normal patterns. Additionally, we enhance the performance through a self-paced enhancement scheme specifically for UGAD tasks. Experiments conducted on various real-world benchmark datasets with organic anomalies validate the effectiveness of our proposed method compared to state-of-the-art competitors.

The Ames mutagenicity test is a gold standard assay for the safety assessment of new chemicals. However, many in silico models rely on challenging-to-interpret ensemble strategies and molecular fingerprint data which neglects gestalt molecular structure. To improve upon these models, we propose AmesFormer, a graph transformer neural network which shows state-of-the-art performance when paired with our new Ames dataset. We briefly review the current state of Ames modelling with a focus on graph neural networks. We then benchmark AmesFormer on a standardised test dataset against 22 other Ames models, achieving state of the art (SOTA) performance. We then uniquely report the calibration performance of our model and attempts to improve it using temperature scaling. We support our findings with reference to other models from the literature and with developments in machine learning (ML) and graph theory. Overall, we present a high-performance, accessible, and open-source computational model for Ames mutagenicity, with significant potential for regulatory and drug development applications

Transformer architectures achieve state-of-the-art performance on a wide range of sequence modeling tasks, but their notion of context is largely static: the effective semantic representation of a segment is determined at the moment of inference and is not retroactively revised when future information arrives. This paper introduces the Topic Integrator, a mechanism designed for Recursive Transformer Models (RTMs) that treats context as a sequence of Collision Blocks-dense clusters of associated tokens represented as local graphs. We define a cluster-graph-based topic integral that summarizes each block as a Topic Vector, and augment it with an Enhanced Reconsideration System (ERS) implementing temporally decayed, recursive backward updates. In this view, context is maintained as an asynchronous, blockchain-like memory ledger: new blocks may exert "semantic torque" on previous ones, leading to dynamic, context-annotated definitions rather than static tags. We formalize the Topic Integrator, provide basic stability and convergence results, outline algorithms for efficient implementation, and sketch an evaluation protocol for large language models.

Predicting the unprecedented, nonlinear nature of COVID-19 presents a significant public health challenge. Recent advances in deep learning, such as graph neural networks (GNNs), recurrent neural networks (RNNs), and Transformers, have enhanced predictions by modeling regional interactions, managing autoregressive time series, and identifying long-term dependencies. However, prior works often feature shallow integration of these models, leading to simplistic graph embeddings and inadequate analysis across different graph types. Additionally, excessive reliance on historical COVID-19 data limits the potential of utilizing future data, such as lagged policy information. To address these challenges, we introduce ReGraFT, a novel sequence-to-sequence (Seq2Seq) model designed for robust long-term forecasting of COVID-19. ReGraFT integrates multigraph-gated recurrent units (MG-GRU) with adaptive graphs, leveraging data from individual states, including infection rates, policy changes, and interstate travel. First, ReGraFT employs adaptive MGGRU cells within an RNN framework to capture interregional dependencies, dynamically modeling complex transmission dynamics. Second, the model features a self-normalizing priming (SNP) layer using Scaled Exponential Linear Units (SeLU) to enhance stability and accuracy across short, medium, and long-term forecasts. Third, ReGraFT systematically compares and integrates various graph types, such as fully connected layers, pooling mechanisms, and attention-based structures, to provide a nuanced representation of interregional relationships. By incorporating lagged COVID-19 policy data, ReGraFT refines forecasts, demonstrating a 2.79% reduction in the root mean square error (RMSE) compared to state-of-the-art models. This work provides accurate long-term predictions, aiding in better public health decisions. Our code is available at https://github.com/mfriendly/ReGraFT.

Efficiently predicting drug response remains a challenge in the realm of drug discovery. By leveraging two types of multiomics data, transcriptomics, and genomics, we create a comprehensive representation of target cells and enable drug response prediction for personalized medicine. To address this issue, we propose four novel model architectures that combine graph transformers with varying positions of multiheaded self-attention mechanisms. A majority of our architectures utilize multiple transformer models, one with a graph attention mechanism and the other with a multiheaded self-attention mechanism, to generate latent representations of both the drug and omics data, respectively. Unlike previous approaches that apply attention only to one data type, either drug or genomics, our model architectures employ this technique for both, with a goal to procure more comprehensive latent representations. The latent representations are then concatenated and input into a fully connected network to predict the IC-50 score, a measure of cell drug response. We experiment with all four of these architectures and extract results from them. The novel model without the multiheaded self-attention mechanism seems to give us the most accurate results on our holdout set. Our study greatly contributes to the future of drug discovery and precision medicine by looking to optimize the time and accuracy of drug response prediction, as well as using multiomics data for a personalized approach to the problem.

<p>Efficiently predicting drug response remains a challenge in the realm of drug discovery. By leveraging two types of multiomics data, transcriptomics, and genomics, we create a comprehensive representation of target cells and enable drug response prediction for personalized medicine. To address this issue, we propose four novel model architectures that combine graph transformers with varying positions of multiheaded self-attention mechanisms. A majority of our architectures utilize multiple transformer models, one with a graph attention mechanism and the other with a multiheaded self-attention mechanism, to generate latent representations of both the drug and omics data, respectively. Unlike previous approaches that apply attention only to one data type, either drug or genomics, our model architectures employ this technique for both, with a goal to procure more comprehensive latent representations. The latent representations are then concatenated and input into a fully connected network to predict the IC-50 score, a measure of cell drug response. We experiment with all four of these architectures and extract results from them. The novel model without the multiheaded self-attention mechanism seems to give us the most accurate results on our holdout set. Our study greatly contributes to the future of drug discovery and precision medicine by looking to optimize the time and accuracy of drug response prediction, as well as using multiomics data for a personalized approach to the problem. </p>

Abstract Motivation Estimation of protein complex structure accuracy is an essential step in protein complex structure prediction and is also important for users to select good structural models for various applications, such as protein function analysis and drug design. Despite the success of structure prediction methods such as AlphaFold2 and AlphaFold3, predicting the quality of predicted complex structures (structural models) and selecting top ones from large model pools remains challenging. Results We present GATE, a novel method that uses graph transformers on pairwise model similarity graphs to predict the quality (accuracy) of complex structural models. By integrating single-model and multi-model quality features, GATE captures both the characteristics of individual models and the geometric similarity between them to make robust predictions. On the dataset of the 15th Critical Assessment of Protein Structure Prediction (CASP15), GATE achieved the highest Pearson’s correlation (0.748) and the lowest ranking loss (0.1191) compared to existing methods. In the blind CASP16 experiment, GATE was ranked 4th according to the overall sum of z-scores of multiple metrics based on both TM-score and Oligo-GDTTS scores. In terms of per-target average metrics based on TM-score, GATE achieved a Pearson’s correlation of 0.7076 (1st place among all methods), a Spearman’s correlation of 0.4514 (3rd place), a ranking loss of 0.1221 (3rd place), and an Area Under the Curve (AUC) score of 0.6680 (3rd place), highlighting its strong, balanced ability of estimating complex model accuracy and selecting good models. Availability The source code of GATE is freely available at https://github.com/BioinfoMachineLearning/GATE .