bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2024.07.25.605219

ChromBERT: Uncovering Chromatin State Motifs in the Human Genome Using a BERT-based Approach

Lee, S.·Lin, C.·Chen, C.-Y.·Nakato, R.
View original source ↗Export metadata ↓

Abstract

Chromatin states, which are defined by specific combinations of histone post-translational modifications, are fundamental to gene regulation and cellular identity. Despite their importance, comprehensive patterns within chromatin state sequences, which could provide insights into key biological functions, remain largely unexplored. In this study, we introduce ChromBERT, a BERT-based model specifically designed to detect distinct chromatin state patterns as "motifs." We pre-trained ChromBERT on 15-state chromatin annotations from 127 human cell and tissue types from the ROADMAP consortium. This pre-trained model can be fine-tuned for various downstream tasks, and obtained high-attention chromatin state patterns are extracted as motifs. To account for the variable-length nature of chromatin state motifs, ChromBERT uses Dynamic Time Warping to cluster similar motifs and identify meaningful representative patterns. In this study, we evaluated the performance of the model on several tasks, including binary and quantitative gene expression prediction, cell type classification, and three-dimensional genome feature classification. Our analyses yielded biologically grounded results and revealed the associated chromatin state motifs. This workflow facilitates the discovery of specific chromatin state patterns across different biological contexts and offers a new framework for exploring the dynamics of epigenomic states.

Explore related subjects

bioinformaticsPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Lee, S., Lin, C., Chen, C.-Y., Nakato, R.. 2024-07-26. ChromBERT: Uncovering Chromatin State Motifs in the Human Genome Using a BERT-based Approach. https://doi.org/10.1101/2024.07.25.605219

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Inferring cascade drivers of VEXAS syndrome by a causal machine learning tool CauNagi

VEXAS syndrome is an adult-onset severe autoinflammatory disease caused by somatic mutations in UBA1, yet the cascade mechanisms linking primitive hematopoietic abnormalities to mature myeloid dysfunctions remain largely unknown. Identifying master regulators of a progressive disease, a black-box process, from complex transcriptomic data also remains challenging. To address this challenge, we developed CauNagi, a computational framework for prioritizing cascade candidate regulators (CCRs). CauNagi integrates a causal representation learning module derived from CausCell with an iterative deep learning backbone adapted from UNAGI; in addition, CauNagi extends these two components with a unique downstream module for CCRs analysis designed to characterize regulatory propagation across hierarchical cellular states. Mechanistically, CauNagi iteratively integrates causal disentangled representation learning with (1) disease-stage cell-state trajectory reconstruction and (2) dynamic regulatory analysis. Benchmarking on single-cell transcriptomic datasets showed that CauNagi preserved cell-type structure in idiopathic pulmonary fibrosis (IPF) and enriched known acute myeloid leukemia(AML)-associated genes among its top-ranked global regulators. When applied to VEXAS syndrome, CauNagi readily revealed inflammatory responses, endoplasmic reticulum stress, and myeloid bias, consistent with the disease features. Furthermore, the CCRs analysis module of CauNagi assisted us in identifying 36 causal drivers, with SPI1, NFKB1, STAT3, and FOS prioritized as high-confidence regulatory hubs linking aberrant myeloid differentiation and inflammatory programs. These findings were further supported by an independent single-cell transcriptomic dataset from a murine VEXAS model. Overall, CauNagi provides a computationally efficient and systematic framework for identifying candidate causal regulators. Beyond hematopoietic diseases, CauNagi may also be applicable to other progressive disorders for which multistage single-cell transcriptomic datasets are available. CauNagi is available at https://github.com/steamed-stuffed-bun/CauNagi.

bioinformatics↗
bioRxiv2026

Inferential boundaries of age prediction: why prediction does not establish biological age measurement

Chronological-age clocks reconstruct age from biological measurements, yet their outputs are interpreted as biological age, gaps as ageing acceleration and intervention-associated decreases as rejuvenation. We show that age supervision identifies an age-task statistic, not a biological-age construct, and establish how this distinction changes biomarker construction and validation. Even at the population optimum, the same observable distribution and age-prediction performance admit incompatible biological-age interpretations. Resolving this ambiguity requires assumptions or evidence beyond the age task. Squared-error age loss penalizes within-age output dispersion without defining its biological direction. Given age and background, a gap re-expresses the compressed score; exact age recovery eliminates it even when heterogeneity remains in the measurements. Shared biological covariance permits genuine prognostic value without establishing construct identity. After allogeneic haematopoietic stem-cell transplantation, recipient-blood scores showed excess donor-lineage affiliation under a score-pairing null. In NHANES, age-trained scores improved held-out five-year mortality prediction beyond age and background, yet direct modelling of source measurements and mortality supervision at matched scalar capacity yielded further gains. The intended biological object must therefore guide study design, measurement selection and representation; validation must establish the claimed measurement relation rather than rely on age-prediction success alone.

bioinformatics↗
bioRxiv2026

DisenTE: Sparse Pattern-Context Modeling for Interpretable Translation-Efficiency Matrix Completion

Partially observed object-by-context matrices arise across data-rich science, where dominant object effects can obscure smaller but informative context-dependent variation. We study this problem in a translation-efficiency atlas of 9,494 5' UTRs across 78 cellular and tissue contexts. We present DisenTE, a sequence-conditioned neural model that combines separate sequence and context branches with a sparse low-rank pattern-context channel. Each module pairs a sequence-derived activation with context-specific deployment weights, forming a dictionary whose sequence and context components can be examined separately. Under five-fold within-panel entry masking, DisenTE achieves a UTR-centered residual Spearman correlation of 0.641 +/- 0.005, compared with 0.304 +/- 0.003 for the strongest reference model. The learned dictionary retains 11 of 20 candidate modules. CTM 6 has the largest overlap with an external TOP set and a cap-proximal pyrimidine pattern; CTMs 5 and 7 also overlap the set but have purine-containing consensuses. The evidence supports CTM 6 as a TOP sequence anchor and CTMs 5 and 7 as TOP-set-associated factors. On this dataset, DisenTE improves completion over the evaluated references and provides module-level summaries of its fitted context-dependent variation.

bioinformatics↗