BERT-CGAE: A Hybrid BERT-based CNN-GRU-Attention Ensemble Framework for Text Classification

Abstract

Text classification is a fundamental task in natural language processing, requiring models to accurately interpret semantic and syntactic patterns in textual data. The proposed framework, BERT-EnsemText, introduces a hybrid approach that combines advanced feature extraction with ensemble learning for robust and accurate classification. Existing methods often rely on static word embeddings or single-model architectures, which struggle to capture contextual nuances and tend to overfit on complex or domain-specific datasets. These approaches also lack adaptive mechanisms to leverage diverse decision boundaries effectively. To address these limitations, BERT-EnsemText integrates BERT-based contextual embeddings with a CNN-GRU-Attention feature extractor to jointly capture local, sequential, and salient contextual features. The extracted features are processed through multiple base classifiers including GEPSVM, XGBoost, and Logistic Regression. An attention-based meta-learner dynamically weights classifier outputs using confidence scores, enabling adaptive and robust ensemble decisions. This framework can be applied to a wide range of text classification tasks such as sentiment analysis, topic categorization, and intent detection, especially in domains with complex language patterns. Experimental results show that BERT-EnsemText significantly improves classification accuracy, generalization, and robustness compared to conventional single-model and static embedding-based baselines.