CAFA6 Protein Function Prediction

IA-Weighted Performance (Overall)0.7147
Test F-max
0.6996
Test AUC
0.7267
Valid F-max
1,157
Total Models

Overview

This study developed a protein function prediction pipeline for the CAFA6 (Critical Assessment of Functional Annotation) challenge. We employ a Hierarchical Binary Classification approach utilizing the Gene Ontology (GO) hierarchy structure, with LightGBM models and GPU acceleration for efficient training.

Key Features

📊

Dynamic Dataset Construction

Per-GO-term dataset construction with 6:2:2 train/valid/test split within each term.

🌳

Hierarchical Model Training

Models trained for all GO terms with ≥50 positive samples, from leaf nodes to root.

⚡

LightGBM with GPU

Gradient boosting models with GPU acceleration for fast training on large-scale data.

🔄

Parallel Processing

Joblib-based parallel model training to maximize computational efficiency.

📈

IA-Weighted Evaluation

Information Accretion (IA) weights re-normalized for each subset with dual F-max evaluation.

✓

True Path Rule

Ancestor propagation ensures GO hierarchy consistency in final predictions.

Model Strategy

🎯 Training Strategy

Minimum 50 positive samples required per GO term
6:2:2 split yields Train(30) : Valid(10) : Test(10)
Models exist at all hierarchy levels (not just leaf nodes)

🔮 Prediction Strategy

Predict using all trained models (leaf, intermediate, root levels)
Apply True Path Rule for ancestor propagation
Partial scores achievable even when leaf model unavailable

Pipeline Steps

Data Loading

Load GO ontology, training sequences, and annotations

Feature Extraction

Extract amino acid composition, physicochemical properties, and ESM-2 embeddings

IA Weight Calculation

Compute Information Accretion weights for each GO term

Model Training

Train LightGBM binary classifiers for each eligible GO term

Prediction

Generate predictions with True Path Rule propagation

Evaluation

Calculate IA-weighted and unweighted F-max scores

Results

The pipeline successfully trained 1,157 hierarchical binary classification models for protein function prediction across three GO ontologies: Biological Process (BP), Molecular Function (MF), and Cellular Component (CC). The model achieved an IA-weighted test F-max of 0.7147 and AUC of 0.6996.

Per-Ontology Performance (Simple Average)

Ontology	Models	Valid F1	Test F1	Valid AUC
BP	799	0.5928	0.5796	0.6745
MF	158	0.6611	0.6385	0.7168
CC	200	0.7323	0.7189	0.7478

Key Findings

Achieved IA-weighted test F-max of 0.7147, demonstrating strong generalization
CC ontology shows highest per-term performance (Test F1: 0.7189) due to localized function space
BP ontology has the most models (799) but lower per-term F1 (0.5796) due to functional diversity
Train-to-test gap (0.9709 → 0.7147) indicates reasonable generalization without severe overfitting
MF shows balanced performance (Test F1: 0.6385) with moderate model count (158)

Technical Specifications

1,157

Total Models

1,320

Feature Dimensions

~4.5 hours

Training Time

NVIDIA T4

GPU