MAT-HPO Library
Multi-Agent Transformer Hyperparameter Optimization
🤖 Multi-Agent HPO
Three specialized agents optimize different hyperparameter types simultaneously using transformer networks and reinforcement learning.
🚀 Quick Example
from MAT_HPO_LIB import MAT_HPO_Optimizer, BaseEnvironment, HyperparameterSpace
from MAT_HPO_LIB.utils import DefaultConfigs
# 1. Define your optimization environment
class MyEnvironment(BaseEnvironment):
def step(self, hyperparams):
""" Core optimization logic"""
f1, auc, gmean = train_and_evaluate(hyperparams)
done = check_stopping_condition()
return f1, auc, gmean, done
# 2. Define hyperparameter space with agent assignment
space = HyperparameterSpace()
space.add_continuous('class_weight_0', 0.5, 3.0, agent=0) # Problem-specific
space.add_discrete('hidden_size', [64, 128, 256, 512], agent=1) # Architecture
space.add_continuous('learning_rate', 1e-5, 1e-2, agent=2) # Training
space.add_discrete('batch_size', [16, 32, 64, 128], agent=2) # Training
# 3. Run optimization
config = DefaultConfigs.standard() # 100 optimization steps
optimizer = MAT_HPO_Optimizer(MyEnvironment(), space, config)
results = optimizer.optimize()
# View results
print(" Best hyperparameters:", results['best_hyperparameters'])
️ Agent Specialization
🤖 Agent 0
Problem-Specific
Class weights, regularization, loss functions,
domain-specific parameters
🏗️ Agent 1
Architecture
Hidden layers, network size, activation functions, model
structure
⚙️ Agent 2
Training
Batch size, learning rate, optimizers, training schedule
🧠 LLM-Enhanced Optimization
Revolutionary AI-powered hyperparameter optimization using Large Language Models for intelligent parameter suggestions. Based on real usage in SPNV2 ECG classification system.
from MAT_HPO_LIB import FullControlHPO
# Production-grade LLM enhancement (exact usage from SPNV2)
optimizer = FullControlHPO(
task_type="time_series_classification",
max_trials=30,
# LLM Configuration (from SPNV2)
llm_enabled=True,
llm_model="llama3.2:3b",
llm_base_url="http://localhost:11434",
mixing_strategy="adaptive", # or "fixed_alpha"
alpha=0.3, # LLM/RL mixing ratio
slope_threshold=0.01, # For adaptive strategy
llm_cooldown_episodes=6, # ICBEB: 6, PTBXL: 4
min_episodes_before_llm=6, # ICBEB: 6, PTBXL: 4
# Advanced RL Parameters (dataset-specific)
replay_buffer_size=150, # ICBEB: 150, PTBXL: 200
batch_size=12, # ICBEB: 12, PTBXL: 16
actor_lr=0.0008, # ICBEB: 0.0008, PTBXL: 0.001
critic_lr=0.0015, # ICBEB: 0.0015, PTBXL: 0.002
# System Configuration
seed=42,
device="cuda", # Auto-detect CUDA
verbose=True,
output_dir="./mat_hpo_logs",
dataset_name="ICBEB_Fold1",
task_description="ECG classification on ICBEB dataset with 9 classes"
)
# Optimize with custom environment (SPNV2 pattern)
results = optimizer.optimize(
X_train, y_train, X_val, y_val,
X_test, y_test,
custom_environment=your_environment,
custom_space=your_hyperparameter_space
)
print(f" Best F1: {results['best_performance']['f1']:.4f}")
Fixed Alpha Strategy
Stable LLM/RL mixing ratio (e.g., 30% LLM + 70% RL) - Used in production SPNV2
Adaptive Strategy
Triggers LLM when RL performance slope falls below threshold
️ Full Control Interface
Complete parameter control - all RL and LLM settings exposed
Installation
# Clone the repository
git clone https://github.com/VM230705/MAT-HPO-Library.git
cd MAT-HPO-Library
# Install dependencies
pip install torch numpy scikit-learn
# ✅ Test installation
python test_working_examples.py
⚙️ Configuration
from MAT_HPO_LIB.utils import DefaultConfigs
# Quick test (10 steps) - Perfect for debugging
config = DefaultConfigs.quick_test()
# Standard optimization (100 steps) - Recommended for most use cases
config = DefaultConfigs.standard()
# CPU-only mode - For environments without GPU
config = DefaultConfigs.cpu_only()
Results
Optimization results are automatically saved:
best_hyperparams.json- Best hyperparameters foundoptimization_results.json- Complete optimization resultsstep_log.jsonl- Step-by-step optimization log
🆕 NEW: Custom Metrics & Rewards
MAT-HPO now supports flexible custom metrics for any domain - time series, regression, or custom tasks!
- ✅ Track any metrics (MASE, SMAPE, MAE, RMSE, etc.)
- ✅ Define custom reward functions
- ✅ No library modification needed
- ✅ Perfect for time series forecasting