Configuration

Configure Chisel CLI for optimal performance with GPU types, environment variables, and optimization settings

Configure Chisel CLI for optimal performance across different workloads, from development to production-scale training.

GPU Configuration

GPU Type Selection

Choose the right GPU configuration based on your specific requirements:

By Use Case

Development & Prototyping

Recommended: A100_80GB_1

Perfect for:

• Code development and debugging
• Algorithm prototyping
• Small dataset experimentation
• Model inference testing

dev_app = ChiselApp("development", gpu=GPUType.A100_80GB_1)

@dev_app.capture_trace(trace_name="prototype")
def prototype_function(data):
    import torch
    device = "cuda" if torch.cuda.is_available() else "cpu"
    # Development code here
    return result

Training Small-Medium Models

Recommended: A100_80GB_2

Optimal for:

• Most PyTorch models
• Computer vision tasks
• NLP models up to ~7B parameters
• Data parallel training

train_app = ChiselApp("training", gpu=GPUType.A100_80GB_2)

@train_app.capture_trace(trace_name="training")
def train_model(data_loader, model):
    import torch
    from torch.nn.parallel import DataParallel
    
    if torch.cuda.device_count() > 1:
        model = DataParallel(model)
    
    return model.cuda()

Large Model Training

Recommended: A100_80GB_4 or A100_80GB_8

Required for:

• Large language models (10B+ parameters)
• High-resolution image models
• Complex multi-modal architectures
• Pipeline parallel training

large_app = ChiselApp("large-training", gpu=GPUType.A100_80GB_8)

@large_app.capture_trace(trace_name="large_model")
def train_large_model(model, data_loader):
    import torch
    from torch.nn.parallel import DistributedDataParallel as DDP
    
    # Setup for distributed training
    model = DDP(model)
    return model

High-Throughput Inference

Recommended: A100_80GB_4

Best for:

• Batch inference workloads
• Real-time serving at scale
• Multiple model deployment
• Concurrent request handling

inference_app = ChiselApp("inference", gpu=GPUType.A100_80GB_4)

@inference_app.capture_trace(trace_name="batch_inference")
def batch_inference(data_batches, models):
    import torch
    
    results = []
    for batch, model in zip(data_batches, models):
        with torch.no_grad():
            result = model(batch.cuda())
            results.append(result.cpu())
    
    return results

Application Configuration

ChiselApp Options

Configure your ChiselApp for different scenarios:

Basic Configuration

from chisel import ChiselApp, GPUType

app = ChiselApp(
    name="my-app",              # Required: App identifier
    upload_dir=".",             # Upload directory (default: current)
    gpu=GPUType.A100_80GB_2     # GPU configuration
)

Trace Configuration

Optimize tracing for debugging and performance monitoring:

Basic Tracing

@app.capture_trace(trace_name="my_operation")
def basic_function():
    # Basic tracing - minimal overhead
    pass

Environment Variables

Core Environment Variables

CHISEL_ACTIVATED

Purpose: Activates GPU functionality
Set by: chisel command
Values: “1” (active) or unset (inactive)

import os

if os.environ.get("CHISEL_ACTIVATED") == "1":
    print("🚀 Chisel is activated - using cloud GPU")
else:
    print("💻 Running locally")

CHISEL_BACKEND_URL

Purpose: Override backend URL
Set by: User (optional)
Default: “http://localhost:8000”

# Use production backend
export CHISEL_BACKEND_URL="https://api.herdora.com"
chisel python my_script.py

# Use staging backend
export CHISEL_BACKEND_URL="https://staging-api.herdora.com"
chisel python my_script.py

CHISEL_API_KEY

Purpose: Direct API key provision
Set by: User or CI/CD system
Usage: Bypasses stored credentials

# CI/CD usage
export CHISEL_API_KEY="your_api_key_here"
chisel python my_script.py

# Or in Python
import os
os.environ["CHISEL_API_KEY"] = "your_api_key_here"

CHISEL_BACKEND_RUN

Purpose: Indicates backend execution
Set by: Backend system
Values: “1” when running on cloud GPU

import os

def is_running_on_cloud():
    return os.environ.get("CHISEL_BACKEND_RUN") == "1"

if is_running_on_cloud():
    job_id = os.environ.get("CHISEL_JOB_ID", "unknown")
    print(f"Running on cloud GPU (Job: {job_id})")

Environment Configuration Files

.env File

# .env file for project configuration
CHISEL_BACKEND_URL=https://api.herdora.com
TORCH_CUDA_ARCH_LIST=8.0
PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128
CUDA_VISIBLE_DEVICES=0,1,2,3

Load in Python:

from dotenv import load_dotenv
load_dotenv()

app = ChiselApp("configured-app")

Performance Optimization

Memory Management

GPU Memory Optimization

Optimize GPU memory usage for better performance:

import torch
import gc

@app.capture_trace(trace_name="memory_optimized", profile_memory=True)
def memory_efficient_processing(large_dataset):
    device = "cuda" if torch.cuda.is_available() else "cpu"
    
    # Clear cache at start
    torch.cuda.empty_cache()
    
    # Process in chunks to manage memory
    batch_size = 1000  # Adjust based on available memory
    results = []
    
    for i in range(0, len(large_dataset), batch_size):
        batch = large_dataset[i:i+batch_size]
        batch_tensor = torch.tensor(batch, device=device)
        
        # Process batch
        with torch.cuda.amp.autocast():  # Use mixed precision
            batch_result = process_batch(batch_tensor)
        
        # Move result to CPU immediately to free GPU memory
        results.append(batch_result.cpu())
        
        # Clean up GPU memory
        del batch_tensor, batch_result
        
        # Periodic memory cleanup
        if i % (batch_size * 10) == 0:
            torch.cuda.empty_cache()
            gc.collect()
    
    return torch.cat(results)

Multi-GPU Configuration

Configure multiple GPUs for maximum throughput:

import torch
from torch.nn.parallel import DataParallel, DistributedDataParallel

@app.capture_trace(trace_name="multi_gpu_setup")
def setup_multi_gpu_model(model):
    device_count = torch.cuda.device_count()
    
    if device_count == 0:
        print("❌ No GPU available")
        return model
    
    elif device_count == 1:
        print("🎯 Using single GPU")
        return model.cuda()
    
    else:
        print(f"🚀 Using {device_count} GPUs with DataParallel")
        model = DataParallel(model)
        return model.cuda()

# For distributed training (advanced)
@app.capture_trace(trace_name="distributed_setup")
def setup_distributed_training():
    import torch.distributed as dist
    
    # Initialize distributed training
    dist.init_process_group(backend='nccl')
    
    # Get local rank and world size
    local_rank = int(os.environ.get('LOCAL_RANK', 0))
    world_size = int(os.environ.get('WORLD_SIZE', 1))
    
    torch.cuda.set_device(local_rank)
    
    return local_rank, world_size

Mixed Precision Training

Use mixed precision for faster training with less memory:

from torch.cuda.amp import autocast, GradScaler

@app.capture_trace(trace_name="mixed_precision_training")
def train_with_mixed_precision(model, data_loader, optimizer, criterion):
    scaler = GradScaler()
    model.train()
    
    for batch_idx, (inputs, targets) in enumerate(data_loader):
        inputs, targets = inputs.cuda(), targets.cuda()
        
        optimizer.zero_grad()
        
        # Forward pass with autocast
        with autocast():
            outputs = model(inputs)
            loss = criterion(outputs, targets)
        
        # Backward pass with gradient scaling
        scaler.scale(loss).backward()
        scaler.step(optimizer)
        scaler.update()
        
        if batch_idx % 100 == 0:
            print(f'Batch {batch_idx}, Loss: {loss.item():.4f}')
    
    return model

Batch Size Optimization

Find optimal batch size for your GPU memory:

def find_optimal_batch_size(model, sample_input, max_memory_gb=80):
    """Find the largest batch size that fits in GPU memory."""
    model = model.cuda()
    batch_size = 32
    
    while batch_size <= 2048:
        try:
            # Create batch of given size
            batch_input = sample_input.repeat(batch_size, 1, 1, 1).cuda()
            
            # Forward pass
            with torch.no_grad():
                output = model(batch_input)
            
            # Check memory usage
            memory_used = torch.cuda.max_memory_allocated() / (1024**3)
            print(f"Batch size {batch_size}: {memory_used:.2f}GB")
            
            # Stop if we're using more than 90% of available memory
            if memory_used > max_memory_gb * 0.9:
                optimal_batch_size = batch_size // 2
                print(f"Optimal batch size: {optimal_batch_size}")
                return optimal_batch_size
            
            batch_size *= 2
            
        except RuntimeError as e:
            if "out of memory" in str(e):
                optimal_batch_size = batch_size // 2
                print(f"Optimal batch size: {optimal_batch_size}")
                return optimal_batch_size
            else:
                raise e
    
    return batch_size

# Usage
optimal_batch = find_optimal_batch_size(my_model, sample_input)

PyTorch Optimization

A100 Specific Optimizations

import torch
import os

# A100 architecture optimization
os.environ['TORCH_CUDA_ARCH_LIST'] = '8.0'

# Memory management
torch.cuda.set_per_process_memory_fraction(0.9)
os.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'max_split_size_mb:128'

# Enable optimized attention (PyTorch 2.0+)
torch.backends.cuda.enable_flash_sdp(True)
torch.backends.cuda.enable_mem_efficient_sdp(True)

Development Configuration

Local Development Setup

Development Mode

import os

# Development configuration
def setup_development_environment():
    """Configure Chisel for development."""
    
    # Disable Chisel for local development
    if os.getenv('DEVELOPMENT'):
        print("💻 Development mode - Chisel inactive")
        return False
    
    # Enable debug mode
    if os.getenv('CHISEL_DEBUG'):
        import logging
        logging.basicConfig(level=logging.DEBUG)
        torch.autograd.set_detect_anomaly(True)
        os.environ['CUDA_LAUNCH_BLOCKING'] = '1'
    
    return True

# Usage
if setup_development_environment():
    app = ChiselApp("dev-app")
else:
    # Create no-op app for local development
    class NoOpApp:
        def capture_trace(self, **kwargs):
            return lambda func: func
    
    app = NoOpApp()

Best Practices

Follow these configuration best practices for optimal Chisel CLI usage:

Configuration Checklist

1. GPU Selection

   • Start with A100_80GB_1 for development
   • Scale up based on actual memory requirements
   • Monitor GPU utilization to ensure efficient usage

2. Upload Optimization

   • Keep upload directories under 100MB
   • Use .gitignore to exclude unnecessary files
   • Include only required code and dependencies

3. Memory Management

   • Use mixed precision training when possible
   • Process large datasets in chunks
   • Clear GPU memory explicitly when needed

4. Environment Variables

   • Use .env files for project-specific configuration
   • Set CHISEL_DEBUG=1 for development debugging
   • Configure backend URLs appropriately for each environment

5. Performance Monitoring

   • Use profile_memory=True for optimization
   • Monitor job costs and GPU utilization
   • Benchmark different configurations for your workload

# Configuration validation script
def validate_chisel_configuration():
    """Validate Chisel CLI configuration."""
    
    issues = []
    
    # Check upload directory size
    upload_dir = "."
    size_mb = get_directory_size(upload_dir)
    if size_mb > 100:
        issues.append(f"Upload directory too large: {size_mb:.1f}MB > 100MB")
    
    # Check GPU memory requirements
    estimated_memory = estimate_memory_requirement(model_params)
    gpu_memory = get_gpu_memory(gpu_type)
    if estimated_memory > gpu_memory * 0.8:
        issues.append(f"Insufficient GPU memory: {estimated_memory:.1f}GB > {gpu_memory:.1f}GB")
    
    # Check authentication
    from chisel.auth import is_authenticated
    if not is_authenticated():
        issues.append("Authentication required")
    
    if issues:
        print("❌ Configuration issues found:")
        for issue in issues:
            print(f"   - {issue}")
        return False
    else:
        print("✅ Configuration validated successfully")
        return True

# Run validation
validate_chisel_configuration()

Related Documentation

API Reference

Complete API documentation for ChiselApp and GPU types

Examples

Working examples with different configurations

Troubleshooting

Configuration troubleshooting and optimization tips

Performance Guide

Advanced performance optimization techniques