> ## Documentation Index > Fetch the complete documentation index at: https://docs.mibyanai.com/llms.txt > Use this file to discover all available pages before exploring further. # Lambda Labs — On-demand GPU cloud instances for ML training > On-demand GPU cloud instances for ML training Commands, package names, and image names on this page come from the open-source project that Mibyan Desktop is built on, and can differ from the Mibyan Desktop installer. For the supported Mibyan install and update path, see [Install and update](/products/desktop-guide/install-and-update). On-demand GPU cloud instances for ML training. ## Skill metadata | | | | - | - | | Source | Optional — install with `mibyan skills install official/mlops/lambda-labs` | | Path | `optional-skills/mlops/lambda-labs` | | Version | `1.0.0` | | Author | Orchestra Research | | License | MIT | | Dependencies | `lambda-cloud-client>=1.0.0` | | Platforms | linux, macos, windows | | Tags | `Infrastructure`, `GPU Cloud`, `Training`, `Inference`, `Lambda Labs` | ## Reference: full SKILL.md The following is the complete skill definition that Mibyan loads when this skill is triggered. This is what the agent sees as instructions when the skill is active. # Lambda Labs GPU Cloud Guide to running ML workloads on Lambda Labs GPU cloud with on-demand instances and 1-Click Clusters. ## When to use Lambda Labs **Use Lambda Labs when:** * Need dedicated GPU instances with full SSH access * Running long training jobs (hours to days) * Want simple pricing with no egress fees * Need persistent storage across sessions * Require high-performance multi-node clusters (16-512 GPUs) * Want pre-installed ML stack (Lambda Stack with PyTorch, CUDA, NCCL) **Key features:** * **GPU variety**: B200, H100, GH200, A100, A10, A6000, V100 * **Lambda Stack**: Pre-installed PyTorch, TensorFlow, CUDA, cuDNN, NCCL * **Persistent filesystems**: Keep data across instance restarts * **1-Click Clusters**: 16-512 GPU Slurm clusters with InfiniBand * **Simple pricing**: Pay-per-minute, no egress fees * **Global regions**: 12+ regions worldwide **Use alternatives instead:** * **Modal**: For serverless, auto-scaling workloads * **SkyPilot**: For multi-cloud orchestration and cost optimization * **RunPod**: For cheaper spot instances and serverless endpoints * **Vast.ai**: For GPU marketplace with lowest prices ## Quick start ### Account setup 1. Create account at [https://lambda.ai](https://lambda.ai) 2. Add payment method 3. Generate API key from dashboard 4. Add SSH key (required before launching instances) ### Launch via console 1. Go to [https://cloud.lambda.ai/instances](https://cloud.lambda.ai/instances) 2. Click "Launch instance" 3. Select GPU type and region 4. Choose SSH key 5. Optionally attach filesystem 6. Launch and wait 3-15 minutes ### Connect via SSH ```bash theme={null} # Get instance IP from console ssh ubuntu@ # Or with specific key ssh -i ~/.ssh/lambda_key ubuntu@ ``` ## GPU instances ### Available GPUs | GPU | VRAM | Price/GPU/hr | Best For | | - | - | - | - | | B200 SXM6 | 180 GB | \$4.99 | Largest models, fastest training | | H100 SXM | 80 GB | \$2.99-3.29 | Large model training | | H100 PCIe | 80 GB | \$2.49 | Cost-effective H100 | | GH200 | 96 GB | \$1.49 | Single-GPU large models | | A100 80GB | 80 GB | \$1.79 | Production training | | A100 40GB | 40 GB | \$1.29 | Standard training | | A10 | 24 GB | \$0.75 | Inference, fine-tuning | | A6000 | 48 GB | \$0.80 | Good VRAM/price ratio | | V100 | 16 GB | \$0.55 | Budget training | ### Instance configurations ``` 8x GPU: Best for distributed training (DDP, FSDP) 4x GPU: Large models, multi-GPU training 2x GPU: Medium workloads 1x GPU: Fine-tuning, inference, development ``` ### Launch times * Single-GPU: 3-5 minutes * Multi-GPU: 10-15 minutes ## Lambda Stack All instances come with Lambda Stack pre-installed: ```bash theme={null} # Included software - Ubuntu 22.04 LTS - NVIDIA drivers (latest) - CUDA 12.x - cuDNN 8.x - NCCL (for multi-GPU) - PyTorch (latest) - TensorFlow (latest) - JAX - JupyterLab ``` ### Verify installation ```bash theme={null} # Check GPU nvidia-smi # Check PyTorch python -c "import torch; print(torch.cuda.is_available())" # Check CUDA version nvcc --version ``` ## Python API ### Installation ```bash theme={null} pip install lambda-cloud-client ``` ### Authentication ```python theme={null} import os import lambda_cloud_client # Configure with API key configuration = lambda_cloud_client.Configuration( host="https://cloud.lambdalabs.com/api/v1", access_token=os.environ["LAMBDA_API_KEY"] ) ``` ### List available instances ```python theme={null} with lambda_cloud_client.ApiClient(configuration) as api_client: api = lambda_cloud_client.DefaultApi(api_client) # Get available instance types types = api.instance_types() for name, info in types.data.items(): print(f"{name}: {info.instance_type.description}") ``` ### Launch instance ```python theme={null} from lambda_cloud_client.models import LaunchInstanceRequest request = LaunchInstanceRequest( region_name="us-west-1", instance_type_name="gpu_1x_h100_sxm5", ssh_key_names=["my-ssh-key"], file_system_names=["my-filesystem"], # Optional name="training-job" ) response = api.launch_instance(request) instance_id = response.data.instance_ids[0] print(f"Launched: {instance_id}") ``` ### List running instances ```python theme={null} instances = api.list_instances() for instance in instances.data: print(f"{instance.name}: {instance.ip} ({instance.status})") ``` ### Terminate instance ```python theme={null} from lambda_cloud_client.models import TerminateInstanceRequest request = TerminateInstanceRequest( instance_ids=[instance_id] ) api.terminate_instance(request) ``` ### SSH key management ```python theme={null} from lambda_cloud_client.models import AddSshKeyRequest # Add SSH key request = AddSshKeyRequest( name="my-key", public_key="ssh-rsa AAAA..." ) api.add_ssh_key(request) # List keys keys = api.list_ssh_keys() # Delete key api.delete_ssh_key(key_id) ``` ## CLI with curl ### List instance types ```bash theme={null} curl -u $LAMBDA_API_KEY: \ https://cloud.lambdalabs.com/api/v1/instance-types | jq ``` ### Launch instance ```bash theme={null} curl -u $LAMBDA_API_KEY: \ -X POST https://cloud.lambdalabs.com/api/v1/instance-operations/launch \ -H "Content-Type: application/json" \ -d '{ "region_name": "us-west-1", "instance_type_name": "gpu_1x_h100_sxm5", "ssh_key_names": ["my-key"] }' | jq ``` ### Terminate instance ```bash theme={null} curl -u $LAMBDA_API_KEY: \ -X POST https://cloud.lambdalabs.com/api/v1/instance-operations/terminate \ -H "Content-Type: application/json" \ -d '{"instance_ids": [""]}' | jq ``` ## Persistent storage ### Filesystems Filesystems persist data across instance restarts: ```bash theme={null} # Mount location /lambda/nfs/ # Example: save checkpoints python train.py --checkpoint-dir /lambda/nfs/my-storage/checkpoints ``` ### Create filesystem 1. Go to Storage in Lambda console 2. Click "Create filesystem" 3. Select region (must match instance region) 4. Name and create ### Attach to instance Filesystems must be attached at instance launch time: * Via console: Select filesystem when launching * Via API: Include `file_system_names` in launch request ### Best practices ```bash theme={null} # Store on filesystem (persists) /lambda/nfs/storage/ ├── datasets/ ├── checkpoints/ ├── models/ └── outputs/ # Local SSD (faster, ephemeral) ~/ (instance home) └── working/ # Temporary files ``` ## SSH configuration ### Add SSH key ```bash theme={null} # Generate key locally ssh-keygen -t ed25519 -f ~/.ssh/lambda_key # Add public key to Lambda console # Or via API ``` ### Multiple keys ```bash theme={null} # On instance, add more keys echo 'ssh-rsa AAAA...' >> ~/.ssh/authorized_keys ``` ### Import from GitHub ```bash theme={null} # On instance ssh-import-id gh:username ``` ### SSH tunneling ```bash theme={null} # Forward Jupyter ssh -L 8888:localhost:8888 ubuntu@ # Forward TensorBoard ssh -L 6006:localhost:6006 ubuntu@ # Multiple ports ssh -L 8888:localhost:8888 -L 6006:localhost:6006 ubuntu@ ``` ## JupyterLab ### Launch from console 1. Go to Instances page 2. Click "Launch" in Cloud IDE column 3. JupyterLab opens in browser ### Manual access ```bash theme={null} # On instance jupyter lab --ip=0.0.0.0 --port=8888 # From local machine with tunnel ssh -L 8888:localhost:8888 ubuntu@ # Open http://localhost:8888 ``` ## Training workflows ### Single-GPU training ```bash theme={null} # SSH to instance ssh ubuntu@ # Clone repo git clone https://github.com/user/project cd project # Install dependencies pip install -r requirements.txt # Train python train.py --epochs 100 --checkpoint-dir /lambda/nfs/storage/checkpoints ``` ### Multi-GPU training (single node) ```python theme={null} # train_ddp.py import torch import torch.distributed as dist from torch.nn.parallel import DistributedDataParallel as DDP def main(): dist.init_process_group("nccl") rank = dist.get_rank() device = rank % torch.cuda.device_count() model = MyModel().to(device) model = DDP(model, device_ids=[device]) # Training loop... if __name__ == "__main__": main() ``` ```bash theme={null} # Launch with torchrun (8 GPUs) torchrun --nproc_per_node=8 train_ddp.py ``` ### Checkpoint to filesystem ```python theme={null} import os checkpoint_dir = "/lambda/nfs/my-storage/checkpoints" os.makedirs(checkpoint_dir, exist_ok=True) # Save checkpoint torch.save({ 'epoch': epoch, 'model_state_dict': model.state_dict(), 'optimizer_state_dict': optimizer.state_dict(), 'loss': loss, }, f"{checkpoint_dir}/checkpoint_{epoch}.pt") ``` ## 1-Click Clusters ### Overview High-performance Slurm clusters with: * 16-512 NVIDIA H100 or B200 GPUs * NVIDIA Quantum-2 400 Gb/s InfiniBand * GPUDirect RDMA at 3200 Gb/s * Pre-installed distributed ML stack ### Included software * Ubuntu 22.04 LTS + Lambda Stack * NCCL, Open MPI * PyTorch with DDP and FSDP * TensorFlow * OFED drivers ### Storage * 24 TB NVMe per compute node (ephemeral) * Lambda filesystems for persistent data ### Multi-node training ```bash theme={null} # On Slurm cluster srun --nodes=4 --ntasks-per-node=8 --gpus-per-node=8 \ torchrun --nnodes=4 --nproc_per_node=8 \ --rdzv_backend=c10d --rdzv_endpoint=$MASTER_ADDR:29500 \ train.py ``` ## Networking ### Bandwidth * Inter-instance (same region): up to 200 Gbps * Internet outbound: 20 Gbps max ### Firewall * Default: Only port 22 (SSH) open * Configure additional ports in Lambda console * ICMP traffic allowed by default ### Private IPs ```bash theme={null} # Find private IP ip addr show | grep 'inet ' ``` ## Common workflows ### Workflow 1: Fine-tuning LLM ```bash theme={null} # 1. Launch 8x H100 instance with filesystem # 2. SSH and setup ssh ubuntu@ pip install transformers accelerate peft # 3. Download model to filesystem python -c " from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained('meta-llama/Llama-2-7b-hf') model.save_pretrained('/lambda/nfs/storage/models/llama-2-7b') " # 4. Fine-tune with checkpoints on filesystem accelerate launch --num_processes 8 train.py \ --model_path /lambda/nfs/storage/models/llama-2-7b \ --output_dir /lambda/nfs/storage/outputs \ --checkpoint_dir /lambda/nfs/storage/checkpoints ``` ### Workflow 2: Batch inference ```bash theme={null} # 1. Launch A10 instance (cost-effective for inference) # 2. Run inference python inference.py \ --model /lambda/nfs/storage/models/fine-tuned \ --input /lambda/nfs/storage/data/inputs.jsonl \ --output /lambda/nfs/storage/data/outputs.jsonl ``` ## Cost optimization ### Choose right GPU | Task | Recommended GPU | | - | - | | LLM fine-tuning (7B) | A100 40GB | | LLM fine-tuning (70B) | 8x H100 | | Inference | A10, A6000 | | Development | V100, A10 | | Maximum performance | B200 | ### Reduce costs 1. **Use filesystems**: Avoid re-downloading data 2. **Checkpoint frequently**: Resume interrupted training 3. **Right-size**: Don't over-provision GPUs 4. **Terminate idle**: No auto-stop, manually terminate ### Monitor usage * Dashboard shows real-time GPU utilization * API for programmatic monitoring ## Common issues | Issue | Solution | | - | - | | Instance won't launch | Check region availability, try different GPU | | SSH connection refused | Wait for instance to initialize (3-15 min) | | Data lost after terminate | Use persistent filesystems | | Slow data transfer | Use filesystem in same region | | GPU not detected | Reboot instance, check drivers | ## References * **[Advanced Usage](https://github.com/NousResearch/hermes-agent/blob/main/optional-skills/mlops/lambda-labs/references/advanced-usage.md)** - Multi-node training, API automation * **[Troubleshooting](https://github.com/NousResearch/hermes-agent/blob/main/optional-skills/mlops/lambda-labs/references/troubleshooting.md)** - Common issues and solutions ## Resources * **Documentation**: [https://docs.lambda.ai](https://docs.lambda.ai) * **Console**: [https://cloud.lambda.ai](https://cloud.lambda.ai) * **Pricing**: [https://lambda.ai/instances](https://lambda.ai/instances) * **Support**: [https://support.lambdalabs.com](https://support.lambdalabs.com) * **Blog**: [https://lambda.ai/blog](https://lambda.ai/blog) This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.