> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mibyanai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Axolotl — Axolotl: YAML LLM fine-tuning (LoRA, DPO, GRPO)

> Axolotl: YAML LLM fine-tuning (LoRA, DPO, GRPO)

Axolotl: YAML LLM fine-tuning (LoRA, DPO, GRPO).

## Skill metadata

| | |
| - | - |
| Source | Optional — install with `mibyan skills install official/mlops/axolotl` |
| Path | `optional-skills/mlops/training/axolotl` |
| Version | `1.0.0` |
| Author | Orchestra Research |
| License | MIT |
| Dependencies | `axolotl`, `torch`, `transformers`, `datasets`, `peft`, `accelerate`, `deepspeed` |
| Platforms | linux, macos |
| Tags | `Fine-Tuning`, `Axolotl`, `LLM`, `LoRA`, `QLoRA`, `DPO`, `KTO`, `ORPO`, `GRPO`, `YAML`, `HuggingFace`, `DeepSpeed`, `Multimodal` |

## Reference: full SKILL.md

<Info>
  The following is the complete skill definition that Mibyan loads when this skill is triggered. This is what the agent sees as instructions when the skill is active.
</Info>

# Axolotl Skill

## What's inside

Expert guidance for fine-tuning LLMs with Axolotl — YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support.

Assistance with axolotl development, generated from official documentation.

## When to Use This Skill

This skill should be triggered when:

* Working with axolotl
* Asking about axolotl features or APIs
* Implementing axolotl solutions
* Debugging axolotl code
* Learning axolotl best practices

## Quick Reference

### Common Patterns

**Pattern 1:** To validate that acceptable data transfer speeds exist for your training job, running NCCL Tests can help pinpoint bottlenecks, for example:

```
./build/all_reduce_perf -b 8 -e 128M -f 2 -g 3
```

**Pattern 2:** Configure your model to use FSDP in the Axolotl yaml. For example:

```
fsdp_version: 2
fsdp_config:
  offload_params: true
  state_dict_type: FULL_STATE_DICT
  auto_wrap_policy: TRANSFORMER_BASED_WRAP
  transformer_layer_cls_to_wrap: LlamaDecoderLayer
  reshard_after_forward: true
```

**Pattern 3:** The context\_parallel\_size should be a divisor of the total number of GPUs. For example:

```
context_parallel_size
```

**Pattern 4:** For example: - With 8 GPUs and no sequence parallelism: 8 different batches processed per step - With 8 GPUs and context\_parallel\_size=4: Only 2 different batches processed per step (each split across 4 GPUs) - If your per-GPU micro\_batch\_size is 2, the global batch size decreases from 16 to 4

```
context_parallel_size=4
```

**Pattern 5:** Setting save\_compressed: true in your configuration enables saving models in a compressed format, which: - Reduces disk space usage by approximately 40% - Maintains compatibility with vLLM for accelerated inference - Maintains compatibility with llmcompressor for further optimization (example: quantization)

```
save_compressed: true
```

**Pattern 6:** Note It is not necessary to place your integration in the integrations folder. It can be in any location, so long as it’s installed in a package in your python env. See this repo for an example: [https://github.com/axolotl-ai-cloud/diff-transformer](https://github.com/axolotl-ai-cloud/diff-transformer)

```
integrations
```

**Pattern 7:** Handle both single-example and batched data. - single example: sample\[‘input\_ids’] is a list\[int] - batched data: sample\[‘input\_ids’] is a list\[list\[int]]

```
utils.trainer.drop_long_seq(sample, sequence_len=2048, min_sequence_len=2)
```

### Example Code Patterns

**Example 1** (python):

```python theme={null}
cli.cloud.modal_.ModalCloud(config, app=None)
```

**Example 2** (python):

```python theme={null}
cli.cloud.modal_.run_cmd(cmd, run_folder, volumes=None)
```

**Example 3** (python):

```python theme={null}
core.trainers.base.AxolotlTrainer(
    *_args,
    bench_data_collator=None,
    eval_data_collator=None,
    dataset_tags=None,
    **kwargs,
)
```

**Example 4** (python):

```python theme={null}
core.trainers.base.AxolotlTrainer.log(logs, start_time=None)
```

**Example 5** (python):

```python theme={null}
prompt_strategies.input_output.RawInputOutputPrompter()
```

## Reference Files

This skill includes comprehensive documentation in `references/`:

* **api.md** - Api documentation
* **dataset-formats.md** - Dataset-Formats documentation
* **other.md** - Other documentation

Use `view` to read specific reference files when detailed information is needed.

## Working with This Skill

### For Beginners

Start with the getting\_started or tutorials reference files for foundational concepts.

### For Specific Features

Use the appropriate category reference file (api, guides, etc.) for detailed information.

### For Code Examples

The quick reference section above contains common patterns extracted from the official docs.

## Resources

### references/

Organized documentation extracted from official sources. These files contain:

* Detailed explanations
* Code examples with language annotations
* Links to original documentation
* Table of contents for quick navigation

### scripts/

Add helper scripts here for common automation tasks.

### assets/

Add templates, boilerplate, or example projects here.

## Notes

* This skill was automatically generated from official documentation
* Reference files preserve the structure and examples from source docs
* Code examples include language detection for better syntax highlighting
* Quick reference patterns are extracted from common usage examples in the docs

## Updating

To refresh this skill with updated documentation:

1. Re-run the scraper with the same configuration
2. The skill will be rebuilt with the latest information


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.