Table 5 QLoRA settings for training
From: Evaluating clinical AI summaries with large language models as judges
Model | Strategy | Quantization | LoRA Rank | LoRA Alpha | LoRA Drop-Out |
|---|---|---|---|---|---|
Mixtral 8 × 22B | SFT | 4-bit | 8 | 8 | 0.1 |
Mixtral 8 × 22B | DPO | 4-bit | 8 | 8 | 0.1 |
Llama 3.1 8B | SFT | 4-bit | 64 | 256 | 0.0 |
Llama 3.1 8B | DPO | 4-bit | 8 | 8 | 0.1 |
