LLM Fine-Tuning Rehberi: Modeli Kendi Verinle Eğit

Fine-tuning = model permanently öğrensin. Stil, format, jargon, task-specific. RAG değil knowledge — behavior değişikliği için.

Fine-Tune Ne İçin?

✅ İyi:
- Tutarlı output format (JSON, XML)
- Brand voice / stil
- Domain jargon
- Spesifik task accuracy
- Lower latency (smaller model)
- Reduced prompt length

❌ Kötü:
- Yeni bilgi öğretmek (RAG kullan)
- Hızla değişen veri
- Tek seferlik
- < 50 example
- Prompt engineering yetiyorsa

🟡 Belki:
- Cost reduction (cheap model fine-tune > expensive big model)
- Privacy (own model > API)

OpenAI Fine-Tuning

from openai import OpenAI

client = OpenAI()

# 1. Prepare data (JSONL)
"""
{"messages": [
  {"role": "system", "content": "Sen Türk AI asistanısın"},
  {"role": "user", "content": "Merhaba"},
  {"role": "assistant", "content": "Merhaba! Size nasıl yardımcı olabilirim?"}
]}
"""

# 2. Upload
file = client.files.create(
    file=open("training.jsonl", "rb"),
    purpose="fine-tune"
)

# 3. Start fine-tune
job = client.fine_tuning.jobs.create(
    training_file=file.id,
    model="gpt-4o-mini-2024-07-18",
    hyperparameters={
        "n_epochs": 3,
        "batch_size": "auto",
        "learning_rate_multiplier": "auto"
    },
    suffix="my-app-v1"
)

# 4. Check status
status = client.fine_tuning.jobs.retrieve(job.id)

# 5. Use fine-tuned model
response = client.chat.completions.create(
    model="ft:gpt-4o-mini:org:my-app-v1:abc123",
    messages=[...]
)

OpenAI Pricing (2026)

Model	Training $/M token	Input $/M	Output $/M
GPT-4o-mini	$3	$0.30	$1.20
GPT-4o	$25	$3.75	$15
GPT-3.5-turbo	$8	$3	$6

LoRA (Open Source)

# Hugging Face PEFT
from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "meta-llama/Llama-3.1-8B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# LoRA config
lora_config = LoraConfig(
    r=16,                # Rank
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

model = get_peft_model(model, lora_config)
# Trainable params: 16M (vs 8B total)

LoRA: Low-Rank Adaptation. Sadece küçük matrix’ler train, model çoğu frozen.

QLoRA (4-bit Quantized)

from transformers import BitsAndBytesConfig

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True
)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto"
)

# LoRA + 4-bit base = QLoRA
# 70B model 24GB GPU'da train edilebilir

QLoRA: Llama 3.1 70B → RTX 3090 (24GB) fine-tune mümkün.

Detay: LLM Quantization

Training Workflow

from transformers import Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir="./output",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    warmup_steps=100,
    learning_rate=2e-4,
    fp16=False,
    bf16=True,
    logging_steps=10,
    save_strategy="epoch",
    eval_strategy="epoch",
    evaluation_strategy="epoch",
    save_total_limit=2,
    load_best_model_at_end=True,
    report_to="wandb",
    push_to_hub=True,
    hub_model_id="myusername/llama-3-tr-finetune"
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    tokenizer=tokenizer
)

trainer.train()
trainer.push_to_hub()

Dataset Preparation

"Quality > Quantity:

Minimum:
- 50 examples (format)
- 500 (style)
- 1000+ (new capability)

Format:
- JSON / JSONL
- Instruction-response pairs
- Multi-turn conversations
- Chat template (model-specific)

Sample example:
{
  "instruction": "Bu Türkçe haberin başlığını yaz",
  "input": "Maraş'taki depremde ...",
  "output": "Maraş Depremi: Son Durum"
}

Best practices:
- Diverse examples
- Clean (no errors)
- Realistic distribution
- Train/val split (90/10)
- Augment (paraphrase)
"

Tools

"Tooling:

Cloud platforms:
- Hugging Face AutoTrain
- Together.ai
- Replicate
- Modal
- RunPod
- Lambda Labs

Frameworks:
- Hugging Face Trainer (high-level)
- TRL (PPO, DPO, ORPO)
- Axolotl (config-based)
- LLaMA-Factory
- Unsloth (2x faster)

Monitoring:
- Weights & Biases (W&B)
- Tensorboard
- LangSmith
- MLflow
"

Use Case: Türkçe Stil

"Brand voice fine-tune:

Goal: Markamın stilinde yazıyor

Dataset:
- 500 örnek "user soru → biz cevap"
- Tutarlı tone
- Brand-specific vocab
- Common scenarios

Train:
- GPT-4o-mini fine-tune
- 3 epoch
- ~$50 total cost

Sonuç:
- Brand voice consistent
- Less prompt engineering
- Cheaper inference (smaller model)
"

Use Case: Domain-Specific

"Medical / Legal jargon:

Klasik LLM: 'generally speaking...'
Fine-tuned: domain-aware response

Dataset:
- 5000 örnek
- Medical/legal terminology
- Citation format
- Hedge appropriate

Result:
- Higher accuracy
- Domain credibility
- Lower hallucination on jargon

⚠️ Genel LLM bilgisini override eder
RAG ile ne zaman kullan?
- Fine-tune: format + stil
- RAG: factual knowledge
"

DPO / RLHF Alternatif

"Modern alignment (post-2024):

DPO (Direct Preference Optimization):
- Reward model'siz
- Pair (chosen, rejected) data
- Daha basit RLHF'den
- Same quality

ORPO (Odds Ratio):
- Reference model'siz
- Even simpler
- 2024 yeni

KTO (Kahneman-Tversky Optimization):
- Binary feedback (good/bad)
- No paired needed
- Production friendly

Stack:
- TRL library (Hugging Face)
- Built-in DPO, ORPO, KTO trainers
"

Detay: RLHF Nasıl Çalışır

Evaluation

"""
Multiple methods:

1. Holdout test set:
   - Hold out 10% data
   - Metric: accuracy, F1, BLEU, ROUGE

2. LLM-as-judge:
   - GPT-4o evaluate
   - Pairwise comparison
   - Rubric-based scoring

3. Human eval:
   - Subset rated
   - Inter-annotator agreement
   - Final benchmark

4. Production A/B:
   - 5% traffic fine-tuned
   - 95% baseline
   - Compare metrics

5. Specific benchmark:
   - Domain test
   - TR LLM Leaderboard
   - MMLU-TR
"""

Production Deployment

"Inference options:

1. OpenAI fine-tune:
   - Same API
   - Just change model ID
   - Auto-scale

2. Self-hosted:
   - vLLM (production serving)
   - TGI (Text Generation Inference)
   - SGLang
   - Together.ai (managed)

3. Edge:
   - GGUF (Llama.cpp)
   - MLX (Apple Silicon)
   - WebLLM (browser)

Cost compare:
- API: pay-per-token (no upfront)
- Self-host: GPU $$$ + ops
- Hybrid: peak API + base self-host
"

Common Pitfalls

"Yaygın hatalar:

1. Catastrophic forgetting:
   - Fine-tune yaparken base bilgi kaybedebilir
   - Mitigation: low learning rate, fewer epochs, regularization

2. Overfitting:
   - Train data ezberle
   - Val accuracy düşer
   - Fix: more diverse data, early stopping

3. Data leakage:
   - Test data train'de
   - Inflated metric
   - Fix: strict split

4. Wrong base model:
   - Too small: no capacity
   - Too large: overfit
   - Match task complexity

5. No baseline:
   - Just fine-tune blind
   - Compare with prompt engineering first
"

Cost Analysis

"Realistic cost (TR project):

Project: Türkçe customer support
Data: 5000 conversations
Goal: smaller cheaper model, brand voice

Option A: GPT-4o-mini fine-tune (OpenAI)
- Training: 5M token × $3 = $15
- Total: ~$50 (including iteration)
- Inference: $0.30/M input, $1.20/M output

Option B: Llama 3.1 8B LoRA (self-host)
- GPU rent (RunPod): $0.5/hour × 8 = $4
- Training: $20 (including iteration)
- Inference: A100 hosting $1.5/hr or vLLM
- Better long-run if scale

Option C: Llama 3.1 70B QLoRA
- 24GB GPU (3090 / A100)
- $50 training
- Higher quality
- Larger inference cost
"

Türkçe Fine-Tuning

"TR-specific tips:

Base model:
- ytu-ce-cosmos/Turkish-Llama-8b (TR pretraining)
- Mistral, Llama 3 multilingual
- Trendyol/Trendyol-LLM-7b
- Existing fine-tunes leverage

Tokenizer:
- TR efficient tokenizer prefer
- BPE issue (long Türkçe words)
- Subword balance

Data:
- TR conversation
- TR instruction tune
- Common Crawl TR
- Wikipedia TR
- Trendyol/DergiPark

Eval:
- Turkish LLM Leaderboard
- MMLU-TR
- TRMMLU
- Custom benchmark
"

Sonraki Adımlar

• RAG Rehberi
• LLM Quantization
• RLHF Nasıl Çalışır
• Hugging Face
• LangChain Agent

Özet

Fine-tuning = stil + format + spec task. OpenAI managed (easy, $50) veya LoRA self-host (control, custom). Anahtar: RAG önce dene, data quality matters, eval framework şart. QLoRA = 70B model laptop’ta.