%pip install -U transformers torch tqdm tiktoken markdown

from datasets import load_dataset

eli5 = load_dataset("eli5_category", split="train[:5000]")

eli5 = eli5.train_test_split(test_size=0.2)

/Users/LSoica/work/AI/blog/.venv/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Downloading data: 100%|██████████| 62.3M/62.3M [00:00<00:00, 66.7MB/s]
Downloading data: 100%|██████████| 5.00M/5.00M [00:00<00:00, 90.5MB/s]
Downloading data: 100%|██████████| 1.76M/1.76M [00:00<00:00, 110MB/s]
Downloading data: 100%|██████████| 3.85M/3.85M [00:00<00:00, 95.6MB/s]
Generating train split: 100%|██████████| 91772/91772 [00:03<00:00, 23245.99 examples/s]
Generating validation1 split: 100%|██████████| 5446/5446 [00:00<00:00, 26980.41 examples/s]
Generating validation2 split: 100%|██████████| 2375/2375 [00:00<00:00, 28917.25 examples/s]
Generating test split: 100%|██████████| 5411/5411 [00:00<00:00, 29139.00 examples/s]

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("distilbert/distilgpt2")


eli5 = eli5.flatten()
eli5["train"][0]

{'q_id': '7h2ns1',
 'title': 'How does different data traveling on the same cable not get lost with all the other data?',
 'selftext': '',
 'category': 'Technology',
 'subreddit': 'explainlikeimfive',
 'answers.a_id': ['dqnmb02'],
 'answers.text': ['Data is encapsulated into packets with headers and trailers that identify it. Sometimes it does get lost though. When 2 devices establish a connection, they decide on packet numbering. If I send you a packet that says it contains data 1500 - 1600 you expect that my next packet starts with 1700. If it doesn’t, then you your response to me is essentially “I need 1700”.'],
 'answers.score': [10],
 'answers.text_urls': [[]],
 'title_urls': ['url'],
 'selftext_urls': ['url']}

def preprocess_function(examples):
    return tokenizer([" ".join(x) for x in examples["answers.text"]])

tokenized_eli5 = eli5.map(
    preprocess_function,
    batched=True,
    num_proc=4,
    remove_columns=eli5["train"].column_names,
)

block_size = 128

def group_texts(examples):
    # Concatenate all texts.
    concatenated_examples = {k: sum(examples[k], []) for k in examples.keys()}
    total_length = len(concatenated_examples[list(examples.keys())[0]])
    # We drop the small remainder, we could add padding if the model supported it instead of this drop, you can
    # customize this part to your needs.
    if total_length >= block_size:
        total_length = (total_length // block_size) * block_size
    # Split by chunks of block_size.
    result = {
        k: [t[i : i + block_size] for i in range(0, total_length, block_size)]
        for k, t in concatenated_examples.items()
    }
    result["labels"] = result["input_ids"].copy()
    return result

lm_dataset = tokenized_eli5.map(group_texts, batched=True, num_proc=4)

from transformers import DataCollatorForLanguageModeling

tokenizer.pad_token = tokenizer.eos_token
data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)

Map (num_proc=4):   0%|          | 0/4000 [00:00<?, ? examples/s]Token indices sequence length is longer than the specified maximum sequence length for this model (1209 > 1024). Running this sequence through the model will result in indexing errors
Token indices sequence length is longer than the specified maximum sequence length for this model (1186 > 1024). Running this sequence through the model will result in indexing errors
Token indices sequence length is longer than the specified maximum sequence length for this model (2070 > 1024). Running this sequence through the model will result in indexing errors
Token indices sequence length is longer than the specified maximum sequence length for this model (1601 > 1024). Running this sequence through the model will result in indexing errors
Map (num_proc=4): 100%|██████████| 4000/4000 [00:00<00:00, 4995.93 examples/s]
Map (num_proc=4):   0%|          | 0/1000 [00:00<?, ? examples/s]Token indices sequence length is longer than the specified maximum sequence length for this model (1246 > 1024). Running this sequence through the model will result in indexing errors
Token indices sequence length is longer than the specified maximum sequence length for this model (1477 > 1024). Running this sequence through the model will result in indexing errors
Token indices sequence length is longer than the specified maximum sequence length for this model (2922 > 1024). Running this sequence through the model will result in indexing errors
Map (num_proc=4):  25%|██▌       | 250/1000 [00:00<00:00, 1413.37 examples/s]Token indices sequence length is longer than the specified maximum sequence length for this model (2575 > 1024). Running this sequence through the model will result in indexing errors
Map (num_proc=4): 100%|██████████| 1000/1000 [00:00<00:00, 3896.43 examples/s]
Map (num_proc=4): 100%|██████████| 4000/4000 [00:01<00:00, 3049.55 examples/s]
Map (num_proc=4): 100%|██████████| 1000/1000 [00:00<00:00, 5714.65 examples/s]

from transformers import AutoModelForCausalLM, TrainingArguments, Trainer

model = AutoModelForCausalLM.from_pretrained("distilbert/distilgpt2")

training_args = TrainingArguments(
    output_dir="my_awesome_eli5_clm-model",
    eval_strategy="epoch",
    learning_rate=2e-5,
    weight_decay=0.01,
    push_to_hub=True,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=lm_dataset["train"],
    eval_dataset=lm_dataset["test"],
    data_collator=data_collator,
    tokenizer=tokenizer,
)

trainer.train()

/var/folders/59/c32_bthx48jd9m2ym5m3tnpw0000j7/T/ipykernel_46283/2004268791.py:13: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `Trainer.__init__`. Use `processing_class` instead.
  trainer = Trainer(
 13%|█▎        | 500/3975 [04:26<30:20,  1.91it/s]

{'loss': 3.9787, 'grad_norm': 4.818574905395508, 'learning_rate': 1.748427672955975e-05, 'epoch': 0.38}

 25%|██▌       | 1000/3975 [08:52<25:56,  1.91it/s]

{'loss': 3.9528, 'grad_norm': 3.8537325859069824, 'learning_rate': 1.4968553459119497e-05, 'epoch': 0.75}

                                                   
 33%|███▎      | 1325/3975 [12:26<23:20,  1.89it/s]

{'eval_loss': 3.8328182697296143, 'eval_runtime': 41.027, 'eval_samples_per_second': 60.497, 'eval_steps_per_second': 7.58, 'epoch': 1.0}

 38%|███▊      | 1500/3975 [14:00<22:34,  1.83it/s]

{'loss': 3.913, 'grad_norm': 3.8344168663024902, 'learning_rate': 1.2452830188679246e-05, 'epoch': 1.13}

 50%|█████     | 2000/3975 [18:25<16:49,  1.96it/s]

{'loss': 3.8573, 'grad_norm': 4.064958572387695, 'learning_rate': 9.937106918238994e-06, 'epoch': 1.51}

import math

eval_results = trainer.evaluate()
print(f"Perplexity: {math.exp(eval_results['eval_loss']):.2f}")

100%|██████████| 311/311 [00:40<00:00,  7.62it/s]

Perplexity: 45.71

prompt = "Somatic hypermutation allows the immune system to"

from transformers import pipeline

generator = pipeline("text-generation", model="my_awesome_eli5_clm-model")
generator(prompt)

Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.

[{'generated_text': 'Somatic hypermutation allows the immune system to adapt to a more drastic response. When the immune system is overwhelmed, the immune system does NOT adapt to new stimuli (an event) and does not respond to new stimuli.This technique is called hyp'}]

from datasets import load_dataset
own_dataset = load_dataset("text", data_files={"train": ["llm-fine-tuning-docs/docs/README.md"]}, split="train")

own_dataset = own_dataset.train_test_split(test_size=0.2)

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("distilbert/distilgpt2")

def preprocess_function(examples):
    return tokenizer([" ".join(x) for x in examples["text"]])

tokenized_own_dataset = own_dataset.map(
    preprocess_function,
    batched=True,
    num_proc=4,
    remove_columns=own_dataset["train"].column_names,
)

block_size = 128

def group_texts(examples):
    # Concatenate all texts.
    concatenated_examples = {k: sum(examples[k], []) for k in examples.keys()}
    total_length = len(concatenated_examples[list(examples.keys())[0]])
    if total_length >= block_size:
        total_length = (total_length // block_size) * block_size
    # Split by chunks of block_size.
    result = {
        k: [t[i : i + block_size] for i in range(0, total_length, block_size)]
        for k, t in concatenated_examples.items()
    }
    result["labels"] = result["input_ids"].copy()
    return result

lm_dataset = tokenized_own_dataset.map(group_texts, batched=True, num_proc=4)

Map (num_proc=4):  19%|█▉        | 21000/110255 [00:00<00:01, 54426.58 examples/s]Token indices sequence length is longer than the specified maximum sequence length for this model (1498 > 1024). Running this sequence through the model will result in indexing errors
Map (num_proc=4):  37%|███▋      | 41000/110255 [00:00<00:01, 47593.35 examples/s]Token indices sequence length is longer than the specified maximum sequence length for this model (2635 > 1024). Running this sequence through the model will result in indexing errors
Token indices sequence length is longer than the specified maximum sequence length for this model (1434 > 1024). Running this sequence through the model will result in indexing errors
Map (num_proc=4):  44%|████▍     | 49000/110255 [00:01<00:01, 51860.43 examples/s]Token indices sequence length is longer than the specified maximum sequence length for this model (1265 > 1024). Running this sequence through the model will result in indexing errors
Map (num_proc=4): 100%|██████████| 110255/110255 [00:02<00:00, 53913.21 examples/s]
Map (num_proc=4):  76%|███████▌  | 21000/27564 [00:00<00:00, 33824.33 examples/s]Token indices sequence length is longer than the specified maximum sequence length for this model (1139 > 1024). Running this sequence through the model will result in indexing errors
Map (num_proc=4): 100%|██████████| 27564/27564 [00:00<00:00, 37830.87 examples/s]
Map (num_proc=4): 100%|██████████| 110255/110255 [00:06<00:00, 17160.73 examples/s]
Map (num_proc=4): 100%|██████████| 27564/27564 [00:01<00:00, 16234.17 examples/s]

from transformers import DataCollatorForLanguageModeling

tokenizer.pad_token = tokenizer.eos_token
data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)

from transformers import AutoModelForCausalLM, TrainingArguments, Trainer

model = AutoModelForCausalLM.from_pretrained("distilbert/distilgpt2")

training_args = TrainingArguments(
    output_dir="my_awesome_eli5_clm-model",
    eval_strategy="epoch",
    learning_rate=2e-5,
    weight_decay=0.01,
    push_to_hub=True,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=lm_dataset["train"],
    eval_dataset=lm_dataset["test"],
    data_collator=data_collator,
    tokenizer=tokenizer,
)

trainer.train()

/var/folders/59/c32_bthx48jd9m2ym5m3tnpw0000j7/T/ipykernel_2713/2004268791.py:13: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `Trainer.__init__`. Use `processing_class` instead.
  trainer = Trainer(
  2%|▏         | 500/25233 [04:33<3:39:04,  1.88it/s]

{'loss': 1.0723, 'grad_norm': 3.0989766120910645, 'learning_rate': 1.9603693575872867e-05, 'epoch': 0.06}

  4%|▍         | 1000/25233 [09:04<3:37:09,  1.86it/s]

{'loss': 0.8407, 'grad_norm': 2.9212241172790527, 'learning_rate': 1.9207387151745732e-05, 'epoch': 0.12}

  6%|▌         | 1500/25233 [13:36<3:35:21,  1.84it/s]

{'loss': 0.784, 'grad_norm': 2.680738687515259, 'learning_rate': 1.8811080727618597e-05, 'epoch': 0.18}

  8%|▊         | 2000/25233 [18:16<3:30:21,  1.84it/s]

{'loss': 0.7412, 'grad_norm': 3.1172192096710205, 'learning_rate': 1.8414774303491462e-05, 'epoch': 0.24}

 10%|▉         | 2500/25233 [22:51<3:35:08,  1.76it/s]

{'loss': 0.7058, 'grad_norm': 2.4785091876983643, 'learning_rate': 1.8018467879364324e-05, 'epoch': 0.3}

prompt = "Somatic hypermutation allows the immune system to"

from transformers import pipeline

generator = pipeline("text-generation", model="my_awesome_eli5_clm-model")
generator(prompt)

Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.

[{'generated_text': 'Somatic hypermutation allows the immune system to t i m e   |                                 '}]

from transformers import AutoModelForCausalLM, TrainingArguments, Trainer
from peft import LoraConfig, get_peft_model

model = AutoModelForCausalLM.from_pretrained("distilbert/distilgpt2").to("mps")

print(model)

GPT2LMHeadModel(
  (transformer): GPT2Model(
    (wte): Embedding(50257, 768)
    (wpe): Embedding(1024, 768)
    (drop): Dropout(p=0.1, inplace=False)
    (h): ModuleList(
      (0-5): 6 x GPT2Block(
        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
        (attn): GPT2SdpaAttention(
          (c_attn): Conv1D(nf=2304, nx=768)
          (c_proj): Conv1D(nf=768, nx=768)
          (attn_dropout): Dropout(p=0.1, inplace=False)
          (resid_dropout): Dropout(p=0.1, inplace=False)
        )
        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
        (mlp): GPT2MLP(
          (c_fc): Conv1D(nf=3072, nx=768)
          (c_proj): Conv1D(nf=768, nx=3072)
          (act): NewGELUActivation()
          (dropout): Dropout(p=0.1, inplace=False)
        )
      )
    )
    (ln_f): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
  )
  (lm_head): Linear(in_features=768, out_features=50257, bias=False)
)

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("distilbert/distilgpt2")

def preprocess_function(examples):
    return tokenizer([" ".join(x) for x in examples["text"]])

tokenized_own_dataset = own_dataset.map(
    preprocess_function,
    batched=True,
    num_proc=4,
    remove_columns=own_dataset["train"].column_names,
)

block_size = 128

def group_texts(examples):
    # Concatenate all texts.
    concatenated_examples = {k: sum(examples[k], []) for k in examples.keys()}
    total_length = len(concatenated_examples[list(examples.keys())[0]])
    if total_length >= block_size:
        total_length = (total_length // block_size) * block_size
    # Split by chunks of block_size.
    result = {
        k: [t[i : i + block_size] for i in range(0, total_length, block_size)]
        for k, t in concatenated_examples.items()
    }
    result["labels"] = result["input_ids"].copy()
    return result

lm_dataset = tokenized_own_dataset.map(group_texts, batched=True, num_proc=4)

from transformers import DataCollatorForLanguageModeling

tokenizer.pad_token = tokenizer.eos_token
data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)

batch_size = 32

training_args = TrainingArguments(
    output_dir="my_awesome_eli5_clm-model",
    eval_strategy="epoch",
    learning_rate=1e-5,
    weight_decay=0.0001,
    push_to_hub=False,
    per_device_train_batch_size=batch_size,
)

peft_config = LoraConfig(
        r=64,
        lora_alpha=128,
        lora_dropout=0.1,
        # bias="none",
        # task_type="CAUSAL_LM",
        # target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
)

print(f"Trainable parameters for the base model: {model.num_parameters(only_trainable=True)}")

model = get_peft_model(model, peft_config)

print(f"Trainable parameters for the LoRA model: {model.num_parameters(only_trainable=True)}")

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=lm_dataset["train"],
    eval_dataset=lm_dataset["test"],
    data_collator=data_collator,
    # tokenizer=tokenizer,
)

trainer.train()

Trainable parameters for the base model: 81912576
Trainable parameters for the LoRA model: 1179648

  0%|          | 38/50463 [00:43<16:01:46,  1.14s/it]
  8%|▊         | 500/6309 [13:36<2:34:54,  1.60s/it]
  8%|▊         | 500/6309 [13:36<2:34:54,  1.60s/it]

{'loss': 1.4383, 'grad_norm': 0.24702320992946625, 'learning_rate': 9.207481375812333e-06, 'epoch': 0.24}

 16%|█▌        | 1000/6309 [26:56<2:20:18,  1.59s/it]
 16%|█▌        | 1000/6309 [26:56<2:20:18,  1.59s/it]

{'loss': 1.2987, 'grad_norm': 0.4560306668281555, 'learning_rate': 8.414962751624663e-06, 'epoch': 0.48}

 24%|██▍       | 1500/6309 [40:04<2:09:28,  1.62s/it]
 24%|██▍       | 1500/6309 [40:04<2:09:28,  1.62s/it]

{'loss': 1.2303, 'grad_norm': 0.4419333040714264, 'learning_rate': 7.622444127436995e-06, 'epoch': 0.71}

 32%|███▏      | 2000/6309 [53:29<1:52:40,  1.57s/it]
 32%|███▏      | 2000/6309 [53:29<1:52:40,  1.57s/it]

{'loss': 1.1944, 'grad_norm': 0.6028203964233398, 'learning_rate': 6.829925503249327e-06, 'epoch': 0.95}

 33%|███▎      | 2103/6309 [56:14<1:49:34,  1.56s/it]


 33%|███▎      | 2103/6309 [1:01:00<1:49:34,  1.56s/it]

prompt = "Somatic hypermutation allows the immune system to"

from transformers import pipeline

generator = pipeline("text-generation", model="my_awesome_eli5_clm-model")
generator(prompt)

Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.

[{'generated_text': 'Somatic hypermutation allows the immune system to r a n g   n a m e   t o   s u p p o r t / h e a d l e r s   i n   S c'}]

from transformers import AutoModelForCausalLM, TrainingArguments, Trainer
from peft import LoraConfig, get_peft_model

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B").to("mps")

print(model)

Qwen2ForCausalLM(
  (model): Qwen2Model(
    (embed_tokens): Embedding(151936, 896)
    (layers): ModuleList(
      (0-23): 24 x Qwen2DecoderLayer(
        (self_attn): Qwen2SdpaAttention(
          (q_proj): Linear(in_features=896, out_features=896, bias=True)
          (k_proj): Linear(in_features=896, out_features=128, bias=True)
          (v_proj): Linear(in_features=896, out_features=128, bias=True)
          (o_proj): Linear(in_features=896, out_features=896, bias=False)
          (rotary_emb): Qwen2RotaryEmbedding()
        )
        (mlp): Qwen2MLP(
          (gate_proj): Linear(in_features=896, out_features=4864, bias=False)
          (up_proj): Linear(in_features=896, out_features=4864, bias=False)
          (down_proj): Linear(in_features=4864, out_features=896, bias=False)
          (act_fn): SiLU()
        )
        (input_layernorm): Qwen2RMSNorm((896,), eps=1e-06)
        (post_attention_layernorm): Qwen2RMSNorm((896,), eps=1e-06)
      )
    )
    (norm): Qwen2RMSNorm((896,), eps=1e-06)
    (rotary_emb): Qwen2RotaryEmbedding()
  )
  (lm_head): Linear(in_features=896, out_features=151936, bias=False)
)

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B")

def preprocess_function(examples):
    return tokenizer([" ".join(x) for x in examples["text"]])

tokenized_own_dataset = own_dataset.map(
    preprocess_function,
    batched=True,
    num_proc=4,
    remove_columns=own_dataset["train"].column_names,
)

block_size = 128

def group_texts(examples):
    # Concatenate all texts.
    concatenated_examples = {k: sum(examples[k], []) for k in examples.keys()}
    total_length = len(concatenated_examples[list(examples.keys())[0]])
    if total_length >= block_size:
        total_length = (total_length // block_size) * block_size
    # Split by chunks of block_size.
    result = {
        k: [t[i : i + block_size] for i in range(0, total_length, block_size)]
        for k, t in concatenated_examples.items()
    }
    result["labels"] = result["input_ids"].copy()
    return result

lm_dataset = tokenized_own_dataset.map(group_texts, batched=True, num_proc=4)

from transformers import DataCollatorForLanguageModeling

tokenizer.pad_token = tokenizer.eos_token
data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)

Map (num_proc=4): 100%|██████████| 110255/110255 [00:02<00:00, 49880.67 examples/s]
Map (num_proc=4): 100%|██████████| 27564/27564 [00:00<00:00, 42227.31 examples/s]
Map (num_proc=4): 100%|██████████| 110255/110255 [00:02<00:00, 43033.31 examples/s]
Map (num_proc=4): 100%|██████████| 27564/27564 [00:00<00:00, 41738.77 examples/s]

batch_size = 8

training_args = TrainingArguments(
    output_dir="my_awesome_eli5_clm-model",
    eval_strategy="epoch",
    learning_rate=1e-5,
    weight_decay=0.0001,
    push_to_hub=False,
    per_device_train_batch_size=batch_size,
)

peft_config = LoraConfig(
        r=64,
        lora_alpha=128,
        lora_dropout=0.1,
        # bias="none",
        # task_type="CAUSAL_LM",
        # target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
)

print(f"Trainable parameters for the base model: {model.num_parameters(only_trainable=True)}")

model = get_peft_model(model, peft_config)

print(f"Trainable parameters for the LoRA model: {model.num_parameters(only_trainable=True)}")

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=lm_dataset["train"],
    eval_dataset=lm_dataset["test"],
    data_collator=data_collator,
    # tokenizer=tokenizer,
)

trainer.train()

Trainable parameters for the base model: 494032768
'NoneType' object has no attribute 'cadam32bit_grad_fp32'
Trainable parameters for the LoRA model: 4325376

/Users/LSoica/work/AI/blog/.venv/lib/python3.12/site-packages/bitsandbytes/cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
  warn("The installed version of bitsandbytes was compiled without GPU support. "
  5%|▌         | 500/9615 [39:28<4:37:32,  1.83s/it]

{'loss': 2.0717, 'grad_norm': 3.0585858821868896, 'learning_rate': 9.479979199167968e-06, 'epoch': 0.16}

 10%|█         | 1000/9615 [3:29:29<24:59:46, 10.45s/it]

{'loss': 1.766, 'grad_norm': 3.2948787212371826, 'learning_rate': 8.959958398335933e-06, 'epoch': 0.31}

 16%|█▌        | 1500/9615 [5:47:00<116:44:43, 51.79s/it] /Users/LSoica/work/AI/blog/.venv/lib/python3.12/site-packages/peft/utils/other.py:689: UserWarning: Unable to fetch remote file due to the following error (ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')), '(Request ID: ee555c7e-30f5-4709-9944-a739ffe482a3)') - silently ignoring the lookup for the file config.json in Qwen/Qwen2.5-0.5B.
  warnings.warn(
/Users/LSoica/work/AI/blog/.venv/lib/python3.12/site-packages/peft/utils/save_and_load.py:243: UserWarning: Could not find a config file in Qwen/Qwen2.5-0.5B - will assume that the vocabulary was not modified.
  warnings.warn(

{'loss': 1.6554, 'grad_norm': 3.529787063598633, 'learning_rate': 8.439937597503902e-06, 'epoch': 0.47}

 21%|██        | 2000/9615 [8:24:12<81:16:02, 38.42s/it]

{'loss': 1.6076, 'grad_norm': 3.6108458042144775, 'learning_rate': 7.919916796671867e-06, 'epoch': 0.62}

 26%|██▌       | 2500/9615 [9:45:17<3:20:55,  1.69s/it]   /Users/LSoica/work/AI/blog/.venv/lib/python3.12/site-packages/peft/utils/other.py:689: UserWarning: Unable to fetch remote file due to the following error (ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')), '(Request ID: 67465ae8-bf4a-4ea0-9da0-bd78880fa4fd)') - silently ignoring the lookup for the file config.json in Qwen/Qwen2.5-0.5B.
  warnings.warn(
/Users/LSoica/work/AI/blog/.venv/lib/python3.12/site-packages/peft/utils/save_and_load.py:243: UserWarning: Could not find a config file in Qwen/Qwen2.5-0.5B - will assume that the vocabulary was not modified.
  warnings.warn(

prompt = "What is a LLM?"
messages = [
    {"role": "system", "content": "You are an AI assistant created by LS."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

Setting `pad_token_id` to `eos_token_id`:None for open-end generation.

A LLM is a program that can be used to generate human readable text based on a sequence of input tokens. It is a type of natural language processing (nlp) algorithm that can be used to generate human readable text based on a sequence of input tokens. LLMs are commonly used for a variety of tasks, such as text generation, question answering, and information retrieval. They are often used in natural language processing applications where a human user needs to be able to understand and interpret the generated text. LLMs can be trained on large amounts of text data to improve their performance and accuracy.

Fine-tuning a LLM on your own docs¶

Natural Language Processing¶

Hugging Face Datasets¶

Peft¶

Install libraries¶

Load the data¶

Preprocess¶

Train¶

Evaluate¶

Inference¶

Build our own dataset¶

Now let's fine-tune the model with QLoRA¶

Infer¶

Now let's fine-tune Qwen/Qwen2.5-0.5B model with QLoRA¶

Infer¶