GPT

GPT-2

Just as good as BERT. But the selling point is Zero-shot Learning.

1.5B parameters (10x larger than GPt)

Next, see GPT-3.

model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")
 
prompt = "GPT2 is a model developed by OpenAI."
 
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
 
gen_tokens = model.generate(
    input_ids,
    do_sample=True,
    temperature=0.9,
    max_length=100,
)
gen_text = tokenizer.batch_decode(gen_tokens)[0]