Baking is a good metaphor, though lately it also feels a bit like making liquor. Distillation is a surprisingly important part of training.

after pre-training you have an llm that understands language and behaves a bit like gpt 3.5 or newer 'base' models

where it will be pretty good at predicting the next token

think: "What is the best city?" might continue with "What is the best programming language?" instead of answering the question

to increase chances of an answer you'd start with "The best city is"

(strong llms will even be able to do a conversation but they are not specifically trained for it yet)

in post-training the llm is trained with input/output pairs that nudge it further into the direction of a back and forth with users or into how it can use tools and so on

there is an art to both parts of training

the reason for why current models are so useful is because there was a lot of progress since gpt 3.5 in both pre- and post- training that got us to where we are now

(pls correct me if I got it wrong)

would love to hear from people familiar with pre- and post- re where you think future gains will more likely come from

It's really simple, simplier than I even expected.

The biggest win for AI dev efficiency is cutting down what gets loaded into context. Semantically matching tasks to the top tools helps a lot.