thx - yes, after reading myself what I have written, i need to ask for Sorry for writing it like it is :)

I think my problem is with the imagination and knowledge transfer:

As i remember, in the year ~2010 +/- 2-3y, machine learning became (again) POI for technology. I remember convolutional networks, deepQ, Genetic algorithms, etc being in the press. I remember Michael Schmidt, a biology student at that time (~roundabout~), being as "the influential data scientist of the year" at forbe's cover. MS did symbolic regression combined with genetic algorithm and founded nutopian that was developing this propriatary "eureqa" algorithm.

At that time, PyTorch was "created" by Facebook as an answer to google's tensorflow. I remember a guy trained on PyTorch a model that could play Nintendo's Mario. I wanted to understand how to do it, but it was toooo much for my brain. What i remember:

- one needs a gym that stands for the world the agent "lives in". The gym/world is data, like the input of a screenshot, or a gym/world for Stock Trading might have historical courses, prices etc.. also rewards, penalties are defined in the gym.

- the agent living in this gym/world have "actions" - left,right,buy, sell, ...

- RL training is then to let that agent "move through world/gym and perform some of its and agent's actions, where the outcome is rewarded or not. agent learns through rewards.."

Gym and RL training can be PyTorch or TensorFlow. In my understanding that gym is only a definition of what data is avaible, shape of data, etc. Then, the agent needs to be defined too - i remember Stable Baselines 3..

and then the RL training is just agent does x -> gym checks for outcome -> reward/penalty & adaption of weights -> inference -> repeat

This is cleary imaginable for me, even its not fully correct - there is an agent, there is a world, there is method to act&compare/data manipulation of weights - there is PyTorch that offers the tools to build a world, agent and to do the weight manipulations ..

OP have implementations of different Archs in PyTorch and my knowledge implodes. I cant imagine what is the "world/gym" in here? What is the data and actionables , whats the agent and whats its role/actions - and how would I use this implementations??

can I clone the repo, pick the one arch i want - and, when i would feed in data, after training, i will get the model that i can use for inference???

I definitely have a knot in my brain because of this!

Thank you for taking your time and trying to understand what my previous comment meant :) Are there some usefull sources for seeing how the learning is actually done (despite the tokenization) - and the skills training?? how are models trained to find the loophole that allows them to break out and reach their objectives "illegaly" - thats bugs my head since weeks!

OpenArch – PyTorch implementations of modern LLM architectures