Data to train RL agent (PPO)

I have 2 arrays which are 8001x2 size. one is input and other is output array.

now can i use these two arrays to train my RL agent ? (PPO agent)

i saw the example of using data to train RL agent on mathworks site but their data contains state actions rewards and all the other information as well. is it not possible with just the input and output array to train my RL agent ?

[Ayush Aniket]

Hi Sourabh,

Can you elaborate on the problem you are trying to solve using your dataset?

From your description, it seems that you already have an output data that you would want a Machine Learning Model to learn based on the input. This falls under the category of Supervised Learning. To accomplish such task there are several other functions available in MATLAB.

RL is used for un-supervised learning tasks, wherein the training data has to be in the form of experience such that the ML model (RL agent) can learn by interacting with the environment and observing its response.

[Sourabh]

I was using tf model before and now i want to use output data which i obtained from giving a step input to same tf model. That is the only difference.

Can't i replace tf model with data of same tf model??

Answer 1

[Shivansh]

Hi Sourabh,

The Proximal Policy Optimization (PPO) agent or any other Reinforcement Learning agent unlike supervised learning requires explicit states, actions, and rewards etc.

You can try to extract the information from the output arrays regarding the rewards and actions but that might require customising the entire Reinforcement learning model and you might not be able to use any existing example model.

I will recommend to use the Reinforcement Learning after properly modelling the problem for desired results. If you have the inputs and labels, you can try using a supervised learning model.

I hope it helps!