quadrotor control with Q-learning

Hello,

I am trying to stabilise a quadrotor using q-learning and no simulink.

I have come accross some documentations in which the whole system was subdivided into four subsystems. One for the longitudinal, lateral,yaw and altitude. Each one on their continous state space form. I was told that if I manage the stabilise one of the subsystems, I would be able to stabilise the whole.

For example I am working on the longitudinal subsystem. And the state space matrices are given below. X(k+1)=AX(k)+BU(k)

Now before I start implementing anything, I started by discretizing the system, the applying the Q-learning, but i cannot manage to reach convergence. Does anyone has an idea on how I can speed up the learning and reach convergence?

Thank you

A=[0 1  0  0;
   0 0 9.8 0;
   0 0  0  1;
   0 0  0  0];

B=[0 ;0 ;0 ;Ix];

goal=[0 0 0 0];
if s==goal % where s is the next state
   reward= 100
else
   reward=-1*abs(x1)-1*abs(t1)-0.1*abs(t2)-0.1*abs(x2) % where x1=x, x2=dx, t1= theta, t2= dtheta
end