Since this is one equation in one unknown, you can use fzero to solve this.
>> f = @(x) cos(x)*cos(3*x)-0.5707;
>> result = fzero(f, 0);
>> check = f(result)
However, it appears your calculator computes the cosine of angles in degrees. The cos function in MATLAB computes the cosine of an angle in radians. To compute the cosine of an angle in degrees in MATLAB, use the cosd function instead of cos.