how do i solve least squares problems with newtons method?

I was wondering if it is possible to solve least squares problems using matlab for 100% of the way. What i mean:

I wrote this code for finding the variables of the polynomial f(x)=a*x+b that minimize the sum of (yi-f(xi))^2 for the given x,y.

x=linspace(-5, 5,100);

y = [zeros(1,50) + 0.2*rand(1,50), ones(1,50)-0.2*rand(1,50)];

sol=[1;1];

sumx=0;

sumy=0;

sumxsq=0;

sumysq=0;

sumxy=0;

for i=1:100

sumx=sumx+x(i);

sumy=sumy+y(i);

sumxsq=sumxsq+x(i).^2;

sumysq=sumysq+y(i).^2;

sumxy=sumxy+x(i).*y(i);

F=[sol(1)*sumxsq+sol(2)*sumx-sumxy; sol(1)*sumx+101*sol(2)-sumy];

J=[sumxsq,sumx;sumx,101];

sol=sol-J\F;

endfor

sol

As you can see, I calculated F and J for the Newton-Raphson method on paper using the theory of least squares. But there is a problem with that, as it requires you to solve a lot of things on paper and it changes for each starting f(x) that you want to fit. I tried doing the same thing for a starting f that is given by: s(z)=1/(1+exp(-z)), f(x)=s(a*x+b), and i had trouble with the complexity of the equation as i couldnt find F and J easily. So, is there anything i missed in this example and its actually easy to calculate, or can i use newtons method in another way to solve this problem without calculating anything on paper. Thanks in advance and sorry of the long post!

[Torsten]

Look up Gauss-Newton method.

[apostolos georgantopoulos]

Is there no way to solve this problem with Newton-Raphson?

[Torsten]

If you have problems writing your own code, use one of the MATLAB solvers for nonlinear regression, e.g. lsqcurvefit.

[apostolos georgantopoulos]

It was specified that it should be solved using Newton-Raphson method (and later also the altered Newton-Raphson where you keep the Jacobian stable) on the book that i found this specific problem in and there are no similar examples. I don't mean to be too needy or anything, I'm just really baffled on how to solve this with this particular function whose derivatives are so complicated. Appreciate any info.

[Danae Katsanta]

I had to solve the excact problem. I found this one:

function s = jacobian(f, x, tol)

% f is a multivariable function handle, x is a starting point

if nargin == 2

tol = 10^(-5);

end

while 1

% if x and f(x) are row vectors, we need transpose operations here

y = x' - jacob(f, x)\f(x)'; % get the next point

if norm(f(y))<tol % check error tolerate

s = y';

return;

end

x = y';

end

function j = jacob(f, x) % approximately calculate Jacobian matrix

k = length(x);

j = zeros(k, k);

for m = 1: k

x2 = x;

x2(m) =x(m)+0.001;

j(m, :) = 1000*(f(x2)-f(x)); % partial derivatives in m-th row

end