How to related PCA output to the original data?

Hello,

I'm new to PCA and I would like to learn the outcome of pca function. I have read the document and checked others works but I'm a bit confused on how to related the results (wcoeff, latent, explained) to the original data. For example, I'm using the example from the document as below. I understand the (wcoeff) presents the eigenvector vectors. the (latent) presents the eigenvalues. the (explained) is the percentage of the total variance explained by each principal component. NOW, how are all these information are related to the main data which is the ingredients here? how do I know from looking at the results in (explained) that the 55 % is related to which variables or columns in the ingredients matrix?

load hald
[wcoeff,~,latent,~,exp
lained] = pca(ingredients,'VariableWeights','variance')

wcoeff = 4×4

-2.7998 2.9940 -3.9736 1.4180

-8.7743 -6.4411 4.8927 9.9863

2.5240 -3.8749 -4.0845 1.7196

9.1714 7.5529 3.2710 11.3273

latent = 4×1

2.2357

1.5761

0.1866

0.0016

explained = 4×1

55.8926

39.4017

4.6652

0.0406

[Adam]

The 55% is related to the first eigenvector, not to any of the original input variables. You can relate back to your original data via the wcoeff matrix whose rows correspond to the original variables.

Also the 'score' output which is missing from there gives you the mapped data from the original variables onto the new coordinate system of principal components.

[Yaser Khojah]

Dear Adamn

I have calcauted the scores as below:

score =

1.4672 1.9030 0.5300 0.0385

2.1358 0.2384 0.2902 -0.0298

-1.1299 0.1839 0.0107 -0.0937

0.6599 1.5768 -0.1792 -0.0331

-0.3588 0.4835 0.7401 0.0192

-0.9666 0.1699 -0.0857 -0.0122

-0.9307 -2.1348 0.1730 0.0083

2.2321 -0.6917 -0.4597 0.0226

0.3515 -1.4322 0.0316 -0.0450

-1.6625 1.8281 -0.8512 0.0198

1.6402 -1.2951 -0.4942 0.0314

-1.6926 -0.3922 0.0198 0.0372

-1.7457 -0.4375 0.2746 0.0368

I understand the PCA helps to reduce the dimensionality. So, how can I know which variables from the main data has the largest impact? can you please show me which column represent the values in explained? This is really important part for me and I have not found any answer for it.

[Adam]

If you normalise the eigenvectors then the magnitude of each component represents the relative contribution of the original variable of the same index.

[Yaser Khojah]

Can you please illustrate that in my example?

According to the manual: Each column of coeff contains coefficients for one principal component, and the columns are in descending order of component variance.They are not relative to the original data?

[Adam]

Yes, each column is a Principal component and each row is the contribution of the original data variable to that Principal component. So the first row of coeff is the contribution of the first input variable to each Principal Component.

[Yaser Khojah]

I guess you did not understand my qeustion? Another way to ask.

Which variables from my dataset are related to PC1, PC2, PC3 and PC4?

For example:

explained = 4×1

55.8926 Which variable?

39.4017 Which variable?

4.6652 Which variable?

0.0406 Which variable?

Here is the explinaiton of each componete which relates to PC and nothing is related to original data in terms of which variables are related to the PC.

• coeff: contains coefficients for one principal component, and the columns are in descending order of component variance
• score: Rows of score correspond to observations, and columns correspond to components.
• explained: the percentage of the total variance explained by each principal component
• latent: Principal component variances, that is the eigenvalues of the covariance matrix of X, returned as a column vector.

I tried the following:

load hald
[coeff,score,latent,tsquared,explained] = pca(ingredients)

coeff =
   -0.0678   -0.6460    0.5673    0.5062
   -0.6785   -0.0200   -0.5440    0.4933
    0.0290    0.7553    0.4036    0.5156
    0.7309   -0.1085   -0.4684    0.4844

    score =
   36.8218   -6.8709   -4.5909    0.3967
   29.6073    4.6109   -2.2476   -0.3958
  -12.9818   -4.2049    0.9022   -1.1261
   23.7147   -6.6341    1.8547   -0.3786
   -0.5532   -4.4617   -6.0874    0.1424
  -10.8125   -3.6466    0.9130   -0.1350
  -32.5882    8.9798   -1.6063    0.0818
   22.6064   10.7259    3.2365    0.3243
   -9.2626    8.9854   -0.0169   -0.5437
   -3.2840  -14.1573    7.0465    0.3405
    9.2200   12.3861    3.4283    0.4352
  -25.5849   -2.7817   -0.3867    0.4468
  -26.9032   -2.9310   -2.4455    0.4116
latent =
  517.7969
   67.4964
   12.4054
    0.2372
tsquared =
    5.6803
    3.0758
    6.0002
    2.6198
    3.3681
    0.5668
    3.4818
    3.9794
    2.6086
    7.4818
    4.1830
    2.2327
    2.7216
explained =
   86.5974
   11.2882
    2.0747
    0.0397
    
% compared against:
covarianceMatrix = cov(ingredients);
[V,D] = eig(covarianceMatrix);

V =
    0.5062    0.5673    0.6460   -0.0678
    0.4933   -0.5440    0.0200   -0.6785
    0.5156    0.4036   -0.7553    0.0290
    0.4844   -0.4684    0.1085    0.7309
D =
    0.2372         0         0         0
         0   12.4054         0         0
         0         0   67.4964         0
         0         0         0  517.7969

So D here I believe is matching the covarianceMatrix which its is column meets the orignal data in (ingredients). In addtion, the first row in D represent the first column in covarianceMatrix and so on. However, latent order is not matching D. I know it matches the PC only and how someone like me can relate the laten output to the original data (ingredients)

[Adam]

coeff.^2

will give you the proportions of each original variable for each principal component.

You can then possibly use the latent values to normalise this against the eigenvalues for each of those principal components.

[Gabrielle Hedrick]

I am reading this thread because I am wondering the same thing. Adam, just to make sure I understand your answer, if I have my original data set as a 2074x6, where I have 6 variables (columns) and 2074 observations (rows), and get the following coeff matrix:

It does mean that variable 5 contributes the most to PC1, and variable 1 to PC2, and so on, right?

[Adam]

Yes, those are your eigenvectors down the columns, so consider each one as a vector in 6 dimensional space. Usually they would be normalised, but the relative values still give the relative strengths and variable 5 contributes to that overall vector more than all the others, e.g. ~3.7 times that of variable 1

[Gabrielle Hedrick]

Thank you!

[Salma Hassan]

i don't understand variable 2 has 11.3 it is greater than variable 5

plz explain this...thank you

[Adam]

The variables are in the rows of the coeff matrix. The columns are the principal components eigenvectors.

[NN]

i have gone through the discussion to understand PCA .I am doing a forecasting problem with neural network and used the below syntax for finding out PCA components for reducing the dimension of training and testing data .

coeff = pca(X)

Can i use the output of this command (coeff matrix) as new training and testing data for neural network ad use it for forecasting?

I have 9 input features for forecasting.How can i plot the contribution rates of each feature and prinicpal components against the variance to know the contribution of features and dimension reduction?

kindly help

[Paul Gershon]

What about negative numbers in the a specific column of the PCA matrix? 5/-2 = -2.5 and -5/2 = -2.5 also. How to assess?

[Paul Gershon]

Is the answer to square all numbers in the coeff matrix and then do the division for relative contribution? For example:

PC1 PC2

var1: -2 0.3

var2: 5 3.8

Square it:

PC1 PC2

var1: 4 0.09

var2: 25 14.4

Now relative contribution of var2 to PC1 = 25/4? or is it 25/29? (the denominator is the sum for the column)?

Thanks!

[Paul Gershon]

..Or is the answer not to square the coefficients, but simply to take absolute values because sign simply shows whether the point is pushed negative on the PC axis by the sign of the coefficient, but the sign is irrelevant to magnitude of contribution?

Thanks!