Hi,
As per my understanding, you are using MATLAB's "comm.DPD" system object but you observed that the output of your amplifier remains highly nonlinear. This suggests that either:
- The chosen predistortion method does not match the amplifier's characteristics.
 - The predistorter is not trained adequately for the specific amplifier model.
 
Please refer to the following steps to resolve the issue:
Model Matching: Ensure the predistorter matches the nonlinearity model of the amplifier. For Rapp and Saleh models:
- Rapp model introduces AM-AM distortion (soft compression) with a parameter controlling the transition sharpness.
 - Saleh model introduces both AM-AM and AM-PM distortions.
 
Train the Predistorter for Adaptive Correction: Use the "comm.DPD" object, implement a feedback loop to iteratively correct errors. Please refer to the following code snippet:
dpd = comm.DPD('MemoryOrder', 3, 'PolynomialOrder', 5);
dpd.Weights = [initial_guess]; 
for iteration = 1:maxIterations
    x_predistorted = dpd(x_input);
    y_amplified = amplifier(x_predistorted);
    error = desired_output - y_amplified;
    dpd = adaptDPD(dpd, error, learningRate);
Once the predistortion works for your Saleh/Rapp models, you can export data from Cadence (input-output samples of the amplifier). Repeat the predistortion process.
Hope this helps!