Remove duplicate rows in CSV file

12 views (last 30 days)
hello dear mathworkers,
I have a dataset consist of approximatlly 4 millions records, and i want to remove the duplicated rows or records, can any one help me with the way, i am using matlab 2018a . thanks in advance
  7 Comments
madhan ravi
madhan ravi on 24 Jul 2019
Mohammed: Alex's solution should have solved your problem.

Sign in to comment.

Accepted Answer

Alex Mcaulley
Alex Mcaulley on 23 Jul 2019
Since all is numeric data, you can use:
data = xlsread('kdd.xlsx');
datanew = unique(data,'rows');
  2 Comments
Shameer Parmar
Shameer Parmar on 23 Jul 2019
This is not working, because non of data is similar.. I dont find duplicate entries in this sheet provided by Mohammad Alsajri..
using your command, the 'data' and 'datanew' both are getting exact same..
Alex Mcaulley
Alex Mcaulley on 23 Jul 2019
This code works!
I guess the excel provided by Mohammad is just a small portion of the dataset (4 million of rows).

Sign in to comment.

More Answers (0)

Categories

Find more on Language Fundamentals in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!