gunzip.m is very slow

6 views (last 30 days)
mathworks2011
mathworks2011 on 26 Oct 2011
Commented: Reema Alhassan on 15 Jul 2018
I am running MATLAB 2010b, win7, 64 bit, quod core, 16GB RAM.
I have a large number of csv.gz files, between 200MB and 6GB in size on my local drive.
myPathData = C:\myFile.csv.gz
I need to extract the file. using tic/toc, for a 200MB file
f = char(gunzip(myPathData));
the time taken is 152 minutes!
This is too slow for me.
I can manually extract the file using WINRAR in 13 mins. I try and call winrar from MATLAB but it fails:
cd('C:\Program Files\WinRAR\');
[status,result] = system(['UnRAR.exe' ' ' 'e' ' ' myPathData]);
cd('V:\MatlabCode');
with the error message "csv.gz is not RAR archive" and "No files to extract".
I see the problem has also been posed here:
Does anyone know how I can dramatically speed up the unzipping? (My only thought is to write something in C# (like below) and then call that exe from system.m). Surely there is a better way?
Many Thanks!
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.IO.Compression;
namespace ohMyGodICantBelieveIt
{
public class unzipper
{
//http://msdn.microsoft.com/en-us/library/system.io.compression.gzipstream.aspx
public static void Decompress(FileInfo fi)
{
// Get the stream of the source file.
using (FileStream inFile = fi.OpenRead())
{
// Get original file extension, for example "doc" from report.doc.gz.
string curFile = fi.FullName;
string origName = curFile.Remove(curFile.Length - fi.Extension.Length);
//Create the decompressed file.
using (FileStream outFile = File.Create(origName))
{
using (GZipStream Decompress = new GZipStream(inFile, CompressionMode.Decompress))
{
// Copy the decompression stream into the output file.
Decompress.CopyTo(outFile);
}
}
}
}
}
}

Accepted Answer

mathworks2011
mathworks2011 on 1 Nov 2011
no one replied:(
I wrote my own stuff in C#. It now takes around 3 mins and not 152...
  1 Comment
Reema Alhassan
Reema Alhassan on 15 Jul 2018
Hi could you share your code of C ? Thank you

Sign in to comment.

More Answers (0)

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!