How can I increase timeout when using readtable to get csv data from url?
16 views (last 30 days)
Show older comments
Peter Valent
on 12 Dec 2023
Commented: Peter Valent
on 12 Dec 2023
I want to use readtable function to read time series csv file from internet via an url. It works for smaller datasets but when I try to download longer datasets I get an error that the server is not responding and that it might be wise to increase the timeout from 5s to a larger number. How can I do it? I can do it with webread function but not with readtable.
Example that works (short dataset):
url = 'https://dataset.api.hub.geosphere.at/v1/station/historical/klima-v1-1h?parameters=RSX&start=1976-01-01T00%3A00%3A00.000Z&end=1976-12-31T23%3A00%3A00.000Z&station_ids=710&output_format=csv&filename=RH_710';
% Options for reading
opts = delimitedTextImportOptions("NumVariables",3,'Delimiter',{','});
opts.Encoding = 'latin1';
opts.VariableNames = ["dateS", "stationID","dataV"];
opts.VariableTypes = ["string","double","double"];
opts.ExtraColumnsRule = "ignore";
opts.EmptyLineRule = "read";
opts.ConsecutiveDelimitersRule = "join";
% Read data using the identified decimal separator
opts.DataLines = [2,Inf];
opts.VariableTypes = ["string","double","double"];
dataTb = readtable(url,opts);
Example that does not work (long dataset):
url = 'https://dataset.api.hub.geosphere.at/v1/station/historical/klima-v1-1h?parameters=RSX&start=1976-01-01T00%3A00%3A00.000Z&end=2020-12-31T23%3A00%3A00.000Z&station_ids=710&output_format=csv&filename=RH_710';
% Options for reading
opts = delimitedTextImportOptions("NumVariables",3,'Delimiter',{','});
opts.Encoding = 'latin1';
opts.VariableNames = ["dateS", "stationID","dataV"];
opts.VariableTypes = ["string","double","double"];
opts.ExtraColumnsRule = "ignore";
opts.EmptyLineRule = "read";
opts.ConsecutiveDelimitersRule = "join";
% Read data using the identified decimal separator
opts.DataLines = [2,Inf];
opts.VariableTypes = ["string","double","double"];
dataTb = readtable(url,opts);
Thank you
0 Comments
Accepted Answer
Jasper Gerritsen
on 12 Dec 2023
You can have webread use your readtable options and increase the Timeout variable to e.g. 20 seconds as follows:
myReadTable = @(x) readtable(x, opts);
webopts = weboptions('ContentReader', myReadTable, 'Timeout', 20);
dataTb = webread(url, webopts);
Kind regards
More Answers (0)
See Also
Categories
Find more on Web Services in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!