how to use urlread in post method

5 views (last 30 days)
Ingrid
Ingrid on 11 Dec 2014
Answered: Ingrid on 16 Dec 2014
I have a website from which I want to download data. However, the url-link only refers to a form on which two parameters needs to be filled in. I checked with the Web Developer Toolbox that these parameter request are sent through the POST method so they do not show up in the url-link.
So I thought I would just use the following command in matlab:
x = urlread('http://www.vmm.be/webrap/ibmcognos/cgi-bin/cognosisapi.dll?b_action=cognosViewer&ui.action=run&ui.object=%2fcontent%2ffolder[%40name%3d%27Water%27]%2ffolder[%40name%3d%27Afvalwater%27]%2freport[%40name%3d%27Individuele%20analyseresultaten%20per%20RWZI%27]&ui.name=Individuele%20analyseresultaten%20per%20RWZI&run.outputFormat=HTML&run.prompt=false&ui.backURL=%2fwebrap%2fibmcognos%2fcgi-bin%2fcognosisapi.dll%3fb_action%3dxts.run%26m%3dportal%2fcc.xts%26m_folder%3di5DDA04E5A00C4B6AB6DF44BB4FAD7CEC&p_RwziNr=51&run.prompt=false','post',{'p_Jaar';'2010';'p_Prompt Rapportpakket';'AW - debiet'});
Why is this not working? I got the parameter names from "Display Form Details" The difference I have noticed with the example given with urlread is that when checking the "View Form Information" the type is not text as expected but input. This is probably because the input needs to be chosen from a dropdown list, but I have no idea how I can incorporate this in the matlab command so I hope someone here can help me with this because I really need to download a lot of data (say for 500 stations, each 10 years so you can imagine I do not want to manually copy paste this!)
I already tried to search for many keywords on this problem and one of the things that popped up was multipart/form-data but I do not really know if this is applicable to my case and if so how I can use this information
The link given as an example is for one example station (namely number 51), I already know that I can go to different stations by just changing this number so I think this should not pose a problem

Accepted Answer

Ingrid
Ingrid on 16 Dec 2014
Turns out it was not possible to do what I wanted in Matlab so I had to migrate to Selenium-C#-chrome to get the job done. In this code I can extract the data and write to a .txt file from which I can read in the data easily in Matlab to do the necessary post-processing.

More Answers (0)

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!