advanced web scraping from Investing

5 views (last 30 days)
cristhian elguera
cristhian elguera on 31 Oct 2019
Hello!. I am doing a homework where I want to exctract information from trading a website. In this case investing.com
I want to build a live plot about an specific FX (example "AUD-CAD"). And for do that, I need to extract the values of candles. (for example every 5 secs)
https://m.investing.com/currencies/aud-cad
tradedd.png
(From image)
left = candles
right = values of every candles
to be able to see every value is necesary to do click on "time lapse", in this case 5 minutes. and I can see in information from page a new xhr type. The problem is the url from this new file. I can not enter to this URL, then this information not apear if I write ctrl+u (see HTML code).
I would like to know If there are a way for be able to extract this values automatically using web scraping.
more information about this response.
General:
Request URL: https://m.investing.com/instrument/services/getChart?volume_series=yes&events=no&candle_count=40&pair_interval=300&chart_type=candlestick&pair_id_for_news=0&pair_id=47&symbol=AUD%252FCAD&showECEvents=false
Request Method: GET
Status Code: 200 OK
Remote Address: 142.54.22.221:443
Referrer Policy: unsafe-url
Response Header:
Accept-Ranges: bytes
Age: 0
Cache-Control: no-store, no-cache, must-revalidate
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 840
Content-Security-Policy: upgrade-insecure-requests; block-all-mixed-content
Content-Type: application/json; charset=UTF-8
Date: Thu, 31 Oct 2019 20:24:03 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Server: nginx/1.16.1
Set-Cookie: ses_id=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; domain=m.investing.com
Vary: Accept-Encoding,User-Agent
Via: 1.1 varnish
X-Powered-By: PHP/7.1.8
X-Varnish: 1807797307
Request Headers:
Accept: application/json, text/javascript, */*; q=0.01
Accept-Encoding: gzip, deflate, br
Accept-Language: es-ES,es;q=0.9,en;q=0.8
Connection: keep-alive
Cookie: adBlockerNewUserDomains=1566366442; _ga=GA1.2.490180893.1566366443; __qca=P0-2008130179-1566366444101; _hjid=e4e61cb6-dac9-468b-90e7-881beebd50a4; _fbp=fb.1.1571268764166.678986027; __gads=ID=1f0f77699f259394:T=1571268765:S=ALNI_MauXf9rYg7Z-VvUMeshYUfPzibvgA; OB-USER-TOKEN=964583a6-3407-4116-ae8c-5c9198205262; _gid=GA1.2.2103139025.1572506355; PHPSESSID=r1ah8kjp16nvu5u2qi8bdoq2sc; StickySession=id.46489073211.560_m.investing.com_StickyMobile; searchedResultsMobile=[{"href":"/currencies/aud-cad","pairId":47,"title":"AUD/CAD","subtitle":"Australian Dollar Canadian Dollar","footerTitle":"Currency"}]; geoC=PE; fpros_popup_mob=1572555384; _gat=1; _gat_allMobileSitesTracker=1; _gat_allSitesTracker=1; nyxDorf=MjY0bmI0N3U2YmhnZzY4JGIyMGg2LzMwYmNlYg%3D%3D
Host: m.investing.com
Referer: https://m.investing.com/currencies/aud-cad
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36
X-Requested-With: XMLHttpRequest
Query String parameters:
volume_series: yes
events: no
candle_count: 40
pair_interval: 300
chart_type: candlestick
pair_id_for_news: 0
pair_id: 47
symbol: AUD%2FCAD
showECEvents: false

Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!