How to use Matlab to download data from an AWS S3 bucket

38 views (last 30 days)
I have been using python scripts to download from a public Amzon S3 bucket to my local computer. I would like to create a Matlab script to do the same. So far my Matlab scripts do not work. The key steps in the python scripts are easy (see below)
import s3fs
import os
import numpy as np
fs = s3fs.S3FileSystem(anon=True)
fs.ls('s3://noaa-goes16/')
sourcestub='noaa-goes16/ABI-L2-CPSF/2020/238/' (gets a list of particular files in a folder)
files = np.array(fs.ls(sourcestr)) (skipped some steps but these are the essentials
Basically the python script goes to the bucket and logs on to the public account with no need for a password. Retrieves the files using a loop one file at a time.
How can I replicate this functionality in Matlab. The data in the AWS help provided by Mathworks talks mostly about uploading data to a personal S3 bucket (not FREE!) on AWS. The python script does not require me to pay any fees as it is a public account apparently owned by the US Govt

Accepted Answer

Yongjian Feng
Yongjian Feng on 11 Aug 2021
Try this?
https://www.mathworks.com/help/cloudcenter/ug/transfer-data-to-amazon-s3-buckets.html
  2 Comments
Stephen Forczyk
Stephen Forczyk on 11 Aug 2021
The data I am trying to download is public access I don't have an AWS account to download from python. So after reading the web page I don't see how I can complete entries such as
set AWS_ACCESS_KEY_ID="YOUR_AWS_ACCESS_KEY_ID"
set AWS_SECRET_ACCESS_KEY="YOUR_AWS_SECRET_ACCESS_KEY"
set AWS_DEFAULT_REGION="us-east-1"
Can the first two items be set to blank?
Yongjian Feng
Yongjian Feng on 11 Aug 2021
Maybe you need to set the first two env vars to be blank. Please give it a try.

Sign in to comment.

More Answers (0)

Categories

Find more on Downloads in Help Center and File Exchange

Products


Release

R2020a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!