ASF Sentinel-1 Metadata Download tool

Related tags

Downloaderasfsmd
Overview

ASF Sentinel-1 Metadata Download tool

Copyright: 2021-2022 Antonio Valentino

Small Python tool (asfsmd) that allows to download XML files containing Sentinel-1 products metadata from the ASF archive.

Sentinel-1 products are stored in the ASF arcive as ZIP files that are quite large because they comntain both the products annotations and the binary image data.

The asfsmd tool is able to retrieve only the relatively samll annotation files (in XML format) without downloading the entire ZIP archive.

asfsmd exploits the httpio Python package for reading HTTP resources as random-access file-like objects. In order to do it the remote server must support the Range header.

This approach allows to open the ZIP archive remotely, inspects contents, and download only the pieces of data that are actually necessary to the user.

Performnces of this approach are quite poor but, in the specific case of Sentinel-1 products, the entire process results to be faster than downloading the entire ZIP archive and extracting only annotation files.

Command Line Interface

$ python3 asfsmd.py --help

usage: asfsmd [-h] [--version]
              [--loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
              [-q] [-v] [-d] [-f] [-o OUTDIR] [-u USERNAME] [-p PASSWORD]
              INPUT [INPUT ...]

ASF Sentinel-1 Metadata Download tool. Small Python tool (`asfsmd`) that
allows to download XML files containing Sentinel-1 products metadata from
the ASF archive. Sentinel-1 products are stored in the ASF arcive as ZIP
files that are quite large because they comntain both the products
annotations and the binary image data. The `asfsmd` tool is able to
retrieve only the relatively samll annotation files (in XML format) without
downloading the entire ZIP archive.

positional arguments:
  INPUT                 Sentinel-1 product name(s). If the '-f' flag is set
                        then the argument is interpreted as the filename
                        containing the list of products.

options:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        logging level (default: WARNING)
  -q, --quiet           suppress standard output messages, only errors are
                        printed to screen
  -v, --verbose         print verbose output messages
  -d, --debug           print debug messages
  -f, --file-list       read the list of products form file. The file is
                        expected to contain one product name per line.
  -o OUTDIR, --outdir OUTDIR
                        path of the output directory (default='.')
  -u USERNAME, --username USERNAME
                        username for ASF authentication. If not provided
                        the tool attempts to retireve the authentication
                        parameters for the user's '.netrc' file looking
                        for the host 'urs.earthdata.nasa.gov'
  -p PASSWORD, --password PASSWORD
                        password for ASF authentication. If not provided
                        the tool attempts to retireve the authentication
                        parameters for the user's '.netrc' file looking
                        for the host 'urs.earthdata.nasa.gov'
  --block-size BLOCK_SIZE
                        httpio block size in bytes (default: 1024)

License

The asfsmd package is distributed under the MIT License.

You might also like...
Tool to download Netflix in 4k

Netflix-4K-Script Tool to download Netflix in 4k You will need to get a L1 CDM that is whitelsited with Netflix CDM In this script are downgraded

A cli tool to download purchased products from the DLsite.

dlsite-downloader A cli tool to download purchased products from the DLsite. How can I use? This program runs with configurations defined at settings.

Tool To download 4KHDR DV SDR from AppleTV
Tool To download 4KHDR DV SDR from AppleTV

# APPLE-TV 4K Downloader Tool To download 4K HDR DV SDR from AppleTV Hello Fellow Developers/ ! Hi! My name is WVDUMP. I am Leaking the scripts to

The tool allows to download a list of tiktok sounds

dependencies: pip install requests how to use LAUCH THE PROGRAM file (option f)

Tool To download Amazon 4k SDR HDR 1080, CDM IS Not Included

WV-AMZN-4K-RIPPER Tool To download Amazon 4k SDR HDR 1080, CDM IS Not Included For CDM You can Mail :- [email protected] Note : CDM is not free L

Tool To download - Amazon - Netflix- Disney+ - VideoLand - Boomerang - RTE.ie
Tool To download - Amazon - Netflix- Disney+ - VideoLand - Boomerang - RTE.ie

vinetrimmer Widevine Decryption Script for Python Modules Amazon Netflix (with [email protected] support) Disney+ VideoLand Boomerang RTE.ie Hello Fellow Dev

A modern CLI to download animes automatically from Twist
A modern CLI to download animes automatically from Twist

Kurby Kurby is a nice and simple CLI that use Twist website, and their huge collection to download animes for free and automatically Animes from Twist

Download courses from khanacademy.org

khan-dl A python script to download courses from Khan Academy using youtube-dl and beautifulsoup4.

1Fichier Download Manager.
1Fichier Download Manager.

1fichier-dl 1Fichier Download Manager. Features ⭐ Manage your downloads ⭐ Bypass time limits Credits All icons, including the app icon, were provided

Comments
  • Consider alternative backends to httpio

    Consider alternative backends to httpio

    The httpio package is very small seems to fit quite well the needs of this project but:

    • it seems to be a not-active project (last commit 2019)
    • it is necessary to have a specialisation of the httpio.SyncHTTPIOFile class to be able to use this package in asfsmd

    Other alternatives exists that could replace httpio as shall be assessed:

    • [X] RemoteZip

      Very interesting, it can be used out of the box and allows to simplify the codebase quite a lot. The performance to get the list of files included in the ZIP archive seems to be better (x2) than the ones that one can get with httpio but the test consisting in the download of annotations of 5 S1 products seems to be 12% slower than httpio.

    • [X] pyremotezip

      Not maintained upstream, Python2 only.

    • [X] smart-open

      Interesting project bur slower (x3-x4) than httpio to retrieve the list of files contained in the ZIP archive.

    • [X] fsspec

      Extremely interesting and feature complete. Used also in Dask. Direct use of the ZipFileSystem is very slow. The use of the HttpFileSystem + zipfile from the Python stdlib seems to work fine, but it is 33% slower than httpio

    opened by avalentino 2
  • Add filter for polarization and additional calibration/noise/rfi downloads

    Add filter for polarization and additional calibration/noise/rfi downloads

    This adds two functions:

    1. Filter the download by polarization (e.g. so you can grab only 3 'vv' files instead of the 6 for vv/vh)
    2. Download the calibration, noise, and RFI xml files

    I also made a separate branch to further filter to a single subswath by checking iw{number}, but these are the additions I've already found useful when testing https://github.com/opera-adt/s1-reader

    opened by scottstanie 1
Releases(v1.3.0)
  • v1.3.0(Dec 18, 2022)

  • v1.2.0(Dec 5, 2022)

    • Refactoring to convert the `asfsmd.py`` module into a package.
    • Support multiple backends for remote file access: httpio, fsspec, remotezip. The httpio based implementation seems to be slightly faster w.r.t. the other ones.
    • Fix issue with the management of default values for the make_patterns function.
    • Improve the management of the download of large files (chunking and progress).
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Dec 3, 2022)

    • Now it is possible to customize the selection of files to be downloaded. Beyond the manifest and the annotation files, now it is also possible to download:

      • calibration annotations
      • noise annotations
      • rfi annotations
      • measurement files

      Moreover now it is possible to select a specific beams or polarizations. Patch developed by @scottstanie and @avalentino.

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Dec 3, 2022)

Owner
Antonio Valentino
Antonio Valentino
A scriptable music downloader for Qobuz, Tidal, and Deezer

streamrip A scriptable stream downloader for Qobuz, Tidal, and Deezer. Features Downloads tracks, albums, playlists, discographies, and labels from Qo

967 Jan 03, 2023
Downloads state flags from wikipedia for states/regions from all countries

world-state-flags Downloads state flags from wikipedia for states/regions from all countries This data is NOT curated Uses https://github.com/dr5hn/co

João Ribeiro Bezerra 2 Dec 15, 2022
YouTube to MP3 or 4, you get to choose...

UTubeToMP YouTube to MP3 or 4, you get to choose... If you don't wanna git clone andor dont wanna install python. Here: Repl.it Instructions: Pretty s

1 Jan 29, 2022
AI Dungeon Catalog Archive Toolkit

AI Dungeon Content Archive Toolkit (AID CAT) AID CAT is a command-line utility that will allow you to download JSON backups of: Your private and publi

Mimi 31 Oct 26, 2022
Tool To download - Amazon - Netflix- Disney+ - VideoLand - Boomerang - RTE.ie

vinetrimmer Widevine Decryption Script for Python Modules Amazon Netflix (with [email protected]

9 Dec 31, 2021
Download songs and playlists from Spotify for free!

spotify-to-mp3-converter You can basically understand the process with just this image but for clarity, these are the steps. Before using the exe down

2 Jan 25, 2022
Code for "Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions"

Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions Codebase for the "Adversarial Motion Priors Make Good Substitutes for Com

Alejandro Escontrela 54 Dec 13, 2022
Download h3t4y for later read

h3nt4y_dl Download h3nt4y for later read Tải h3nt4y về đọc thôi nào các bạn ơiiiiiiii! (Tải từ h**taivn nhé) Usage: python get_that_ht4i.py New versio

1 Dec 03, 2021
apkizer is a mass downloader for android applications for all available versions.

apkizer apkizer collects all available versions of an Android application from apkpure.com Purpose Sometimes mobile applications can be useful to dig

Kamil Onur Özkaleli 41 Dec 16, 2022
Aline file downloader automator!

AlineDorker Aline is used for donwloading files with google dorking , dowloading specified links such as dorks. Dependences: python3 installed pip ins

27 Nov 16, 2022
Downloads photos you saved from a specific profile.

instagram-download-saved A python script that downloads photos you saved from a specific profile. The only dependency is instaloader, an open-source p

Aviv 1 Dec 19, 2021
Gogoanime-dl - Gogoanime downloader for downloading anime.

gogoanime-dl With this script, you can download episodes of your favorite anime from Gogoanime. The current site that's developed against is https://w

1 Jan 06, 2022
This repository contains code for a youtube-dl GUI written in PyQt.

youtube-dl-GUI This repository contains code for a youtube-dl GUI written in PyQt. It is based on youtube-dl which is a Video downloading script maint

M.Yasoob Ullah Khalid ☺ 191 Jan 02, 2023
A manga download script written in python.

manga-dlp python script to download mangas Description A manga download script written in python. It only supports mangadex.org for now. But support f

Ivan Schaller 15 Nov 28, 2022
Pantheon - The fastest YouTube downloader.

A Youtube downloader written in Python3, using HTTP requests and an API.

Billy 38 Nov 21, 2022
Youtube-music - Youtube music with python

youtube-music fzf on https://github.com/junegunn/fzf python3 ytb.py [no/yes] yes

direskyfer 0 Feb 03, 2022
Download the resources of the Blue Archive easily!

blue-archive-bundle-downloader Download the resources of the Blue Archive easily! Known issue In Windows It works only if the console is "fullscreen"

Ryu juheon 7 Apr 08, 2022
This is a Text Data Analysis Project Involving (YouTube Case Study).

Text_Data_Analysis This is a Text Data Analysis Project Involving (YouTube Case Study). Problem Statement = Sentiment Analysis. Package1: There are m

1 Mar 05, 2022
Tool To download 4KHDR DV SDR from AppleTV

# APPLE-TV 4K Downloader Tool To download 4K HDR DV SDR from AppleTV Hello Fellow Developers/ ! Hi! My name is WVDUMP. I am Leaking the scripts to

5 Dec 25, 2021
A toolkit to automatically crawl the paper list and download paper pdfs of ACL Ahthology.

ACL-Anthology-Crawler A toolkit to automatically crawl the paper list and download paper pdfs of ACL Anthology

Ray GG 9 Oct 09, 2022