Bulk download tool for the MyMedia platform

Overview

MyMedia Bulk Content Downloader

This is a bulk download tool for the MyMedia platform.

USE ONLY WHERE ALLOWED BY THE COPYRIGHT OWNER. NOT AFFILIATED WITH ANYTHING/ANYONE/ANY ORGANISATION.

Usage

0. Prerequisites

To use this tool, you will have to have all of the following installed.

As long as they are installed correctly, you can put them wherever you want, I have not hardcoded any default locations.

a. Python 3

This tool is written in Python and you need to have Python installed for it to work.

You can get Python from its official website. Download the installer, then finish the install before continuing.

b. The Selenium Python package

This is what I use to interact with Google Chrome. It comes with its own Chrome installation, so no need to install it seperately, and it shouldn't change anything on your regular Chrome.

Open your favorite terminal emulator (see the FAQ for how to do this) and type the following commands

py -m ensurepip --upgrade

pip3 install selenium

c. ffmpeg

MyMedia stores videos in a way that makes them difficult to download and use. ffmpeg is a tool that is used (among other things) to convert video formats. This tool uses it to convert that format to mp4 files.

You can download and install ffmpeg from its official website. Please make sure it is in your "path". See the FAQ if you don't know what that means, or if you don't know how to put it in your path.

1. Download the tool

Secondary (right) click here then select "Save Link As" to download it. You can put it wherever you want, but if you put it in your Downloads folder, you can just go off the examples in this guide.

If you're using Mac and don't know what a secondary click is, just click the link normally, then save the resulting page with ⌘+S

2. Get a list of all the videos you want to download

Create a text file (not a Word document) of all the MyMedia URLs (the thingy in the address bar when you're watching the video) you want to get the videos for, one URL per line. Make a note of where you saved this file

3. Create a folder where all your downloaded videos will be put in

Again, make a note of where this folder is

4. Open a terminal emulator and navigate to where you downloaded the tool

Please see the FAQs if you don't know how to do this

5. Run the tool

On Windows (Using cmd)

Type .\mymedia-downloader.py (path to the list) (path to the folder to put all the videos with the trailing \)

Replace the bits in the brackets with the paths. You can drag-and-drop the file and the folder to the terminal if you don't know how to find the path to a file, you will just need to add a \ after the second one. (This not a regular slash, it is a backslah. You type it by pressing the key to the left of the shift key on most English keyboards.)

On Mac or Linux

Type ./mymedia-downloader.py (path to the list) (path to the folder to put all the videos with the trailing /)

Replace the bits in the brackets with the paths. You can probably drag-and-drop the file and the folder to the terminal if you don't know how to find the path to a file, you might need to add a / after the second one.

The reason I'm saying probably is because Linux comes in a variety of flavours, and yours might not support this. I assume if you're using Linux, you know what a path is and how to find one. Otherwise google "find absolute path in (name of your Linux distro)".

6. Log in with your UTORid and password

Wait for the Chrome window to pop up, then click Log In and authenticate. I do not ever see your UTORid and password, all the communication happens directly between you and the university. Please see the source code if you don't believe me.

7. Wait until the screen says "Done."

After that, you should see all the videos you wanted to download in the folder you provided. Unfortunately I have not implemented naming these files correctly, so they are just numbered from 0 onwards. I might come back at this and fix that at some point.

Frequently Asked Querstions

Why should I trust you to enter my UTORid and password?

Great question. In theory, you shouldn't. However, I have provided the source code and you can see if you speak Python that the login process is identical to what would happen if you used Google Chrome yourself. After you log in, I hide the Chrome window to make things more efficient and create less of an eye sore, and download all the videos. If you don't speak Python, please feel free to show this to a friend who does. (Hint: Anyone who studies anything adjacent to computer science, data science or engineering should know Python after their first semester.)

Are you affiliated with (inster organisation here)?

No. I am not. I created this tool on my own, all errors are mine only and I do not claim to represent anyone. This tool is not "official" in any way, shape or form.

I am affiliated with (insert organisation here) and want to yell at you for creating this.

I have made my absolute best to make sure I'm not violating any EULA's or trademarks etc. It does not exploit any vulnerabilities, it simply automates a task that would be very tedious to do by hand. If you disagree and think that I'm either breaking some law/agreement/etc. or if you think this tool will create harm, please contact me. I'm sure you can find my UofT email, email me there. If you can't, please create an issue here.

What is a terminal emulator and how do I open/use one?

Using a terminal is like having a text message conversation with your computer. You tell it (in a very specific format) to do things, press enter, and it does them for you, and replies with the result.

On Windows (cmd)

Open the start menu, then type cmd.exe and press enter. This will open "Command Prompt". You can type cd then where you want to go to, to navigate to a folder. Again, cou can drag-and-drop folders to the command prompt to place avoid having to type their "paths" out. For example, to go to your Downloads folder, you can type cd Downloads right after launching command prompt. You will notice the bit before where you type your command change from C:\Users\yourusername to C:\Users\yourusername\Downloads. If you have changed the location of your Downloads folder, this will not work. I trust if you know how to do that, you know how to use command prompt and can figure this out on your own (or just drag-and-drop the folder.)

On Mac

Read this. You can use the cd gommand to move around. For example, you can type cd ~/Downloads to go to your downloads folder

On Linux

If you run Linux, you probably know how to do this. If you don't, google "(your distribution name) how to use terminal".

How do I put something on my path?

Your path is where your computer looks for things to run.

On Mac/Linux

Run export (path to the thing you want to add to your path) in your terminal each time

On Windows

Read this and do it once. You can remove what you added when you're done if you'd like.

Owner
Ege Feyzioglu
Ege Feyzioglu
Library to scrape and clean web pages to create massive datasets.

lazynlp A straightforward library that allows you to crawl, clean up, and deduplicate webpages to create massive monolingual datasets. Using this libr

Chip Huyen 2.1k Jan 06, 2023
Scrapes mcc-mnc.com and outputs 3 files with the data (JSON, CSV & XLSX)

mcc-mnc.com-webscraper Scrapes mcc-mnc.com and outputs 3 files with the data (JSON, CSV & XLSX) A Python script for web scraping mcc-mnc.com Link: mcc

Anton Ivarsson 1 Nov 07, 2021
A Python web scraper to scrape latest posts from official Coinbase's Blog.

Coinbase Blog Scraper A Python web scraper to scrape latest posts from official Coinbase's Blog. IDEA It scrapes up latest blog posts from https://blo

Lucas Villela 3 Feb 18, 2022
A Scrapper with python

Scrapper-en-python Scrapper des données signifie récuperer des données pour les traiter ou les analyser. En python, il y'a 2 grands moyens de scrapper

Lun4rIum 1 Dec 05, 2021
原神爬虫 抓取原神界面圣遗物信息

原神圣遗物半自动爬虫 说明 直接抓取原神界面中的圣遗物数据 目前只适配了背包页面的抓取 准确率:97.5%(普通通用接口,对 40 件随机圣遗物识别,统计完全正确的数量为 39) 准确率:100%(4k 屏幕,普通通用接口,对 110 件圣遗物识别,统计完全正确的数量为 110) 不排除还有小错误的

hwa 28 Oct 10, 2022
Binance harvester - A Python 3 script to harvest data from the Binance socket stream and calculate popular TA indicators and produce lists of top trending coins

Binance harvester - A Python 3 script to harvest data from the Binance socket stream and calculate popular TA indicators and produce lists of top trending coins

68 Oct 08, 2022
Deep Web Miner Python | Spyder Crawler

Webcrawler written in Python. This crawler does dig in till the 3 level of inside addressed and mine the respective data accordingly

Karan Arora 17 Jan 24, 2022
A package designed to scrape data from Yahoo Finance.

yahoostock A package designed to scrape data from Yahoo Finance. Installation The most simple installation method is through PIP. pip install yahoosto

Rohan Singh 2 May 28, 2022
Divar.ir Ads scrapper

Divar.ir Ads Scrapper Introduction This project first asynchronously grab Divar.ir Ads and then save to .csv and .xlsx files named data.csv and data.x

Iman Kermani 4 Aug 29, 2022
A Python package that scrapes Google News article data while remaining undetected by Google.

A Python package that scrapes Google News article data while remaining undetected by Google. Our scraper can scrape page data up until the last page and never trigger a CAPTCHA (download stats: https

Geminid Systems, Inc 6 Aug 10, 2022
Scraping and visualising India's real-time COVID-19 data from the MOHFW dataset.

COVID19-WEB-SCRAPER Open Source Tech Lab - Project [SEMESTER IV] OSTL Assignments OSTL Assignments - 1 OSTL Assignments - 2 Project COVID19 India Data

AMEY THAKUR 8 Apr 28, 2022
This repo has the source code for the crawler and data crawled from auto-data.net

This repo contains the source code for crawler and crawled data of cars specifications from autodata. The data has roughly 45k cars

Tô Đức Anh 5 Nov 22, 2022
一款利用Python来自动获取QQ音乐上某个歌手所有歌曲歌词的爬虫软件

QQ音乐歌词爬虫 一款利用Python来自动获取QQ音乐上某个歌手所有歌曲歌词的爬虫软件,默认去除了所有演唱会(Live)版本的歌曲。 使用方法 直接运行python run.py即可,然后输入你想获取的歌手名字,然后静静等待片刻。 output目录下保存生成的歌词和歌名文件。以周杰伦为例,会生成两

Yang Wei 11 Jul 27, 2022
京东茅台抢购

截止 2021/2/1 日,该项目已无法使用! 京东:约满即止,仅限京东实名认证用户APP端抢购,2月1日10:00开始预约,2月1日12:00开始抢购(京东APP需升级至8.5.6版本及以上) 写在前面 本项目来自 huanghyw - jd_seckill,作者的项目地址我找不到了,找到了再贴上

abee 73 Dec 03, 2022
Current Antarctic large iceberg positions derived from ASCAT and OSCAT-2

Iceberg Locations Antarctic large iceberg positions derived from ASCAT and OSCAT-2. All data collected here are from the NASA SCP website Overview Thi

Joel Hanson 5 Jul 27, 2022
SearchifyX, predecessor to Searchify, is a fast Quizlet, Quizizz, and Brainly webscraper with various stealth features.

SearchifyX SearchifyX, predecessor to Searchify, is a fast Quizlet, Quizizz, and Brainly webscraper with various stealth features. SearchifyX lets you

28 Dec 20, 2022
HappyScrapper - Google news web scrapper with python

HappyScrapper ~ Google news web scrapper INSTALLATION ♦ Clone the repository ♦ O

Jhon Aguiar 0 Nov 07, 2022
自动完成每日体温上报(Github Actions)

体温上报助手 简介 每天 10:30 GMT+8 自动完成体温上报,如想修改定时运行的时间,可修改 .github/workflows/SduHealthReport.yml 中 schedule 属性。 如果当日有异常,请手动在小程序端/PC 端填写!

Teng Zhang 23 Sep 15, 2022
ChromiumJniGenerator - Jni Generator module extracted from Chromium project

ChromiumJniGenerator - Jni Generator module extracted from Chromium project

allenxuan 4 Jun 12, 2022
Free-Game-Scraper is a useful script that allows you to track down free games and DLCs on many platforms.

Game Scraper Free-Game-Scraper is a useful script that allows you to track down free games and DLCs on many platforms. Join the discord About The Proj

KursK 2 Mar 28, 2022