On Anytime Learning At Macroscale

Learning from sequential data dumps

(key) Requirements

Python 3.7
Pytorch 1.9.0
Hydra 1.1.0 (pip install hydra-core & pip install hydra-submitit-launcher)

Structure

├── crlapi           
  ├── benchmark.py    # Creates the data stream, feeds it to the model and evaluates it
  ├── core.py         # Abstract classes for 
  ├── logger.py   
  ├── sl
    ├── architectures
      ├── ...         # NN architectures used in this project
    ├── clmodels
      ├── ...         # Models (e.g. Single, gEns, ..., )
    ├── streams
      ├── ...         # CIFAR and MNIST stream implementatins

Running Experiments

To run experiments, you need to call the dataset specific run file, and you need to pass the configuration of the run. We have place the configurations in the previous directory (../configs). The config structure is as follows

    ├── configs
        ├── mnist
           ├── run.py                 # run file
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ├── test_finetune_mlp.yaml # This is the "Single Model"
           ... 
        ├── cifar
           ├── run.py                 # run file
           ├── test_finetune_vgg.yaml # This is the "Single Model"
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ...

To run an e.g. mnist gMoE run, the command is (launched from the directory just above (so cd ..)

PYTHONPATH=./ python configs/mnist/run.py -cn test_usage_gmoe n_megabatches=2 replay=1 clmodel.max_epochs=200

Important arguments

n_megabatches : controls the number of megabatches. So n_megabatches=1 is your regular full dataset training
replay : whether to use replay or not
clmodel.init_from_scratch : whether to reinitialize the model at every MB. Should only be used when replay=1
device : use cuda or cpu depending on your hardware

License

alma is released under the MIT license. See LICENSE for additional details about it. See also our Terms of Use and Privacy Policy.

Anytime Learning At Macroscale

Related tags

Overview

On Anytime Learning At Macroscale

(key) Requirements

Structure

Running Experiments

Important arguments

License

Owner

Meta Research

K-means clustering is a method used for clustering analysis, especially in data mining and statistics.

Python package for concise, transparent, and accurate predictive modeling

Polyglot Machine Learning example for scraping similar news articles.

Empyrial is a Python-based open-source quantitative investment library dedicated to financial institutions and retail investors

A logistic regression model for health insurance purchasing prediction

A linear equation solver using gaussian elimination. Implemented for fun and learning/teaching.

Metric learning algorithms in Python

A simple guide to MLOps through ZenML and its various integrations.

A simple machine learning python sign language detection project.

Implemented four supervised learning Machine Learning algorithms

Stock Price Prediction Bank Jago Using Facebook Prophet Machine Learning & Python

A single Python file with some tools for visualizing machine learning in the terminal.

A Time Series Library for Apache Spark

Generate music from midi files using BPE and markov model

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

Scikit learn library models to account for data and concept drift.

PyCaret is an open-source, low-code machine learning library in Python that automates machine learning workflows.

A library of sklearn compatible categorical variable encoders

scikit-multimodallearn is a Python package implementing algorithms multimodal data.

#30DaysOfStreamlit is a 30-day social challenge for you to build and deploy Streamlit apps.