The dynamics of representation learning in shallow, non-linear autoencoders

The package is written in python and uses the pytorch implementation to ML. Repository src contains the source files.

train_autoencoder.py
Trains (online and off-line) a shallow autoencoder both on a synthetic dataset and on benchmark datasets. Includes an implementation of the analytical equations allowing to track the dynamics of online training at all times. Included benchmark datasets : Cifar 10 (in gray scale) and FashionMNIST

Example of command to train online, integrating the analytical equations, on a synthetic dataset:
python3 train_autoencoder.py --D 500 --K 2 --dataset sinusoidal --analytical_updates 1

Example of command to train on finite dataset:
python3 train_autoencoder.py --dataset fmnist

truncated_vanilla_SGD.py
Trains an AE using different learning rules for reconstruction:

sanger's rule
vanilla SGD
truncated version of SGD introduced in the article Includes the implementation of the analytical equations tracking the dynamics of learning.

Example of command to train online, integrating the analytical equations, on a synthetic dataset:
python3 truncated_vanilla_SGD.py --K 5 --low_rank 5 --D 500 --analytical 1

The dynamics of representation learning in shallow, non-linear autoencoders

Related tags

Overview

The dynamics of representation learning in shallow, non-linear autoencoders

Owner

Maria Refinetti

Official git for "CTAB-GAN: Effective Table Data Synthesizing"

Active learning for Mask R-CNN in Detectron2

NaturalProofs: Mathematical Theorem Proving in Natural Language

Metric learning algorithms in Python

Supervised Sliding Window Smoothing Loss Function Based on MS-TCN for Video Segmentation

A Weakly Supervised Amodal Segmenter with Boundary Uncertainty Estimation

ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representation from common sense knowledge graphs.

[CVPR'22] Official PyTorch Implementation of Collaborative Transformers for Grounded Situation Recognition

Implementation of Auto-Conditioned Recurrent Networks for Extended Complex Human Motion Synthesis

Code for the KDD 2021 paper 'Filtration Curves for Graph Representation'

An Open Source Machine Learning Framework for Everyone

A framework for joint super-resolution and image synthesis, without requiring real training data

Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation

Distributed Evolutionary Algorithms in Python

Some methods for comparing network representations in deep learning and neuroscience.

The source code for the Cutoff data augmentation approach proposed in this paper: "A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation".

AI-UPV at IberLEF-2021 EXIST task: Sexism Prediction in Spanish and English Tweets Using Monolingual and Multilingual BERT and Ensemble Models

LF-YOLO (Lighter and Faster YOLO) is used to detect defect of X-ray weld image.

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning using 🤗 transformers

The code for SAG-DTA: Prediction of Drug–Target Affinity Using Self-Attention Graph Network.