Long Expressive Memory (LEM)

Last update: Dec 17, 2022

Overview

Long Expressive Memory for Sequence Modeling

This repository contains the implementation to reproduce the numerical experiments of the paper Long Expressive Memory for Sequence Modeling

Requirements

To setup the environment, install the requirements using python 3.7.10 with

pip install -r requirements.txt

Experiments

This repository contains the codes to reproduce the results of the following experiments for the proposed LEM:

Adding Task
EigenWorms
FitzHugh-Nagumo
Google12 (V2)
Heart rate prediction
noisy CIFAR-10
Permuted Sequential MNIST
PennTree Bank (word- and char-level)
Sequential MNIST

All datasets get either synthetically generated or downloaded automatically, providing a full data pipline for each experiment.
Note that the PTB experiments are based on the open-source tensorflow project LAnguage Modelling Benchmarks (lamb) for tuning and testing Tensorflow language models. In order to reproduce these experiments, please follow the steps explained in the README of the PTB directory.

Results

The results of LEM for each of the experiments are:

Experiment	Result
EigenWorms	92.3% mean test accuracy
FitzHugh-Nagumo	0.002 test L^2 loss
Google12 (V2)	95.7% test accuracy
Heart rate prediction	0.85 test L^2 loss
noisy CIFAR-10	60.5% test accuracy
Permuted Sequential MNIST	96.6% test accuracy
PennTree Bank char-level (single layer)	1.25 test bits-per-character
PennTree Bank word-level (single layer)	72.8 test perplexity
Sequential MNIST	99.5% test accuracy

Long Expressive Memory (LEM)

Related tags

Overview

Long Expressive Memory for Sequence Modeling

Requirements

Experiments

Results

Owner

Konstantin Rusch

Automatically measure the facial Width-To-Height ratio and get facial analysis results provided by Microsoft Azure

A simple interface for editing natural photos with generative neural networks.

RetinaNet-PyTorch - A RetinaNet Pytorch Implementation on remote sensing images and has the similar mAP result with RetinaNet in MMdetection

"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

(ICCV 2021 Oral) Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation.

DetCo: Unsupervised Contrastive Learning for Object Detection

Transfer style api - An API to use with Tranfer Style App, where you can use two image and transfer the style

Cave Generation using metaballs in Blender. Originally created by sdfgeoff, Edited by Myself (Archie Jaskowicz).

Image Data Augmentation in Keras

Implementation of ML models like Decision tree, Naive Bayes, Logistic Regression and many other

Source code for CVPR 2021 paper "Riggable 3D Face Reconstruction via In-Network Optimization"

An efficient and effective learning to rank algorithm by mining information across ranking candidates. This repository contains the tensorflow implementation of SERank model. The code is developed based on TF-Ranking.

Pytorch implementation of the paper DocEnTr: An End-to-End Document Image Enhancement Transformer.

Unofficial PyTorch implementation of Neural Additive Models (NAM) by Agarwal, et al.

OpenDILab RL Kubernetes Custom Resource and Operator Lib

Awesome Artificial Intelligence, Machine Learning and Deep Learning as we learn it

The Video-based Accident Detection System built in Python

ChebLieNet, a spectral graph neural network turned equivariant by Riemannian geometry on Lie groups.

Code for "Discovering Non-monotonic Autoregressive Orderings with Variational Inference" (paper and code updated from ICLR 2021)

Variational autoencoder for anime face reconstruction