Ladder network is a deep learning algorithm that combines supervised and unsupervised learning

Related tags

Game Developmentladder
Overview

This repository contains source code for the experiments in a paper titled Semi-Supervised Learning with Ladder Networks by A Rasmus, H Valpola, M Honkala, M Berglund, and T Raiko.

Required libraries

Install Theano, Blocks Stable 0.2, Fuel Stable 0.2

Refer to the Blocks installation instructions for details but use tag v0.2 instead. Something along:

pip install git+git://github.com/mila-udem/[email protected]
pip install git+git://github.com/mila-udem/[email protected]

Fuel comes with Blocks, but you need to download and convert the datasets. Refer to the Fuel documentation. One might need to rename the converted files.

fuel-download mnist
fuel-convert mnist --dtype float32
fuel-download cifar10
fuel-convert cifar10
Alternatively, one can use the environment.yml file that is provided in this repo to create an conda environment.
  1. First install anaconda from https://www.continuum.io/downloads. Then,
  2. conda env create -f environment.yml
  3. source activate ladder
  4. The environment should be good to go!

Models in the paper

The following commands train the models with seed 1. The reported numbers in the paper are averages over several random seeds. These commands use all the training samples for training (--unlabeled-samples 60000) and none are used for validation. This results in a lot of NaNs being printed during the trainining, since the validation statistics are not available. If you want to observe the validation error and costs during the training, use --unlabeled-samples 50000.

MNIST all labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 1000,1,0.01,0.01,0.01,0.01,0.01 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_full
# Bottom
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,0,0,0,0,0,0 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_bottom
# Gamma model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,2 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 60000 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_all_baseline
MNIST 100 labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_full
# Bottom-only
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 5000,0,0,0,0,0,0 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_bottom
# Gamma
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,0.5 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 100 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_100_baseline
MNIST 1000 labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --f-local-noise-std 0.2 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_full
# Bottom-only
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,0,0,0,0,0,0 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_bottom
# Gamma model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,10 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 1000 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_1000_baseline
MNIST 50 labels
# Full model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --labeled-samples 50 --unlabeled-samples 60000 --seed 1 -- mnist_50_full
MNIST convolutional models
# Conv-FC
run.py train --encoder-layers convv:1000:26:1:1-convv:500:1:1:1-convv:250:1:1:1-convv:250:1:1:1-convv:250:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec gauss --denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_conv_fc
# Conv-Small, Gamma
run.py train --encoder-layers convf:32:5:1:1-maxpool:2:2-convv:64:3:1:1-convf:64:3:1:1-maxpool:2:2-convv:128:3:1:1-convv:10:1:1:1-globalmeanpool:6:6-fc:10 --decoder-spec 0-0-0-0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,0,0,0,1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1  -- mnist_100_conv_gamma
# Conv-Small, supervised baseline. Overfits easily, so keep training short.
run.py train --encoder-layers convf:32:5:1:1-maxpool:2:2-convv:64:3:1:1-convf:64:3:1:1-maxpool:2:2-convv:128:3:1:1-convv:10:1:1:1-globalmeanpool:6:6-fc:10 --decoder-spec 0-0-0-0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0,0,0,0 --num-epochs 20 --lrate-decay 0.5 --f-local-noise-std 0.45 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_conv_baseline
CIFAR models
# Conv-Large, Gamma
./run.py train --encoder-layers convv:96:3:1:1-convf:96:3:1:1-convf:96:3:1:1-maxpool:2:2-convv:192:3:1:1-convf:192:3:1:1-convv:192:3:1:1-maxpool:2:2-convv:192:3:1:1-convv:192:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec 0-0-0-0-0-0-0-0-0-0-0-0-gauss --dataset cifar10 --act leakyrelu --denoising-cost-x 0,0,0,0,0,0,0,0,0,0,0,0,4.0 --num-epochs 70 --lrate-decay 0.86 --seed 1 --whiten-zca 3072 --contrast-norm 55 --top-c False --labeled-samples 4000 --unlabeled-samples 50000 -- cifar_4k_gamma
# Conv-Large, supervised baseline. Overfits easily, so keep training short.
./run.py train --encoder-layers convv:96:3:1:1-convf:96:3:1:1-convf:96:3:1:1-maxpool:2:2-convv:192:3:1:1-convf:192:3:1:1-convv:192:3:1:1-maxpool:2:2-convv:192:3:1:1-convv:192:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec 0-0-0-0-0-0-0-0-0-0-0-0-0 --dataset cifar10 --act leakyrelu --denoising-cost-x 0,0,0,0,0,0,0,0,0,0,0,0,0 --num-epochs 20 --lrate-decay 0.5 --seed 1 --whiten-zca 3072 --contrast-norm 55 --top-c False --labeled-samples 4000 --unlabeled-samples 50000 -- cifar_4k_baseline
Evaluating models with testset

After training a model, you can infer the results on a test set by performing the evaluate command. An example use after training a model:

./run.py evaluate results/mnist_all_bottom0
Owner
Curious AI
Deep good. Unsupervised better.
Curious AI
GitHub Repository
用于 blivechat 的图形界面

blivechat GUI 用于 blivechat 的图形界面。 有朋友在搞 Vtuber，像 blivechat 类似的项目能通过自定义 CSS 的方式在 OBS 上添加一个非常好看的聊天栏。但是想要在桌面端看到弹幕的话得要再开一个浏览器页面，十分不方便。就想写一个背景透明的浮窗浏览器。 挺喜欢

Silence 11 Dec 29, 2022
Un semplice Snake game , come negli anni 90!

Project-SnakeGame Un semplice Snake game , come negli anni 90! ITA VI porto un semplice giochino per i nostalgini degli anni 90 , ispirato al vecchio

Matt K Lawrence 1 Oct 17, 2021
Scalable computer implemented in the game of life.

scalable-gol-computer This is a computer built in Conway’s game of life. It supports variable sizes of 8, 16 and 32 bit. Maximum program size: 256 lin

Nicolas Loizeau 19 Nov 19, 2022
Guess number game with PyQt5

Guess-Number-Project Guess number game with PyQt5 you can choose a number in your mind and then computer will guess a nummber and you guide the comput

MohammadAli.HBA 1 Nov 11, 2021
Arcade-like space shooter game written entirely in python

E.T.-Attack Arcade-like space shooter game written entirely in python Project description A space shooter game - inspired by the legendary game Space

Sven Eschlbeck 2 Dec 17, 2022
This is Minesweeper coded in Python. It has almost all features from the main game

Minesweeper This is Minesweeper coded in Python. It has almost all features from the main game Use right click to open tile, right click on an open ti

3 Jul 12, 2022
This is a simple Tic-Tac-Toe game.

Tic-Tac-Toe Nosso famoso e tradicional Jogo da Velha, mas agora em Python. Development setup Para rodar o programa, basta instalar python em sua maqui

João Assalim 1 Oct 10, 2022
This a secret santa game organizer that assigns secret santa randomly to each participant and then sends an automated mail to each santa with details of his/her secret santa child.

Before executing the script, make sure to turn on 'Less Secure App access' option from your gmail ID that will be used to send out the mails to all participants of the game. To do so, get going with

DEV_FINWIZ 10 Dec 06, 2022
Client-Server design (guess the closest number to the average score game)

Multiplayer game (enter the number closest to the average) Design Client-Server design The client's side is responsible for sending numbers from the g

Adam Piszczek 0 Jun 29, 2022
a game of life implementation in python

gameoflife-py python implementation of game of life Installing As long as you have bash and curl installed and are on Linux the install script should

Raghav 5 Jun 09, 2021
Overview: copain, your friendly AI framework to learn and play games

Overview: copain, your friendly AI framework to learn and play games Interface fceux with python and run reinforcement learning. Compatibility Current

fcharras 1 Dec 16, 2021
A multiplayer RPG Discord bot, where you play as a god.

To run Ensure your Python is up to date, and install packages from requirements.txt Duplicate secrets-template.yaml, and name it secrets.yaml Insert y

4 Jan 18, 2022
The main objective of the game is to destroy multiple waves of asteroids with the help of a blaster mounted on the spaceship.

Astronomia: let the exploration begin The main objective of the game is to destroy multiple waves of asteroids with the help of a blaster mounted on t

Aryan Nath 8 Nov 18, 2022
Minesweeper clone with 3 modes of difficulty, UI scaling and custom games feature.

Insect Sweeper Minesweeper clone with 3 modes of difficulty, UI scaling and custom games feature. Mines are replaced with random insects that a player

Piotr Data 1 Nov 05, 2021
Simple Game created using Python & PyGame, as my Beginner Python Project!

Space Invaders This is a simple SPACE INVADER game create using PYGAME whihc have sound and lot's of keyboard functions. Prerequisites More Experience

Gaurav Pandey 2 Jan 08, 2022
🪨 📄 ✂ game in python with recursion

🪨 📄 ✂ Game Rock Paper Scissor game in python with recursion ⚙️ Developer's Guide Things you need to get started with this code:- Download python3 fr

Atul Anand 3 Jul 25, 2022
An ongoing process to make a physics engine using python.

Simple_Physics_Engine An ongoing process to make a physics engine using python. I am using this goal as a way to learn python in and out. I am trying

Jon Sherrick 1 Jan 18, 2022
This is a simple game of rock-paper-scissors developed in Python

This is a simple game of rock-paper-scissors developed in Python. It allows two players to play with one another on different command lines through networking.

NAMAN JAIN 3 Oct 21, 2022
Ice-Walker-Game - This repository is about the Ice Walker game made in Python.

Ice-Walker-Game Ce dépot contient le jeu Ice Walker programmé en Python. Les différentes grilles du jeu sont contenues dans le sous-dossier datas. Vou

Mohamed Amine SABIL 1 Jan 02, 2022
AI plays games with python

AI-plays-games- To use it, you first need to create an img file and save the pic

Murat Ali Avcu 7 Oct 16, 2022