BalancingGroups

Code to replicate the experimental results from Simple data balancing baselines achieve competitive worst-group-accuracy.

Replicating the main results

Set environment variables

export DATASETS_PATH=/path/to/datasets
export SLURM_PATH=/path/to/slurm/logs

Download and extract datasets

Waterbirds to $DATASETS_PATH/waterbirds
CelebA to $DATASETS_PATH/celeba
CivilComments to $DATASETS_PATH/civilcomments
MultiNLI to $DATASETS_PATH/multinli

Generate dataset metadata

cd metadata/
python generate_metadata_waterbirds.py
python generate_metadata_celeba.py
python generate_metadata_civilcomments.py
python generate_metadata_multinli.py
cd ..

Launch jobs

# Launching 1400 combo seeds = 50 hparams for 4 datasets for 7 algorithms
# Each combo seed is ran 5 times to compute error bars, totalling 7000 jobs
./train.py --output_dir main_sweep --num_hparams_seeds 1400

Parse results

./parse.py main_sweep

License

This source code is released under the CC-BY-NC license, included here.

Simple data balancing baselines for worst-group-accuracy benchmarks.

Related tags

Overview

BalancingGroups

Replicating the main results

Set environment variables

Download and extract datasets

Generate dataset metadata

Launch jobs

Parse results

License

Owner

Facebook Research

Machine Learning Algorithms

A handy tool for common machine learning models' hyper-parameter tuning.

Anomaly Detection and Correlation library

Bayesian Additive Regression Trees For Python

CD) in machine learning projectsImplementing continuous integration & delivery (CI/CD) in machine learning projects

DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning.

Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.

A collection of Scikit-Learn compatible time series transformers and tools.

Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort

monolish: MONOlithic Liner equation Solvers for Highly-parallel architecture

Transpile trained scikit-learn estimators to C, Java, JavaScript and others.

Automatically build ARIMA, SARIMAX, VAR, FB Prophet and XGBoost Models on Time Series data sets with a Single Line of Code. Now updated with Dask to handle millions of rows.

A collection of machine learning examples and tutorials.

Given the names and grades for each student in a class N of students, store them in a nested list and print the name(s) of any student(s) having the second lowest grade.

Machine Learning Techniques using python.

AutoOED: Automated Optimal Experiment Design Platform

ELI5 is a Python package which helps to debug machine learning classifiers and explain their predictions

flexible time-series processing & feature extraction

Banpei is a Python package of the anomaly detection.

hgboost - Hyperoptimized Gradient Boosting