Utils to quickly evaluate many 🤗 models on the GLUE tasks

Last update: Dec 22, 2021

Related tags

Overview

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Setup

wget https://raw.githubusercontent.com/huggingface/transformers/master/examples/pytorch/text-classification/run_glue.py
conda env create -f environment.yaml
conda activate glue-utils

Evaluate multiple models on every glue task:

python runner.py \
    path/to/model_1/ \
    path/to/model_2/ \
    path/to/model_3/ \
    --gpu_idx 1

This will store results in path/to/model_N/glue/[task_name]/, for each model, for each task.

To quickly gather all results, run:

python gather_scores.py \
    path/to/model_1/glue/ \
    path/to/model_2/glue/ \
    path/to/model_3/glue/

Owner

Przemyslaw K. Joniak

Grad student at The University of Tokyo

GitHub Repository

solsim is the Solana complex systems simulator. It simulates behavior of dynamical systems—DeFi protocols, DAO governance, cryptocurrencies, and more—built on the Solana blockchain

12 Jul 13, 2022

Hashcrack - A non-object oriented open source, Software for Windows/Linux made in Python 3

Multi Force This project is a non-object oriented open source, Software for Wind

3 Jan 02, 2023

Group P-11's submission for the University of Waterloo's 2021 Engineering Competition (Programming section).

P-11-WEC2021 Group P-11's submission for the University of Waterloo's 2021 Engineering Competition (Programming section). Part I Compute typing time f

1 May 14, 2022

Incident Response Process and Playbooks | Goal: Playbooks to be Mapped to MITRE Attack Techniques

PURPOSE OF PROJECT That this project will be created by the SOC/Incident Response Community Develop a Catalog of Incident Response Playbook for every

987 Jan 02, 2023

Oblique Strategies for Python

3 Feb 17, 2022

Encode and decode cancro lang files to and from brainfuck

cancrolang Encode and decode cancro lang files to and from brainfuck. examples python3 main.py -f hello.cancro --run Hello World! the interpreter is n

1 Dec 20, 2021

A replacement of qsreplace, accepts URLs as standard input, replaces all query string values with user-supplied values and stdout.

Bhedak A replacement of qsreplace, accepts URLs as standard input, replaces all query string values with user-supplied values and stdout. Works on eve

84 Dec 31, 2022

Streamlit Component, for a Chatbot UI

st-chat Streamlit Component, for a Chat-bot UI, example app authors - @yashppawar & @YashVardhan-AI Installation Install streamlit-chat with pip pip i

99 Jan 07, 2023

Some shitty programs just to brush up on my understanding of binary conversions.

Binary Converters Some shitty programs just to brush up on my understanding of binary conversions. Supported conversions formats = "unsigned-binary" |

2 Jan 09, 2022

Your self-hosted bookmark archive. Free and open source.

Your self-hosted bookmark archive. Free and open source. Contents About LinkAce Support Setup Contribution About LinkAce LinkAce is a self-hosted arch

1.7k Jan 03, 2023

A command line interface tool converting starknet warp transpiled outputs into readable cairo contracts.

warp-to-cairo warp-to-cairo is a simple tool converting starknet warp outputs (NethermindEth/warp) outputs into readable cairo contracts. The warp out

5 Jun 10, 2022

A dashboard for your code. A build system.

NOTICE: THIS REPO IS NO LONGER UPDATED Changes Changes is a build coordinator and reporting solution written in Python. The project is primarily built

763 Sep 09, 2022

Have an idea for a Python package? Register the name on PyPI 💡

Register Package Names on PyPI Have an idea for a Python package? Thought of a great name? Register it on PyPI, before someone else does! A tool that

1 Jul 15, 2022

An interactive tool with which to explore the possible imaging performance of candidate ngEHT architectures.

ngEHTexplorer An interactive tool with which to explore the possible imaging performance of candidate ngEHT architectures. Welcome! ngEHTexplorer is a

7 Jan 28, 2022

SpellingBeeSolver - This program generates solutions to NYT style spelling bee problems.

SpellingBeeSolver This program generates solutions to NYT style spelling bee problems. The initial version of this program is being written in Python

1 Jan 01, 2022

Use this function to get list of routes for particular journey

route-planner Functions api_processing Use this function to get list of routes for particular journey. Function has three parameters: Origin Destinati

2 Nov 28, 2021

This is a simple web interface for SimplyTranslate

SimplyTranslate Web This is a simple web interface for SimplyTranslate List of Instances You can find a list of instances here: SimplyTranslate Projec

4 Dec 14, 2022

Extremely unfinished animation toolset for Blender 3.

AbraTools Alpha IMPORTANT: Code is a mess. Be careful using it in production. Bug reports, feature requests and PRs are appreciated. Download AbraTool

15 Dec 17, 2022

Dockernized ZeroTierOne controller with zero-ui web interface.

docker-zerotier-controller Dockernized ZeroTierOne controller with zero-ui web interface. 中文讨论 Customize ZeroTierOne's controller planets Modify patch

209 Jan 04, 2023

This is a vscode extension with a Virtual Assistant that you can play with when you are bored or you need help..

VS Code Virtual Assistant This is a vscode extension with a Virtual Assistant that you can play with when you are bored or you need help. Its currentl

6 Aug 22, 2021