LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

A collection of pre-trained StyleGAN2 models trained on different datasets at different resolution.

Semi-supevised Semantic Segmentation with High- and Low-level Consistency

BaseCls BaseCls 是一个基于 MegEngine 的预训练模型库，帮助大家挑选或训练出更适合自己科研或者业务的模型结构

NeurIPS-2021: Neural Auto-Curricula in Two-Player Zero-Sum Games.

VD-BERT: A Unified Vision and Dialog Transformer with BERT

Fast algorithms to compute an approximation of the minimal volume oriented bounding box of a point cloud in 3D.

A PyTorch implementation of PointRend: Image Segmentation as Rendering

A simple consistency training framework for semi-supervised image semantic segmentation

Hierarchical Aggregation for 3D Instance Segmentation (ICCV 2021)

Simulation of the solar system using various nummerical methods

Robot Hacking Manual (RHM). From robotics to cybersecurity. Papers, notes and writeups from a journey into robot cybersecurity.

MMGeneration is a powerful toolkit for generative models, based on PyTorch and MMCV.

Learning Energy-Based Models by Diffusion Recovery Likelihood

Import Python modules from dicts and JSON formatted documents.

ktrain is a Python library that makes deep learning and AI more accessible and easier to apply

A deep neural networks for images using CNN algorithm.

LightSeq is a high performance training and inference library for sequence processing and generation implemented in CUDA

Code for "The Box Size Confidence Bias Harms Your Object Detector"

【steal piano】GitHub偷情分析工具！

SegNet-Basic with Keras