Code for Massive-scale Decoding for Text Generation using Lattices

Last update: Dec 18, 2022

Related tags

Overview

Massive-scale Decoding for Text Generation using Lattices

TL;DR: a new search algorithm to construct lattices encoding many generation options; two key technical contributions: (1) best-first search, (2) path recombination.

Visualization

We provide a few examples in the vis folder and on my homepage. You need to download the html files to view and interact with the model outputs.

The complete set of outputs are available on Box.

Getting started

model contains all of the methods, including baselines like beam search, nucleus sampling, and our methods.
evaluation contains scripts for evaluation.
command are the prompts and shells we use to run the experiment.

Beam Search:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4  -beam_size 16 -min_len 10 -max_len 35   -model bs

Best-first Search:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4  -beam_size 16 -min_len 10 -max_len 35   -model astar_baseline

Best-first Search with Recomb:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4 -beam_size 16 -min_len 10 -max_len 35 -model astar -merge imp  -avg_score 0.75  -adhoc

Best-first Search with Zip:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4 -beam_size 16 -min_len 10 -max_len 35 -model astar -merge zip  -avg_score 0.75  -adhoc

More detailed instructions coming soon!

Citation

@misc{xu-durrett-2021-massive,
    title={Massive-scale Decoding for Text Generation using Lattices},
    author={Jiacheng Xu and Greg Durrett},
    year={2021},
    eprint={2112.07660},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

Contact

[email protected]

Code for Massive-scale Decoding for Text Generation using Lattices

Related tags

Overview

Massive-scale Decoding for Text Generation using Lattices

Visualization

Getting started

Citation

Contact

Owner

Jiacheng Xu

Annealed Flow Transport Monte Carlo

An educational resource to help anyone learn deep reinforcement learning.

SAGE: Sensitivity-guided Adaptive Learning Rate for Transformers

Real-time Joint Semantic Reasoning for Autonomous Driving

Shōgun

Code for Recurrent Mask Refinement for Few-Shot Medical Image Segmentation (ICCV 2021).

HAR-stacked-residual-bidir-LSTMs - Deep stacked residual bidirectional LSTMs for HAR

Code for "The Box Size Confidence Bias Harms Your Object Detector"

This repository is based on Ultralytics/yolov5, with adjustments to enable rotate prediction boxes.

Transparent Transformer Segmentation

Pytorch implementation for Semantic Segmentation/Scene Parsing on MIT ADE20K dataset

A Distributional Approach To Controlled Text Generation

Official implementation of the Implicit Behavioral Cloning (IBC) algorithm

GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models

FastReID is a research platform that implements state-of-the-art re-identification algorithms.

Taming Transformers for High-Resolution Image Synthesis

An implementation of the BADGE batch active learning algorithm.

PyTorch Personal Trainer: My framework for deep learning experiments

PyTorch implementation for ACL 2021 paper "Maria: A Visual Experience Powered Conversational Agent".

Gray Zone Assessment