This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples Above is an adversarial example: the slightly pert

Anish Athalye 838 Dec 18, 2022
Sonoff NSPanel protocol and hacking information. Tasmota Berry driver for NSPanel

NSPanel Hacking Sonoff NSPanel protocol and hacking information and Tasmota Berry driver. NSPanel protocol manual Tasmota driver nspanel.be Installati

blakadder 98 Dec 26, 2022
A security system to warn you when people enter your room 🎥

Get Out My Room v0.1 I hate people coming in my room when i'm not there. Get Out My Room is a simple security system that sends notifications with vid

ScriptLine 1 Jan 11, 2022
macOS Initial Access Payload Generator

Mystikal macOS Initial Access Payload Generator Related Blog Post: https://posts.specterops.io/introducing-mystikal-4fbd2f7ae520 Usage: Install Xcode

Leo Pitt 206 Dec 31, 2022
Python tool for exploiting CVE-2021-35616

OracleOTM Python tool for exploiting CVE-2021-35616 The script works in modules, which I implemented in the following order: ► Username enumeration ►

11 Dec 06, 2022
Security offerings for AWS Control Tower

Caylent Security Catalyst Reference Architecture Examples This repository contains solutions for Caylent's Security Catalyst. The Security Catalyst is

Steven Connolly 1 Oct 22, 2021
IDA Pro Python plugin to analyze and annotate Linux kernel alternatives

About This is an IDA Pro (Interactive Disassembler) plugin allowing to automatically analyze and annotate Linux kernel alternatives (content of .altin

Open Source Security, Inc. 16 Oct 12, 2022
STATS305C: Applied Statistics III (Spring, 2022)

STATS305C: Applied Statistics III Instructor: Scott Linderman TA: Matt MacKay, James Yang Term: Spring 2022 Stanford University Course Description: Pr

Scott Linderman 14 Aug 11, 2022
Now patched 0day for force reseting an accounts password

Animal Jam 0day No-Auth Force Password Reset via API Now patched 0day for force reseting an accounts password Used until patched to cause anarchy. Pro

IRIS 10 Nov 17, 2022
A BurpSuite extension to parse 5GC NF OpenAPI 3.0 files to assess 5G core networks

5GC_API_parse Description 5GC API parse is a BurpSuite extension allowing to assess 5G core network functions, by parsing the OpenAPI 3.0 not supporte

PentHertz 57 Dec 16, 2022
Docker Compose based system for running remote browsers (including Flash and Java support) connected to web archives

pywb Remote Browsers This repository provides a simple configuration for deploying any pywb with remote browsers provided by OWT/Shepherd Remote Brows

Webrecorder 10 Jul 28, 2022
An intranet tool for easily intranet pentesting

IntarKnife v1.0 a tool can be used in intarnet for easily pentesting moudle hash spray U can use this tool to spray hash on a webshell IntraKnife.exe

4 Nov 24, 2021
🏃 Python Solutions of All Problems in FHC 2021 (In Progress)

FacebookHackerCup-2021 Python solutions of Facebook Hacker Cup 2021. Solution begins with * means it will get TLE in the largest data set (total compu

kamyu 14 Oct 15, 2022
SPV SecurePasswordVerification

SPV SecurePasswordVerification Its is python module for doing a secure password verification without sharing the password directly. Features The passw

Merwin 1 Feb 12, 2022
Fast subdomain scanner, Takes arguments from a Json file ("args.json") and outputs the subdomains.

Fast subdomain scanner, Takes arguments from a Json file ("args.json") and outputs the subdomains. File Structure core/ colors.py db/ wordlist.txt REA

whoami security 4 Jul 02, 2022
Universal Radio Hacker: Investigate Wireless Protocols Like A Boss

The Universal Radio Hacker (URH) is a complete suite for wireless protocol investigation with native support for many common Software Defined Radios.

Dr. Johannes Pohl 9k Jan 03, 2023
A simple multi-threaded distributed SSH brute-forcing tool written in Python.

OrbitalDump A simple multi-threaded distributed SSH brute-forcing tool written in Python. How it Works When the script is executed without the --proxi

K4YT3X 408 Jan 03, 2023
The probability of having the password you want in the PassMaker is +90%!!

PasswordMaker Strong listing password Introduction The probability of having the password you want in the tool is +90%!! How to Install Open the termi

MasterBurnt 4 Sep 05, 2021
A windows post exploitation tool that contains a lot of features for information gathering and more.

Crowbar - A windows post exploitation tool Status - ✔️ This project is now considered finished. Any updates from now on will most likely be new script

29 Nov 20, 2022
Password List Maker

Red-Key Red-Key Password List Maker Version 1.1.2 Created By FireKing255 -=Features=- Create Random Password List Create Password List Create Password

FireKing255 7 Dec 26, 2021