Almost State-of-the-art Text Generation library

Related tags

Text Data & NLPpythontensorflowtext-analysistext-generationpytorchlstm-neural-networkstext-gen
Overview

Ps: we are adding transformer model soon

Text Gen 🐐

Downloads python tensorflow PyPI

Almost State-of-the-art Text Generation library

Text gen is a python library that allow you build a custom text generation model with ease 😄 Something sweet built with Tensorflow and Pytorch(coming soon) - This is the brain of Rosalove ai (https://rosalove.xyz/)

How to use it

Install text-gen

pip install -U text-gen

import the library

from text_gen import ten_textgen as ttg

Load your data. your data must be in a text format.

Download the example data from the example folder

load data

data = 'rl.csv'
text = ttg.loaddata(data)

build our Model Architeture

pipeline = ttg.tentext(text)
seq_text = pipeline.sequence(padding_method = 'pre')
configg = pipeline.configmodel(seq_text, lstmlayer = 128, activation = 'softmax', dropout = 0.25)

train model

model_history = pipeline.fit(loss = 'categorical_crossentropy', optimizer = 'adam', batch = 300, metrics = 'accuracy', epochs = 500, verbose = 0, patience = 10)

generate text using the phrase

pipeline.predict('hello love', word_length = 200, segment = True)

plot loss and accuracy

pipeline.plot_loss_accuracy()

Hyper parameter optimization

Tune your model to know the best optimizer, activation method to use.

pipeline.hyper_params(epochs = 500)
pipeline.saveModel('model')

use a saved model for prediction

#the corpus is the train text file
ttg.load_model_predict(corpus = corpus, padding_method = 'pre', modelname = '../input/model2/model2textgen.h5', sample_text = 'yo yo', word_length = 100)

Give us a star ⭐ 🐉

If you want to contribute, take a look at the issues and the Futurework.md file

Contributors

Comments
  • use pipenv for managing dependencies

    use pipenv for managing dependencies

    Consider using (pipenv)[https://pypi.org/project/pipenv/] to pin your dependencies. This would allow contributors to easily reproduce the project without messing up the dependencies and its also good on the long run for maintainability

    opened by paularah 1
  • [Snyk] Security upgrade pillow from 6.2.2 to 8.3.2

    [Snyk] Security upgrade pillow from 6.2.2 to 8.3.2

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 661/1000
    Why? Recently disclosed, Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-PILLOW-1319443 | pillow:
    6.2.2 -> 8.3.2
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
  • Read on how to create a simple python library

    Read on how to create a simple python library

    https://towardsdatascience.com/how-to-build-your-first-python-package-6a00b02635c9

    https://medium.com/analytics-vidhya/how-to-create-a-python-library-7d5aea80cc3f

    opened by Emekaborisama 0
  • [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    This PR was automatically created by Snyk using the credentials of a real user.


    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
tensorflow 1.14.0 requires protobuf, which is not installed.
tensorflow-serving-api 1.12.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires protobuf, which is not installed.
GPyOpt 1.2.6 requires GPy, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- medium severity | 551/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-WHEEL-3180413 | wheel:
    0.30.0 -> 0.38.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Regular Expression Denial of Service (ReDoS)

    opened by Emekaborisama 0
  • [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    This PR was automatically created by Snyk using the credentials of a real user.


    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
tensorflow 1.14.0 requires grpcio, which is not installed.
tensorflow 1.14.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires grpcio, which is not installed.
parameter-sherpa 1.0.6 requires pymongo, which is not installed.
parameter-sherpa 1.0.6 requires GPyOpt, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- medium severity | 551/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-WHEEL-3092128 | wheel:
    0.30.0 -> 0.38.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Regular Expression Denial of Service (ReDoS)

    opened by Emekaborisama 0
  • [Snyk] Fix for 23 vulnerabilities

    [Snyk] Fix for 23 vulnerabilities

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
parameter-sherpa 1.0.6 requires scikit-learn, which is not installed.
GPy 1.10.0 requires paramz, which is not installed.
GPy 1.10.0 requires cython, which is not installed.
GPy 1.10.0 has requirement scipy<1.5.0,>=1.3.0, but you have scipy 1.2.3.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1055461 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1055462 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 509/1000
    Why? Has a fix available, CVSS 5.9 | Out-of-bounds Write
    SNYK-PYTHON-PILLOW-1059090 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1080635 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-PILLOW-1080654 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081494 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081501 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081502 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 654/1000
    Why? Has a fix available, CVSS 8.8 | Heap-based Buffer Overflow
    SNYK-PYTHON-PILLOW-1082329 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Insufficient Validation
    SNYK-PYTHON-PILLOW-1082750 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090584 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090586 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090587 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090588 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1292150 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1292151 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 566/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.6 | Buffer Overflow
    SNYK-PYTHON-PILLOW-1316216 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-Bounds
    SNYK-PYTHON-PILLOW-574573 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574574 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574575 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574576 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 469/1000
    Why? Has a fix available, CVSS 5.1 | Buffer Overflow
    SNYK-PYTHON-PILLOW-574577 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit low severity | 506/1000
    Why? Proof of Concept exploit, Has a fix available, CVSS 3.7 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-SCIKITLEARN-1079100 | scikit-learn:
    0.20.4 -> 0.24.2
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
Releases(v1.9.0)
Owner
Emeka boris ama
Machine Learning Engineer, Data Scientist, Youtuber and Advocacy
Emeka boris ama
GitHub Repository
Using Bert as the backbone model for lime, designed for NLP task explanation (sentence pair text classification task)

Lime Comparing deep contextualized model for sentences highlighting task. In addition, take the classic explanation model "LIME" with bert-base model

JHJu 2 Jan 18, 2022
leaking paid token generator that was a shit lmao for 100$ haha

Discord-Token-Generator-Leaked leaking paid token generator that was a shit lmao for 100$ he selling it for 100$ wth here the code enjoy don't forget

Keevo 5 Apr 15, 2022
Simple Annotated implementation of GPT-NeoX in PyTorch

Simple Annotated implementation of GPT-NeoX in PyTorch This is a simpler implementation of GPT-NeoX in PyTorch. We have taken out several optimization

labml.ai 101 Dec 03, 2022
iSTFTNet : Fast and Lightweight Mel-spectrogram Vocoder Incorporating Inverse Short-time Fourier Transform

iSTFTNet : Fast and Lightweight Mel-spectrogram Vocoder Incorporating Inverse Short-time Fourier Transform This repo try to implement iSTFTNet : Fast

Rishikesh (ऋषिकेश) 126 Jan 02, 2023
Code associated with the "Data Augmentation using Pre-trained Transformer Models" paper

Data Augmentation using Pre-trained Transformer Models Code associated with the Data Augmentation using Pre-trained Transformer Models paper Code cont

44 Dec 31, 2022
Generating Korean Slogans with phonetic and structural repetition

LexPOS_ko Generating Korean Slogans with phonetic and structural repetition Generating Slogans with Linguistic Features LexPOS is a sequence-to-sequen

Yeoun Yi 3 May 23, 2022
Spam filtering made easy for you

spammy Author: Tasdik Rahman Latest version: 1.0.3 Contents 1 Overview 2 Features 3 Example 3.1 Accuracy of the classifier 4 Installation 4.1 Upgradin

Tasdik Rahman 137 Dec 18, 2022
Repository for Project Insight: NLP as a Service

Project Insight NLP as a Service Contents Introduction Features Installation Setup and Documentation Project Details Demonstration Directory Details H

Abhishek Kumar Mishra 286 Dec 06, 2022
Official implementations for various pre-training models of ERNIE-family, covering topics of Language Understanding & Generation, Multimodal Understanding & Generation, and beyond.

English|简体中文 ERNIE是百度开创性提出的基于知识增强的持续学习语义理解框架，该框架将大数据预训练与多源丰富知识相结合，通过持续学习技术，不断吸收海量文本数据中词汇、结构、语义等方面的知识，实现模型效果不断进化。ERNIE在累积 40 余个典型 NLP 任务取得 SOTA 效果，并在 G

5.4k Jan 03, 2023
This is the 25 + 1 year anniversary version of the 1995 Rachford-Rice contest

Rachford-Rice Contest This is the 25 + 1 year anniversary version of the 1995 Rachford-Rice contest. Can you solve the Rachford-Rice problem for all t

13 Sep 20, 2022
A Flask Sentiment Analysis API, with visual implementation

The Sentiment Analysis Api was created using python flask module,it allows users to parse a text or sentence throught the (?text) arguement, then view the sentiment analysis of that sentence. It can

Ifechukwudeni Oweh 10 Jul 17, 2022
Submit issues and feature requests for our API here.

AIx GPT API Submit issues and feature requests for our API here. See https://apps.aixsolutionsgroup.com for more info. Python Quick Start pip install

AIx Solutions 7 Mar 27, 2022
EMNLP'2021: Can Language Models be Biomedical Knowledge Bases?

BioLAMA BioLAMA is biomedical factual knowledge triples for probing biomedical LMs. The triples are collected and pre-processed from three sources: CT

DMIS Laboratory - Korea University 41 Nov 18, 2022
Legal text retrieval for python

legal-text-retrieval Overview This system contains 2 steps: generate training data containing negative sample found by mixture score of cosine(tfidf)

Nguyễn Minh Phương 22 Dec 06, 2022
This project uses word frequency and Term Frequency-Inverse Document Frequency to summarize a text.

Text Summarizer This project uses word frequency and Term Frequency-Inverse Document Frequency to summarize a text. Team Members This mini-project was

1 Nov 16, 2021
fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.

fastNLP fastNLP是一款轻量级的自然语言处理（NLP）工具包，目标是快速实现NLP任务以及构建复杂模型。 fastNLP具有如下的特性： 统一的Tabular式数据容器，简化数据预处理过程； 内置多种数据集的Loader和Pipe，省去预处理代码; 各种方便的NLP工具，例如Embedd

fastNLP 2.8k Jan 01, 2023
End-to-end MLOps pipeline of a BERT model for emotion classification.

image source EmoBERT-MLOps The goal of this repository is to build an end-to-end MLOps pipeline based on the MLOps course from Made with ML, but this

Dimitre Oliveira 4 Nov 06, 2022
This is a project built for FALLABOUT2021 event under SRMMIC, This project deals with NLP poetry generation.

FALLABOUT-SRMMIC 21 POETRY-GENERATION HINGLISH DESCRIPTION We have developed a NLP(natural language processing) model which automatically generates a

7 Sep 28, 2021
XLNet: Generalized Autoregressive Pretraining for Language Understanding

Introduction XLNet is a new unsupervised language representation learning method based on a novel generalized permutation language modeling objective.

Zihang Dai 6k Jan 07, 2023
Generate a cool README/About me page for your Github Profile

Github Profile README/ About Me Generator 💯 This webapp lets you build a cool README for your profile. A few inputs + ~15 mins = Your Github Profile

Rahul Banerjee 179 Jan 07, 2023