Divide full port scan results and use it for targeted Nmap runs

Overview

DivideAndScan

Divide Et Impera And Scan (and also merge the scan results)

version python codacy pypi docker twitter


DivideAndScan is used to efficiently automate port scanning routine by splitting it into 3 phases:

  1. Discover open ports for a bunch of targets.
  2. Run Nmap individually for each target with version grabbing and NSE actions.
  3. Merge the results into a single Nmap report (different formats available).

For the 1st phase a fast port scanner is intended to be used (Masscan / RustScan), whose output is parsed and stored in a database (TinyDB). Next, during the 2nd phase individual Nmap scans are launched for each target with its set of open ports (multiprocessing is supported) according to the database data. Finally, in the 3rd phase separate Nmap outputs are merged into a single report in different formats (XML / HTML / Simple text / Grepable) with nMap_Merger.

Potential use cases:

  • Pentest engagements / red teaming with a large scope to enumerate.
  • Cybersecurity wargames / training CTF labs.
  • OSCP certification exam.

How It Works

how-it-works.png

How to Install

Prerequisites

To successfully divide and scan we need to get some good port scanning tools.

πŸ“‘ Note: if you don't feel like messing with dependecies on your host OS, skip to the Docker part.

Nmap

sudo apt install nmap sudo xsltproc -y
sudo nmap --script-updatedb

Masscan

cd /tmp
git clone https://github.com/robertdavidgraham/masscan.git
cd masscan
make
sudo make install
cd && rm -rf /tmp/masscan

RustScan

cd /tmp

wget -qO- https://api.github.com/repos/RustScan/RustScan/releases/latest \
| grep "browser_download_url.*amd64.deb" \
| cut -d: -f2,3 \
| tr -d \" \
| wget -qO rustscan.deb -i-

sudo dpkg -i rustscan.deb
cd && rm /tmp/rustscan.deb

sudo wget https://gist.github.com/snovvcrash/c7f8223cc27154555496a9cbb4650681/raw/a76a2c658370d8b823a8a38a860e4d88051b417e/rustscan-ports-top1000.toml -O /root/.rustscan.toml

Installation

DivideAndScan is available on PyPI as divideandscan, though I recommend installing it from GitHub with pipx in order to always have the bleeding-edge version:

~$ pipx install -f "git+https://github.com/snovvcrash/DivideAndScan.git"
~$ das

For debbugging purposes you can set up a dev environment with poetry:

~$ git clone https://github.com/snovvcrash/DivideAndScan
~$ cd DivideAndScan
~$ poetry install
~$ poetry run das

πŸ“‘ Note: DivideAndScan uses sudo to run all the port scanners, so it will ask for the password when scanning commands are invoked.

Using from Docker

You can run DivideAndScan in a Docker container as follows:

~$ docker run -it --rm --name das -v `pwd`:/app snovvcrash/divideandscan

Since the tool requires some input data and produces some output data, you should specify your current working directory as the mount point at /app within the container. You may want to set an alias to make the base command shorter:

~$ alias das='docker run -it --rm --name das -v `pwd`:/app snovvcrash/divideandscan'
~$ das

How to Use

how-to-use.png

More examples coming soon at snovvcrash.rocks...

0. Preparations

Make a new directory to start DivideAndScan from. The tool will create subdirectories to store the output, so I recommend launching it from a clean directory to stay organized:

~$ mkdir divideandscan
~$ cd divideandscan

1. Filling the DB

Provide the add module a command for either Masscan or RustScan to discover open ports in a desired range.

⚠️ Warning: please, make sure that you understand what you're doing, because nearly all port scanning tools can damage the system being tested if used improperly.

# Masscan example
~$ das add -db testdb masscan '--rate 1000 -iL hosts.txt --open -p1-65535'
# RustScan example
~$ das add -db testdb rustscan '-b 1000 -t 2000 -u 5000 -a hosts.txt -r 1-65535 -g --no-config'

When the module completes its work, a hidden directory .db is created in CWD containig the database file and raw scan results.

2. Targeted Scanning

Launch targeted Nmap scans with the scan module. You can adjust the scan surface with either -hosts or -ports option:

# Scan by hosts
~$ das scan -db testdb -hosts all -oA report1
~$ das scan -db testdb -hosts 192.168.1.0/24,10.10.13.37 -oA report1
~$ das scan -db testdb -hosts hosts.txt -oA report1
# Scan by ports
~$ das scan -db testdb -ports all -oA report2
~$ das scan -db testdb -ports 22,80,443,445 -oA report2
~$ das scan -db testdb -ports ports.txt -oA report2

To start Nmap simultaneously in multiple processes, specify the -parallel switch and set number of workers with the -proc option (if no value is provided, it will default to the number of processors on the machine):

~$ das scan -db testdb -hosts all -oA report -parallel [-proc 4]

The output format is selected with -oX, -oN, -oG and -oA options for XML+HTML formats, simple text format, grepable format and all formats respectively. When the module completes its work, a hidden directory .nmap is created in CWD containig Nmap raw scan reports.

Also, you can inspect the contents of the database with -show option before actually launching the scans:

~$ das scan -db testdb -hosts all -show

3 (Optional). Merging the Reports

In order to generate a report independently of the scan module, you should use the report module. It will search for Nmap raw scan reports in the .nmap directory and process and merge them based on either -hosts or -ports option:

# Merge outputs by hosts
~$ das report -db testdb -hosts all -oA report1
~$ das report -db testdb -hosts 192.168.1.0/24,10.10.13.37 -oA report1
~$ das report -db testdb -hosts hosts.txt -oA report1
# Merge outputs by ports
~$ das report -db testdb -ports all -oA report2
~$ das report -db testdb -ports 22,80,443,445 -oA report2
~$ das report -db testdb -ports ports.txt -oA report2

πŸ“‘ Note: keep in mind that the report module does not search the DB when processing the -hosts or -ports options, but looks for Nmap raw reports directly in .nmap directory instead; it means that -hosts 127.0.0.1 argument value will be successfully resolved only if .nmap/127-0-0-1.* file exists, and -ports 80 argument value will be successfully resolved only if .nmap/port80.* file exists.

Help

usage: das [-h] {add,scan,report} ...

 -----------------------------------------------------------------------------------------------
|  ________  .__      .__    .___        _____              .____________                       |
|  \______ \ |__|__  _|__| __| _/____   /  _  \   ____    __| _/   _____/ ____ _____    ____    |
|   |    |  \|  \  \/ /  |/ __ |/ __ \ /  /_\  \ /    \  / __ |\_____  \_/ ___\\__  \  /    \   |
|   |    `   \  |\   /|  / /_/ \  ___//    |    \   |  \/ /_/ |/        \  \___ / __ \|   |  \  |
|  /_______  /__| \_/ |__\____ |\___  >____|__  /___|  /\____ /_______  /\___  >____  /___|  /  |
|          \/                 \/    \/        \/     \/      \/       \/     \/     \/     \/   |
|  {@snovvcrash}            {https://github.com/snovvcrash/DivideAndScan}             {vX.Y.Z}  |
 -----------------------------------------------------------------------------------------------

positional arguments:
  {add,scan,report}
    add              run a full port scan {masscan,rustscan} and add the output to DB
    scan             run targeted Nmap scans against hosts and ports from DB
    report           merge separate Nmap outputs into a single report in different formats

optional arguments:
  -h, --help         show this help message and exit

Psst, hey buddy... Wanna do some organized p0r7 5c4nn1n6?

ToDo

Support

If this tool has been useful for you, feel free to buy me a beer coffee!

beer.png

You might also like...
It helps to use fixtures in pytest.mark.parametrize

pytest-lazy-fixture Use your fixtures in @pytest.mark.parametrize. Installation pip install pytest-lazy-fixture Usage import pytest @pytest.fixture(p

An AWS Pentesting tool that lets you use one-liner commands to backdoor an AWS account's resources with a rogue AWS account - or share the resources with the entire internet 😈
An AWS Pentesting tool that lets you use one-liner commands to backdoor an AWS account's resources with a rogue AWS account - or share the resources with the entire internet 😈

An AWS Pentesting tool that lets you use one-liner commands to backdoor an AWS account's resources with a rogue AWS account - or share the resources with the entire internet 😈

This file will contain a series of Python functions that use the Selenium library to search for elements in a web page while logging everything into a file

element_search with Selenium (Now With docstrings 😎 ) Just to mention, I'm a beginner to all this, so it it's very possible to make some mistakes The

WIP SAT benchmarking tooling, written with only my personal use in mind.

SAT Benchmarking Some early work in progress tooling for running benchmarks and keeping track of the results when working on SAT solvers and related t

A command-line tool and Python library and Pytest plugin for automated testing of RESTful APIs, with a simple, concise and flexible YAML-based syntax

1.0 Release See here for details about breaking changes with the upcoming 1.0 release: https://github.com/taverntesting/tavern/issues/495 Easier API t

Turn any OpenAPI2/3 and Postman Collection file into an API server with mocking, transformations and validations.

Prism is a set of packages for API mocking and contract testing with OpenAPI v2 (formerly known as Swagger) and OpenAPI v3.x. Mock Servers: Life-like

To automate the generation and validation tests of COSE/CBOR Codes and it's base45/2D Code representations

To automate the generation and validation tests of COSE/CBOR Codes and it's base45/2D Code representations, a lot of data has to be collected to ensure the variance of the tests. This respository was established to collect a lot of different test data and related test cases of different member states in a standardized manner. Each member state can generate a folder in this section.

Test django schema and data migrations, including migrations' order and best practices.

django-test-migrations Features Allows to test django schema and data migrations Allows to test both forward and rollback migrations Allows to test th

An Instagram bot that can mass text users, receive and read a text, and store it somewhere with user details.
An Instagram bot that can mass text users, receive and read a text, and store it somewhere with user details.

Instagram Bot πŸ€– July 14, 2021 Overview πŸ‘ A multifunctionality automated instagram bot that can mass text users, receive and read a message and store

Comments
  • Runtime error FileNotFoundError: [Errno 2] No such file or directory: '/root/.das/db/main.json'

    Runtime error FileNotFoundError: [Errno 2] No such file or directory: '/root/.das/db/main.json'

    I just cloned the repo and run the following commands :

    /tmp/DivideAndScan# alias das='docker run -it --rm --name das -v ~/.das:/root/.das -v pwd:/app divideandscan:latest'

    /tmp/DivideAndScan# das scan -hosts 185.165.116.100-185.165.116.200 -oA report1

    and here is the error that I faced :

    Traceback (most recent call last): File "/usr/local/bin/das", line 8, in sys.exit(main()) File "/usr/local/lib/python3.9/dist-packages/das/divideandscan.py", line 158, in main sr = ScanRun(str(P), args.hosts, args.ports) File "/usr/local/lib/python3.9/dist-packages/das/scan.py", line 35, in init self.db = TinyDB(db_path) File "/usr/local/lib/python3.9/dist-packages/tinydb/database.py", line 94, in init self._storage: Storage = storage(*args, **kwargs) File "/usr/local/lib/python3.9/dist-packages/tinydb/storages.py", line 102, in init touch(path, create_dirs=create_dirs) File "/usr/local/lib/python3.9/dist-packages/tinydb/storages.py", line 31, in touch with open(path, 'a'): FileNotFoundError: [Errno 2] No such file or directory: '/root/.das/db/main.json'

    opened by neginsadeghi 1
  • Bump cryptography from 38.0.2 to 38.0.3

    Bump cryptography from 38.0.2 to 38.0.3

    Bumps cryptography from 38.0.2 to 38.0.3.

    Changelog

    Sourced from cryptography's changelog.

    38.0.3 - 2022-11-01

    
    * Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL 3.0.7,
      which resolves *CVE-2022-3602* and *CVE-2022-3786*.
    

    .. _v38-0-2:

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump urllib3 from 1.26.4 to 1.26.5

    Bump urllib3 from 1.26.4 to 1.26.5

    Bumps urllib3 from 1.26.4 to 1.26.5.

    Release notes

    Sourced from urllib3's releases.

    1.26.5

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Fixed deprecation warnings emitted in Python 3.10.
    • Updated vendored six library to 1.16.0.
    • Improved performance of URL parser when splitting the authority component.

    If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

    Changelog

    Sourced from urllib3's changelog.

    1.26.5 (2021-05-26)

    • Fixed deprecation warnings emitted in Python 3.10.
    • Updated vendored six library to 1.16.0.
    • Improved performance of URL parser when splitting the authority component.
    Commits
    • d161647 Release 1.26.5
    • 2d4a3fe Improve performance of sub-authority splitting in URL
    • 2698537 Update vendored six to 1.16.0
    • 07bed79 Fix deprecation warnings for Python 3.10 ssl module
    • d725a9b Add Python 3.10 to GitHub Actions
    • 339ad34 Use pytest==6.2.4 on Python 3.10+
    • f271c9c Apply latest Black formatting
    • 1884878 [1.26] Properly proxy EOF on the SSLTransport test suite
    • See full diff in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Releases(v0.2.0)
Owner
snovvcrash
Penetration tester
snovvcrash
buX Course Enrollment Automation

buX automation BRACU - buX course enrollment automation Features: Automatically enroll into multiple courses at a time. Find courses just entering cou

Mohammad Shakib 1 Oct 06, 2022
This project is used to send a screenshot by email of your MyUMons schedule using Selenium python lib (headless mode)

MyUMonsSchedule Use MyUMonsSchedule python script to send a screenshot by email (Gmail) of your MyUMons schedule. If you use it on Windows, take care

Pierre-Louis D'Agostino 6 May 12, 2022
Testing Calculations in Python, using OOP (Object-Oriented Programming)

Testing Calculations in Python, using OOP (Object-Oriented Programming) Create environment with venv python3 -m venv venv Activate environment . venv

William Koller 1 Nov 11, 2021
A suite of benchmarks for CPU and GPU performance of the most popular high-performance libraries for Python :rocket:

A suite of benchmarks for CPU and GPU performance of the most popular high-performance libraries for Python :rocket:

Dion HΓ€fner 255 Jan 04, 2023
0hh1 solver for the web (selenium) and also for mobile (adb)

0hh1 - Solver Aims to solve the '0hh1 puzzle' for all the sizes (4x4, 6x6, 8x8, 10x10 12x12). for both the web version (using selenium) and on android

Adwaith Rajesh 1 Nov 05, 2021
Selenium Manager

SeleniumManager I'm fed up with always having to struggle unnecessarily when I have to use Selenium on a new machine, so I made this little python mod

Victor Vague 1 Dec 24, 2021
a socket mock framework - for all kinds of socket animals, web-clients included

mocket /mΙ”ΛˆkΙ›t/ A socket mock framework for all kinds of socket animals, web-clients included - with gevent/asyncio/SSL support ...and then MicroPytho

Giorgio Salluzzo 249 Dec 14, 2022
A pytest plugin to run an ansible collection's unit tests with pytest.

pytest-ansible-units An experimental pytest plugin to run an ansible collection's unit tests with pytest. Description pytest-ansible-units is a pytest

Community managed Ansible repositories 9 Dec 09, 2022
Useful additions to Django's default TestCase

django-test-plus Useful additions to Django's default TestCase from REVSYS Rationale Let's face it, writing tests isn't always fun. Part of the reason

REVSYS 546 Dec 22, 2022
Doggo Browser

Doggo Browser Quick Start $ python3 -m venv ./venv/ $ source ./venv/bin/activate $ pip3 install -r requirements.txt $ ./sobaki.py References Heavily I

Alexey Kutepov 9 Dec 12, 2022
Code for "SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism"

SUGAR Code for "SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism" Overview train.py: the cor

41 Nov 08, 2022
frwk_51pwn is an open-sourced remote vulnerability testing and proof-of-concept development framework

frwk_51pwn Legal Disclaimer Usage of frwk_51pwn for attacking targets without prior mutual consent is illegal. frwk_51pwn is for security testing purp

51pwn 4 Apr 24, 2022
Thin-wrapper around the mock package for easier use with pytest

pytest-mock This plugin provides a mocker fixture which is a thin-wrapper around the patching API provided by the mock package: import os class UnixF

pytest-dev 1.5k Jan 05, 2023
d4rk Ghost is all in one hacking framework For red team Pentesting

d4rk ghost is all in one Hacking framework For red team Pentesting it contains all modules , information_gathering exploitation + vulnerability scanning + ddos attacks with 12 methods + proxy scraper

d4rk sh4d0w 15 Dec 15, 2022
This file will contain a series of Python functions that use the Selenium library to search for elements in a web page while logging everything into a file

element_search with Selenium (Now With docstrings 😎 ) Just to mention, I'm a beginner to all this, so it it's very possible to make some mistakes The

2 Aug 12, 2021
To automate the generation and validation tests of COSE/CBOR Codes and it's base45/2D Code representations

To automate the generation and validation tests of COSE/CBOR Codes and it's base45/2D Code representations, a lot of data has to be collected to ensure the variance of the tests. This respository was

160 Jul 25, 2022
API mocking with Python.

apyr apyr (all lowercase) is a simple & easy to use mock API server. It's great for front-end development when your API is not ready, or when you are

Umut Seven 55 Nov 25, 2022
pytest_pyramid provides basic fixtures for testing pyramid applications with pytest test suite

pytest_pyramid pytest_pyramid provides basic fixtures for testing pyramid applications with pytest test suite. By default, pytest_pyramid will create

Grzegorz ŚliwiΕ„ski 12 Dec 04, 2022
UX Analytics & A/B Testing

UX Analytics & A/B Testing

Marvin EDORH 1 Sep 07, 2021
pytest plugin to test mypy static type analysis

pytest-mypy-testing β€” Plugin to test mypy output with pytest pytest-mypy-testing provides a pytest plugin to test that mypy produces a given output. A

David Fritzsche 21 Dec 21, 2022