Differences

This shows you the differences between two versions of the page.

--- lab:stack_of_nucs:ansible_playbook_-_hashtopolis_installation [2023/05/08 16:16] – [Windows] user
+++ lab:stack_of_nucs:ansible_playbook_-_hashtopolis_installation [2024/05/06 02:11] (current) – removed user
@@ Line 1: / Line 1: @@
-====== Ansible Playbook - Hashtopolis Installation ======
-This is a bonus step for our [[start|Stack of NUCs]] lab. We are going to create and run an Ansible playbook to set up [[https://hashtopolis.org/|Hashtopolis]]. Cyber security professionals use Hashtopolis to create a cluster of systems all running Hashcat, a versatile password hash cracking tool.
-It is important to discuss GPUs at this point.
-  * hashcat is no longer CPU-only; it uses GPUs and CPUs via OpenCL
-  * if your NUCs have a supported GPU, great it; otherwise you will be using OpenCL and CPU
-    * in my lab this worked out to only 61039 kH/s for md5 and 330 H/s for mode 1880 (Unix)
-  * if you have an NVIDIA GPU install hashcat-nvidia for better performance
-  * because of this complexity, we are installing the ''hashcat'' package and its requirements instead of the traditional hashtopolis way of simply copying the binary and running it
-There are two pieces to set up:
-  * server - central server distributes the keyspace of a task, aggregates jobs, and collects results in MySQL database
-    * communicates over HTTPS with agent machines
-    * passes over files, binaries and task commands
-  * agents - act on the commands, execute the hash cracking application, and report "founds" to the server
-Purpose:
-  * Demonstrate a running a cluster of hash cracking nodes
-References:
-  * [[https://github.com/hashtopolis]]
-  * [[https://jakewnuk.com/posts/hashtopolis-infrastructure/]]
-  * [[https://github.com/peterezzo/hashtopolis-docker]]
-  * [[https://resources.infosecinstitute.com/topic/hashcat-tutorial-beginners/]]
-  * [[https://infosecscout.com/install-hashcat-on-ubuntu/]]
-====== Step 1 - Start a project folder for Hashtopolis ======
-  - Log in to the Ansible control node ([[NUC 2]])
-  - Create directory ''/home/ansible/my-project/hashtopolis'' and change to it
-    * ''mkdir hashtopolis''
-    * ''cd hashtopolis''
-  - Create the inventory, putting one of the worker nodes in the ''[server]'' section and the rest in the ''[agents]'' section
-    * ''/home/ansible/my-project/hashtopolis/inventory''
-    * <file yaml inventory>
-[all:vars]
-ansible_python_interpreter=/usr/bin/python3
-ansible_user='ansible'
-ansible_become=true
-ansible_become_method=sudo
-[server]
-[agents]
-</file>
-  - Create the ''ansible.cfg'' file
-    * ''/home/ansible/my-project/hashtopolis/ansible.cfg''
-    * <file yaml ansible.cfg>
-[defaults]
-inventory = inventory
-</file>
-  - Create the ''apache.conf.j2'' file
-    * <file bash apache.conf.j2>
-<VirtualHost *:{{ http_port }}>
-   ServerAdmin webmaster@localhost
-   ServerName {{ http_host }}
-   ServerAlias www.{{ http_host }}
-   DocumentRoot /var/www/{{ http_host }}
-   ErrorLog ${APACHE_LOG_DIR}/error.log
-   CustomLog ${APACHE_LOG_DIR}/access.log combined
-   <Directory /var/www/{{ http_host }}>
-         Options -Indexes
-   </Directory>
-   <IfModule mod_dir.c>
-       DirectoryIndex index.php index.html index.cgi index.pl  index.xhtml index.htm
-   </IfModule>
-</VirtualHost>
-</file>
-  - Create the ''info.php.j2'' file
-    * <file php info.php.j2>
-<?php
-phpinfo();
-</file>
-  - Create the ''.my.cnf.j2'' file
-    * <file bash .my.cnf.j2>
-[client]
-port		= 3306
-socket		= /var/run/mysqld/mysqld.sock
-user=root
-password={{ mysql_root_password }}
-</file>
-====== Step 2 - Install Server ======
-The server runs on a LAMP stack. This playbook installs the LAMP stack and git clones the Hashtopolis server application.
-TODO: add php.ini tweaking to the playbook, as necessary
-  - Create the playbook ''/home/ansible/my-project/hashtopolis/hashtopolis-server.yml''
-    * <file yaml hashtopolis-server.yml>
----
-- hosts: localhost
-  tasks:
-    - name: Download wordlist common passwords
-      get_url:
-        url: https://raw.githubusercontent.com/danielmiessler/SecLists/master/Passwords/Common-Credentials/10-million-password-list-top-100000.txt
-        dest: /home/ansible/my-project/hashtopolis/10-million-password-list-top-100000.txt
-    - name: Download wordlist rockyou.txt
-      get_url:
-        url: http://downloads.skullsecurity.org/passwords/rockyou.txt.bz2
-        dest: /home/ansible/my-project/hashtopolis/rockyou.txt.bz2
-    - name: Download oneruletorulethemall
-      get_url:
-        url: https://raw.githubusercontent.com/NotSoSecure/password_cracking_rules/master/OneRuleToRuleThemAll.rule
-        dest: /home/ansible/my-project/hashtopolis/OneRuleToRuleThemAll.rule
-- hosts: server
-  become: true
-  vars:
-    mysql_root_password: "my_sql_root_password"
-    app_user: "ansible"
-    http_host: "hashtopolis"
-    http_conf: "hashtopolis.conf"
-    http_port: "80"
-    disable_default: true
-    hashtopolis_password: "my_hashtopolis_password"
-  tasks:
-    - name: Install prerequisites
-      apt:
-        pkg:
-          - aptitude
-          - git
-          - phpmyadmin
-    # Apache Configuration
-    - name: Install LAMP Packages
-      apt:
-        pkg:
-          - apache2
-          - mysql-server
-          - python3-pymysql
-          - php
-          - php-pear
-          - php-mysql
-          - libapache2-mod-php
-    - name: Create document root
-      file:
-        path: "/var/www/{{ http_host }}"
-        state: directory
-        owner: "{{ app_user }}"
-        mode: '0755'
-    - name: Set up Apache virtualhost
-      template:
-        src: "apache.conf.j2"
-        dest: "/etc/apache2/sites-available/{{ http_conf }}"
-      notify: Reload Apache
-    - name: Enable new site
-      shell: /usr/sbin/a2ensite {{ http_conf }}
-      notify: Reload Apache
-    - name: Disable default Apache site
-      shell: /usr/sbin/a2dissite 000-default.conf
-      when: disable_default
-      notify: Reload Apache
-    # MySQL Configuration
-    - name: start and enable mysql service
-      service:
-        name: mysql
-        state: started
-        enabled: true
-    - name: manage MySQL root password
-      become: true
-      template:
-        src: /home/ansible/my-project/hashtopolis/.my.cnf.j2
-        dest: /root/.my.cnf
-    - name: Sets the hashtopolis password
-      mysql_user:
-        name: hashtopolis
-        password: "{{ hashtopolis_password }}"
-        priv: "*.*:ALL"
-        login_user: root
-        login_password: "{{ mysql_root_password }}"
-        state: present
-    - name: Removes all anonymous user accounts
-      mysql_user:
-        name: ''
-        host_all: true
-        state: absent
-        login_user: root
-        login_password: "{{ mysql_root_password }}"
-    - name: Removes the MySQL test database
-      mysql_db:
-        name: test
-        state: absent
-        login_user: root
-        login_password: "{{ mysql_root_password }}"
-    - name: Create new databases
-      mysql_db:
-        name:
-          - hashtopolis
-        state: present
-        login_user: root
-        login_password: "{{ mysql_root_password }}"
-    # UFW Configuration
-    - name: "UFW - Allow HTTP on port {{ http_port }}"
-      ufw:
-        rule: allow
-        port: "{{ http_port }}"
-        proto: tcp
-    # PHP Info Page
-    - name: Sets Up PHP Info Page
-      template:
-        src: "info.php.j2"
-        dest: "/var/www/{{ http_host }}/info.php"
-    - name: Clone Hashtopolis github repository
-      git:
-        repo: https://github.com/s3inlc/hashtopolis.git
-        dest: /home/ansible/repos/
-        clone: true
-        update: true
-    - name: Copy hastopolis/src/* to /var/www/hashtopolis
-      copy:
-        src: /home/ansible/repos/src/
-        dest: "/var/www/{{ http_host }}/"
-        remote_src: true
-        owner: www-data
-        group: www-data
-    - name: Copy wordlist 100k
-      copy:
-        src: /home/ansible/my-project/hashtopolis/10-million-password-list-top-100000.txt
-        dest: /var/www/hashtopolis/import/10-million-password-list-top-100000.txt
-        owner: www-data
-        group: www-data
-    - name: Copy wordlist rockyou.txt
-      copy:
-        src: /home/ansible/my-project/hashtopolis/rockyou.txt.bz2
-        dest: /var/www/hashtopolis/import/rockyou.txt.bz2
-        owner: www-data
-        group: www-data
-    - name: Decompress rockyou.txt
-      command: /usr/bin/bunzip2 /var/www/hashtopolis/import/rockyou.txt.bz2
-    - name: Copy oneruletorulethemall
-      copy:
-        src: /home/ansible/my-project/hashtopolis/OneRuleToRuleThemAll.rule
-        dest: /var/www/hashtopolis/import/OneRuleToRuleThemAll.rule
-        owner: www-data
-        group: www-data
-  handlers:
-    - name: Reload Apache
-      service:
-        name: apache2
-        state: reloaded
-    - name: Restart Apache
-      service:
-        name: apache2
-        state: restarted
-</file>
-  - Run the playbook
-    * ''ansible-playbook hashtopolis-server.yml''
-====== Step 3 - Configure Hashtopolis Server ======
-  - Configure the server using the Web UI
-    * open web browser and point to the server's IP address
-    * complete the installation gui to configure the server
-      * server hostname: localhost
-      * server port: 3306
-      * mysql user: hashtopolis
-      * mysql password: my_hashtopolis_password
-      * database name: hashtopolis
-    * create a login account when prompted
-  - Allow voucher reuse
-    * Click Config > Server
-    * Click Server
-    * Check "Vouchers can be used multiple times and will not be deleted automatically."
-    * Click Save Changes
-  - Import word lists
-    * Click Files
-    * Click Wordlists
-    * Under Import files select 10-million-password-list-top-100000.txt and rockyou.txt
-    * Click Import
-  - Import rule
-    * Click Files
-    * Click Rules
-    * Under Import files select OneRuleToRuleThemAll.rule
-    * Click Import
-  - After configuration is complete, remove the install directory.
-     * <file yaml remove-hashtopolis-installer.yml>
----
-- hosts: server
-  become: true
-  vars:
-    http_host: "hashtopolis"
-  tasks:
-    - name: Remove install directory
-      file:
-        path: "/var/www/{{ http_host }}/install"
-        state: "absent"
-</file>
-    * ''ansible-playbook remove-hashtopolis-installer.yml''
-====== Step 4 - Generate Voucher Codes ======
-  - Log in and create enough vouchers for all your worker nodes
-    * Click Agents > New
-    * Under Vouchers, and next to the New voucher button, click Create
-    * Repeat to generate vouchers for all your workers
-    * Save these voucher codes to ''/home/ansible/my-project/hashtopolis/vouchers.txt''
-====== Step 5 - Install Agents ======
-Intel CPUs require this runtime: "OpenCL Runtime for Intel Core and Intel Xeon Processors" (16.1.1 or later)
-  * [[https://github.com/intel/compute-runtime/releases]]
-  * hmmmm ''sudo apt install intel-opencl-icd''
-  * [[http://registrationcenter-download.intel.com/akdlm/irc_nas/vcp/15532/l_opencl_p_18.1.0.015.tgz]]
-Testing: ''sudo crackers/1/hashcat.bin -a6 -m0 hashlists/1 ?d?d?d?d?d?d?d?d''
-  - Create a j2 template for the agent configuration file
-    * <file json config.json.j2>
-{
-  "files-path": "/home/ansible/files",
-  "crackers-path": "/home/ansible/crackers",
-  "hashlists-path": "/home/ansible/hashlists",
-  "zaps-path": "/home/ansible",
-  "preprocessors-path": "/home/ansible/preprocessors",
-  "url": "http://{{ server_ip }}/api/server.php",
-  "voucher": "{{ vouchers[play_hosts.index(inventory_hostname)]}}",
-  "token": "",
-  "uuid": ""
-}
-</file>
-    * See more options for this config file at [[https://github.com/hashtopolis/agent-python]]
-  - Create unit file for the new hashtopolis-agent service
-    * <file systemd hashtopolis-agent.service>
-[Unit]
-Description=Hashtopolis Agent
-After=network.target
-[Service]
-Type=simple
-ExecStart=/usr/bin/python3 /home/ansible/hashtopolis.zip
-Restart=on-failure
-StandardOutput=syslog
-StandardError=syslog
-SyslogIdentifier=hashtopolis-agent
-WorkingDirectory=/home/ansible
-[Install]
-WantedBy=multi-user.target
-</file>
-  - Create playbook to install the agent
-    * <file yaml hashtopolis-agent.yml>
----
-- hosts: server
-- hosts: agents
-  become: true
-  vars:
-    server_ip: "{{groups['server'].0}}"
-    vouchers: "{{ lookup('file', '/home/ansible/my-project/hashtopolis/vouchers.txt').splitlines() }}"
-  tasks:
-    - name: Install prerequisites
-      apt:
-        pkg:
-          - git
-          - zip
-          - curl
-          - hashcat
-          - python3
-          - python3-psutil
-          - python3-requests
-          - pciutils
-    - name: Pull agent
-      get_url:
-        url: "http://{{ server_ip }}/agents.php?download=1"
-        dest: /home/ansible/
-    - name: Create config file
-      template:
-        src: "/home/ansible/my-project/hashtopolis/config.json.j2"
-        dest: "/home/ansible/config.json"
-    - name: Create systemd unit file
-      copy:
-        src: /home/ansible/my-project/hashtopolis/hashtopolis-agent.service
-        dest: /etc/systemd/system
-        owner: root
-        mode: 644
-    - name: Just force systemd to reread configs (2.4 and above)
-      systemd:
-        daemon_reload: true
-    - name: Start hashtopolis-agent service
-      systemd:
-        name: hashtopolis-agent
-        enabled: true
-        state: started
-</file>
-  - ''ansible-playbook hashtopolis-agent.yml''
-  - If some agents are not coming on-line, check the ''config.json'' for a missing voucher. Put in a voucher code and ''sudo systemctl restart hashtopolis-agent.service''
-====== Step 6 - Confirm Agents are Up and Running ======
-  - Check the services using a playbook
-    * <file yaml check-agent-service.yml>
----
-- hosts: agents
-  tasks:
-    - name: Get Service Status
-      ansible.builtin.systemd:
-        name: "hashtopolis-agent"
-      register: hta_service_status
-    - debug:
-        var: hta_service_status.status.ActiveState
-</file>
-  - Log in to the Hashtopolis dashboard and view the agents
-  - Edit each agent "Trust" setting by checking the box for "Trust agent with secret data"
-====== Step 7 - Create Sample md5 Password Hashes ======
-  - Create a list of passwords you want to crack
-    * Use a variety of passwords
-      * poor passwords
-      * kids passwords ([[https://www.dinopass.com/]])
-      * short leet passwords
-      * short truly random passwords
-        * <code bash>
-sudo apt install pwgen -ypwgen
-pwgen 5 1
-pwgen 7 1
-</code>
-  - Put the passwords in a file ''passwords.txt''
-    * <file bash passwords.txt>
-Butterfly123!
-returnofthejedi
-J@sonHouse
-sillywombat11
-mi$tyHelp55
-January2022
-P@$$w0rd
-Ewug4
-ieMuth6
-covidsucks
-</file>
-  - Create a list of md5 hashes of these passwords (we are cracking with very old NUCs after all) in the file ''hashes.txt''
-      * <file bash hash-passwords.sh>
-#!/bin/bash
-file="passwords.txt"
-output="hashes.txt"
-while read -r line
-do
-    /bin/echo -n "$line" | md5sum | cut -d' ' -f 1 >> "$output"
-done < $file
-</file>
-      * <file bash hashes.txt>
-c67bd5694775b082d3d858a1882afb1
-dd02e107e35921b778bd4d61be734b1
-ffda037af61805f2d797d4cafce12
-f81e55522520a7646c6d5a1c643cda9c
-f43b4850a2ecd83471d7e938d54a636
-d745dd658451e50f969e132e86de9f
-c53e479b03b3220d3d56da88c4cace20
-fd350c0534cff7b3e4bc6b99b11c1286
-cfbf38310f0bb6972addb10494db51db
-dc3dac9e39bfa97fdf180899b9e81f
-</file>
-  - SORT the file
-    * ''sort -o hashes.txt hashes.txt''
-    * <file bash hashes.txt>
-dc3dac9e39bfa97fdf180899b9e81f
-d745dd658451e50f969e132e86de9f
-f43b4850a2ecd83471d7e938d54a636
-c67bd5694775b082d3d858a1882afb1
-dd02e107e35921b778bd4d61be734b1
-ffda037af61805f2d797d4cafce12
-c53e479b03b3220d3d56da88c4cace20
-cfbf38310f0bb6972addb10494db51db
-f81e55522520a7646c6d5a1c643cda9c
-fd350c0534cff7b3e4bc6b99b11c1286
-</file>
-  - Upload the hashes.txt file to Hashtopolis
-    * Lists > New hashlist
-      * Name: hashes.txt
-      * Hashtype: 0 (md5)
-      * Hashlist format: Text File
-      * Hash source: Upload
-      * File to upload: Click Choose File, then select the file
-      * Click Create hashlist
-====== Step 8 - Create Tasks to Crack the Hashes ======
-  - Tasks > New Task
-    * Name: demo
-    * Hashlist: hashes.txt
-    * Command:
-      * Click Rules then check (under T) OneRuleToRuleThemAll.rule
-      * Click Wordlists then check (under T) rockyou.txt
-      * Command should be: "#HL# -r OneRuleToRuleThemAll.rule rockyou.txt"
-    * Priority: leave 10 (greater than 0)
-    * Maximum number of agents: leave 0
-    * Task notes: demo
-    * Color: A00000
-    * Click Create Task
-    * Under Assigned agents
-      * For each node click Assign
-      * WARNING if the task assignment fails, modify the agent(s) to be "trusted" with secret data
-  - Wait for your job to complete
-    * Click Lists > Cracks to view cracked passwords
-      * First to be cracked:
-        * P@$$w0rd
-        * Butterfly123!
-        * January2022
-        * covidsucks
-        * sillywombat11
-        * Ewug4
-      * Consider what the difference would be without using the rule or with using the smaller word list
-  - Tasks > New Task
-    * Name: brute7
-    * Hashlist: hashes.txt
-    * Command:
-      * ''-a3 #HL# ?a?a?a?a?a?a?a''
-    * Priority: 10
-    * Maximum number of agents: leave 0
-    * Task notes: brute force
-    * Color: 00A000
-    * Click Create Task
-  - Tasks > New Task
-    * Name: brute8
-    * Hashlist: hashes.txt
-    * Command:
-      * ''-a3 #HL# ?a?a?a?a?a?a?a?a''
-    * Priority: 10
-    * Maximum number of agents: leave 0
-    * Task notes: brute force
-    * Color: 00A000
-    * Click Create Task
-  - Create more tasks for longer lengths if you'd like
-====== Step 9 - Uninstall Hashtopolis ======
-  - Create the file /home/ansible/my-project/fah/remove-hashtopolis.yml
-    * <file yaml remove-hashtopolis.yml>
----
-- hosts: server
-  become: true
-  become_user: root
-  tasks:
-    - name: Stop and disable services
-      service:
-        name: "{{ item }}"
-        state: stopped
-        enabled: false
-      with_items:
-        - apache2
-        - mysql
-    - name: Remove packages
-      apt:
-        name:
-          - apache2
-          - mysql
-          - python3-pymysql
-          - php
-          - php-pear
-          - php-mysql
-          - libapache2-mod-php
-        state: absent
-        autoclean: true
-        purge: true
-    - name: Remove directory
-      file:
-        state: absent
-        path: /var/www/hashtopolis
-- hosts: agents
-  become: true
-  become_user: root
-  tasks:
-    - name: Stop and disable services
-      service:
-        name: "{{ item }}"
-        state: stopped
-        enabled: false
-      with_items:
-        - hashtopolis-agent
-    - name: Remove packages
-      apt:
-        name:
-          - git
-          - zip
-          - curl
-          - hashcat
-          - python3-psutil
-          - python3-requests
-          - pciutils
-        state: absent
-        autoclean: true
-        purge: true
-</file>
-  - Run the playbook: ''ansible-playbook remove-hashtopolis.yml''
-NOTE After running playbook to remove Hashtoplis, I would that upon reinstalling the server, the Hashtopolis server PHP stopped working. The following are commands to fix that issue.
-<code bash>
-sudo apt install php-fpm
-sudo a2enmod proxy_fcgi setenvif
-sudo a2enconf php8.1-fpm
-sudo systemctl restart apache2
-</code>
-====== Optional ======
-Try cracking other hashes.
-===== Ubuntu type 1800 =====
-  - On [[NUC 1]]
-    - Create a user account name ''tryhackme'' on [[NUC 1]] with a simple password, like ''password''
-    - Dump the hash for the user ''tryhackme''
-      * ''sudo grep tryhackme/etc/shadow | cut -f 2 -d":"''
-  - In Hashtopolis
-    - Create a lastlist
-      * name ''Unix''
-      * paste in the hash (starts with and includes "$6$")
-        * you can also use the example hash from [[https://hashcat.net/wiki/doku.php?id=example_hashes]]
-      * hashtype 1800 - sha512crypt, SHA512(UNIX)
-      * No check for salted hashes, separator
-      * No check for salt is in hex (only when salted hashes)
-    - Create a task
-      * name unix
-      * hashlist Unix
-      * worklist rockyou.txt
-      * priority 5
-      * attack command: ''#HL# rockyou.txt''
-===== Windows =====
-Note that I'm still working out the specifics
-Reference: [[https://hashcat.net/wiki/doku.php?id=example_hashes]]. You can use the provided hashes for testing.
-Issues:
-  * benchmarks are seeming to get stuck
-  * agents error out
-  * plaintext comes back blank
-  - On the Windows 10 PC
-    - Add an account with the a simple password, such as ''password''
-      * net user add person /active:yes /add
-      * net localgroup administrators /add person
-        * making this user an administrator makes it show up easier to find in the password hash dump
-      * ''net user person *''
-        * set the password to something easy like ''password''
-    - Extract the SAM and SYSTEM registry hives
-      * Open a shell as Administrator
-        * ''reg save hklm\sam c:\sam''
-        * ''reg save hklm\system c:\system''
-        * the last parameter is the location to copy the file two
-  - On [[NUC 1]]
-    - Copy the SAM and SYSTEM registry hives to [[NUC 1]]
-    - Install impacket-secretsdump
-      * ''sudo apt install python3-impacket -y''
-    - Dump the system keys and hashes
-      * If the files are named "sam" and "system"
-        * ''impacket-secretsdump -sam SAM -system SYSTEM LOCAL''
-    - For your test user (i.e. ''person'') there are two hashes
-      * one for LM authentication, (deprecated and only populated with a value to ensure the syntax remains constant)
-      * other is the NTLM string
-      * For example, for person:
-        * LM aad3b435b51404eeaad3b435b51404ee = means LM is not being stored
-          * might this actually be NTLM hash? let's try
-        * NTLM 8846f7eaee8fb117ad06bdd830b7586c
-        * Warning 31d6cfe0d16ae931b73c59d7e0c089c0 is blank password, and it means weren't open to pull the hashes successfully. samdump2 for example, gives this hash.
-  - In Hashtopolis
-    - Create new hashlists
-      * LM
-        * Name: LM
-        * Hashtype: 3000 - LM
-        * Paste in text, the LM hash you dumped
-        * Create
-        * Trying Runtime Benchmark
-      * NTLM
-        * Name: NTLM
-        * Hashtype: 1000 - NTLM
-        * Paste in text, the LM hash you dumped
-        * Create
-    - Create new tasks
-      * LM
-        * Name: LM
-        * Hashlist: LM
-        * Enable rule OneRuleToRuleThemAll.rule
-        * Enable worklist rockyou.txt
-        * Priority: 10
-        * Attack command: ''#HL# rockyou.txt -r OneRuleToRuleThemAll.rule''
-      * NTLM
-        * Name: LM
-        * Hashlist: LM
-        * Enable rule OneRuleToRuleThemAll.rule
-        * Enable worklist rockyou.txt
-        * Priority: 9
-        * Attack command: ''#HL# rockyou.txt -r OneRuleToRuleThemAll.rule''
-        * If the plaintext comes back blank....??