Skip to content

Install Ingest Prerequisites

natedogs911 edited this page Apr 21, 2016 · 1 revision

Installing prerequisites should be done in a directory created under the /home// directory. It is recommended to create a temporary folder called “src” so that it can be deleted easily after successful validation.

mkdir src
cd src

First copy the modified nfdump source code and tshark source code to the /src directory. For nfdump, follow the steps below to build and install it.

# a GCC compiler is required for this step
sudo yum -y groupinstall “Development Tools”
git clone https://github.com/Open-Network-Insight/oni-nfdump.git
cd oni-nfdump
./install_nfdump.sh
cd ..

Install the prerequisites for the ingest queue (if the version in your yum repository is the same or later, you may also use yum to install a precompiled binary).

wget --no-check-certificate https://bootstrap.pypa.io/get-pip.py
sudo -H python get-pip.py
sudo -H pip install watchdog
wget --no-check-certificate https://pypi.python.org/packages/source/p/pika/pika-0.10.0b2.tar.gz
tar -zxvf pika-0.10.0b2.tar.gz
cd pika-0.10.0b2
sudo python setup.py install
cd ..
wget https://www.rabbitmq.com/releases/erlang/erlang-17.4-1.el6.x86_64.rpm
sudo rpm -i erlang-17.4-1.el6.x86_64.rpm
wget --no-check-certificate https://www.rabbitmq.com/releases/rabbitmq-server/v3.5.3/rabbitmq-server-3.5.3-1.noarch.rpm	
sudo rpm -i rabbitmq-server-3.5.3-1.noarch.rpm

For tshark, follow the steps on the web site to install it. Tshark must be downloaded and built from Wireshark page

Full instructions for compiling Wireshark can be found here instructions for compiling

#install prerequisites
sudo yum -y install gtk2-devel gtk+-devel bison qt-devel qt5-qtbase-devel
sudo yum -y groupinstall "Development Tools"
sudo yum -y install libpcap-devel
#if you can not locate the libpcap-devel package you must compile from source
wget http://www.tcpdump.org/release/libpcap-1.7.4.tar.gz
tar xvf libpcap-1.7.4.tar.gz
cd libpcap-1.7.4
./configure --prefix=/usr
sudo make install
cd ..
#compile Wireshark
wget https://1.na.dl.wireshark.org/src/wireshark-2.0.1.tar.bz2
tar xvf wireshark-2.0.1.tar.bz2 
cd wireshark-2.0.1
./configure --with-gtk2
make
sudo make install
cd ..

The screen utility is used to capture output from the ingest component for logging, troubleshooting, etc. You can check if screen is installed on the node.

which screen

If screen is not available, install it.

[soluser@edge-node] sudo yum install screen
  • Home
  • [Overview of Open Network Insight](Overview of Open Network Insight)
    • [Technical Overview](Technical Overview)
  • [Planning Guide](Planning Guide)
    • [Deployment Option 1: Pure Hadoop](Pure Hadoop)
    • [Deployment Option 2: Hybrid Hadoop / Virtual](Hybrid Hadoop)
  • Development Guide
  • [Deployment Guide](Deployment Guide)
  • [Installation Guide](Installation Guide)
    • Step 0: Initial Configuration
      • [Configure User Accounts](Configure User Accounts)
      • [Edit Solution Configuration](Edit Solution Configuration).
      • [Setup HDFS](Setup HDFS)
    • Step 1: Ingest Component Installation
      • [Install Prerequisites](Install Ingest Prerequisites)
      • [Install and Configure Ingest](Install and Configure Ingest)
      • [Validate](Validate ingest)
    • Step 2: Machine Learning
      • [Install Prerequisites](Install ML Prerequisites).
      • [Install and Configure ML](Install and Configure ML)
      • [Validate](Validate ML)
    • Step 3: Operational Analytics Component Installation
      • [Install Prerequisites](Install OA Prerequisites).
      • [Install OA solution](Install OA Solution)
      • [Validate](Validate OA)
  • [User Guide](User Guide)
    • [Suspicious Connects – Analyst View](Suspicious Connects)
    • [Threat Investigation – Analyst View](Threat Investigation)
    • [Ingest Summary – Analyst View](Ingest Summary)

Clone this wiki locally