Edit Solution Configuration

Edit .conf Copy the template config file to /etc and then edit it for your machine configuration. We recommend only changing the HUSER, UINODE, LUSER, and NODES variables.

[soluser@node-04]$ git clone  https://github.com/Open-Network-Insight/oni-setup.git
[soluser@node-04]$ cd setup 
[soluser@node-04 setup]$ sudo cp <solution>.conf /etc/.
[soluser@node-04 setup]$ sudo vim /etc/<solution>.conf

Once the file has been edited, copy it to the three nodes named as UINODE, MLNODE, and GWNODE in the config file.

Below is what the default configuration file looks like:

#node configuration
NODES=('node-01' 'node-02')
UINODE='node03'
MLNODE='node04'
GWNODE='node16'

#HDFS - base user and data source config
HUSER='/user/duxbury'
DSOURCES=('flow' 'dns')
DFOLDERS=('binary' 'csv' 'hive' 'stage')
DPATH=${HUSER}/${DSOURCE}/${DFOLDER}/y=${YR}/m=${MH}/d=${DY}
HPATH=${HUSER}/${DSOURCE}/${DFOLDER}/lda'${FDATE}
DBNAME=’duxbury’

KRB_AUTH=false
KINITPATH=
KINITOPTS=
KEYTABPATH=
KRB_USER=

#LOCAL FS base user and data source config
LUSER='/home/duxbury'
LPATH=${LUSER}/ml/${FDATE}
RPATH=${LUSER}/ipython/user/${FDATE}
LDAPATH=${LUSER}/ml/lda-c-parallel
LIPATH=${LUSER}/ingest

SPK_EXEC='400'
SPK_EXEC_MEM='2048m'

The following variables are needed throughout the ML pipeline:

• NODES – a space delimited list of the Data Nodes that will run the C/MPI part of the pipeline. Be very careful to keep * the variable in the format (‘host1’ ‘host2’ ‘host3’ …).
• UINODE – the node that runs the Operational Analytics will run (aka, user interface node).
• MLNODE- the node that runs the ML pipeline, controlling the other nodes.
• GWNODE – the node that runs the ingest process.
• DBNAME – the name of the database used by the solution
• HUSER – HDFS user path that will be the base path for the solution; this is usually the same user that you created to run the solution
• DSOURCES – data sources enabled in this installation
• DFOLDERS – built-in paths for the directory structure in HDFS
• DPATH – the path to the flow records in Hive; this will be dynamically built within the pipeline with values for ${YR}, ${MH} and ${DY}
• HPATH –
• KRB_AUTH – (default: false) turn Kerberos authentication features on/off
• KINITPATH -
• KINITOPTS -
• KEYTABPATH -
• KRB_USER -
• LUSER – the local filesystem path for the solution, ‘/home/solution-user/’
• LPATH – the local path for the ML intermediate and final results, dynamically built when the pipeline runs
• RPATH – the path on the Operational Analytics node where the pipeline output will be delivered
• LDAPATH –
• LIPATH –
• SPK_EXEC – number of spark executors
• SPK_EXEC_MEM – size (in MB) of spark executor