1 - Processing Users
Processing User Setup
Stroom and Stroom Proxy should be run under a processing user (we assume stroomuser below).
Create user
You may want to allow normal accounts to sudo to this account for maintenance (visudo).
Create service script
Create a service script to start/stop on server startup (as root).
Paste/type the following content into vi.
#!/bin/bash
#
# stroomuser This shell script takes care of starting and stopping
# the stroomuser subsystem (tomcat6, etc)
#
# chkconfig: - 86 14
# description: stroomuser is the stroomuser sub system
STROOM_USER=stroomuser
DEPLOY_DIR=/home/${STROOM_USER}/stroom-deploy
case $1 in
start)
/bin/su ${STROOM_USER} ${DEPLOY_DIR}/stroom-deploy/start.sh
;;
stop)
/bin/su ${STROOM_USER} ${DEPLOY_DIR}/stroom-deploy/stop.sh
;;
restart)
/bin/su ${STROOM_USER} ${DEPLOY_DIR}/stroom-deploy/stop.sh
/bin/su ${STROOM_USER} ${DEPLOY_DIR}/stroom-deploy/start.sh
;;
esac
exit 0
Now initialise the script.
Setup user’s environment
Setup env.sh to include JAVA_HOME to point to the installed directory of the JDK (this will be platform specific).
In vi add the following lines.
# User specific aliases and functions
export JAVA_HOME=/usr/lib/jvm/java-1.8.0
export PATH=${JAVA_HOME}/bin:${PATH}
Setup the user’s profile to include source the env script.
In vi add the following lines.
# User specific aliases and functions
. ~/env.sh
Verify Java installation
Assuming you are using Stroom without using docker and have installed Java, verify that the processing user can use the Java installation.
The shell output below may show a different version of Java to the one you are using.
2 - Securing Stroom
NOTE This document was written for stroom v4/5. Some parts may not be applicable for v6+.
Firewall
The following firewall configuration is recommended:
- Outside cluster drop all access except ports HTTP 80, HTTPS 443, and any other system ports your require SSH, etc
- Within cluster allow all access
This will enable nodes within the cluster to communicate on:
- 8080 - Stroom HTTP.
- 8081 - Stroom HTTP (admin).
- 8090 - Stroom Proxy HTTP.
- 8091 - Stroom Proxy HTTP (admin).
- 3306 - MySQL
MySQL
TODO
Update this for MySQL 8It is recommended that you run mysql_secure_installation to set a root password and remove test database:
mysql_secure_installation (provide a root password)
- Set root password? [Y/n] Y
- Remove anonymous users? [Y/n] Y
- Disallow root login remotely? [Y/n] Y
- Remove test database and access to it? [Y/n] Y
- Reload privilege tables now? [Y/n] Y
3 - Java Key Store Setup
TODO
This is out of date for stroom 7.In order that the java process communicates over https (for example Stroom Proxy forwarding onto Stroom) the JVM requires relevant keystore’s setting up.
As the processing user copy the following files to a directory stroom-jks in the processing user home directory :
- CA.crt - Certificate Authority
- SERVER.crt - Server certificate with client authentication attributes
- SERVER.key - Server private key
As the processing user perform the following:
- First turn your keys into der format:
cd ~/stroom-jks
SERVER=<SERVER crt/key PREFIX>
AUTHORITY=CA
openssl x509 -in ${SERVER}.crt -inform PEM -out ${SERVER}.crt.der -outform DER
openssl pkcs8 -topk8 -nocrypt -in ${SERVER}.key -inform PEM -out ${SERVER}.key.der -outform DER
- Import Keys into the Key Stores:
Stroom_UTIL_JAR=`find ~/*app -name 'stroom-util*.jar' -print | head -1`
java -cp ${Stroom_UTIL_JAR} stroom.util.cert.ImportKey keystore=${SERVER}.jks keypass=${SERVER} alias=${SERVER} keyfile=${SERVER}.key.der certfile=${SERVER}.crt.der
keytool -import -noprompt -alias ${AUTHORITY} -file ${AUTHORITY}.crt -keystore ${AUTHORITY}.jks -storepass ${AUTHORITY}
- Update Processing User Global Java Settings:
PWD=`pwd`
echo "export JAVA_OPTS=\"-Djavax.net.ssl.trustStore=${PWD}/${AUTHORITY}.jks -Djavax.net.ssl.trustStorePassword=${AUTHORITY} -Djavax.net.ssl.keyStore=${PWD}/${SERVER}.jks -Djavax.net.ssl.keyStorePassword=${SERVER}\"" >> ~/env.sh
Any Stroom or Stroom Proxy instance will now additionally pickup the above JAVA_OPTS settings.
4 - MySQL Setup
TODO
This needs updating to MySQL 8. Stroom v7 requires MySQL 8.Prerequisites
- MySQL 8.0.x server installed (e.g. yum install mysql-server)
- Processing User Setup
A single MySQL database is required for each Stroom instance. You do not need to setup a MySQL instance per node in your cluster.
Check Database installed and running
[root@stroomdb ~]# /sbin/chkconfig --list mysqld
mysqld 0:off 1:off 2:on 3:on 4:on 5:on 6:off
[root@stroomdb ~]# mysql --user=root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
...
mysql> quit
The following commands can be used to auto start mysql if required:
Overview
MySQL configuration can be simple to complex depending on your requirements.
For a very simple configuration you simply need an out-of-the-box mysql
install and create a database user account.
Things get more complicated when considering:
- Security
- Master Slave Replication
- Tuning memory usage
- Running Stroom Stats in a different database to Stroom
- Performance Monitoring
Simple Install
Ensure the database is running, create the database and access to it
[stroomuser@host stroom-setup]$ mysql --user=root
Welcome to the MySQL monitor. Commands end with ; or \g.
...
mysql> create database stroom;
Query OK, 1 row affected (0.02 sec)
mysql> grant all privileges on stroom.* to 'stroomuser'@'host' identified by 'password';
Query OK, 0 rows affected (0.00 sec)
mysql> create database stroom_stats;
Query OK, 1 row affected (0.02 sec)
mysql> grant all privileges on stroom_stats.* to 'stroomuser'@'host' identified by 'password';
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
Advanced Security
It is recommended to run /usr/bin/mysql_secure_installation to remove test database and accounts.
./stroom-setup/mysql_grant.sh is a utility script that creates accounts for you to use within a cluster (or single node setup). Run to see the options:
[stroomuser@host stroom-setup]$ ./mysql_grant.sh
usage : --name=<instance name (defaults to my for /etc/my.cnf)>
--user=<the stroom user for the db>
--password=<the stroom password for the db>
--cluster=<the file with a line per node in the cluster>
--user=<db user> Must be set
N.B. name is used when multiple mysql instances are setup (see below).
You need to create a file cluster.txt with a line for each member of your cluster (or single line in the case of a one node Stroom install). Then run the utility script to lock down the server access.
[stroomuser@host ~]$ hostname >> cluster.txt
[stroomuser@host ~]$ ./stroom-setup/mysql_grant.sh --name=mysql56_dev --user=stroomuser --password= --cluster=cluster.txt
Enter root mysql password :
--------------
flush privileges
--------------
--------------
delete from mysql.user where user = 'stroomuser'
--------------
...
...
...
--------------
flush privileges
--------------
[stroomuser@host ~]$
Advanced Install
The below example uses the utility scripts to create 3 custom mysql server instances on 2 servers:
- server1 - master stroom,
- server2 - slave stroom, stroom_stats
As root on server1:
yum install "mysql56-mysql-server"
Create the master database:
[root@node1 stroomuser]# ./stroom-setup/mysqld_instance.sh --name=mysqld56_stroom --port=3106 --server=mysqld56 --os=rhel6
--master not set ... assuming master database
Wrote base files in tmp (You need to move them as root). cp /tmp/mysqld56_stroom /etc/init.d/mysqld56_stroom; cp /tmp/mysqld56_stroom.cnf /etc/mysqld56_stroom.cnf
Run mysql client with mysql --defaults-file=/etc/mysqld56_stroom.cnf
[root@node1 stroomuser]# cp /tmp/mysqld56_stroom /etc/init.d/mysqld56_stroom; cp /tmp/mysqld56_stroom.cnf /etc/mysqld56_stroom.cnf
[root@node1 stroomuser]# /etc/init.d/mysqld56_stroom start
Initializing MySQL database: Installing MySQL system tables...
OK
Filling help tables...
...
...
Starting mysql56-mysqld: [ OK ]
Check Start up Settings Correct
[root@node2 stroomuser]# chkconfig mysqld off
[root@node2 stroomuser]# chkconfig mysql56-mysqld off
[root@node1 stroomuser]# chkconfig --add mysqld56_stroom
[root@node1 stroomuser]# chkconfig mysqld56_stroom on
[root@node2 stroomuser]# chkconfig --list | grep mysql
mysql56-mysqld 0:off 1:off 2:off 3:off 4:off 5:off 6:off
mysqld 0:off 1:off 2:off 3:off 4:off 5:off 6:off
mysqld56_stroom 0:off 1:off 2:on 3:on 4:on 5:on 6:off
mysqld56_stats 0:off 1:off 2:on 3:on 4:on 5:on 6:off
Create a text file will all members of the cluster:
[root@node1 stroomuser]# vi cluster.txt
node1.my.org
node2.my.org
node3.my.org
node4.my.org
Create the grants:
[root@node1 stroomuser]# ./stroom-setup/mysql_grant.sh --name=mysqld56_stroom --user=stroomuser --password=password --cluster=cluster.txt
As root on server2:
[root@node2 stroomuser]# yum install "mysql56-mysql-server"
[root@node2 stroomuser]# ./stroom-setup/mysqld_instance.sh --name=mysqld56_stroom --port=3106 --server=mysqld56 --os=rhel6 --master=node1.my.org --user=stroomuser --password=password
--master set ... assuming slave database
Wrote base files in tmp (You need to move them as root). cp /tmp/mysqld56_stroom /etc/init.d/mysqld56_stroom; cp /tmp/mysqld56_stroom.cnf /etc/mysqld56_stroom.cnf
Run mysql client with mysql --defaults-file=/etc/mysqld56_stroom.cnf
[root@node2 stroomuser]# cp /tmp/mysqld56_stroom /etc/init.d/mysqld56_stroom; cp /tmp/mysqld56_stroom.cnf /etc/mysqld56_stroom.cnf
[root@node1 stroomuser]# /etc/init.d/mysqld56_stroom start
Initializing MySQL database: Installing MySQL system tables...
OK
Filling help tables...
...
...
Starting mysql56-mysqld: [ OK ]
Check Start up Settings Correct
[root@node2 stroomuser]# chkconfig mysqld off
[root@node2 stroomuser]# chkconfig mysql56-mysqld off
[root@node1 stroomuser]# chkconfig --add mysqld56_stroom
[root@node1 stroomuser]# chkconfig mysqld56_stroom on
[root@node2 stroomuser]# chkconfig --list | grep mysql
mysql56-mysqld 0:off 1:off 2:off 3:off 4:off 5:off 6:off
mysqld 0:off 1:off 2:off 3:off 4:off 5:off 6:off
mysqld56_stroom 0:off 1:off 2:on 3:on 4:on 5:on 6:off
Create the grants:
[root@node1 stroomuser]# ./stroom-setup/mysql_grant.sh --name=mysqld56_stroom --user=stroomuser --password=password --cluster=cluster.txt
Make the slave database start to follow:
[root@node2 stroomuser]# cat /etc/mysqld56_stroom.cnf | grep "change master"
# change master to MASTER_HOST='node1.my.org', MASTER_PORT=3106, MASTER_USER='stroomuser', MASTER_PASSWORD='password';
[root@node2 stroomuser]# mysql --defaults-file=/etc/mysqld56_stroom.cnf
mysql> change master to MASTER_HOST='node1.my.org', MASTER_PORT=3106, MASTER_USER='stroomuser', MASTER_PASSWORD='password';
mysql> start slave;
As processing user on server1:
[stroomuser@node1 ~]$ mysql --defaults-file=/etc/mysqld56_stroom.cnf --user=stroomuser --password=password
mysql> create database stroom;
Query OK, 1 row affected (0.00 sec)
mysql> use stroom;
Database changed
mysql> create table test (a int);
Query OK, 0 rows affected (0.05 sec)
As processing user on server2 check server replicating OK:
[stroomuser@node2 ~]$ mysql --defaults-file=/etc/mysqld56_stroom.cnf --user=stroomuser --password=password
mysql> show create table test;
+-------+----------------------------------------------------------------------------------------+
| Table | Create Table |
+-------+----------------------------------------------------------------------------------------+
| test | CREATE TABLE `test` (`a` int(11) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
+-------+----------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
As root on server2:
[root@node2 stroomuser]# /home/stroomuser/stroom-setup/mysqld_instance.sh --name=mysqld56_stats --port=3206 --server=mysqld56 --os=rhel6 --user=statsuser --password=password
[root@node2 stroomuser]# cp /tmp/mysqld56_stats /etc/init.d/mysqld56_stats; cp /tmp/mysqld56_stats.cnf /etc/mysqld56_stats.cnf
[root@node2 stroomuser]# /etc/init.d/mysqld56_stats start
[root@node2 stroomuser]# chkconfig mysqld56_stats on
Create the grants:
[root@node2 stroomuser]# ./stroom-setup/mysql_grant.sh --name=mysqld56_stats --database=stats --user=stroomstats --password=password --cluster=cluster.txt
As processing user create the database:
[stroomuser@node2 ~]$ mysql --defaults-file=/etc/mysqld56_stats.cnf --user=stroomstats --password=password
Welcome to the MySQL monitor. Commands end with ; or \g.
....
mysql> create database stats;
Query OK, 1 row affected (0.00 sec)