Vinod Pandey

Appenlight Setup in Centos

 931  5 mins

Creating appenlight User

1

sudo useradd appenlight -r -s /sbin/nologin

Python 3.6 installation & supervisor installation

Install below pre-requisites:

1

sudo yum -y install gcc openssl-devel glibc-headers bzip2-devel wget vim git unzip

Download python

1
2

cd /tmp/
wget https://www.python.org/ftp/python/3.6.6/Python-3.6.6.tgz

Python Alternate Installation

1
2
3
4
5
6

tar xzf Python-3.6.6.tgz
cd Python-3.6.6
./configure --enable-optimizations
sudo make altinstall

sudo ln -sfn /usr/local/bin/python3.6 /usr/bin/python3.6

Supervisor installation

1
2
3
4
5
6

pip3.6 install supervisor

sudo ln -sfn /usr/local/bin/supervisord /usr/bin/supervisord
sudo ln -sfn /usr/local/bin/supervisorctl /usr/bin/supervisorctl

sudo mkdir -p /etc/supervisor/conf.d/

create configuration file

1

sudo sh -c "/usr/local/bin/echo_supervisord_conf >> /etc/supervisord.conf"

config update

1
2
3
4
5

sudo vim /etc/supervisord.conf

add below lines:
[include]
files = /etc/supervisor/conf.d/*.conf

start supervisor

1

sudo /usr/bin/supervisord -c /etc/supervisord.conf

Redis Installation

1
2
3
4
5
6
7
8

sudo yum install -y tcl
cd /tmp/
wget http://download.redis.io/releases/redis-3.2.6.tar.gz
tar xzf redis-3.2.6.tar.gz
cd redis-3.2.6
make
make test 
sudo make install

Execute install script to generate redis config file,  /etc/init.d script, install them and run redis server.

1
2
3
4
5

sudo REDIS_PORT=6379 \
     REDIS_CONFIG_FILE=/etc/redis/6379.conf \
     REDIS_LOG_FILE=/var/log/redis_6379.log \
     REDIS_DATA_DIR=/var/lib/redis/6379 \
     REDIS_EXECUTABLE=/usr/local/bin/redis-server ./utils/install_server.sh

Set overcommit_memory to 1 (always overcommit, never check)

1
2
3
4
5
6
7
8
9

sudo sysctl vm.overcommit_memory=1
sudo sysctl -w fs.file-max=100000

# to make the change permanent
# open sysctl.conf for editing
sudo vim /etc/sysctl.conf

# add below line
vm.overcommit_memory=1

Redis guidances  on why vm.overcommit_memory should be set to 1.

Start redis on system restart

1

sudo chkconfig --add redis_6379

Postgres Installation

Install Postres using rpm :

1
2

sudo yum -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-6-x86_64/pgdg-redhat-repo-latest.noarch.rpm
sudo yum -y install postgresql95-server postgresql95-contrib postgresql95 libpqxx-devel postgresql-devel

Initialize database environment:

1

sudo service postgresql-9.5 initdb

Configure it to start at boot up and start the software:

1
2

sudo chkconfig postgresql-9.5 on
sudo service postgresql-9.5 start

psql setup

1
2

sudo mv /usr/bin/psql /usr/bin/psql-bk
sudo ln -sfn /usr/pgsql-9.5/bin/psql /usr/bin/psql

enable password based login in postgres

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

sudo vim /var/lib/pgsql/9.5/data/pg_hba.conf

# add appenligt md5 row keeping the order same

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     peer
# IPv4 local connections:
host    appenlight      appenlight      127.0.0.1/32            md5
host    all             all             127.0.0.1/32            ident


service  postgresql-9.5 restart

elastic search 6.x installation

install java

1

sudo yum -y install java-1.8.0-openjdk

install elastic search

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

sudo vi /etc/yum.repos.d/elasticsearch.repo

[Elasticsearch-6]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

sudo yum -y install elasticsearch

sudo service elasticsearch start
sudo service elasticsearch status
curl http://localhost:9200/

virtual environment setup & appenlight configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

sudo mkdir /opt/appenlight/
cd /opt/appenlight/

sudo wget https://github.com/AppEnlight/appenlight/archive/master.zip
sudo unzip master.zip

python3.6 -m venv venv
source venv/bin/activate

pip install appenlight
pip install ae_uptime_ce

# generate production.ini
appenlight-make-config production.ini

# update elastic search path
# we are assuming elastic search is running on docker host machine
# change accordingly
elasticsearch.nodes = http://localhost:9200

# disable registration
appenlight.disable_registration = true

# update password from test to something more secure
# and set the same in below postgres command
sqlalchemy.url = postgresql://appenlight:test@127.0.0.1:5432/appenlight

# Creating database for appenlight in postgres
# change password from 'test' to something more secure

sudo -u postgres psql template1
template1=# CREATE DATABASE appenlight;
template1=# CREATE USER appenlight WITH PASSWORD 'test';
template1=# GRANT ALL PRIVILEGES ON DATABASE appenlight to appenlight;
template1=# \q


# setup database structure
appenlight-migratedb -c production.ini

# elastic search configutation

appenlight-reindex-elasticsearch -t all -c production.ini


# Create base database objects and admin account
appenlight-initializedb -c production.ini

# Generate static assets
appenlight-static -c production.ini

supervisor config

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

sudo mkdir -p /var/log/gunicorn/

sudo vim /etc/supervisor/conf.d/appenlight.conf

[program:appenlight]
directory=/opt/appenlight/
command=/opt/appenlight/venv/bin/pserve production.ini 
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/var/log/gunicorn/appenlight-std.log
stderr_logfile=/var/log/gunicorn/appenlight-err.log

[program:appenlight-worker]
directory=/opt/appenlight/
command=/opt/appenlight/venv/bin/celery worker -A appenlight.celery -Q "reports,logs,metrics,default" --ini production.ini
user=appenlight
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/var/log/gunicorn/appenlight-worker-std.log
stderr_logfile=/var/log/gunicorn/appenlight-worker-err.log

[program:appenlight-beat]
directory=/opt/appenlight/
command=/opt/appenlight/venv/bin/celery beat -A appenlight.celery --ini production.ini
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/var/log/gunicorn/appenlight-beat-std.log
stderr_logfile=/var/log/gunicorn/appenlight-beat-err.log


sudo supervisorctl shutdown

sudo supervisord -c /etc/supervisord.conf

sudo supervisorctl status

access: http://localhost:6543/

apache setup

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

<VirtualHost *:80>
  ServerName api.example.com
  ServerAlias api.example.com
  DocumentRoot /srv/api.example.com/public_html
  ErrorLog /var/log/httpd/api.example.com_error_log
  CustomLog /var/log/httpd/api.example.com_access_log combined
  ScriptAlias /cgi-bin/ /srv/api.example.comcgi-bin/
  DirectoryIndex index.html index.htm index.php index.php4 index.php5

  <Directory /srv/api.example.com/public_html>
    Options -Indexes +IncludesNOEXEC +SymLinksIfOwnerMatch +ExecCGI
    allow from all
    AllowOverride All
  </Directory>

  ProxyPass / http://localhost:6543/ retry=0
  ProxyPassReverse / http://localhost:6543/

  ProxyPreserveHost On
  RequestHeader set X-Forwarded-Proto "https" env=HTTPS
  SetOutputFilter INFLATE;DEFLATE

</VirtualHost>

django client

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

pip install appenlight-client==0.6.16

########### appenlight client #####################
import socket
import appenlight_client.client as e_client
APPENLIGHT = e_client.get_config({
    'appenlight.api_key':'api_key', 
    'appenlight.transport_config': 'http://localhost:6543?threaded=1&timeout=5&verify=0',
    'appenlight.server_name': socket.gethostname()
})

MIDDLEWARE_CLASSES.insert(0, 'appenlight_client.django_middleware.AppenlightMiddleware')

###################################################

Miscellaneous

Below is the docker container used for testing above scripts. Make sure that the allocated RAM to docker is more than 5GB (Docker > Preferences > Resouces - Memory). Otherwise the container will be limited to the max docker RAM size.

1

docker run -t -d --memory="5g" --name appenlight -p 80:80 -p 6543:6543 centos:6.8