[Nginx + Tomcat] 무중단 배포 구현하기 2

How to do

1. nginx 설치

https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-centos-7

sudo yum install epel-release -y
sudo yum install nginx -y
sudo systemctl start nginx

nginx.conf 설정

#서브도메인에 prod를 포함하고, 로컬 머신으로 부터 요청이 올경우 중단이 되지 않는 인스턴스로 응답
	server {        
	        listen       80;
	        server_name ~^prod+.*$ localhost;
	
	        include /etc/nginx/conf.d/*.inc;        # for switch configuration

	        location / {
	                proxy_pass $prod_url;
	                proxy_set_header X-Real-IP $remote_addr;
	                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	                proxy_set_header Host $http_host;
	        }
	}
    
#서브도메인에 deploy를 포함하면 배포로 간주하고 소스 배포가 가능한 설정으로 응답
	server {
 	       listen       80;
 	       server_name ~^deploy+.*$;

 	       client_max_body_size 200M;              # for deploy sourcecode
 	       include /etc/nginx/conf.d/*.inc;        # for switch configuration

   	     location / {
   	        	client_max_body_size 200M;      # for deploy sourcecode
   	        	proxy_connect_timeout 1200;     # for deploy sourcecode
       	        	proxy_send_timeout 1200;        # for deploy sourcecode
       	        	proxy_read_timeout 1200;        # for deploy sourcecode
                	send_timeout 1200;              # for deploy sourcecode

                	proxy_pass $deploy_url;         # for deploy sourcecode

                	proxy_set_header X-Real-IP $remote_addr;
                	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                	proxy_set_header Host $http_host;
	        }
	}

conf.d/prod-url.inc 와 conf.d/deploy-url.inc 추가

conf.d/prod-url.inc

set $prod_url http://127.0.0.1:8080;

conf.d/deploy-url.inc

set $deploy_url http://127.0.0.1:8081;

nginx.conf 기반으로 해당 머신에 private dns 설정

ex) 1.1.1.1 서버에 아래 두 서브도메인을 등록(ip마지막 번호를 서브도메인에 사용)

deploy*.* > deploy1.domain.com > http://127.0.0.1:8081;

prod*.* > prod1.domain.com > http://127.0.0.1:8080;


deploy1.domain.com로 접속하면 8081 포트를 사용하는 인스턴스로 reverse proxy
prod1.domain.com로 접속하면 8080 포트를 사용하는 인스턴스로 reverse proxy

2. tomcat 설치(2 instance)

JDK install

yum list java*jdk-devel
yum install java-1.7.0-openjdk-devel.x86_64 -y

Tomcat user add

adduser -s /sbin/nologin tomcat
chown tomcat.tomcat /usr/local/tomcat7* -R (after install finished)

Tomcat install

yum install wget -y
wget http://apache.mirror.cdnetworks.com/tomcat/tomcat-7/v7.0.86/bin/apache-tomcat-7.0.86.tar.gz
tar xzf apache-tomcat-7* -C /usr/local/
mv /usr/local/apache-tomcat-7* /usr/local/tomcat7_0
(copy from 0 » cp tomcat7_0/ tomcat7_1 -rf)

Tomcat configuration for 2 instance

vi /usr/local/tomcat7_0/bin/catalina.sh

(after esac)
CATALINA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms4096m -Xmx4096m -XX:ParallelGCThreads=2 -XX:-UseConcMarkSweepGC -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:PermSize=1G -XX:MaxPermSize=2G -XX:+DisableExplicitGC"


(after #Get standard environment variables)

========================================Instance 1===========================================

export CATALINA_HOME=/usr/local/tomcat7_0
export TOMCAT_HOME=/usr/local/tomcat7_0
export CATALINA_BASE=/usr/local/tomcat7_0
CATALINA_PID=/usr/local/tomcat7_0/bin/tomcat.pid

========================================Instance 2===========================================

export CATALINA_HOME=/usr/local/tomcat7_1
export TOMCAT_HOME=/usr/local/tomcat7_1
export CATALINA_BASE=/usr/local/tomcat7_1
CATALINA_PID=/usr/local/tomcat7_1/bin/tomcat.pid

vi /usr/local/tomcat7_0/conf/server.xml

========================================Instance 1===========================================
<?xml version='1.0' encoding='utf-8'?>
<Server port="8005" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <Listener className="org.apache.catalina.core.JasperListener" />
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <Service name="Catalina">

    <Connector port="8080" protocol="HTTP/1.1" packetSize="65536"
               connectionTimeout="20000"
               redirectPort="8443" />

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />


    <Engine name="Catalina" defaultHost="localhost">

      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>
      <Valve className="org.apache.catalina.valves.RemoteIpValve"
        remoteIpHeader="X-Forwarded-For"
        protocolHeader="X-Forwarded-Proto"
        protocolHeaderHttpsValue="https"
        />
      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log." suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>

========================================Instance 2===========================================

<?xml version='1.0' encoding='utf-8'?>
<Server port="8004" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <Listener className="org.apache.catalina.core.JasperListener" />
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <Service name="Catalina">

    <Connector port="8081" protocol="HTTP/1.1" packetSize="65536"
               connectionTimeout="20000"
               redirectPort="8443" />

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8010" protocol="AJP/1.3" redirectPort="8444" />


    <Engine name="Catalina" defaultHost="localhost">

      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>
      <Valve className="org.apache.catalina.valves.RemoteIpValve"
        remoteIpHeader="X-Forwarded-For"
        protocolHeader="X-Forwarded-Proto"
        protocolHeaderHttpsValue="https"
        />
      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log." suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>

systemctl service regist(centos7)

tomcat7_0 instance
vi /usr/lib/systemd/system/tomcat_0.service

[Unit]
Description=Tomcat7 Service
After=syslog.target
After=network.target

[Service]
Type=forking
User=tomcat
ExecStart=/usr/local/tomcat7_0/bin/catalina.sh start
ExecStop=/usr/local/tomcat7_0/bin/catalina.sh stop

[Install]
WantedBy=multi-user.target

tomcat7_1 instance
vi /usr/lib/systemd/system/tomcat_1.service

[Unit]
Description=Tomcat7 Service
After=syslog.target
After=network.target

[Service]
Type=forking
User=tomcat
ExecStart=/usr/local/tomcat7_1/bin/catalina.sh start
ExecStop=/usr/local/tomcat7_1/bin/catalina.sh stop

[Install]
WantedBy=multi-user.target

systemctl start tomcat_0
systemctl enable tomcat_0.service

systemctl start tomcat_1
systemctl enable tomcat_1.service

systemctl service regist(centos6)

touch /etc/init.d/tomcat7_0
chmod 755 /etc/init.d/tomcat7_0
vi /etc/init.d/tomcat7_0

#!/bin/bash  
#JAVA_HOME=/usr/java/jdk
#export JAVA_HOME
#JRE_HOME=/usr/java/jre
#export JRE_HOME
#PATH=$JAVA_HOME/bin:$PATH  
#export PATH
CATALINA_HOME="/usr/local/tomcat7_0"
NAME="$(basename $0)"
case $1 in  
start)  
sh $CATALINA_HOME/bin/startup.sh  
;;   
stop)     
sh $CATALINA_HOME/bin/shutdown.sh  
;;   
status)
if [ -f "/var/run/${NAME}.pid" ]; then
	read kpid < /var/run/${NAME}.pid
	if [ -d "/proc/${kpid}" ]; then
		echo "${NAME} (pid ${kpid}) is running..."
	fi
else
	pid="$(/usr/bin/pgrep -d , java)"
	if [ -z "$pid" ]; then
		echo "${NAME} is stopped"
	else
		echo "${NAME} (pid $pid) is running..."
	fi
fi
;;
restart)  
sh $CATALINA_HOME/bin/shutdown.sh  
sh $CATALINA_HOME/bin/startup.sh  
;;   
version)  
sh $CATALINA_HOME/bin/version.sh  
;;
*)
echo "Usage: $0 {start|stop|restart|status|version}"
;;
esac      
exit 0

3. jenkins job 설정

* 젠킨스 job으로 부터 원격지의 명령어를 실행시키기 위해 원격지서버에 유저 권한 설정

sudo visudo
jenkins ALL=(ALL) NOPASSWD: ALL

* 배포 전 서비스중인 포트 확인

echo "> Now Running Port Check"
NOW_RUNNING_PORT=$(curl -s http://127.0.0.1/systemApi/port)
echo "> $NOW_RUNNING_PORT"

if [[ $NOW_RUNNING_PORT == "8080" ]]
then
        echo "now : 8080"
        IDLE_PORT=8081
        PROD_PORT=8080

        sudo service tomcat7_1 start
        sleep 5

elif [[ $NOW_RUNNING_PORT == "8081" ]]
then
        echo "now : 8081"
        IDLE_PORT=8080
        PROD_PORT=8081

        sudo service tomcat7_0 start
        sleep 5
else
        echo "none"
        IDLE_PORT=8081
        PROD_PORT=8080
fi

echo "> delpoy.*.* trans to : $IDLE_PORT"
echo "set \$prod_url http://127.0.0.1:${PROD_PORT};" | sudo tee /etc/nginx/conf.d/prod-url.inc
echo "set \$deploy_url http://127.0.0.1:${IDLE_PORT};" | sudo tee /etc/nginx/conf.d/deploy-url.inc

echo "> Nginx Change PROD Port : $NOW_RUNNING_PORT to $PROD_PORT"
echo "> Nginx Change Deploy Port to : $IDLE_PORT"
echo "> Nginx Reload"
sudo service nginx reload

sleep 10

* 배포 후 배포된 서버로 스위칭

echo "> After Deploy : Switch To New Version And Stop IDLE Port"
NOW_RUNNING_PORT=$(curl -s http://127.0.0.1/systemApi/port)
echo "> Will be Stop : $NOW_RUNNING_PORT"

if [[ $NOW_RUNNING_PORT == "8080" ]]
then
        echo "now : 8080"
        IDLE_PORT=8080
        PROD_PORT=8081

        echo "8080 stop and log backup and clean source"
        sudo service tomcat7_0 stop
        sleep 5
        sudo cp /usr/local/tomcat7_0/logs /usr/local/tomcat7_0/logs$(date '+_%Y%m%d_%H%M') 
        sleep 5
        sudo rm /usr/local/tomcat7_0/logs/* -rf
        sudo rm /usr/local/tomcat7_0/webapps/ROOT* -rf

elif [[ $NOW_RUNNING_PORT == "8081" ]]
then
        echo "now : 8081"
        IDLE_PORT=8081
        PROD_PORT=8080

        echo "8081 stop and log backup and clean source"
        sudo service tomcat7_1 stop
        sleep 5
        sudo cp /usr/local/tomcat7_1/logs /usr/local/tomcat7_1/logs$(date '+_%Y%m%d_%H%M') 
        sleep 5
        sudo rm /usr/local/tomcat7_0/logs/* -rf
        sudo rm /usr/local/tomcat7_1/webapps/ROOT* -rf

else
        echo "Running Port Check Fail"
        IDLE_PORT=8081
        PROD_PORT=8080
fi

echo "> prod.*.* trans to : $PROD_PORT"
echo "set \$prod_url http://127.0.0.1:${PROD_PORT};" | sudo tee /etc/nginx/conf.d/prod-url.inc
echo "set \$deploy_url http://127.0.0.1:${IDLE_PORT};" | sudo tee /etc/nginx/conf.d/deploy-url.inc

echo "> Nginx Change PROD Port : $NOW_RUNNING_PORT to $PROD_PORT"
echo "> Nginx Reload"
sudo service nginx reload

'DevOps > Deployment' 카테고리의 다른 글

golang sqlboiler (0)	2023.06.02
쉘스크립트 팁 (0)	2022.08.22
[Nginx + Tomcat] 무중단 배포 구현하기 (0)	2018.03.20

Custom + Maker

[Nginx + Tomcat] 무중단 배포 구현하기 2

How to do

1. nginx 설치

2. tomcat 설치(2 instance)

JDK install

Tomcat user add

Tomcat install

Tomcat configuration for 2 instance

systemctl service regist(centos7)

systemctl service regist(centos6)

3. jenkins job 설정

'DevOps > Deployment' 카테고리의 다른 글

티스토리툴바

[Nginx + Tomcat] 무중단 배포 구현하기 2

How to do

1. nginx 설치

2. tomcat 설치(2 instance)

JDK install

Tomcat user add

Tomcat install

Tomcat configuration for 2 instance

systemctl service regist(centos7)

systemctl service regist(centos6)

3. jenkins job 설정

'DevOps > Deployment' 카테고리의 다른 글

관련글

티스토리툴바