[rhel6.5]Nginx + tomcat + memcached 的配置

今天药忘吃喽~ 2022-05-30 20:18 64阅读 0赞

*  **安装环境**
    
    > 操作系统:`rhel6.5`  
    > 主机:`server6.com server3.com server4.com`  
    > 软件: `jdk-7u79-linux-x64.tar.gz apache-tomcat-7.0.37.tar.gz`  
    > 内核版本: `2.6.32-431.el6.x86_64`
 *  **需要注意的几点**
 *  1.服务至少需要三台主机;
 *  2.作为`Nginx`调度器的是`server6.com` `IP`地址是`172.25.23.6`;
 *  3.`server3.com`作为`tomcat+memcached`服务器,`server4.com`的作用是一样的;
 *  4.`server3.com`的`IP`是`172.25.23.3`, `server4.com`的`IP`是`172.25.23.4`;
 *  `Tomcat`是使用`Java`开发的,所以需要配置`Java`的环境变量;

**\*首先在`server3`上面完成配置,使`test.jsp`页面在`Nginx+Tomcat`可以正常访问**  
\* 关闭`server3`上面不需要的服务;

[root@server4 ~]# ssh 172.25.23.3 '/etc/init.d/iptables stop'
    [root@server4 ~]# ssh 172.25.23.3 '/etc/init.d/httpd stop'
    Stopping httpd: [FAILED]
    [root@server4 ~]# ssh 172.25.23.3 '/etc/init.d/mysqld stop'
    Stopping mysqld:  [  OK  ]
    [root@server4 ~]# ssh 172.25.23.3 'chkconfig mysqld off'

*  将`server6`上面的软件包发送给`server3`

[root@server6 ~]# scp jdk-7u79-linux-x64.tar.gz apache-tomcat-7.0.37.tar.gz 172.25.23.3:~/
    jdk-7u79-linux-x64.tar.gz                     100%  146MB  14.6MB/s   00:10    
    apache-tomcat-7.0.37.tar.gz                   100% 7631KB   7.5MB/s   00:00

*  在`server3`上面完成解压,`Java`环境变量的配置,配置`test.jsp`页面

[root@server3 tomcat]# tar xvf jdk-7u79-linux-x64.tar.gz -C /usr/local/
    [root@server3 tomcat]# tar xvf apache-tomcat-7.0.37.tar.gz -C /usr/local/
    [root@server3 tomcat]# ln -sv /usr/local/jdk1.7.0_79/ /usr/local/java
    [root@server3 tomcat]# ln -sv /usr/local/apache-tomcat-7.0.37/ /usr/local/tomcat
    [root@server3 tomcat]# vim /etc/profile
    export JAVA_HOME=/usr/local/java
    export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
    export PATH=$PATH:$JAVA_HOME/bin
    [root@server3 ~]# source /etc/profile
    [root@server3 tomcat]# vim webapps/ROOT/test.jsp 
    [root@server3 tomcat]# cat webapps/ROOT/test.jsp 
    server3 the time is <%=new java.util.Date() %>

*  启动服务,并且查看服务监听的端口

[root@server3 tomcat]# ./bin/startup.sh 
    Using CATALINA_BASE:   /usr/local/tomcat
    Using CATALINA_HOME:   /usr/local/tomcat
    Using CATALINA_TMPDIR: /usr/local/tomcat/temp
    Using JRE_HOME:        /usr/local/java
    Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar

![这里写图片描述][70]

*  **在server6.com**上面进行操作
 *  修改`Nginx`的配置文件

[root@server6 ~]# vim /usr/local/lnmp/nginx/conf/nginx.conf
     location / {
         root   html;
         index  index.html index.jsp index.php index.htm;
     }
    
      location ~ \.jsp$ {
            proxy_pass   http://172.25.23.3:8080;
      }

*  检测服务配置文件的合法性,并且启动服务

[root@server6 ~]# nginx -t
    nginx: the configuration file /usr/local/lnmp/nginx/conf/nginx.conf syntax is ok
    nginx: configuration file /usr/local/lnmp/nginx/conf/nginx.conf test is successful
    [root@server6 ~]# nginx

*  通过浏览器查看`test.jsp`是否可以正常访问到  
    ![这里写图片描述][70 1]
 *  **接下来完成在server4.com**上面的配置,实现`Nginx`对于双节点`Tomcat`服务器的调度
 *  首先需要将`server6.com`上面的软件复制到`server4.com`主机上面

root@server6 ~]# scp apache-tomcat-7.0.37.tar.gz jdk-7u79-linux-x64.tar.gz 172.25.23.4:~/
    apache-tomcat-7.0.37.tar.gz                   100% 7631KB   7.5MB/s   00:01    
    jdk-7u79-linux-x64.tar.gz                     100%  146MB  73.2MB/s   00:02

*  在`server4.com`上面进行软件的解压,和创建软连接

[root@server4 ~]# tar xf jdk-7u79-linux-x64.tar.gz -C /usr/local/
    [root@server4 ~]# tar xf apache-tomcat-7.0.37.tar.gz -C /usr/local/
    [root@server4 ~]# ln -sv /usr/local/apache-tomcat-7.0.37/ /usr/local/tomcat
    `/usr/local/tomcat' -> `/usr/local/apache-tomcat-7.0.37/'
    [root@server4 ~]# ln -sv /usr/local/jdk1.7.0_79/ /usr/local/java `/usr/local/java' -> `/usr/local/jdk1.7.0_79/'

*  配置`Java`环境变量

[root@server4 ~]# vim /etc/profile
    
    [root@server4 ~]# tail -4 /etc/profile
    
    export JAVA_HOME=/usr/local/java export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
    export PATH=$PATH:$JAVA_HOME/bin
    
    [root@server4 ~]# source /etc/profile
    [root@server4 ~]# echo $PATH
    /usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/mysql/bin:/bin:/root/bin:/usr/local/mysql/bin:/usr/local/java/bin

*  为`server4.com`提供`test.jsp`发布页面

[root@server4 ~]# vim /usr/local/tomcat/webapps/ROOT/test.jsp
    [root@server4 ~]# tail /usr/local/tomcat/webapps/ROOT/test.jsp
    server4 the time is <%=new java.util.Date() %>

*  启动服务,并且查看端口监听

[root@server4 ~]# /usr/local/tomcat/bin/startup.sh 
    Using CATALINA_BASE:   /usr/local/tomcat
    Using CATALINA_HOME:   /usr/local/tomcat
    Using CATALINA_TMPDIR: /usr/local/tomcat/temp
    Using JRE_HOME:        /usr/local/java
    Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar

*  ![这里写图片描述][70 2]
 *  通过浏览器访问,来验证  
    ![这里写图片描述][70 3]
 *  **修改`server6.com`上面的`nginx`的配置文件,使`nginx`可以实现对于`tomcat`的调度管理**

[root@server6 ~]# vim /usr/local/lnmp/nginx/conf/nginx.conf
    http {
            upstream westos {
                    server 172.25.23.3:8080;
                    server 172.25.23.4:8080;
            #       server 172.25.23.6:8080 backup;
    
            }
    
      location ~ \.jsp$ {
           proxy_pass   http://westos;
      }
    
    # server { 
    # listen 80;
    # server_name server6.westos.org;
    # location / { 
    # # root /web1;
    # # index index.html;
    # proxy_pass http://westos;
    # }
    # }

*  验证修改后配置文件的合法性以及`reload`配置文件

*  通过浏览器访问`Nginx`主机,来验证对于`tomcat`服务的调度  
    ![这里写图片描述][70 4]
 *  再次刷新页面之后,验证访问情况  
    ![这里写图片描述][70 5]
 *  手动停止某一个节点上面的`tomcat`服务来验证`Nginx`对于后端服务的检查

[root@server4 tomcat]# ./bin/shutdown.sh 
    Using CATALINA_BASE:   /usr/local/tomcat
    Using CATALINA_HOME:   /usr/local/tomcat
    Using CATALINA_TMPDIR: /usr/local/tomcat/temp
    Using JRE_HOME:        /usr/local/java
    Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar

*  通过浏览器访问,页面刷新在

![这里写图片描述][70 6]

*  **重新编译`Nginx`,并且为`Nginx`提供`sticky`模块**
 *  首先解压软件包`nginx-goodies-nginx-sticky-module-ng-c78b7dd79d0d.tar.gz`

[root@server6 nginx-1.10.1]# tar xvf nginx-goodies-nginx-sticky-module-ng-c78b7dd79d0d.tar.gz

*  重新编译`Nginx`,使用选项如下

[root@server6 nginx-1.10.1]# ./configure --prefix=/usr/local/lnmp/nginx 
    > --with-http_ssl_module 
    > --with-http_stub_status_module 
    > --add-module=/root/nginx-goodies-nginx-sticky-module-ng-c78b7dd79d0d

*  出现这个界面  
    ![这里写图片描述][70 7]
 *  执行`make`

[root@server6 nginx-1.10.1]# make

*  执行`make install`

[root@server6 nginx-1.10.1]# make install

*  编辑`Nginx`配置文件,应用这个模块

[root@server6 nginx-1.10.1]# vim /usr/local/lnmp/nginx/conf/nginx.conf
    http {
            upstream westos {
                    sticky;  //这个为新添加的
                    server 172.25.23.3:8080;
                    server 172.25.23.4:8080;
            # server 172.25.23.6:8080 backup;
    
            }

*  检查配置文件,并且重启服务

[root@server6 nginx-1.10.1]# nginx -t 
    nginx: the configuration file /usr/local/lnmp/nginx/conf/nginx.conf syntax is ok
    nginx: configuration file /usr/local/lnmp/nginx/conf/nginx.conf test is successful
    [root@server6 nginx-1.10.1]# nginx -s reload

*  ![这里写图片描述][70 8]
 *  因为浏览器支持`cookies`机制,所以除非清除`cookies`,否则服务会被一直定义到这台服务器上面
 *  **在`server3.com`和`server4.com`上面安装memcache服务**
 *  `rhe6.5`上面自带`memcache`

[root@server6 nginx-1.10.1]# ssh 172.25.23.3 'yum install memcached -y'
    [root@server6 nginx-1.10.1]# ssh 172.25.23.4 'yum install memcached -y'
    配置文件在 /etc/sysconfig/memcached 
    PORT="11211"
    USER="memcached"
    MAXCONN="1024"
    CACHESIZE="64"
    OPTIONS=""

*  启动两台主机上面的`memcache`

[root@server3 ~]# /etc/init.d/memcached start
    Starting memcached:                                        [  OK  ]
    [root@server3 ~]# ssh 172.25.23.4 '/etc/init.d/memcached start'
    root@172.25.23.4's password: Starting memcached: [ OK ]

*  可以使用`telnet`服务来登录`memcache`

[root@server3 ~]# telnet server3.com 11211
    Trying 172.25.23.3...
    Connected to server3.com.
    Escape character is '^]'.
    add name 0 0 
    ERROR
    add name 0 0 6
    westos
    STORED
    get name
    VALUE name 0 6
    westos
    END

*  修改`server3.com`上面的`test.jsp`页面

[root@server3 ~]# vim /usr/local/tomcat/webapps/ROOT/test.jsp 
    
    %@ page contentType="text/html; charset=GBK" %>
    <%@ page import="java.util.*" %>
    <html><head><title>Cluster App Test</title></head>
    <body>
    Server Info:
    <% out.println(request.getLocalAddr() + " : " + request.getLocalPort()+"<br>");%>
    <% out.println("<br> ID " + session.getId()+"<br>"); String dataName = request.getParameter("dataName"); if (dataName != null && dataName.length() > 0) { String dataValue = request.getParameter("dataValue"); session.setAttribute(dataName, dataValue); } out.print("<b>Session list</b>"); Enumeration e = session.getAttributeNames(); while (e.hasMoreElements()) { String name = (String)e.nextElement(); String value = session.getAttribute(name).toString(); out.println( name + " = " + value+"<br>"); System.out.println( name + " = " + value); } %>
    <form action="test.jsp" method="POST">
    name:<input type=text size=20 name="dataName">
    <br>
    key:<input type=text size=20 name="dataValue">
    <br>
    <input type=submit>
    </form>
    </body>
    </html>

*  使用浏览器访问,达到下面的效果

![这里写图片描述][70 9]

*  将这个页面拷贝一份给`server4.com`

[root@server3 ~]# scp /usr/local/tomcat/webapps/ROOT/test.jsp 172.25.23.4:/usr/local/tomcat/webapps/ROOT/test.jsp

*  **配置`Tomcat`将`session`会话信息存储在多个`Memcache`上面,首先是向其中一个`memcache`上面进行存储,这个`memcache`损坏之后,将数据存储在另一个节点上,用于避免单节点故障**
 *  首先关闭两个节点`server3.com`和`server4.com`上面的`tomcat`服务

[root@server3 lib]# ../bin/shutdown.sh 
    Using CATALINA_BASE:   /usr/local/tomcat
    Using CATALINA_HOME:   /usr/local/tomcat
    Using CATALINA_TMPDIR: /usr/local/tomcat/temp
    Using JRE_HOME:        /usr/local/java
    Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
    
    [root@server4 lib]# ../bin/shutdown.sh 
    Using CATALINA_BASE:   /usr/local/tomcat
    Using CATALINA_HOME:   /usr/local/tomcat
    Using CATALINA_TMPDIR: /usr/local/tomcat/temp
    Using JRE_HOME:        /usr/local/java
    Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar

*  需要下列的`*.jar`文件  
    
    
    > asm-3.2.jar  
    > kryo-1.04.jar  
    > kryo-serializers-0.10.jar  
    > memcached-session-manager-1.6.3.jar  
    > memcached-session-manager-tc6-1.6.3.jar  
    > memcached-session-manager-tc7-1.6.3.jar  
    > minlog-1.2.jar  
    > msm-kryo-serializer-1.6.3.jar  
    > reflectasm-1.01.jar  
    > spymemcached-2.7.3.jar
 *  在`server3.com`上面将这些文件放在`/usr/local/tomcat/lib`

[root@server3 ~]# mv *.jar /usr/local/tomcat/lib/
    [root@server3 ~]# rm -f memcached-session-manager-tc6-1.6.3.jar

*  在`server4.com`上面进行同样的操作

[root@server4 ~]# rm -f memcached-session-manager-tc6-1.6.3.jar 
    [root@server4 ~]# mv *.jar /usr/local/tomcat/lib/

*  修改节点`server3.com`上面的配置文件,并且复制一份给`server4.com`

[root@server3 ~]# vim /usr/local/tomcat/conf/context.xml 
    
        <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
        memcachedNodes="n1:172.25.23.3:11211,n2:172.25.23.4:11211"
        failoverNodes="n1"
        requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
        transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
        />
    
    [root@server3 ~]# scp /usr/local/tomcat/conf/context.xml 172.25.23.4:/usr/local/tomcat/conf/context.xml
    root@172.25.23.4's password: context.xml

*  将`server4.com`上面的配置文件`failoverNode`进行修改

[root@server3 lib]# vim ../conf/context.xml 
     <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
    memcachedNodes="n1:172.25.23.3:11211,n2:172.25.23.4:11211"
    failoverNodes="n2"
     requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"   transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
     />

*  启动`server3.com`和`server4.com`上面的对应服务

[root@server3 lib]# ../bin/startup.sh 
    Using CATALINA_BASE:   /usr/local/tomcat
    Using CATALINA_HOME:   /usr/local/tomcat
    Using CATALINA_TMPDIR: /usr/local/tomcat/temp
    Using JRE_HOME:        /usr/local/java
    Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
    
    [root@server4 lib]# ../bin/startup.sh 
    Using CATALINA_BASE:   /usr/local/tomcat
    Using CATALINA_HOME:   /usr/local/tomcat
    Using CATALINA_TMPDIR: /usr/local/tomcat/temp
    Using JRE_HOME:        /usr/local/java
    Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar

*  查看日志信息,两个节点是否互为备用节点

![节点:server3.com][server3.com]

![节点:server4.com][server4.com]

*  通过浏览器查看,查看是否符合要求  
    ![输入数据之前][70 10]

![输入数据之后][70 11]

*  **对于上面两张图的分析**
 *  1.第一张图`server info`是`172.25.23.4`,按照要求`1=2`再输入之后应该在`server3`主机上,因为`memcache`正常时,优先存储在`server3.com`上面,之后的图,验证了这个过程;
 *  **接下来停止server3.com上面的`memcache`,那么两个节点`tomcat`上面的数据都只能够存储在一个节点上面;**

[root@server3 ~]# /etc/init.d/memcached stop
    Stopping memcached:                                        [  OK  ]

*  看两张图的对比  
    ![这里写图片描述][70 12]

![这里写图片描述][70 13]

*  并且输出了错误信息  
    ![这里写图片描述][70 14]

[70]: /images/20220531/63858be221084348b9ea54ecd3a5268e.png
[70 1]: /images/20220531/e1dc189877744333b827f9ec2521654a.png
[70 2]: /images/20220531/95609795711441289b606f09e49b20f3.png
[70 3]: /images/20220531/2531daf14ab340f2afab662fb26b5612.png
[70 4]: /images/20220531/2aadb240dced4f1abe8b5891af13394d.png
[70 5]: /images/20220531/7e9a2a69e7644869ac0fac5979446c20.png
[70 6]: /images/20220531/69c61690b61346d6b76688c403e19d3f.png
[70 7]: /images/20220531/f5731114ce62418781a0ad4cb4bf9068.png
[70 8]: /images/20220531/26851ab96fdd4da3adf6f794b0ad6b7c.png
[70 9]: /images/20220531/6fcc552c4fae4aaa831a4e807297c709.png
[server3.com]: /images/20220531/2034aee44d68480fb91313136755c527.png
[server4.com]: /images/20220531/79cbe52be7394d6fa050a9fa8a015c23.png
[70 10]: /images/20220531/2d0bca3049904d0a86764195610ef6ee.png
[70 11]: /images/20220531/5ac88c2920e045628a027c6b12c96fe1.png
[70 12]: /images/20220531/b7700bd8d14b4b63b0ba12d6ffa13f7d.png
[70 13]: /images/20220531/d457b9a7eb7b491581858bc8cf8ad14c.png
[70 14]: /images/20220531/c8431a8d68b74a9796a7e9b2fd46d21b.png