使用Nginx和tomcat配置负载均衡时同步session的方法
时间:2015-01-21 21:58 来源:linux.it.net.cn 作者:IT
分别下载
tomcat http://archive.apache.org/dist/tomcat/tomcat-7/v7.0.22/bin/apache-tomcat-7.0.22.tar.gz
nginx http://nginx.org/download/nginx-1.1.14.tar.gz
准备两个虚拟机：
server1 192.168.1.112
server2 192.168.1.64
Tomcat直接解压,运行,使用默认的8080端口
tar zxvf apache-tomcat-7.0.22.tar.gz
cd apache-tomcat-7.0.22/bin
./startup.sh
访问http://192.168.1.112:8080和http://192.168.1.64:8080出现Tomcat首页即可
接下来安装nginx, nginx我就直接安装在server1上
nginx_upstream_jvm_route是一个Nginx的扩展模块，用来实现基于Cookie的SessionSticky的功能，去SVN下载最新版
svn checkout http://nginx-upstream-jvm-route.googlecode.com/svn/trunk/ /root/dev/nginx-upstream-jvm-route-read-only
解压nginx
tar zxvf nginx-1.1.14.tar.gz
cd nginx-1.1.14
运行
patch -p0 < /root/dev/nginx-upstream-jvm-route-read-only/jvm_route.patch
./configure --prefix=/etc/nginx --with-http_stub_status_module --add-module=/root/dev/nginx-upstream-jvm-route-read-only/
make
make install
在nginx安装目录下的conf/目录新建一个文件proxy.conf(/etc/nginx/conf/proxy.conf), 内容如下:
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
修改nginx配置文件/etc/nginx/conf/nginx.conf
完整配置如下
http {
include mime.types;
#反向代理配置
include proxy.conf;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream server1{
server 192.168.1.112:8080 srun_id=tomcat1;
server 192.168.1.64:8080 srun_id=tomcat2;
jvm_route $cookie_JSESSIONID|sessionid reverse;
}
server {
listen 80;
server_name localhost;
access_log /var/log/nginx/access.log;
location ~ ^/NginxStatus/ {
stub_status on; #Nginx 状态监控配置
access_log off;
}
location ~ ^/(WEB-INF)/ {
deny all;
}
location / {
root html;
index index.html index.htm;
proxy_pass http://server1;
}
location /doc {
root /usr/share;
autoindex on;
allow 127.0.0.1;
deny all;
}
location /images {
root /usr/share;
autoindex on;
}
具体配置说明请参考《轻量级WEB服务器Nginx》
然后运行nginx
访问http://192.168.1.112/NginxStatus,可以看到nginx状态
然后修改tomcat配置文件, 打开apache-tomcat-7.0.22/conf/server.xml, 找到最下面的<Engine name="Catalina" defaultHost="localhost">节点, 修改为
server1
<Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1">
并插入如下配置
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="224.0.0.4"
port="45564"
frequency="500"
dropTime="3000"/>
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="192.168.1.112"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" />
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
server2
...
<Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat2">
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="224.0.0.4"
port="45564"
frequency="500"
dropTime="3000"/>
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="192.168.1.64"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" />
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
...
jvmRoute="tomcat1"和nginx配置upstream中的srun_id对应
membership中的address＝224.0.0.4为组播IP，集群中的tomcat通信之用
receiver中的address设为本机IP，或auto，如果多个Tomcat在同一台电脑上，则要保证port端口不重复
在两个tomcat的webapps目录下分别新建一个项目test,
/test
/test/index.jsp
/test/WEB-INF/
/test/WEB-INF/web.xml
index.jsp内容
<%@ page language="java" contentType="text/html; charset=UTF-8"
pageEncoding="UTF-8"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Insert title here</title>
</head>
<body>
<%=session.getId() %><br>
<%
String msg = (String)session.getAttribute("msg");
if(null == msg){
session.setAttribute("msg", "Hello!");
}else{
session.setAttribute("msg", msg + 0);
}
%>
<%=session.getAttribute("msg") %>
</body>
</html>
web.xml<?xml version="1.0" encoding="UTF-8"?>
<welcome-file-list>
<welcome-file>index.jsp</welcome-file>
</welcome-file-list>
<distributable/>
</web-app>
注意上面的distributable节点，表示该应用是在集群环境下的。
重启tomcat后在两台机子上访问http://192.168.1.112/test
从SessionId后面带的服务器名可以看到负载均衡的效果。
然后将server1上的tomcat停掉，再刷新页面，可以看到原来访问server1的页面成功地切换到了server2，而sessionId没有变，session中的msg也和原来一样。
(责任编辑：IT)
| 分别下载 tomcat http://archive.apache.org/dist/tomcat/tomcat-7/v7.0.22/bin/apache-tomcat-7.0.22.tar.gz nginx http://nginx.org/download/nginx-1.1.14.tar.gz 准备两个虚拟机： server1 192.168.1.112 server2 192.168.1.64 Tomcat直接解压,运行,使用默认的8080端口 tar zxvf apache-tomcat-7.0.22.tar.gz cd apache-tomcat-7.0.22/bin ./startup.sh 访问http://192.168.1.112:8080和http://192.168.1.64:8080出现Tomcat首页即可 接下来安装nginx, nginx我就直接安装在server1上 nginx_upstream_jvm_route是一个Nginx的扩展模块，用来实现基于Cookie的SessionSticky的功能，去SVN下载最新版 svn checkout http://nginx-upstream-jvm-route.googlecode.com/svn/trunk/ /root/dev/nginx-upstream-jvm-route-read-only 解压nginx tar zxvf nginx-1.1.14.tar.gz cd nginx-1.1.14 运行 patch -p0 < /root/dev/nginx-upstream-jvm-route-read-only/jvm_route.patch ./configure --prefix=/etc/nginx --with-http_stub_status_module --add-module=/root/dev/nginx-upstream-jvm-route-read-only/ make make install 在nginx安装目录下的conf/目录新建一个文件proxy.conf(/etc/nginx/conf/proxy.conf), 内容如下: proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 10m; client_body_buffer_size 128k; proxy_connect_timeout 90; proxy_send_timeout 90; proxy_read_timeout 90; proxy_buffer_size 4k; proxy_buffers 4 32k; proxy_busy_buffers_size 64k; proxy_temp_file_write_size 64k; 修改nginx配置文件/etc/nginx/conf/nginx.conf 完整配置如下 http { include mime.types; #反向代理配置 include proxy.conf; default_type application/octet-stream; sendfile on; keepalive_timeout 65; upstream server1{ server 192.168.1.112:8080 srun_id=tomcat1; server 192.168.1.64:8080 srun_id=tomcat2; jvm_route $cookie_JSESSIONID|sessionid reverse; } server { listen 80; server_name localhost; access_log /var/log/nginx/access.log; location ~ ^/NginxStatus/ { stub_status on; #Nginx 状态监控配置 access_log off; } location ~ ^/(WEB-INF)/ { deny all; } location / { root html; index index.html index.htm; proxy_pass http://server1; } location /doc { root /usr/share; autoindex on; allow 127.0.0.1; deny all; } location /images { root /usr/share; autoindex on; } 具体配置说明请参考《轻量级WEB服务器Nginx》 然后运行nginx 访问http://192.168.1.112/NginxStatus,可以看到nginx状态 然后修改tomcat配置文件, 打开apache-tomcat-7.0.22/conf/server.xml, 找到最下面的<Engine name="Catalina" defaultHost="localhost">节点, 修改为 server1 <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1"> 并插入如下配置 <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8"> <Manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" notifyListenersOnReplication="true"/> <Channel className="org.apache.catalina.tribes.group.GroupChannel"> <Membership className="org.apache.catalina.tribes.membership.McastService" address="224.0.0.4" port="45564" frequency="500" dropTime="3000"/> <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver" address="192.168.1.112" port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6"/> <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter"> <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" /> </Sender> <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/> <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/> <Interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/> </Channel> <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/> <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/> <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/> <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/> </Cluster> server2 ... <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat2"> <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8"> <Manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" notifyListenersOnReplication="true"/> <Channel className="org.apache.catalina.tribes.group.GroupChannel"> <Membership className="org.apache.catalina.tribes.membership.McastService" address="224.0.0.4" port="45564" frequency="500" dropTime="3000"/> <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver" address="192.168.1.64" port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6"/> <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter"> <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" /> </Sender> <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/> <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/> <Interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/> </Channel> <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/> <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/> <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/> <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/> </Cluster> ... jvmRoute="tomcat1"和nginx配置upstream中的srun_id对应 membership中的address＝224.0.0.4为组播IP，集群中的tomcat通信之用 receiver中的address设为本机IP，或auto，如果多个Tomcat在同一台电脑上，则要保证port端口不重复 在两个tomcat的webapps目录下分别新建一个项目test, /test /test/index.jsp /test/WEB-INF/ /test/WEB-INF/web.xml index.jsp内容 <%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <title>Insert title here</title> </head> <body> <%=session.getId() %><br> <% String msg = (String)session.getAttribute("msg"); if(null == msg){ session.setAttribute("msg", "Hello!"); }else{ session.setAttribute("msg", msg + 0); } %> <%=session.getAttribute("msg") %> </body> </html> web.xml<?xml version="1.0" encoding="UTF-8"?> <welcome-file-list> <welcome-file>index.jsp</welcome-file> </welcome-file-list> <distributable/> </web-app> 注意上面的distributable节点，表示该应用是在集群环境下的。 重启tomcat后在两台机子上访问http://192.168.1.112/test 从SessionId后面带的服务器名可以看到负载均衡的效果。 然后将server1上的tomcat停掉，再刷新页面，可以看到原来访问server1的页面成功地切换到了server2，而sessionId没有变，session中的msg也和原来一样。 (责任编辑：IT) |