在Linux下实现consul+upsync+nginx实现动态负载均衡,原理将upstream配置与Nginx本身解耦,实现在线修改upstream信息nginx动态生效。
优势:
1、无需登录服务器配置
2、避免nginx进行reload
3、在线配置,后期更容易实现蓝绿。与Apollo这种类似,但需要借助upsync模块
第一步、安装nginx支持upsync:
测试: nginx version: nginx/1.12.2 可行
--add-module=/tmp/nginx-upsync-module
cd /tmp/ git clone ./configure --prefix=/data1/nginx/release/{{dir_date}} --conf-path=/data1/nginx/release/{{dir_date}}/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --user=nginx --group=nginx --with-http_ssl_module --with-http_v2_module --with-http_dav_module --with-http_stub_status_module --with-threads --with-file-aio --with-stream --with-stream_ssl_module --add-module=/tmp/nginx-upsync-module
第二、安装配置consul
mkdir /data1/consul cd /data1/consul wget https://releases.hashicorp.com/consul/1.7.3/consul_1.7.3_linux_amd64.zip unzip consul_1.7.3_linux_amd64.zip ## 启动consul ##应为集群 consul agent --server --bind=172.16.2.128 --data-dir=/tmp/consule -bootstrap-expect=1 -node=nginx-lb -client 0.0.0.0 -ui
第三、新增一条upstream信息
## 新增一个upstream信息curl -X PUT http://172.16.2.128:8500/v1/kv/upstreams/test/172.16.2.129:8086
第四、配置nginx配置文件
upstream test_server { server 127.0.0.1:11111; upsync 172.16.2.128:8500/v1/kv/upstreams/test upsync_timeout=6m upsync_interval=500ms upsync_type=consul strong_dependency=off; upsync_dump_path /data1/nginx/current/consul/test.ppdapi.com.conf; } server { listen 80; server_name test.ppdai.com; index default.htm index.html index.htm default.html; gzip on; location / { proxy_pass http://test_server; } }
consul的日常管理
触发新增:
curl -X PUT http://172.16.2.128:8500/v1/kv/upstreams/test/172.16.2.129:8086
触发删除:
curl -X DELETE http://172.16.2.128:8500/v1/kv/upstreams/test/172.16.2.129:8086
修改权重:
http://172.16.2.128:8500/v1/kv/upstreams/test/172.16.2.129:8081
json:{"weight":1, "max_fails":2, "fail_timeout":10}
curl -X PUT -d '{"weight":1, "max_fails":2, "fail_timeout":10}'
http://$consul_ip:$port/v1/kv/$dir1/$upstream_name/$backend_ip:$backend_port配置信息参考
upstream xxx_server { server 127.0.0.1:11111; upsync 10.0.24.251:8500/v1/kv/upstreams/xxx upsync_timeout=6m upsync_interval=500ms upsync_type=consul strong_dependency=off; upsync_dump_path /data/nginx/conf/xxx.ppdapi.com.conf; check interval=3000 rise=2 fall=5 timeout=1000 type=http; check_http_send "HEAD http://xxx.ppdapi.com/hs HTTP/1.0\r\n\r\n"; check_http_expect_alive http_2xx http_3xx ; } server { listen 80; server_name xxx.ppdapi.com; server_tokens off; proxy_hide_header X-Powered-By; proxy_hide_header X-AspNet-Version; index default.htm index.html index.htm default.html; gzip on; location / { proxy_pass http://xxx_server; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } }