配置高可用的项目
基于部署考试系统的项目进行高可用
一、前端的高可用
1.先创建三个前端nginx容器，端口不能映射80
# 删除通用的前端容器[root@haproxy ~]# docker ps --allCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES0f3ed10aa6fc java:v0 "/bin/bash" 7 minutes ago Up 7 minutes 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp gracious_tharpc9abaa798fcf mysql:5.7.44 "docker-entrypoint.s…" 12 minutes ago Up 12 minutes 0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp focused_aryabhata# 创建web01容器[root@haproxy ~]# docker run -itd --name web01 -v /root/project_exam_system/web/dist/:/usr/share/nginx/html nginx:latest9c40a2df1d7bc58f73737e8c84d0603e531f3e13173e6e25981f03fa5d9dda91# 进到web01容器[root@haproxy ~]# docker exec -it web01 /bin/bash# 进行访问测试root@9c40a2df1d7b:/# curl localhost<!DOCTYPE html><html lang="en"> <head> <meta charset="UTF-8"> <link rel="icon" href="/favicon.ico"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Vite App</title> <script type="module" crossorigin src="/assets/index-C4kAShR5.js"></script> <link rel="stylesheet" crossorigin href="/assets/index-CSz7ARPP.css"> </head> <body> <div id="app"></div> </body></html># ctrl + p + q 退出不中断root@9c40a2df1d7b:/# read escape sequence# 创建web02容器[root@haproxy ~]# docker run -itd --name web02 -v /root/project_exam_system/web/dist/:/usr/share/nginx/html nginx:latest1dc4276bdfcf974b1f1467e0c7225dcf880ee2da8e1b44caee06fce986f76e99# 进到web02容器[root@haproxy ~]# docker exec -it web02 /bin/bash# 进行访问测试root@1dc4276bdfcf:/# curl localhost<!DOCTYPE html><html lang="en"> <head> <meta charset="UTF-8"> <link rel="icon" href="/favicon.ico"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Vite App</title> <script type="module" crossorigin src="/assets/index-C4kAShR5.js"></script> <link rel="stylesheet" crossorigin href="/assets/index-CSz7ARPP.css"> </head> <body> <div id="app"></div> </body></html># ctrl + p + q 退出不中断root@1dc4276bdfcf:/# read escape sequence# 创建web03容器[root@haproxy ~]# docker run -itd --name web03 -v /root/project_exam_system/web/dist/:/usr/share/nginx/html nginx:latestcea53a5ef797bf2df312454cfb4344b6f21beb42125c20583ba686538a2be858# 进到web03容器[root@haproxy ~]# docker exec -it web03 /bin/bash# 进行访问测试root@cea53a5ef797:/# curl localhost<!DOCTYPE html><html lang="en"> <head> <meta charset="UTF-8"> <link rel="icon" href="/favicon.ico"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Vite App</title> <script type="module" crossorigin src="/assets/index-C4kAShR5.js"></script> <link rel="stylesheet" crossorigin href="/assets/index-CSz7ARPP.css"> </head> <body> <div id="app"></div> </body></html># ctrl + p + q 退出不中断root@cea53a5ef797:/# read escape sequence# 拉取haproxy镜像[root@haproxy ~]# docker pull haproxyUsing default tag: latestlatest: Pulling from library/haproxyDigest: sha256:3ab695ae94fb960da08947bb6d9ccb12e8237d7813c711beaf93a5310d150474Status: Image is up to date for haproxy:latestdocker.io/library/haproxy:latest# 使用haproxy镜像创建容器[root@haproxy ~]# docker run -itd -p5000:5000 haproxy:latest /bin/bash29e4a9f777f0fc6048e4f15b837b593732a96e1db2f6d428d4e732fea2158bbf# 编辑haproxy.cfg文件[root@haproxy ~]# vim haproxy.cfg # 这里是容器中的IP地址，由于配置的是轮询roundrobin，weight 权重其实没有生效server web0 web01:80 check weight 1 maxconn 2000server web1 web02:80 check weight 1 maxconn 2000server web2 web03:80 check weight 1 maxconn 2000# 将修改后的haproxy.cfg传到用haproxy创建的容器中[root@haproxy ~]# docker cp /root/haproxy.cfg 29:/usr/local/etc/haproxySuccessfully copied 5.12kB to 29:/usr/local/etc/haproxy# 挂载这个容器的终端[root@haproxy ~]# docker attach 29# 查看文件haproxy@29e4a9f777f0:~$ ls /usr/local/etc/haproxy/haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg# 在容器中运行这个文件haproxy@29e4a9f777f0:~$ haproxy -f /usr/local/etc/haproxy/haproxy.cfg [NOTICE] (8) : haproxy version is 3.0.3-95a607c[NOTICE] (8) : path to executable is /usr/local/sbin/haproxy[ALERT] (8) : config : [/usr/local/etc/haproxy/haproxy.cfg:67] : 'server proxy-web/web0' : could not resolve address 'web01'.[ALERT] (8) : config : [/usr/local/etc/haproxy/haproxy.cfg:68] : 'server proxy-web/web1' : could not resolve address 'web02'.[ALERT] (8) : config : [/usr/local/etc/haproxy/haproxy.cfg:69] : 'server proxy-web/web2' : could not resolve address 'web03'.[ALERT] (8) : config : Failed to initialize server(s) addr.2.使用浏览器访问：192.168.2.20
出现前端页面
3.使用haproxy对web容器进行高可用
# 删除之前使用haproxy创建的容器[root@haproxy ~]# docker stop 2929[root@haproxy ~]# docker rm 2929# 创建使用haproxy高可用三个web前端的容器[root@haproxy ~]# docker run -itd --link=web01 --link=web02 --link=web03 -p5000:5000 -v /root/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg haproxy:latest 7848bd08f685cf69deb5cd01467ad7c4123b191617001eab100732e4368f0746# -itd 交互 终端 后台# link 锚定web容器# v 将配置文件挂载到容器中 # 查看创建的容器[root@haproxy ~]# docker ps --allCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES7848bd08f685 haproxy:latest "docker-entrypoint.s…" About a minute ago Up About a minute 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp suspicious_villani7a0f856ed053 nginx:latest "/docker-entrypoint.…" 8 minutes ago Up 8 minutes 80/tcp web03db4c5ce94e76 nginx:latest "/docker-entrypoint.…" 8 minutes ago Up 8 minutes 80/tcp web021d99e2005bd9 nginx:latest "/docker-entrypoint.…" 8 minutes ago Up 8 minutes 80/tcp web010f3ed10aa6fc java:v0 "/bin/bash" 48 minutes ago Up 48 minutes 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp gracious_tharpc9abaa798fcf mysql:5.7.44 "docker-entrypoint.s…" 53 minutes ago Up 53 minutes 0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp focused_aryabhata4.浏览访问：192.168.2.20:5000
5.调用haproxy的监控界面
# 删除之前创建的容器[root@haproxy ~]# docker stop 7878[root@haproxy ~]# docker rm 7878# 查看监控界面配置[root@haproxy ~]# vim haproxy.cfg ######## 监控界面配置 #################listen admin_status # 监控界面访问信息 bind 0.0.0.0:8888 mode http # URI相对地址 stats uri /dbs # 统计报告格式 stats realm Global\ statistics # 登录账户信息 stats auth admin:123456# 创建可以监控的高可用容器[root@haproxy ~]# docker run -itd --link=web01 --link=web02 --link=web03 -p5000:5000 -p8888:8888 -v /root/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg haproxy:latest cb32ed855f59a3bfd9770f93f3dce5c12b2fceff30e6b4b09bb8e39ae9cba735# 查看创建的容器[root@haproxy ~]# docker ps --allCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMEScb32ed855f59 haproxy:latest "docker-entrypoint.s…" About a minute ago Up About a minute 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp, 0.0.0.0:8888->8888/tcp, :::8888->8888/tcp happy_chatelet7a0f856ed053 nginx:latest "/docker-entrypoint.…" 20 minutes ago Up 20 minutes 80/tcp web03db4c5ce94e76 nginx:latest "/docker-entrypoint.…" 20 minutes ago Up 20 minutes 80/tcp web021d99e2005bd9 nginx:latest "/docker-entrypoint.…" 21 minutes ago Up 21 minutes 80/tcp web010f3ed10aa6fc java:v0 "/bin/bash" About an hour ago Up About an hour 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp gracious_tharpc9abaa798fcf mysql:5.7.44 "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp focused_aryabhata6.浏览器访问测试
（1）高可用后的前端页面
192.168.2.20:5000
（2）监控页面
192.168.2.20:8888/dbs
二、java容器的高可用
1.java容器的高可用
# 删除通用的java容器[root@haproxy ~]# docker stop 0f0f[root@haproxy ~]# docker rm 0f0f# 查看目前的容器[root@haproxy ~]# docker ps --allCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMEScb32ed855f59 haproxy:latest "docker-entrypoint.s…" 5 minutes ago Up 5 minutes 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp, 0.0.0.0:8888->8888/tcp, :::8888->8888/tcp happy_chatelet7a0f856ed053 nginx:latest "/docker-entrypoint.…" 24 minutes ago Up 24 minutes 80/tcp web03db4c5ce94e76 nginx:latest "/docker-entrypoint.…" 24 minutes ago Up 24 minutes 80/tcp web021d99e2005bd9 nginx:latest "/docker-entrypoint.…" 24 minutes ago Up 24 minutes 80/tcp web01c9abaa798fcf mysql:5.7.44 "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp focused_aryabhata# 创建三个java容器[root@haproxy ~]# docker run -itd --name java0 java:v0beb08466361480064fac9e66426b679e0cb4b9dc00d624c5d90f689b75002b98[root@haproxy ~]# docker run -itd --name java1 java:v07574e7d3756f5d2e5b06a718433b84027559d8c9b82d10c1be2639542541c819[root@haproxy ~]# docker run -itd --name java2 java:v0658ef09ab2fb35a0184a3d5f2f3548bf0fc29834a8db15184d7cf6f98a005a59# 查看创建的容器[root@haproxy ~]# docker ps --allCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES658ef09ab2fb java:v0 "/bin/bash" 37 seconds ago Up 36 seconds java27574e7d3756f java:v0 "/bin/bash" 40 seconds ago Up 40 seconds java1beb084663614 java:v0 "/bin/bash" 2 minutes ago Up 2 minutes java0cb32ed855f59 haproxy:latest "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp, 0.0.0.0:8888->8888/tcp, :::8888->8888/tcp happy_chatelet7a0f856ed053 nginx:latest "/docker-entrypoint.…" 27 minutes ago Up 27 minutes 80/tcp web03db4c5ce94e76 nginx:latest "/docker-entrypoint.…" 27 minutes ago Up 27 minutes 80/tcp web021d99e2005bd9 nginx:latest "/docker-entrypoint.…" 28 minutes ago Up 28 minutes 80/tcp web01c9abaa798fcf mysql:5.7.44 "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp focused_aryabhata# 修改haproxy.cfg配置文件[root@haproxy ~]# vim haproxy.cfg # 在最后一行添加内容listen proxy-java bind 0.0.0.0:8080 mode http # 负载均衡算法 # static-rr 权重, leastconn 最少连接, source 请求IP, 轮询 roundrobin balance roundrobin # 日志格式 option tcplog # 在 mysql 创建一个没有权限的haproxy用户，密码为空。 haproxy用户 # create user 'haproxy'@'%' identified by ''; FLUSH PRIVILEGES; #option mysql-check user haproxy # 这里是容器中的IP地址，由于配置的是轮询roundrobin，weight 权重其实没有生效 server web0 java0:8080 check weight 1 maxconn 2000 server web1 java1:8080 check weight 1 maxconn 2000 server web2 java2:8080 check weight 1 maxconn 2000 #server MYSQL_3 192.168.130.102:3306 check weight 1 maxconn 2000 # 使用keepalive检测死链 # option tcpka########################################## 查看8888端口有没有被占用[root@haproxy ~]# netstat -lntup|grep 8888tcp 0 0 0.0.0.0:8888 0.0.0.0:* LISTEN 22318/docker-proxy tcp6 0 0 :::8888 :::* LISTEN 22323/docker-proxy# 删除之前的web容器的高可用容器[root@haproxy ~]# docker stop cbcb[root@haproxy ~]# docker rm cbcb# 8888端口没有被占用了[root@haproxy ~]# netstat -lntup|grep 8888# 创建web容器和java容器的高可用容器[root@haproxy ~]# docker run -itd --link=web01 --link=web02 --link=web03 --link=java0 --link=java1 --link=java2 -p5000:5000 -p8080:8080 -p8888:8888 -v /root/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg haproxy:latest 289cefc3caaf0154e54f75cc15d175d1b7d322b64fa0425f1a828e213397a4782.监控界面显示java容器是红色的
是因为没有启动服务
3.将三个容器的服务全部启动
[root@haproxy ~]# docker attach java0[root@beb084663614 /]# /usr/local/jdk/bin/java -jar Project_ExamSystem-V1.0.0.war read escape sequence[root@haproxy ~]# docker attach java1[root@7574e7d3756f /]# /usr/local/jdk/bin/java -jar Project_ExamSystem-V1.0.0.war read escape sequence[root@haproxy ~]# docker attach java2[root@658ef09ab2fb /]# /usr/local/jdk/bin/java -jar Project_ExamSystem-V1.0.0.war read escape sequence[root@haproxy ~]#4.浏览器访问测试
（1）进到考试页面：192.168.2.20:5000
（2）进入管理者页面：192.168.2.20:5000/#/admin
（3）haproxy的监控界面：192.168.2.20:8888
5.ab压力测试
# ab压力测试[root@haproxy ~]# yum -y install httpd-tools[root@haproxy ~]# ab -n 100 -c 10 http://192.168.2.20:5000/This is ApacheBench, Version 2.3 <$Revision: 1430300 $>Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking 192.168.2.20 (be patient).....doneServer Software: Server Hostname: 192.168.2.20Server Port: 5000Document Path: /Document Length: 430 bytesConcurrency Level: 10Time taken for tests: 0.066 secondsComplete requests: 100Failed requests: 0Write errors: 0Total transferred: 66300 bytesHTML transferred: 43000 bytesRequests per second: 1504.48 [#/sec] (mean)Time per request: 6.647 [ms] (mean)Time per request: 0.665 [ms] (mean, across all concurrent requests)Transfer rate: 974.09 [Kbytes/sec] receivedConnection Times (ms) min mean[+/-sd] median maxConnect: 0 0 0.1 0 1Processing: 1 5 3.0 4 16Waiting: 1 5 3.0 3 16Total: 1 5 3.0 4 16Percentage of the requests served within a certain time (ms) 50% 4 66% 5 75% 6 80% 8 90% 10 95% 10 98% 15 99% 16 100% 16 (longest request)三、docker-compose 自动化部署
新机器：192.168.2.30
1.部署docker
# 编辑docker脚本[root@docker ~]# vim docker.shcat << EOF | tee /etc/modules-load.d/k8s.conf overlaybr_netfilterEOFmodprobe overlaymodprobe br_netfiltercat << EOF | tee /etc/sysctl.d/k8s.confnet.bridge.bridge-nf-call-iptables = 1net.bridge.bridge-nf-call-ip6tables = 1net.ipv4.ip_forward = 1EOFsysctl --systemyum install -y yum-utils device-mapper-persistent-data lvm2yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repoyum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y# 运行docker脚本[root@docker ~]# source docker.sh# 编辑daemon.json文件[root@docker ~]# vim /etc/docker/daemon.json{ "registry-mirrors": [ "https://do.nark.eu.org", "https://dc.j8.work", "https://docker.m.daocloud.io", "https://dockerproxy.com", "https://docker.mirrors.ustc.edu.cn", "https://docker.nju.edu.cn" ]}# 修改docker.service文件[root@docker ~]# vim /usr/lib/systemd/system/docker.service 13ExecStart=/usr/bin/dockerd# 加载文件[root@docker ~]# systemctl daemon-reload# 重启docker服务[root@docker ~]# systemctl start docker2.安装pip
[root@docker ~]# yum -y install python2-pip3.升级pip
[root@docker ~]# pip install --upgrade pip==20.3 -i https://mirrors.aliyun.com/pypi/simpl4.安装docker-compose
[root@docker ~]# pip install docker-compose -i https://mirrors.aliyun.com/pypi/simple5.创建pes目录以及子目录
[root@docker ~]# mkdir -p pes/{msyql,java,web}[root@docker ~]# tree pespes├── java├── msyql└── web3 directories, 0 files6.在pes目录下创建并且编辑docker-compose.yml文件
[root@docker ~]# cd pes[root@docker pes]# vim docker-compose.yml version: "3"services: web: container_name: web0 image: nginx:latest ports: - "80:80" expose: - 80 volumes: - ./web/src/dist/:/usr/share/nginx/html/ restart: "always"7.把之前部署的考试系统的web目录传过来
# 进到pes目录下[root@docker ~]# cd pes# 创建web/src目录[root@docker pes]# mkdir -p web/src/# 把之前部署的考试系统的web目录传到web/src/下[root@docker pes]# scp -r 192.168.2.20:/root/project_exam_system/web/dist/ web/src[root@docker pes]# lsjava msyql web[root@docker pes]# cd web[root@docker web]# lssrc# 查看传过来的dist文件[root@docker web]# ls src/dist/assets favicon.ico index.html8.docker-compose启动集群
# 拉取nginx镜像[root@docker ~]# docker pull nginx:latest# 查看镜像[root@docker ~]# docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEnginx latest 5ef79149e0ec 2 weeks ago 188MB# docker-compose启动集群[root@docker pes]# docker-compose up -d/usr/lib/python2.7/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography.hazmat.backends import default_backendCreating network "pes_default" with the default driverCreating web0 ... done# 查看用docker-compose创建的容器[root@docker pes]# docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES648da7c000f9 nginx:latest "/docker-entrypoint.…" About a minute ago Up About a minute 0.0.0.0:80->80/tcp, :::80->80/tcp web09.浏览器访问:192.168.2.30(本机IP)
10.使用docker-compose一次性创建多台一样的容器
# 创建多台容器的语法[root@docker pes]# docker-compose scale --helpUsage: scale [options] [SERVICE=NUM...]# 编辑yml文件# 创建相同的容器，不能同名，不可以映射相同的端口，所以注释掉这两行[root@docker pes]# vim docker-compose.yml version: "3"services: web: #container_name: web0 image: nginx:latest #ports: #- "80:80" volumes: - ./web/src/dist/:/usr/share/nginx/html/ restart: "always" # 使用scale选项创建3台相同的web容器[root@docker pes]# docker-compose up --scale web=3 -d/usr/lib/python2.7/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography.hazmat.backends import default_backendRecreating web0 ... doneCreating pes_web_2 ... doneCreating pes_web_3 ... done# 查看创建的容器[root@docker pes]# docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESa84d833c496c nginx:latest "/docker-entrypoint.…" 32 seconds ago Up 31 seconds 80/tcp pes_web_35b59fb581bcb nginx:latest "/docker-entrypoint.…" 32 seconds ago Up 31 seconds 80/tcp pes_web_269b05e3cf0b4 nginx:latest "/docker-entrypoint.…" 32 seconds ago Up 32 seconds 80/tcp pes_web_1#docker-compose暂停集群[root@docker pes]# docker-compose stop/usr/lib/python2.7/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography.hazmat.backends import default_backendStopping pes_web_3 ... doneStopping pes_web_2 ... doneStopping pes_web_1 ... done# 查看容器（空）[root@docker pes]# docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES