龚哥哥 爱生活、做自己!
Keepalived+Haproxy搭建高可用负载均衡
发表于 2016-8-27 | 浏览(6374) | 服务器

Keepalived

简单的是一个路由的软件用C写的这个项目的主要目标是提供简单而强大的设施的负载均衡和高可用性对Linux系统和基于Linux的基础设施。负载均衡架构依赖于众所周知的和广泛使用的Linux虚拟服务器(IPVS)内核模块提供第四层负载均衡。简单的实现了一套检测动态自适应维护和管理服务器根据其健康loadbalanced池。另一方面,高可用性的实现VRRP协议.VRRP路由器故障转移的一个基本的砖。此外,简单的实现了一套钩VRRP有限状态机提供低空和高速协议的相互作用。简单的框架可以单独或一起提供弹性基础设施。

Haproxy

HAProxy提供高可用性、负载均衡以及基于TCP和HTTP应用的代理,支持虚拟主机,它是免费、快速并且可靠的一种解决方案。HAProxy特别适用于那些负载特大的web站点,这些站点通常又需要会话保持或七层处理。HAProxy运行在当前的硬件上,完全可以支持数以万计的并发连接。并且它的运行模式使得它可以很简单安全的整合进您当前的架构中, 同时可以保护你的web服务器不被暴露到网络上。

架构图


1、环境

CentOS 6.5
keepalived 1.2.23
haproxy 1.5.4

2、准备4台服务器

VIP         192.168.0.200
Master      192.168.0.110
Backup      192.168.0.111
Server1     192.168.0.120
Server2     192.168.0.121

3、安装gcc编译器,openssl,wget,如果已经安装则跳过

yum -y install openssl-devel ncurses-devel gcc gcc-c++ make rpm-build wget

4、创建软件存放目录

mkdir /soft

5、安装keepalived

cd /soft
wget http://www.keepalived.org/software/keepalived-1.2.23.tar.gz
tar -zxvf keepalived-1.2.23.tar.gz
cd keepalived-1.2.23
./configure --prefix=/usr/local/keepalived
make
make install

6、将keepalived做成启动脚务

cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
chmod +x /etc/init.d/keepalived

7、配置文件修改  vim /etc/keepalived/keepalived.conf

7.1、MASTER 配置信息

global_defs {
    notification_email
    {
        fuxiang.gong@qq.com
    }
    notification_email_from 17091959688@163.com
    smtp_server smtp.163.com
    stmp_connect_timeout 30
    router_id lnmp_node1
}

# 检测haproxy脚本
vrrp_script chk_haproxy {
        script "/etc/keepalived/check_haproxy.sh"
        interval 2
        weight 2
}

# 服务
vrrp_instance VIP_1 {
    state MASTER    #设置为主服务器
    interface eth0  #监测网络接口
    virtual_router_id 51  #主、备必须一样
    priority 100   #主机级别,值越大优先级越高
    advert_int 1   #VRRP Multicast广播周期秒数

    authentication {
        auth_type PASS  #VRRP认证方式,主备必须一致
        auth_pass 1111  #密码
    }
    track_script {
        chk_haproxy  # 执行监控的服务
    }
    virtual_ipaddress {
        192.168.0.200  #漂移IP地址
    }
}

7.2、BACKUP 配置信息

global_defs { 
    notification_email
    {
        fuxiang.gong@qq.com
    }
    notification_email_from 17091959688@163.com
    smtp_server smtp.163.com
    stmp_connect_timeout 30
    router_id lnmp_node2
}

# 检测haproxy脚本
vrrp_script chk_haproxy {
        script "/etc/keepalived/check_haproxy.sh"
        interval 2
        weight 2
}

# 服务
vrrp_instance VIP_1 {
    state BACKUP    #设置为备用服务器
    interface eth0  #监测网络接口
    virtual_router_id 51  #主、备必须一样
    priority 90   #主、备机取不同的优先级,主机值较大,备份机值较小,值越大优先级越高 
    advert_int 1   #VRRP Multicast广播周期秒数
    authentication {
        auth_type PASS  #VRRP认证方式,主备必须一致
        auth_pass 1111  #密码
    }
    track_script {
        chk_haproxy  # 执行监控的服务
    }
    virtual_ipaddress {
        192.168.0.200  #漂移IP地址
    }
}

7.3、添加Haproxy检测脚本 vim  /etc/keepalived/check_haproxy.sh 添加以下内容

#!/bin/bash
if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then
	/etc/init.d/haproxy start
fi
sleep 2
if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then
	/etc/init.d/keepalived stop
fi

7.4、给check_haproxy.sh脚本赋值运行权限(MASTER和BACKUP一致)

chmod +x /etc/keepalived/check_haproxy.sh

7.5、允许两台服务器vrrp包通过防火墙,如果关闭防火墙则跳过(两台服务器上都配置)

MASTER
  vim /etc/sysconfig/iptables
  -A INPUT -i eth0 -p vrrp -s 192.168.0.111 -j ACCEPT

BACKUP
  vim /etc/sysconfig/iptables
  -A INPUT -i eth0 -p vrrp -s 192.168.0.110 -j ACCEPT

重启防火墙
service iptables restart

8、启动keepalived服务

service keepalived start

8.1、查看服务器多了一个虚拟IP,keepalived配置成功

MASTER  ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:d9:a8:bd brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.110/24 brd 192.168.0.255 scope global eth0
    inet 192.168.0.200/32 scope global eth0
    inet6 fe80::20c:29ff:fed9:a8bd/64 scope link 
       valid_lft forever preferred_lft forever


BACKUP  ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:d9:8f:72 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.109/24 brd 192.168.0.255 scope global eth0
    inet6 fe80::20c:29ff:fed9:8f72/64 scope link 
       valid_lft forever preferred_lft forever

8.2、查看Keepalived日志

tail -f /var/log/messages


9、yum方式安装haproxy

yum install -y haproxy

9.2、查看haproxy版本信息

rpm -qi haproxy 或 haproxy -version

Name        : haproxy                      Relocations: (not relocatable)
Version     : 1.5.4                             Vendor: CentOS
Release     : 3.el6                         Build Date: 2016年05月11日 星期三 03时17分37秒
Install Date: 2016年08月24日 星期三 05时34分08秒      Build Host: worker1.bsys.centos.org
Group       : System Environment/Daemons    Source RPM: haproxy-1.5.4-3.el6.src.rpm
Size        : 2552550                          License: GPLv2+
Signature   : RSA/SHA1, 2016年05月12日 星期四 18时49分33秒, Key ID 0946fca2c105b9de
Packager    : CentOS BuildSystem <http://bugs.centos.org>
URL         : http://www.haproxy.org/
Summary     : HAProxy is a TCP/HTTP reverse proxy for high availability environments
Description :
HAProxy is a TCP/HTTP reverse proxy which is particularly suited for high
availability environments. Indeed, it can:
 - route HTTP requests depending on statically assigned cookies
 - spread load among several servers while assuring server persistence
   through the use of HTTP cookies
 - switch to backup servers in the event a main one fails
 - accept connections to special ports dedicated to service monitoring
 - stop accepting connections without breaking existing ones
 - add, modify, and delete HTTP headers in both directions
 - block requests matching particular patterns
 - persists clients to the correct application server depending on
   application cookies
 - report detailed status as HTML pages to authenticated users from a URI
   intercepted from the application

9.2、查看haproxy位置

rpm -ql haproxy

10、添加独立日志  vim /etc/rsyslog.conf 在底部添加以下配置信息

# haproxy
# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514  # 启动udp,启动端口后将作为服务器工作    
# # Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514  # 启动tcp监听端口    
local2.* /var/log/haproxy.log

10.1、重启日志服务

service rsyslog restart

10.2、vim haproxy.cfg 在global端中需要添加此行

log 127.0.0.1 local2

11、配置防火墙,允许80,1080端口访问,添加以下两行(测试可以直接关闭防火墙)

-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 1080 -j ACCEPT

11.1、重启防火墙

service iptables restart

15、编辑配置文件  vim /etc/haproxy/haproxy.cfg

15.1、一个最简单的http服务的配置

global
log 127.0.0.1 local2  # 定义日志
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
stats socket /var/lib/haproxy/stats
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000

frontend webser #webser为名称
option forwardfor
bind *:80
default_backend webserver
backend webserver
balance roundrobin #使拥roundrobin 算法
server app1 192.168.1.120:80 check
server app2 192.168.1.121:80 check

15.2、haproxy统计页面的输出机制

frontend webser
log 127.0.0.1 local2
option forwardfor
bind *:80
default_backend webserver
backend webserver
cookie node insert nocache
balance roundrobin
server app1 192.168.0.120:80 check cookie node1 intval 2 rise 1 fall 2
server app2 192.168.0.121:80 check cookie node2 intval 2 rise 1 fall 2
listen statistics
bind *:8009 # 自定义监听端口
stats enable # 启用基于程序编译时默认设置的统计报告
stats auth admin:admin # 统计页面用户名和密码设置
stats uri /admin?stats # 自定义统计页面的URL,默认为/haproxy?stats
stats hide-version # 隐藏统计页面上HAProxy的版本信息
stats refresh 30s # 统计页面自动刷新时间
stats admin if TRUE #如果认证通过就做管理功能,可以管理后端的服务器
stats realm Hapadmin # 统计页面密码框上提示文本,默认为Haproxy\ Statistics

15.3、静态与动态请求分离

# web服务
frontend webservs

# 绑定80端口,域名不限
bind *:80

# 定义静态规则
acl url_static path_beg -i /static /images /javascript /stylesheets
acl url_static path_end -i .jpg .gif .png .css .js .html
acl host_static hdr_beg(host) -i img. imgs. video. videos. ftp. image. download.

# 定义动态规则
acl url_php path_end -i .php

# 后端请求归纳
use_backend static if url_static or host_static
use_backend dynamic if url_php

# 默认动态组
default_backend dynamic

# 静态请求处理
backend static
# 分配算法(轮流分配)
balance roundrobin
# 实际处理请求的服务器列表
server node1 192.168.0.120:80 check maxconn 3000

# 动态请求处理
backend dynamic
# 分配算法(轮流分配)
balance roundrobin
# 实际处理请求的服务器列表
server node1 192.168.0.121:80 check maxconn 3000
server node2 192.168.0.122:80 check maxconn 3000

15.4、http完整配置负载均衡

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 30000
listen stats
mode http
bind 0.0.0.0:1080
stats enable
stats hide-version
stats uri /haproxyadmin?stats
stats realm Haproxy\ Statistics
stats auth admin:admin
stats admin if TRUE
frontend http-in
bind *:80
mode http
log global
option httpclose
option logasap #不等待响应结束就记录日志,表示提前记录日志,一般日志会记录响应时长,此不记录响应时长
option dontlognull #不记录空信息
capture request header Host len 20 #记录请求首部的前20个字符
capture request header Referer len 60 #referer跳转引用,就是上一级
default_backend servers

frontend healthcheck
bind :1099 #定义外部检测机制
mode http
option httpclose
option forwardfor
default_backend servers
backend servers
balance roundrobin
server websrv1 192.168.0.120:80 check maxconn 2000
server websrv2 192.168.0.121:80 check maxconn 2000

15.5、MySQL完整配置负载均衡

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
defaults
mode tcp
log global
option httplog
option dontlognull
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 600
listen stats
mode http
bind 0.0.0.0:1080
stats enable
stats hide-version
stats uri /haproxyadmin?stats
stats realm Haproxy\ Statistics
stats auth admin:admin
stats admin if TRUE
frontend mysql
bind *:3306
mode tcp
log global
default_backend mysqlservers
backend mysqlservers
balance leastconn
server dbsrv1 192.168.1.120:3306 check port 3306 intval 2 rise 1 fall 2 maxconn 300
server dbsrv2 192.168.1.121:3306 check port 3306 intval 2 rise 1 fall 2 maxconn 300

16、启动haproxy服务

service haproxy start

17、查看统计页面

http://192.168.0.200:1080/haproxyadmin?stats
用户名和密码 admin

18、查看Haproxy日志

tail -f /var/log/haproxy.log

19、Haproxy配置信息 MASTR 与 BACKUP配置完全相同

20、访问服务器VIP地址会自动分配到不同服务器进行处理

http://192.168.0.200

1、关闭MASTER服务,BACKUP会自动升级为MASTER接替服务。启动MASTER的Keepalived服务,会自动切回原来的MASTER服务器。
2、关闭Haproxy服务,脚本会尝试启动Haproxy服务,如果启动失败则关闭Keepalived服务,让备用服务器接替。

到这里一个完整的web负载均衡服务器就配置完成了,Haproxy主要做服务分配,Keepalived做双机热备,Keepalived还可以配置成双主热备。在keepalived中检测Haproxy是否可用,不可用是否关闭Keepalived服务器,具体可以根据自己业务做处理。

阅读全文

CentOS yum remove卸载软件出错
发表于 2016-8-26 | 浏览(3933) | 服务器

卸载过程中的错误

error: %preun(XXX-1.5.4-3.el6.x86_64) scriptlet failed, exit status 1
Error in PREUN scriptlet in rpm package XXX要卸载的包名

解决方案

yum --setopt=tsflags=noscripts remove XXX要卸载的包名

查看软件包

rpm -ql XXX包名

阅读全文

CentOS使用Keepalived搭建双机热备
发表于 2016-8-23 | 浏览(4425) | 服务器

1、环境

CentOS6.5
keepalived-1.2.23

2、准备两台服务器和一个VIP IP

Master      192.168.0.109
Backup      192.168.0.110
VIP         192.168.0.200

3、安装gcc编译器,openssl,wget,如果已经安装则跳过

yum -y install openssl-devel ncurses-devel gcc gcc-c++ make rpm-build wget

4、目录创建

mkdir /data/soft
mkdir /data/nginx
mkdir /data/www

6、安装keepalived

cd /data/soft
wget http://www.keepalived.org/software/keepalived-1.2.23.tar.gz
tar -zxvf keepalived-1.2.23.tar.gz
cd keepalived-1.2.23
./configure --prefix=/usr/local/keepalived
make
make install

7、将keepalived做成启动脚务

cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
chmod +x /etc/init.d/keepalived

8.1、MASTER配置文件,位置 /etc/keepalived/keepalived.conf

global_defs {
    notification_email
    {
        fuxiang.gong@qq.com
    }
    notification_email_from 17091959688@163.com
    smtp_server smtp.163.com
    stmp_connect_timeout 30
    router_id lnmp_node1
}

# 服务
vrrp_instance LNMP {
    state MASTER    #设置为主服务器
    interface eth0  #监测网络接口
    virtual_router_id 51  #主、备必须一样
    priority 100   #主机级别,值越大优先级越高
    advert_int 1   #VRRP Multicast广播周期秒数

    authentication {
        auth_type PASS  #VRRP认证方式,主备必须一致
        auth_pass 1111  #密码
    }
    virtual_ipaddress {
        192.168.0.200  #VRRP HA虚拟地址
    }
}

8.2、BACKUP配置文件

global_defs { 
    notification_email
    {
        fuxiang.gong@qq.com
    }
    notification_email_from 17091959688@163.com
    smtp_server smtp.163.com
    stmp_connect_timeout 30
    router_id lnmp_node2
}

# 服务
vrrp_instance LNMP {
    state BACKUP    #设置为备用服务器
    interface eth0  #监测网络接口
    virtual_router_id 51  #主、备必须一样
    priority 90   #主、备机取不同的优先级,主机值较大,备份机值较小,值越大优先级越高 
    advert_int 1   #VRRP Multicast广播周期秒数
    authentication {
        auth_type PASS  #VRRP认证方式,主备必须一致
        auth_pass 1111  #密码
    }
    virtual_ipaddress {
        192.168.0.200  #VRRP HA虚拟地址
    }
}

8.3、允许两台服务器vrrp包通过防火墙,如果关闭防火墙则跳过(两台服务器上都配置)

MASTER
  vim /etc/sysconfig/iptables
  -A INPUT -i eth0 -p vrrp -s 192.168.0.111 -j ACCEPT

BACKUP
  vim /etc/sysconfig/iptables
  -A INPUT -i eth0 -p vrrp -s 192.168.0.110 -j ACCEPT

重启防火墙
service iptables restart

9、keepalived操作,这里做启动操作

service keepalived start      启动
service keepalived stop       停止
service keepalived restart    重启

10、查看已经成功添加虚拟网卡

主服务器  ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:d9:a8:bd brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.109/24 brd 192.168.0.255 scope global eth0
    inet 192.168.0.200/32 scope global eth0
    inet6 fe80::20c:29ff:fed9:a8bd/64 scope link 
       valid_lft forever preferred_lft forever


BACKUP服务器  ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:d9:8f:72 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.110/24 brd 192.168.0.255 scope global eth0
    inet6 fe80::20c:29ff:fed9:8f72/64 scope link 
       valid_lft forever preferred_lft forever

11、安装nginx,详细步骤查看博客中其它nginx安装方法文章

12、把nginx访问指向 /data/www目录,在 /data/www 目录中创建index.html内容分别写 MASTER, BACKUP(通过浏览器访问 192.168.0.200 ),默认MASTER,如果关闭MASTER keepalived服务,BACKUP服务器将自动升级为MASTER提供服务。开启MASTER keepalived服务,自动切换MASTER服务器来服务,BACKUP降级为备用服务。

13、查看日志

tail -f /var/log/messages

阅读全文

Linux下一个网卡配置多个ip(虚拟ip)
发表于 2016-8-23 | 浏览(5082) | 服务器

临时设置,关机失效

在同一个网段的虚拟ip(本机,同网段的电脑可以ping通)以下192.168.0.100只能在本机ping通

ifconfig eth0:1 10.0.82.200 netmask 255.255.255.0
ifconfig eth0:1 192.168.0.100 netmask 255.255.255.0

设置永久ip

1、设置子ip配置文件

cd /etc/sysconfig/network-scripts

2、创建新文件 vim ifcfg-eth0:0,添加以下内容,保存退出

DEVICE=eth0:0  # 虚拟网络接口,随意    
ONBOOT=yes  # 系统启动时激活
BOOTPROTO=static  # 使用静态ip地
IPADDR=10.0.82.200  # 该虚拟网络接口的ip别名,随意
NETMASK=255.255.255.0  # 子网掩码,对应ip别名
GATEWAY=10.0.82.1  # 网关,对应ip别名
HWADDR=00:0c:29:d9:a8:bd  # 网卡MAC地址,与 eth0 相同
USERCTL=yes  # 是否给予非root用户设备管理权限

3、重启网卡

service network restart

4、临时清除ip别名

ifconfig eth0:0 down

阅读全文

VMware完整克隆虚拟机后连不上网络
发表于 2016-8-19 | 浏览(4123) | 服务器

完整克隆之后却不能连接到虚拟机了,产生这个问题的原因是克隆之后,新的克隆副本的网卡信息被修改了,所以需要手动设置下。原先使用的eth0网卡已经被使用了所以不能使用。

1、修改 70-persistent-net.rules 文件(删除旧网卡信息,使用新网卡信息)

vi /etc/udev/rules.d/70-persistent-net.rules

上面就是克隆后的文件信息,如果不是克隆的,红色部分没有。因为克隆了,所以就多出来红色部分的了。因为使用的还是旧网卡信息,所以导致了xshell连接不上。这里我们删除黄色框中东西,并将红色最后的eth1改成eth0。同时记住 ATTR{address}的值。

1.jpg

2、修改 ifcfg-eth0文件(将原来的旧网卡的HWADDR改成新网卡的HWADDR)

vi /etc/sysconfig/network-scripts/ifcfg-eth0

HWADDR的值就是上图中ATTR{addres}的值,然后设置一下想要的IP。reboot重启后就可以使用xshell连接了,注释黄色一行,红色一行为新添加。

2.jpg

3、重启服务器即可

init 6

阅读全文

Elasticsearch集群 搜索引擎
发表于 2016-8-17 | 浏览(3586) | 服务器

1、准备,两个ip在同一个网段下,两台linux安装 Elasticsearch

192.168.8.1
192.168.8.2

2、配置文件位置 config/elasticsearch.yml

A、192.168.8.1配置文件

# 开启script
script.inline: on
script.indexed: on
script.file: on

# 绑定的IP地址,可以是IPV4或者IPV6
network.bind_host: 0.0.0.0

# 其它节点交互ip地址
network.publish_host: 192.168.8.1

# 该参数用于同时设置 bind_host 和 publish_host
network.host: 192.168.8.1

# 节点之间tcp通信端口
#transport.tcp.port: 9300

# 开启tcp传输压缩
#transport.tcp.compress: true

# master节点初始化
discovery.zen.ping.unicast.hosts: ["192.168.8.1:9300","192.168.8.2:9300"]

B、192.168.8.2配置文件

# 开启script
script.inline: on
script.indexed: on
script.file: on

# 绑定的IP地址,可以是IPV4或者IPV6
network.bind_host: 0.0.0.0

# 其它节点交互ip地址
network.publish_host: 192.168.8.2

# 该参数用于同时设置 bind_host 和 publish_host
network.host: 192.168.8.2

# 节点之间tcp通信端口
#transport.tcp.port: 9300

# 开启tcp传输压缩
#transport.tcp.compress: true

# master节点初始化
discovery.zen.ping.unicast.hosts: ["192.168.8.1:9300","192.168.8.2:9300"]

3、启动 ElasticSearch ,进入 bin 目录启动

elasticSearch

4、安装 ElasticSearch Head 插件,进入 bin 目录运行

plugin install mobz/elasticsearch-head

5、通过浏览器访问插件

http://192.168.8.1:9200/_plugin/head
http://192.168.8.2:9200/_plugin/head

阅读全文

Elasticsearch入门 搜索引擎
发表于 2016-8-11 | 浏览(5884) | PHP

Elasticsearch最高效开源的搜索引擎框架,我们下面添加几条员工数据进行演示,详情可以在线文档查看,这里用PHP操作Elasticsearch的一些例子,在使用例子的情况下需要先安装Elasticsearch搜索引擎,安装方法可以查看官网,地址:https://www.elastic.co/downloads/elasticsearch

Elasticsearch依赖java,地址:http://www.java.com/

中文文档地址:http://es.xiaoleilu.com/

公共方法,用于操作Elasticsearch

/**
 * [PostCurl post请求]
 * @param  [string]  $url    [请求地址]
 * @param  [string]  $option [参数]
 * @param  [integer] $header [http头部信息]
 * @param  [string]  $type   [请求类型]
 * @return [array]           [返回的数据]
 */
function PostCurl($url, $option = '', $header = '', $type = 'POST')
{
	if(empty($header)) $header = array();
    $curl = curl_init (); // 启动一个CURL会话
    curl_setopt($curl, CURLOPT_URL, $url); // 要访问的地址
    curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE); // 对认证证书来源的检查
    curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, FALSE); // 从证书中检查SSL加密算法是否存在
    curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)'); // 模拟用户使用的浏览器
    if (! empty($option)) {
        if($type == 'PUT')
        {
            $options = json_encode($option);
        } else {
            $options = json_encode($option, JSON_FORCE_OBJECT);
        }
        curl_setopt($curl, CURLOPT_POSTFIELDS, $options); // Post提交的数据包
    }
    curl_setopt($curl, CURLOPT_TIMEOUT, 30); // 设置超时限制防止死循环
    curl_setopt($curl, CURLOPT_HTTPHEADER, $header); // 设置HTTP头
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); // 获取的信息以文件流的形式返回
    curl_setopt($curl, CURLOPT_CUSTOMREQUEST, $type);
    $result = curl_exec($curl); // 执行操作
    curl_close($curl); // 关闭CURL会话
    return json_decode($result, true);
}

定义url地址

$url = 'http://localhost:9200/';

1、添加员工数据

关系数据库     ⇒ 数据库 ⇒ 表     ⇒ 行     ⇒ 列(Columns)

Elasticsearch   ⇒ 索引   ⇒ 类型   ⇒ 文档   ⇒ 字段(Fields)

索引 megacorp

类型 employee

文档 1 ~ 3

字段

// 定义员工数据
$param = array(
	1	=>	array(
			'first_name'	=>	'John',
			'last_name'		=>	'Smith',
			'age'			=>	25,
			'about'			=>	'I love to go rock climbing',
			'interests'		=>	array('sports', 'music')
		),
	2	=>	array(
			'first_name'	=>	'Jane',
			'last_name'		=>	'Smith',
			'age'			=>	32,
			'about'			=>	'I like to collect rock albums',
			'interests'		=>	array('music')
		),
	3	=>	array(
			'first_name'	=>	'Douglas',
			'last_name'		=>	'Fir',
			'age'			=>	35,
			'about'			=>	'I like to build cabinets',
			'interests'		=>	array('forestry')
		),
	);


// 循环将员工数据加入 Elasticsearch 搜索引擎中
$success = 0;
foreach($param as $k=>$v)
{
	$result = PostCurl($url.'megacorp/employee/'.$k, $v, '', 'PUT');
	if(isset($result['_shards']['successful']) && $result['_shards']['successful'] == 1) $success++;
}
echo $success.'<br />';


输出 3

2、获取id为1的员工数据

$result = PostCurl($url.'megacorp/employee/1', '', '', 'GET');
print_r($result);

返回数据
Array
(
    [_index] => megacorp
    [_type] => employee
    [_id] => 1
    [_version] => 28
    [found] => 1
    [_source] => Array
        (
            [first_name] => John
            [last_name] => Smith
            [age] => 25
            [about] => I love to go rock climbing
            [interests] => Array
                (
                    [0] => sports
                    [1] => music
                )

        )

)

3、搜索全部员工

$result = PostCurl($url.'megacorp/employee/_search', '', '', 'GET');
print_r($result);

返回数据
Array
(
    [took] => 4
    [timed_out] => 
    [_shards] => Array
        (
            [total] => 5
            [successful] => 5
            [failed] => 0
        )

    [hits] => Array
        (
            [total] => 3
            [max_score] => 1
            [hits] => Array
                (
                    [0] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 2
                            [_score] => 1
                            [_source] => Array
                                (
                                    [first_name] => Jane
                                    [last_name] => Smith
                                    [age] => 32
                                    [about] => I like to collect rock albums
                                    [interests] => Array
                                        (
                                            [0] => music
                                        )

                                )

                        )

                    [1] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 1
                            [_score] => 1
                            [_source] => Array
                                (
                                    [first_name] => John
                                    [last_name] => Smith
                                    [age] => 25
                                    [about] => I love to go rock climbing
                                    [interests] => Array
                                        (
                                            [0] => sports
                                            [1] => music
                                        )

                                )

                        )

                    [2] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 3
                            [_score] => 1
                            [_source] => Array
                                (
                                    [first_name] => Douglas
                                    [last_name] => Fir
                                    [age] => 35
                                    [about] => I like to build cabinets
                                    [interests] => Array
                                        (
                                            [0] => forestry
                                        )

                                )

                        )

                )

        )

)

4、搜索姓 last_name等于Smith的员工, 轻量的搜索方法

$result = PostCurl($url.'megacorp/employee/_search?q=last_name:Smith', '', '', 'GET');
print_r($result);

返回数据
Array
(
    [took] => 6
    [timed_out] => 
    [_shards] => Array
        (
            [total] => 5
            [successful] => 5
            [failed] => 0
        )

    [hits] => Array
        (
            [total] => 2
            [max_score] => 0.30685282
            [hits] => Array
                (
                    [0] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 2
                            [_score] => 0.30685282
                            [_source] => Array
                                (
                                    [first_name] => Jane
                                    [last_name] => Smith
                                    [age] => 32
                                    [about] => I like to collect rock albums
                                    [interests] => Array
                                        (
                                            [0] => music
                                        )

                                )

                        )

                    [1] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 1
                            [_score] => 0.30685282
                            [_source] => Array
                                (
                                    [first_name] => John
                                    [last_name] => Smith
                                    [age] => 25
                                    [about] => I love to go rock climbing
                                    [interests] => Array
                                        (
                                            [0] => sports
                                            [1] => music
                                        )

                                )

                        )

                )

        )

)

5、使用Query DSL搜索。搜索姓 last_name等于Smith的员工, 轻量的搜索方法

$param = array(
	'query'	=>	array(
			'match'	=>	array('last_name'	=>	'Smith'),
		),
	);
$result = PostCurl($url.'megacorp/employee/_search', $param, '', 'GET');
print_r($result);

返回数据
Array
(
    [took] => 1
    [timed_out] => 
    [_shards] => Array
        (
            [total] => 5
            [successful] => 5
            [failed] => 0
        )

    [hits] => Array
        (
            [total] => 2
            [max_score] => 0.30685282
            [hits] => Array
                (
                    [0] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 2
                            [_score] => 0.30685282
                            [_source] => Array
                                (
                                    [first_name] => Jane
                                    [last_name] => Smith
                                    [age] => 32
                                    [about] => I like to collect rock albums
                                    [interests] => Array
                                        (
                                            [0] => music
                                        )

                                )

                        )

                    [1] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 1
                            [_score] => 0.30685282
                            [_source] => Array
                                (
                                    [first_name] => John
                                    [last_name] => Smith
                                    [age] => 25
                                    [about] => I love to go rock climbing
                                    [interests] => Array
                                        (
                                            [0] => sports
                                            [1] => music
                                        )

                                )

                        )

                )

        )

)

6、更加复杂的搜索,搜索姓 last_name等于Smith的员工, 轻量的搜索方法。年龄大于30岁的限定条

$param = array(
	'query'	=>	array(
		'filtered'	=>	array(
			'query'		=>	array(
				'match'	=>	array('last_name'	=>	'Smith'),
			),

			'filter'	=>	array(
				'range'	=>	array(
					'age'	=>	array('gt'	=>	30)
				),
			),
		),
	),
);
$result = PostCurl($url.'megacorp/employee/_search', $param, '', 'GET');
print_r($result);

返回数据
Array
(
    [took] => 4
    [timed_out] => 
    [_shards] => Array
        (
            [total] => 5
            [successful] => 5
            [failed] => 0
        )

    [hits] => Array
        (
            [total] => 1
            [max_score] => 0.30685282
            [hits] => Array
                (
                    [0] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 2
                            [_score] => 0.30685282
                            [_source] => Array
                                (
                                    [first_name] => Jane
                                    [last_name] => Smith
                                    [age] => 32
                                    [about] => I like to collect rock albums
                                    [interests] => Array
                                        (
                                            [0] => music
                                        )

                                )

                        )

                )

        )

)

7、更加复杂的全文搜索。一项在传统数据库很难实现的功能、我们将会搜索所有喜欢 rock climbing 的员工

$param = array(
	'query'	=>	array(
			'match'	=>	array('about'	=>	'rock climbing'),
		),
	);
$result = PostCurl($url.'megacorp/employee/_search', $param, '', 'GET');
print_r($result);

返回数据
Array
(
    [took] => 7
    [timed_out] => 
    [_shards] => Array
        (
            [total] => 5
            [successful] => 5
            [failed] => 0
        )

    [hits] => Array
        (
            [total] => 2
            [max_score] => 0.16273327
            [hits] => Array
                (
                    [0] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 1
                            [_score] => 0.16273327
                            [_source] => Array
                                (
                                    [first_name] => John
                                    [last_name] => Smith
                                    [age] => 25
                                    [about] => I love to go rock climbing
                                    [interests] => Array
                                        (
                                            [0] => sports
                                            [1] => music
                                        )

                                )

                        )

                    [1] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 2
                            [_score] => 0.016878016
                            [_source] => Array
                                (
                                    [first_name] => Jane
                                    [last_name] => Smith
                                    [age] => 32
                                    [about] => I like to collect rock albums
                                    [interests] => Array
                                        (
                                            [0] => music
                                        )

                                )

                        )

                )

        )

)

8、段落搜索。我们只需要查询到 about 字段只包含 rock climbing 的短语的员工

$param = array(
	'query'	=>	array(
			'match_phrase'	=>	array('about'	=>	'rock climbing'),
		),
	);
$result = PostCurl($url.'megacorp/employee/_search', $param, '', 'GET');
print_r($result);

返回数据
Array
(
    [took] => 2
    [timed_out] => 
    [_shards] => Array
        (
            [total] => 5
            [successful] => 5
            [failed] => 0
        )

    [hits] => Array
        (
            [total] => 1
            [max_score] => 0.23013961
            [hits] => Array
                (
                    [0] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 1
                            [_score] => 0.23013961
                            [_source] => Array
                                (
                                    [first_name] => John
                                    [last_name] => Smith
                                    [age] => 25
                                    [about] => I love to go rock climbing
                                    [interests] => Array
                                        (
                                            [0] => sports
                                            [1] => music
                                        )

                                )

                        )

                )

        )

)

9、高亮我们的搜索。但是添加一个 highlight 参数

$param = array(
	'query'	=>	array(
			'match_phrase'	=>	array('about'	=>	'rock climbing'),
		),
	'highlight'	=>	array(
			'fields'	=>	array(
					'about'	=>	array()
				)
		),
	);
$result = PostCurl($url.'megacorp/employee/_search', $param, '', 'GET');
print_r($result);

返回数据
Array
(
    [took] => 2
    [timed_out] => 
    [_shards] => Array
        (
            [total] => 5
            [successful] => 5
            [failed] => 0
        )

    [hits] => Array
        (
            [total] => 1
            [max_score] => 0.23013961
            [hits] => Array
                (
                    [0] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 1
                            [_score] => 0.23013961
                            [_source] => Array
                                (
                                    [first_name] => John
                                    [last_name] => Smith
                                    [age] => 25
                                    [about] => I love to go rock climbing
                                    [interests] => Array
                                        (
                                            [0] => sports
                                            [1] => music
                                        )

                                )

                            [highlight] => Array
                                (
                                    [about] => Array
                                        (
                                            [0] => I love to go rock climbing
                                        )

                                )

                        )

                )

        )

)

10、统计。例如,找一下员工中最受欢迎的兴趣是什么

$param = array(
	'aggs'	=>	array(
		'all_interests'	=>	array(
			'terms'	=>	array('field'	=>	'interests'),
		),
	),
);
$result = PostCurl($url.'megacorp/employee/_search', $param, '', 'GET');
print_r($result);

返回数据
Array
(
    [took] => 2
    [timed_out] => 
    [_shards] => Array
        (
            [total] => 5
            [successful] => 5
            [failed] => 0
        )

    [hits] => Array
        (
            [total] => 3
            [max_score] => 1
            [hits] => Array
                (
                    [0] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 2
                            [_score] => 1
                            [_source] => Array
                                (
                                    [first_name] => Jane
                                    [last_name] => Smith
                                    [age] => 32
                                    [about] => I like to collect rock albums
                                    [interests] => Array
                                        (
                                            [0] => music
                                        )

                                )

                        )

                    [1] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 1
                            [_score] => 1
                            [_source] => Array
                                (
                                    [first_name] => John
                                    [last_name] => Smith
                                    [age] => 25
                                    [about] => I love to go rock climbing
                                    [interests] => Array
                                        (
                                            [0] => sports
                                            [1] => music
                                        )

                                )

                        )

                    [2] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 3
                            [_score] => 1
                            [_source] => Array
                                (
                                    [first_name] => Douglas
                                    [last_name] => Fir
                                    [age] => 35
                                    [about] => I like to build cabinets
                                    [interests] => Array
                                        (
                                            [0] => forestry
                                        )

                                )

                        )

                )

        )

    [aggregations] => Array
        (
            [all_interests] => Array
                (
                    [doc_count_error_upper_bound] => 0
                    [sum_other_doc_count] => 0
                    [buckets] => Array
                        (
                            [0] => Array
                                (
                                    [key] => music
                                    [doc_count] => 2
                                )

                            [1] => Array
                                (
                                    [key] => forestry
                                    [doc_count] => 1
                                )

                            [2] => Array
                                (
                                    [key] => sports
                                    [doc_count] => 1
                                )

                        )

                )

        )

)

11、如果只想要查询姓 smith 的员工的兴趣汇总情况

$param = array(
	'query'	=>	array(
		'match'	=>	array(
			'last_name'	=>	'smith'
		),
	),
	'aggs'	=>	array(
		'all_interests'	=>	array(
			'terms'	=>	array('field'	=>	'interests'),
		),
	),
);
$result = PostCurl($url.'megacorp/employee/_search', $param, '', 'GET');
print_r($result);

返回数据
Array
(
    [took] => 2
    [timed_out] => 
    [_shards] => Array
        (
            [total] => 5
            [successful] => 5
            [failed] => 0
        )

    [hits] => Array
        (
            [total] => 2
            [max_score] => 0.30685282
            [hits] => Array
                (
                    [0] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 2
                            [_score] => 0.30685282
                            [_source] => Array
                                (
                                    [first_name] => Jane
                                    [last_name] => Smith
                                    [age] => 32
                                    [about] => I like to collect rock albums
                                    [interests] => Array
                                        (
                                            [0] => music
                                        )

                                )

                        )

                    [1] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 1
                            [_score] => 0.30685282
                            [_source] => Array
                                (
                                    [first_name] => John
                                    [last_name] => Smith
                                    [age] => 25
                                    [about] => I love to go rock climbing
                                    [interests] => Array
                                        (
                                            [0] => sports
                                            [1] => music
                                        )

                                )

                        )

                )

        )

    [aggregations] => Array
        (
            [all_interests] => Array
                (
                    [doc_count_error_upper_bound] => 0
                    [sum_other_doc_count] => 0
                    [buckets] => Array
                        (
                            [0] => Array
                                (
                                    [key] => music
                                    [doc_count] => 2
                                )

                            [1] => Array
                                (
                                    [key] => sports
                                    [doc_count] => 1
                                )

                        )

                )

        )

)

12、汇总还允许多个层面的统计。比如我们还可以统计每一个兴趣下的平均年龄

$param = array(
	'aggs'	=>	array(
		'all_interests'	=>	array(
			'terms'	=>	array('field'	=>	'interests'),
			'aggs'	=>	array(
				'avg_age' =>	array(
					'avg'	=>	array(
						'field'	=>	'age'
					),
				),
			),
		),
	),
);
$result = PostCurl($url.'megacorp/employee/_search', $param, '', 'GET');
print_r($result);

返回数据
Array
(
    [took] => 16
    [timed_out] => 
    [_shards] => Array
        (
            [total] => 5
            [successful] => 5
            [failed] => 0
        )

    [hits] => Array
        (
            [total] => 3
            [max_score] => 1
            [hits] => Array
                (
                    [0] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 2
                            [_score] => 1
                            [_source] => Array
                                (
                                    [first_name] => Jane
                                    [last_name] => Smith
                                    [age] => 32
                                    [about] => I like to collect rock albums
                                    [interests] => Array
                                        (
                                            [0] => music
                                        )

                                )

                        )

                    [1] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 1
                            [_score] => 1
                            [_source] => Array
                                (
                                    [first_name] => John
                                    [last_name] => Smith
                                    [age] => 25
                                    [about] => I love to go rock climbing
                                    [interests] => Array
                                        (
                                            [0] => sports
                                            [1] => music
                                        )

                                )

                        )

                    [2] => Array
                        (
                            [_index] => megacorp
                            [_type] => employee
                            [_id] => 3
                            [_score] => 1
                            [_source] => Array
                                (
                                    [first_name] => Douglas
                                    [last_name] => Fir
                                    [age] => 35
                                    [about] => I like to build cabinets
                                    [interests] => Array
                                        (
                                            [0] => forestry
                                        )

                                )

                        )

                )

        )

    [aggregations] => Array
        (
            [all_interests] => Array
                (
                    [doc_count_error_upper_bound] => 0
                    [sum_other_doc_count] => 0
                    [buckets] => Array
                        (
                            [0] => Array
                                (
                                    [key] => music
                                    [doc_count] => 2
                                    [avg_age] => Array
                                        (
                                            [value] => 28.5
                                        )

                                )

                            [1] => Array
                                (
                                    [key] => forestry
                                    [doc_count] => 1
                                    [avg_age] => Array
                                        (
                                            [value] => 35
                                        )

                                )

                            [2] => Array
                                (
                                    [key] => sports
                                    [doc_count] => 1
                                    [avg_age] => Array
                                        (
                                            [value] => 25
                                        )

                                )

                        )

                )

        )

)

删除指定日期之前的数据

curl -XDELETE 'http://localhost:9200/logstash-2017.02.25*'

上面这些已经可以基本掌握Elasticsearch的操作,后面更深入的、分布式集群,等...... 敬请关注~

阅读全文

mac安装pkg-config
发表于 2016-7-26 | 浏览(3954) | 服务器

下载源码包

curl http://pkgconfig.freedesktop.org/releases/pkg-config-0.28.tar.gz -o pkg-config-0.28.tar.gz

解压安装包

tar -xf pkgconfig-0.18.tar.gz

开始安装

./configure  --with-internal-glib
make
sudo make install

阅读全文

CentOS下PHP7安装mysql扩展
发表于 2016-7-12 | 浏览(6737) | 服务器

从PHP7开始mysql扩展正式移除,如果系统db需要mysql方式操作数据库,我们可以手动安装(建议使用PDO操作数据库)

1、下载mysql扩展包

git clone https://github.com/php/pecl-database-mysql     mysql   --recursive
cd mysql
phpize
./configure --with-php-config=/usr/local/php/bin/php-config
make
make install

2、vim /etc/php.ini 添加扩展包

extension=mysql.so

保存后重启php-fpm服务

php -m 查看就有了

阅读全文

终端高亮 vim高亮 彩色 变色
发表于 2016-7-10 | 浏览(3829) | 服务器

mac终端彩色配置   vim ~/.bash_profile   在文件中添加

alias ls="ls -G"
保存后重启终端使用 ls 测试即可

vim语法高亮   vim ~/.vimrc  在文件中添加

彩色
  syntax on

行号
  set nu

底部显示行号和百分比
  set ruler

保存后使用vim打开一个文件瞧一瞧即可看见效果了

阅读全文

TOP