ambari-server迁移

在新的ambari-server 安装  ambari-server
设置java环境
系统基础设置
设置与各个节点之间的ssh免密码登陆
ssh-copy-id -i /root/.ssh/id_rsa.pub datanodeyy-37.ds.xx.cn
ssh-copy-id -i /root/.ssh/id_rsa.pub datanodeyy-xxx.ds.xx.cn
…..  节点太多估计要多花点时间
各个节点的hosts文件要把新的ambari-server 主机名与ip信息添加进去    #节点太多估计要多花点时间   最好用内部DNS服务器就会省事很多 
添加repo库
 [ambari]
name=ambari
baseurl=http://172.16.30.22/AMBARI-2.2.1.0/centos7/2.2.1.0-161/
gpgcheck=0
enabled=1

yum install ambari-server
复杂mysql java连接jar包
cp mysql-connector-java-5.1.34.jar /usr/share/java/mysql-connector-java-5.1.34.jar
cp mysql-connector-java-5.1.34.jar /usr/lib/ambari-server/mysql-connector-java-5.1.34.jar

直接重用原来ambari-server的配置文件
mv /etc/ambari-server/conf  /etc/ambari-server/confold
scp -r root@old.ambari-server.xx.cn:/etc/ambari-server/conf /etc/ambari-server/conf

然后直接就启动ambari-server
ambari-server   start  (因为直接用了原有的配置文件,就不用ambari-server setup了  注意java环境路径,基础环境要跟原来的一样,不然会出错)

直接 https://ambari-serverxxxx.cn:8080     admin/admin  就直接进去了,  但是进去由于ambari-agent没有修改,故会发现所有节点都是丢失心跳状态

修改各个节点的ambari-agent
把/etc/ambari-agent/conf/ambari-agent.ini  中的
hostname=old-ambari-server
改为
hostname=new-ambari-server

重启agent
ambari-agent restart       #节点太多建议用ansiable跑一下就省事很多

over

redis mark

config set client-output-buffer-limit “slave 0 0 0” #防止全量同步时 生成的实例快照过大造成同步失败
config set repl-backlog-size 936870912 #建议时间长一点,全量备份恢复时间过长,主中的临时写入队列放里面,太短的话,可能造成无限的全量同步
config set repl-timeout 240    #建议时间长一点,全量备份恢复时间过长,造成判断为集群断开,造成无限的全量同步,
config set no-appendfsync-on-rewrite yes   #做bgrewriteaof的时候 就不要再主动往 aof文件中追加, 会造成文件Io争抢,造成阻塞. 仅会造成一点点的数据安全隐患,可接受范围
config set appendfsync “no”  #性能最高。由于集群由主从两份,30s的数据安全可保证
config set stop-writes-on-bgsave-error no   #bgsave失败就不允许再写入 这开关关闭影响比较小

config set save “”  #aof 和RDB 选一个就好

echo 1 > /proc/sys/vm/overcommit_memory  # vm.overcommit_memory=1
echo 511 > /proc/sys/net/core/somaxconn
echo never > /sys/kernel/mm/transparent_hugepage/enabled  #redis的页单位的数据量是比较小的,启用大页不方便管理
/etc/security/limits.conf   #ulimit 限制 65535

cluster failover  #手动切换redis cluster的主从关系  后面可加force 强制选项

redis-trib.rb  create  –replicas 1  192.168.1.101:6379  192.168.1.102:6379   192.168.1.103:6379   192.168.1.104:6379   192.168.1.105:6379   192.168.1.106:6379 #create 创建集群  replicas  代表有几个备份
redis-trib.rb add-node 192.168.0.110:6379  192.168.0.120:6379 #增加集群节点

在做bgrewriteaof 太慢失败时, 可先尝试做一次bgsave 再做bgrewriteaof 就会快很多

chrome调试

console中 输出对应DOM的值
$0 返回最后一次点选的DOM节点
先选中对应的DOM节点 再
$0.value

$0~$4则代表了最近5个你选择过的DOM节点。

也可以使用 类似 document.getElementById(‘ext-co-1543’).value;

copy通过此命令可以将在控制台获取到的内容复制到剪贴板
copy($0)
copy(document.body)

monitor & unmonitor
monitor(function),它接收一个函数名作为参数,比如function a,每次a被执行了,都会在控制台输出一条信息,里面包含了函数的名称a及执行时所传入的参数。
而unmonitor(function)便是用来停止这一监听。

console中关于类似jquery选择器的支持:

1. 如果打开的网页本身使用了jQuery,那么它的控制台是可以直接使用所有的jQuery语法的(具体根据jQuery版本而定)。 
2. chrome浏览器的控制台支持一部分jQuery的语法(jq选择器),并非全部。
若网页本身中没有包含jquery,则可以

先添加这两个chrome扩展
jQuery Injector
jQuery Everywhere   没用

直接注入jquery代码由于chrome CSP安全原因,下面这两种办法引入jquery 无用

var importJs=document.createElement('script')  //在页面新建一个script标签
importJs.setAttribute("type","text/javascript")  //给script标签增加type属性
importJs.setAttribute("src", 'http://ajax.microsoft.com/ajax/jquery/jquery-1.4.min.js') //给script标签增加src属性, url地址为cdn公共库里的
document.getElementsByTagName("head")[0].appendChild(importJs) //把importJs标签添加在页面
;(function(d,s){d.body.appendChild(s=d.createElement('script')).src='http://cdn.bootcss.com/jquery/1.11.0/jquery.min.js'})(document);

// https页面先在控制台写如下代码
;(function(d,s){d.body.appendChild(s=d.createElement('script')).src='https://cdn.bootcss.com/jquery/1.11.0/jquery.min.js'})(document);


js断点中,在wath中直接修改js变量值

sql mark

1.查看抢占锁的语句

SELECT * FROM information_schema.innodb_trx where trx_id in (SELECT lock_trx_id FROM information_schema.INNODB_LOCKS WHERE LOCK_TRX_ID IN (SELECT BLOCKING_TRX_ID FROM information_schema.INNODB_LOCK_WAITS) );

php jwt

1.生成签名:

<?php
$message="eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1cmwiOiJodHRwczpcL1wveHh4eC54eHguY25cL3dhcFwvaW5kZXgucGhwIiwidWlkIjoieHh4eCIsImNpdHkiOiJ4eHgiLCJkYXRhX3ZlciI6MiwidHMiOjE1MzcxNzg5NjE5MzEsImNpdHlfY29kZSI6IjUwMDEwMCIsInByb3ZfY29kZSI6IjUwMDAwMCIsInByb3YiOiJ4IiwibGF0IjoyMTkuNjE0NTE1ODA2OTU5MTA1LCJsbmciOjIwNi41MDYyNTMyODI0MzMxMSwibm9uY2UiOiI2QTAyMkI5MS1BRUQ1LTQ3N0MtODkzRC1BODQ3RDkyMjk1RUEifQ==";
$secret='07e4e10fbc774f8ab914b58a2ea26752';
$secret=md5($secret);
$s = hash_hmac('sha256', $message, $secret,true);
$resut=base64_encode($s);
echo $resut;
$saferesult=str_replace('=', '', strtr(base64_encode($s), '+/', '-_'));
var_dump($saferesult);
?>

2.

<?php
$head='{"alg": "HS256","typ": "JWT"}';
$payload='{"sub": "1234567890","name": "John Doe","iat": 151623902}';
$head_base=base64_encode($head);
$payload_base=base64_encode($payload);
$message="{$head_base}.{$payload_base}";
$secret='07e4e10fbc774f8ab914b58a2ea26752';
//$secret=md5($secret);
$s = hash_hmac('sha256', $message, $secret,true);
$resut=base64_encode($s);
echo $resut;
$saferesult=str_replace('=', '', strtr(base64_encode($s), '+/', '-_'));
var_dump($saferesult);
$token=$message.".".$saferesult;
?>

lua dump 变量

function var_dump(data, max_level, prefix)   
	if type(prefix) ~= "string" then   
		prefix = ""  
	end   
	if type(data) ~= "table" then   
		dump_html(prefix .. tostring(data))   
	else  
		dump_html(tostring(data))   
		if max_level ~= 0 then   
			local prefix_next = prefix .. "    "  
			dump_html(prefix .. "{")   
			for k,v in pairs(data) do   
				dump_html(prefix_next .. k .. " = ") 
				if type(v) ~= "table" or (type(max_level) == "number" and max_level <= 1) then   
					dump_html(v)   
				else  
					if max_level == nil then   
						var_dump(v, nil, prefix_next)   
					else  
						var_dump(v, max_level - 1, prefix_next)   
					end   
				end   
			end   
			dump_html(prefix .. "}")   
		end   
	end   
end  


function dump_html(str)
	if str ~= nil then
		ngx.header.content_type = "text/html"
        ngx.say(str)
	end
end

logstash 解析nginx error日志

input {
#    beats {
#        host => "0.0.0.0"
#        port => 5400
#    }

stdin { }
}

filter {
 grok {
   patterns_dir => "/etc/logstash/patterns"
   #match => [ "message" , "%{NGINXACCESS}"]
   match => [ "message" , "%{DATA:timestr} \[%{DATA:error_level}\] (?<nginx_message>(.|\r|\n)*)(?:, client: %{IPORHOST:clientip})(?:, server: %{IPORHOST:nginx_server})(?:, request: \"%{DATA:nginx_request}\")?(?:, upstream: \"%{DATA:nginx_upstream}\")?(?:, host: \"%{DATA:nginx_host}\")?(?:, referrer: \"%{DATA:nginx_referrer}\")?"]
 }

      if [http_x_forwarded_for] == "-" or [http_x_forwarded_for] == "null" {
         mutate {
            update => { "http_x_forwarded_for" => "" }
         }
      }

      if [referer] == "-" or [referer] == "null" {
         mutate {
            update => { "referer" => "" }
         }
      }

    geoip {
      source => "clientip"
    }

    useragent {
      source => "agent"
      target => "agent_fields"
    }

  date {
    match => [ "timestr", "yyyy/MM/dd HH:mm:ss" ]
    timezone => "Asia/Shanghai"
    #target => "newtimestr"
    #locale => "en"
  }

        ruby {
                code => "event.set('index_day', event.get('@timestamp').time.localtime.strftime('%Y%m%d'))"
        }


}

output {
 elasticsearch {
   hosts => ["127.0.0.1:9200"]
   #index => "tek-%{+YYYY.MM.dd}"
   index => "tek-%{index_day}"
   document_type => "nginx_logs"
   template_name => "ta"
 }
 stdout { codec => rubydebug }
}

防止一天的日志分在了两个index中

        ruby {
                code => "event.set('index_day', event.get('@timestamp').time.localtime.strftime('%Y%m%d'))"
        }

kvm网络隔离禁止虚拟机之间通信

nwfilter xml文件默认路径: /etc/libvirt/nwfilter/

cat deny-test.xml
<filter name='deny-test' chain='ipv4' priority='-700'>
  <uuid>fce8ae34-e69e-83bf-262e-30786c1f8079</uuid>
  <rule action='drop' direction='out' priority='200'>
    <ip srcipaddr='172.21.13.102' dstipaddr='172.21.13.107' dstipmask='32'/>
  </rule>
</filter>


virsh nwfilter-define deny-test.xml
virsh nwfilter-list #确认是否添加成功


virsh edit xxx

<interface type='bridge'>
      <mac address='52:54:00:7c:17:86'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <filterref filter='deny-test'/> #add
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

重启虚拟机

宿主机上执行:

ebtables -t nat -L


确认规则有没有添加成功

Bridge table: nat

Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
-i vnet46 -j libvirt-I-vnet46

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

Bridge chain: POSTROUTING, entries: 0, policy: ACCEPT

Bridge chain: libvirt-I-vnet46, entries: 1, policy: ACCEPT
-p IPv4 -j I-vnet46-ipv4

Bridge chain: I-vnet46-ipv4, entries: 1, policy: ACCEPT
-p IPv4 --ip-src 172.21.13.102 --ip-dst 172.21.13.107 -j DROP 

简单 SHELL 

#!/bin/bash
# usage ./1.sh 172.21.13.102 deny-test 
tmpxml=$(mktemp /tmp/ifcfg.XXX)
macaddr="$(virsh domiflist $1 | awk "/bridge\s/ {print \$NF}")"
if [ -z "$macaddr" ]; then 
    echo "vm not exist" 
    exit 2
fi
if [ -z "$2" ]; then 
    echo "nwfilter name is null" 
    exit 2
fi

cat > "$tmpxml" <<EOF
<interface type='bridge'>
    <mac address='$macaddr'/>
    <source bridge='br0'/>
    <model type='virtio'/>
    <filterref filter='$2'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
EOF
virsh update-device "$1" "$tmpxml" --live --persistent --config
rm "$tmpxml"