MongoDB 3.2.x 分片搭建

5台分片服务器建立
解压
tar zxvf /data/mongo.tgz -C /data
mkdir /data/mongo/conf
mkdir /data/mongo/logs
mkdir /data/mongo/databases
mkdir /data/mongo/databases/shard1
mkdir /data/mongo/databases/shard2
mkdir /data/mongo/databases/shard3
vi /data/mongo/conf/shard1.conf
shardsvr=true
replSet=shard1
port=10001
dbpath=/data/mongo/databases/shard1
logpath=/data/mongo/logs/shard1.log
directoryperdb=true
storageEngine=wiredTiger
wiredTigerCacheSizeGB=2
syncdelay=30
wiredTigerCollectionBlockCompressor=zlib
oplogSize=2048
logappend=true
journal=false
slowms=200
fork=true
rest=true
httpinterface=true
vi /data/mongo/conf/shard2.conf
shardsvr=true
replSet=shard2
port=10002
dbpath=/data/mongo/databases/shard2
logpath=/data/mongo/logs/shard2.log
directoryperdb=true
storageEngine=wiredTiger
wiredTigerCacheSizeGB=2
syncdelay=30
wiredTigerCollectionBlockCompressor=zlib
oplogSize=2048
logappend=true
journal=false
slowms=200
fork=true
rest=true
httpinterface=true
vi /data/mongo/conf/shard3.conf
shardsvr=true
replSet=shard3
port=10003
dbpath=/data/mongo/databases/shard3
logpath=/data/mongo/logs/shard3.log
directoryperdb=true
storageEngine=wiredTiger
wiredTigerCacheSizeGB=2
syncdelay=30
wiredTigerCollectionBlockCompressor=zlib
oplogSize=2048
logappend=true
journal=false
slowms=200
fork=true
rest=true
httpinterface=true
5台配置服务器建立
mkdir /data/mongo/conf
mkdir /data/mongo/logs
mkdir /data/mongo/databases
mkdir /data/mongo/databases/config
vi /data/mongo/conf/config.conf
configsvr=true
replSet=config
port=20000
dbpath=/data/mongo/databases/config
logpath=/data/mongo/logs/config.log
directoryperdb=true
logappend=true
storageEngine=wiredTiger
profile=1
slowms=200
fork=true
maxConns=20000

 

N台路由服务器建立(本例一台)
vi /data/mongo/conf/mongos.conf
configdb=config/server1:20000,server2:20000,server3:20000,server4:20000,server5:20000
port=40000
logpath=/data/mongo/logs/mongos.log
logappend=true
maxConns=20000
fork=true
每台服务器确认安装需要的类库
确认是否有库文件/usr/lib64/libnetsnmpagent.so.20
ls -lhtr /usr/lib64/libnetsnmpagent.so.20
没有则安装
yum -y install net-snmp-libs

 

mongodb内存设置
查看:cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
cat /sys/kernel/mm/redhat_transparent_hugepage/defrag
临时设置:
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
永久vi /etc/rc.d/rc.local启动mongo之前加入:
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
   echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
   echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
 
启动mongodb命令
/data/mongo/bin/mongod -f /data/mongo/conf/shard1.conf
/data/mongo/bin/mongod -f /data/mongo/conf/shard2.conf
/data/mongo/bin/mongod -f /data/mongo/conf/shard3.conf
/data/mongo/bin/mongod -f /data/mongo/conf/config.conf
/data/mongo/bin/mongos -f /data/mongo/conf/mongos.conf
配置随系统自启动设置文件:/etc/rc.d/rc.local
重启:shutdown -r now 立刻重启
 
登录mongodb命令
/data/mongo/bin/mongo   --host  127.0.0.1:10001
/data/mongo/bin/mongo   --host  server1:10001
/data/mongo/bin/mongo   --host  server1:10002
/data/mongo/bin/mongo   --host  server1:10003
/data/mongo/bin/mongo   --host  server1:20000
/data/mongo/bin/mongo   --host  server6:40000
 
关闭mongodb命令
/data/mongo/bin/mongod -f /data/mongo/conf/shard1.conf --shutdown
/data/mongo/bin/mongod -f /data/mongo/conf/config.conf --shutdown
或先查看再kill(mongos服务必须用此)
ps -ef|grep mongo|grep -v grep
kill -2 pid

 

初始化配置服务器集群
rs.initiate( {
   _id: "config",
   configsvr: true,
   members: [
      { _id: 0, host: "server1:20000" },
      { _id: 1, host: "server2:20000" },
      { _id: 2, host: "server3:20000" },
      { _id: 3, host: "server4:20000" },
      { _id: 4, host: "server5:20000" }
   ]
} )
或
rs.add("server1:20000");
rs.add("server2:20000");
rs.add("server3:20000");

 

设置分片副本集
shard1执行
rs.initiate({"_id" : "shard1","version" : 1,
        "members" : [{"_id" : 0,"host" : "server1:10001"},{"_id" : 1,"host" : "server2:10001"},{"_id" : 2,"host" : "server3:10001"},{"_id" : 3,"host" : "server4:10001"},{"_id" : 4,"host" : "server5:10001"}]
});
shard2执行
rs.initiate({"_id" : "shard2","version" : 1,
        "members" : [{"_id" : 0,"host" : "server1:10002"},{"_id" : 1,"host" : "server2:10002"},{"_id" : 2,"host" : "server3:10002"},{"_id" : 3,"host" : "server4:10002"},{"_id" : 4,"host" : "server5:10002"}]
});
shard3执行
rs.initiate({"_id" : "shard3","version" : 1,
        "members" : [{"_id" : 0,"host" : "server1:10003"},{"_id" : 1,"host" : "server2:10003"},{"_id" : 2,"host" : "server3:10003"},{"_id" : 3,"host" : "server4:10003"},{"_id" : 4,"host" : "server5:10003"}]
});
 
或添加分片副本:
rs.add("server4:10001");
rs.add("server5:10001");
rs.add("server4:10002");
rs.add("server5:10002");
rs.add("server4:10003");
rs.add("server5:10003");
移除副本
rs.remove("server1:10003");
rs.remove("server5:10003");
添加集群,mongos下执行
sh.addShard("shard1/server1:10001,server2:10001,server3:10001,server4:10001,server5:10001")
sh.addShard("shard2/server1:10002,server2:10002,server3:10002,server4:10002,server5:10002")
sh.addShard("shard3/server1:10003,server2:10003,server3:10003,server4:10003,server5:10003")
或
use admin
db.runCommand({addshard:"shard1/server1:10001,server2:10001,server3:10001,server4:10001,server5:10001"});
db.runCommand({addshard:"shard2/server1:10002,server2:10002,server3:10002,server4:10002,server5:10002"});
db.runCommand({addshard:"shard3/server1:10003,server2:10003,server3:10003,server4:10003,server5:10003"});

 

为集群开启分片(可不开启)
数据库开启分片
sh.enableSharding("database") 或
use admin
db.runCommand( { enableSharding: <database> } )
 
集合开启分片
1.首先选择一个 shard key ,所选择的片键会影响集群的效率.参见 选择片键的注意事项. 获得注意事项.
2.如果集合中已经包含有数据,需要使用ensureIndex() 在片键上创建索引.如果集合是空的,MongoDB会在 sh.shardCollection() 过程中自动创建索引.
3.使用sh.shardCollection()方法来为一个集合启用分片,语法如下:
sh.shardCollection("<database>.<collection>", shard-key-pattern)
示例:
sh.shardCollection("records.people", { "zipcode": 1, "name": 1 } )
sh.shardCollection("people.addresses", { "state": 1, "_id": 1 } )
sh.shardCollection("assets.chairs", { "type": 1, "_id": 1 } )
sh.shardCollection("events.alerts", { "_id": "hashed" } )
或
use admin
db.runCommand({"shardcollection":"foo.bar","key":{"uid":1}});
db.runCommand({"shardcollection":"test.person","key":{_id:'hashed'}})

 

其他操作和常用命令
查看集群分片
sh.status()  db.printShardingStatus()  db.stats() db.serverStatus()
或
use admin
db.runCommand({listshards:1})
或
use config
db.databases.find();
db.shards.find();
 
移除分片
db.runCommand( { removeshard: "shardX" } )
 
移动数据库到分片
db.runCommand({movePrimary:"lessonplan",to:"shard3"})
 
创建用户和角色db.createUser() db.createRole()

 

web访问mongodb
http://server5:41000/databaseName/collectionName/
Centos Mongodb 工具
MobaXterm Personal Edition    多任务命令行工具
NoSQL Manager for MongoDB     MongoDB UI工具
Robomongo MongoDB UI工具
Mongodb数据库同步
系统自带方法
db.copyDatabase(<from_dbname>, <to_dbname>, <from_hostname>);
copyDatabase 源服务器也可能是 slave/secondary。
copyDatabase 不是原子性的操作: 如果在复制的过程中,源服务器数据库修改了,
目标服务器接收的document和更新的源服务器会有所不同。
这个命令必须运行于目标服务器。
这个命令不会给源服务器或者目标服务器加锁。两个服务器都允许进行读写操作。
其他工具
全量同步
mongosync -h server1:27017  –to server2:27020
增量同步
mongosync -h server1:27017 –to server2:27020  –oplog -s 1369406664,1
“初始化”同步(全量+增量+实时)
mongosync -h server1:27017  –to server2:27020  –oplog
同步一段时间范围内的数据
mongosync -h server1:27017  –to server2:27020  –oplog -s 1369811325,1 -t 1369811373,1
常用命令
1、关闭mongo实例   ./mongod –dbpath=/home/mongo/dbs/shard/data/shard1 –shutdown
2、设置副本节点可以读数据:db.getMongo().setSlaveOk();
3、查看分片情况:db.runCommand({listshards:1})
4、更改配置立即生效:rs.reconfig(config, { force: true})
5、查看mongo进程:ps -ef|grep mongod|grep -v grep  ,  ps -ef | grep mongo
6、实时监控:./bin/mongostat -h 10.132.83.230:40000
7、查看集群状态:mongos> db.printShardingStatus() sh.status()  db.serverStatus()
8、手动改变复制源
登录备份节点:use admin
db.adminCommand({“replSetSyncFrom”:”新复制源”})
rs.syncFrom(“新复制源”)
9、主节点降为secondary
use admin
rs.stepDown(10)#单位为 秒
摘自:http://blog.csdn.net/guochunyang/article/details/53585762

发表评论

电子邮件地址不会被公开。