mongodb cluster building

Create mongodb sharding cluster

1, Environmental preparation

1. Server node information [three servers]
    x.x.x.159, x.x.x..160, x.x.x..161
2. Server system
Linux x86? 64 GNU / Linux or centos7 or redhat
3. mongodb version
    mongodb v4.2.5
Download address: https://fastdl.mongodb.org/linux/mongodb-linux-x86_-rhel70-4.2.5.tgz

II. Cluster environment design

1. Three piecewise replication sets
    shard1(x.x.x.159:27017, x.x.x.160:27017, x.x.x.161:27017)
    shard2(x.x.x.159:27018, x.x.x.160:27018, x.x.x.161:27018)
    shard3(x.x.x.159:27019, x.x.x.160:27019, x.x.x.161:27019)
2. A config replication set
    (x.x.x.159:28018, x.x.x.160:28018, x.x.x.161:28018)
3. A mongos node
    (x.x.x.159:27000)

III. mongodb installation

`tar -zxvf mongodb-linux-x86_64-rhel70-4.2.5.tgz`
 mv mongodb-linux-x86_64-rhel70-4.2.5 /opt/mongodb4.2

 

4, Add environment variable

  1. vim /etc/profile
2. Append at the end of the file: export PATH=$PATH:/opt/mongodb/bin
3. The loading file takes effect immediately: source /etc/profile

5, Build mongodb sharding replica set

1. Execute the following commands on the three servers (x.x.x.159, x.x.x.160, x.x.x.161) nodes respectively

2. mongodb partition cluster configuration file directory

      mkdir -p /opt/mongodb4.2/conf

3. partition log directory

    mkdir -p /opt/mongodb4.2/logs

4. Create a cluster security authentication mechanism KeyFile

    openssl rand -base64 736 > /opt/mongodb4.2/conf/mongodb.keyfile

Notice: keyfile must meet the following conditions:

1) At least 6 characters, less than 1024 bytes.
2) White space characters in the file are not considered for authentication.
3) The KeyFile of the connection replica set member and the KeyFile of the start mongos process must have the same content.
4) Must be base64 encoded, but cannot have an equal sign.
5) The file permission must be 600(chmod 600), and no permission can be assigned to group members and other members.

 

5. Create partition data storage directory

    mkdir -p /opt/mongodb4.2/data/db1 [Slice 1]
    mkdir -p /opt/mongodb4.2/data/db2 [Slice 2]
    mkdir -p /opt/mongodb4.2/data/db3 [Slice 3]

6. Create partition profile

  vim /opt/mongodb4.2/conf/mongodb1.conf [Fragment 1 configuration file]
        storage:
            journal:
                enabled: true
            wiredTiger:
                engineConfig:
                    cacaeSizeGB: 2
                    directoryForIndexes: true
        systemLog:
            destination: file
            logAppend: true
            path: /opt/mongodb4.2/logs/mongodb1.log
        net:
            port: 27017
            bindIp: 0.0.0.0
        processManagement:
            fork: true
        replication:
            oplogSizeMB: 4000
            replSetName: jxk1
        sharding:
            clusterRole: shardsvr
        security:
            authorization: enabled
            keyFile: /opt/mongodb4.2/conf/mongodb.keyfile
            clusterAuthMode: keyFile


    vim /opt/mongodb4.2/conf/mongodb2.conf [Fragment 2 profile]
        storage:
            journal:
                enabled: true
            wiredTiger:
                engineConfig:
                    cacaeSizeGB: 2
                    directoryForIndexes: true
        systemLog:
            destination: file
            logAppend: true
            path: /opt/mongodb4.2/logs/mongodb2.log
        net:
            port: 27018
            bindIp: 0.0.0.0
        processManagement:
            fork: true
        replication:
            oplogSizeMB: 4000
            replSetName: jxk2
        sharding:
            clusterRole: shardsvr
        security:
            authorization: enabled
            keyFile: /opt/mongodb4.2/conf/mongodb.keyfile
            clusterAuthMode: keyFile


    vim /opt/mongodb4.2/conf/mongodb3.conf [Fragment 3 configuration file]
        storage:
            journal:
                enabled: true
            wiredTiger:
                engineConfig:
                    cacaeSizeGB: 2
                    directoryForIndexes: true
        systemLog:
            destination: file
            logAppend: true
            path: /opt/mongodb4.2/logs/mongodb3.log
        net:
            port: 27018
            bindIp: 0.0.0.0
        processManagement:
            fork: true
        replication:
            oplogSizeMB: 4000
            replSetName: jxk3
        sharding:
            clusterRole: shardsvr
        security:
            authorization: enabled
            keyFile: /opt/mongodb4.2/conf/mongodb.keyfile
            clusterAuthMode: keyFile

    //Note: the syntax format of the configuration file yuml can not be used if it is copied directly

   

6. Start sharding copy set

  1. On three servers(x.x.x.159,x.x.x.160, x.x.x.161)The node executes the following command
  2. mongod -f /opt/mongodb4.2/conf/mongodb1.conf
  3. mongod -f /opt/mongodb4.2/conf/mongodb2.conf
  4. mongod -f /opt/mongodb4.2/conf/mongodb3.conf

7, Initialize the shard copy set and set the shard copy set account

1. Log in to any node server and use the x.x.x.159 node
2. Mongo port 27017 [login]
3. Enter mongodb command input terminal input command as follows

    var conf = {
        _id:'jxk',
        members:[
            {_id:1, host:'x.x.x.159:27017'},
            {_id:2, host:'x.x.x.160:27017'},
            {_id:3, host:'x.x.x.161:27017'},
        ],
    }
    rs.initiate(conf)
    rs.status()

  

4. Find the main database selected by mongodb and enter the command:

    use admin
    db.createUser({
        user: "username",
        pwd : "password",
        roles:[
            {role:"root", db:"admin"}
        ]
    })
    db.auth("username", "password")

8, Set up the replication set of config node

1. Execute the following commands on the three servers (x.x.x.159, x.x.x.160, x.x.x.161) nodes respectively

2. Create config data storage directory

    mkdir -p /opt/mongodb4.2/mongo-cfg/data

3. Create config log directory

    mkdir -p /opt/mongodb4.2/mongo-cfg/logs

4. Create config replica set configuration file
 

    vim /opt/mongodb4.2/conf/mongodb-cfg.conf
        systemLog:
            destination: file
            logAppend: true
            path: /opt/mongodb4.2/logs/mongodb3.log
        storage:
            journal:
                enabled: true
            dbPath: /opt/mongodb4.2/mongo-cfg/data
            directoryPerDB: true
            wiredTiger:
                engineConfig:
                    cacaeSizeGB: 2
                    directoryForIndexes: true
                collectionConfig:
                    blockCompressor: zstd
                indexConfig:
                    prefixCompression: true
        net:
            port: 28018
            bindIp: x.x.x.159
        replication:
            oplogSizeMB: 4000
            replSetName: configReplSet
        sharding:
            clusterRole: configsvr
        processManagement:
            fork: true 

5. Start the config node replica set

    mongod -f /opt/mongodb4.2/conf/mongodb-cfg.conf

6. Log in to the config node

    mongo -host x.x.x.(159/160/161) -port 28018

7. Initialize config node

        var conf = {
            _id:'configReplSet',
            members:[
                {_id:0, host:"x.x.x.159:28018"},
                {_id:1, host:"x.x.x.160:28018"},
                {_id:2, host:"x.x.x.161:28018"},
            ]
        }
        rs.initiate(conf)
        rs.status()
    

8. Create an authenticated user

        use admin
        db.createUser({
            user: "username",
            pwd : "password",
            roles:[
                {role:"root", db:"admin"}
            ]
        })
        db.auth("username", "password")

  

9. Enable config replica set authentication
Close config replica set
Append in profile
    vim /opt/mongodb4.2/conf/mongodb-cfg.conf

     security:
          authorization: enabled
          keyFile: /opt/mongodb4.2/conf/mongodb.keyfile
          clusterAuthMode: keyFile

 

Start config replica set

9, Build mongos node (x.x.18.159)

1. Build the client on the x.x.x.159 server
2. Create mongos log directory

    mkdir -p /opt/mongodb4.2/mongos/logs

3. Create mongos configuration file
  vim /opt/mongodb4.2/conf/mongos.conf

        systemLog:
            destination: file
            logAppend: true
            path: /opt/mongodb4.2/mongos/logs/mongodb.log
        net:
            port: 27000
            bindIp: 0.0.0.0
        sharding:
            configDB: configReplSet/x.x.x.159:28018,x.x.x.160:28018,x.x.x.160:28018,
        processManagement:
            fork: true  
        security:
            keyFile: /opt/mongodb4.2/conf/mongodb.keyfile
            clusterAuthMode: keyFile

4. Start mongos

    mongos -f /opt/mongodb4.2/conf/mongos.conf

5. Log in and initialize mongos

    mongo -port 27000
    use admin
    db.auth("username", "password")  #The account used here is the account of the config replica set
    db.runCommand({
        addshard:"jxk1/x.x.x.159:27017,x.x.x.160:27017,x.x.x.161:27017,",
        name: "shard1"
    });
    db.runCommand({
        addshard:"jxk2/x.x.x.159:27018,x.x.x.160:27018,x.x.x.161:27018,",
        name: "shard1"
    });
    db.runCommand({
        addshard:"jxk3/x.x.x.159:27019,x.x.x.160:27019,x.x.x.161:27019,",
        name: "shard1"
    })

  

6. Test whether the building is successful

    db.runCommand( {enablesharding:"dbtest"})
    db.runCommand( {shardcollection:"dbtest.one", key:{id:1} })
    use dbtest
    var tmp = [];
    for(var i =0; i<10000000; i++){
        tmp.push({ 'id':i, "name":"lunck dog " + i});
    }
    db.one.insertMany(tmp);
    sh.status()

  

10, Common command summary

1. Slice command

  mongod -f /opt/mongodb4.2/conf/mongodb1.conf start-up
  mongod -f /opt/mongodb4.2/conf/mongodb1.conf --shutdown Close
  OR
  mongo -port 27017
  use admin
  db.shutdownServers()

  

2.config replica set command

    mongod -f /opt/mongodb4.2/conf/mongodb-cfg.conf start-up
    mongod -f /opt/mongodb4.2/conf/mongodb-cfg.conf --shutdown Close

3. mongos command

    mongos -f /opt/mongodb4.2/conf/mongos.conf start-up
    mongo -port 27017 Close
    use admin
    db.shutdownServers()

  

reference material

  https://my.oschina.net/u/563789/blog/3050068
  https://blog.csdn.net/qq_24434491/article/details/102907486
  https://blog.csdn.net/lezeqe/article/details/90518179
  https://blog.csdn.net/tototuzuoquan/article/details/78295040
  https://www.freesion.com/article/6886370039/#8mongo_247
  https://www.cnblogs.com/shaosks/p/5775757.html
  https://www.cnblogs.com/wxmdevelop/p/7341292.html

Tags: Linux MongoDB vim Fragment

Posted on Wed, 29 Apr 2020 01:10:36 -0700 by JonathanV