mongodb 3.0.2与wiredTiger存储引擎安装测试

拼搏现实的明天。 2022-08-09 13:56 221阅读 0赞

mongodb3.0版本，除了之前默认的MMAP存储引擎，还新推出了wiredTiger存储引擎。

据官网测试报告称，mongodb 3.0版本使用wiredTiger存储引擎后，写性能能够提高7-10倍，数据压缩30%-80%，运维降低95%，这三个方面都还是很有吸引力的。

为了进行mongodb 2.6和mongodb 3.0的性能，先搭建一个mongodb 3.0环境，然后用YCSB工具进行测试。

mongodb 3.0.2 与wiredTiger的安装步骤如下：

**1. 下载和安装mongodb 3.0软件**

从mongodb官网下载mongodb 3.0的安装介质  mongodb-linux-x86\_64-rhel62-3.0.2.tgz ，然后登陆的服务器上执行下面的命令安装：

cd /usr/local/

rz mongodb-linux-x86\_64-rhel62-3.0.2.tgz

tar -zxvf mongodb-linux-x86\_64-rhel62-3.0.2.tgz

ln -s mongodb-linux-x86\_64-rhel62-3.0.2 mongodb30

chown -R mongodb:mongodb mongodb-linux-x86\_64-rhel62-3.0.2 mongodb30

**2. 准备mongodb实例目录和配置文件**

创建mongodb实例所需目录：

rm -fR /data/mongodb30/

mkdir -p /data/mongodb30/\{db,logs\}

touch /data/mongodb30/logs/mongodb.log

chown -R mongodb:mongodb /data/mongodb30/

创建配置文件，配置启动参数，尤其注意添加wriedTiger存储引擎相关参数：

vim /data/mongodb30/mongodb30.conf

storageEngine = wiredTiger

wiredTigerCacheSizeGB = 2

syncdelay = 30

wiredTigerCollectionBlockCompressor = snappy

port=38019

dbpath=/data/mongodb30/db

oplogSize=2048

logpath=/data/mongodb30/logs/mongodb.log

logappend=true

fork=true

rest=true

journal = true

上面的参数中，标红的参数是wriedTiger相关参数， storageEngine 是设置存储引擎；wiredTigerCacheSizeGB 是设置mongodb存储引擎所用的内容，默认为系统内存的50%；syncdelay 是设置从内存同步到硬盘的时间间隔，默认为60秒，可以设置的少一些，在mongodb故障时，丢失的日志会少一些；wiredTigerCollectionBlockCompressor 是设定压缩策略 snappy 是一种压缩速度非常快的压缩策略。

由此可见，在mongodb 3.0中想要启用wriedTiger存储引擎，只需要增加该引擎的几个参数即可。

**注：**mongodb实例的启动，可以用命令行启动，也可以用配置文件启动；为了方便操作，并且可以保存启动的的参数，建议使用配置文件的方式启动mongodb实例。

修改实例目录和文件的属主为mongodb：

chown -R mongodb:mongodb /data/mongodb30/

**3. 启动mongodb实例，并登录测试**

使用mongod命令，以配置文件的方式启动mongodb实例：

/usr/local/mongodb30/bin/mongod --config /data/mongodb30/mongodb30.conf

确认启动后的进程状态：

ps -ef | grep mongo

使用mongo命令登录到mongodb实例中：

/usr/local/mongodb30/bin/mongo --port=38019

实际操作过程记录如下：

[root@test07 local]# 
    [root@test07 local]# rm -fR /data/mongodb30/
    [root@test07 local]# mkdir -p /data/mongodb30/{db,logs}
    [root@test07 local]# touch /data/mongodb30/logs/mongodb.log
    [root@test07 local]# chown -R mongodb:mongodb /data/mongodb30/
    [root@test07 local]# 
    [root@test07 local]# vim /data/mongodb30/mongodb30.conf
    [root@test07 local]# 
    [root@test07 local]# 
    [root@test07 local]# cat /data/mongodb30/mongodb30.conf
    storageEngine = wiredTiger 
    wiredTigerCacheSizeGB = 2
    syncdelay = 30
    wiredTigerCollectionBlockCompressor = Snappy
    port=38019
    dbpath=/data/mongodb30/db
    oplogSize=2048
    logpath=/data/mongodb30/logs/mongodb.log
    logappend=true
    fork=true
    rest=true
    journal = true
    [root@test07 local]# 
    [root@test07 local]# chown -R mongodb:mongodb /data/mongodb30/
    [root@test07 local]# 
    [root@test07 local]# su - mongodb
    [mongodb@test07 ~]$ 
    [mongodb@test07 ~]$ /usr/local/mongodb30/bin/mongod --config /data/mongodb30/mongodb30.conf
    2015-10-27T11:06:12.889+0800 F CONTROL  Failed global initialization: BadValue storage.wiredTiger.collectionConfig.blockCompressor must be a string of the format: (none/snappy/zlib)
    [mongodb@test07 ~]$ sed -i 's/Snappy/snappy/g' /data/mongodb30/mongodb30.conf                               
    [mongodb@test07 ~]$ 
    [mongodb@test07 ~]$ /usr/local/mongodb30/bin/mongod --config /data/mongodb30/mongodb30.conf
    2015-10-27T11:07:04.640+0800 I CONTROL  ** WARNING: --rest is specified without --httpinterface,
    2015-10-27T11:07:04.640+0800 I CONTROL  **          enabling http interface
    about to fork child process, waiting until server is ready for connections.
    forked process: 3098
    child process started successfully, parent exiting
    [mongodb@test07 ~]$ 
    [mongodb@test07 ~]$ ps -ef | grep mysql
    mongodb   3344  3063  0 11:42 pts/0    00:00:00 grep mysql
    [mongodb@test07 ~]$ 
    [mongodb@test07 ~]$ 
    [mongodb@test07 ~]$ ps -ef|grep mongodb
    mongodb   2953     1  0 11:00 ?        00:00:16 /usr/local/mongodb26/bin/mongod --config /data/mongodb26/mongodb26.conf
    root      3062  2802  0 11:05 pts/0    00:00:00 su - mongodb
    mongodb   3063  3062  0 11:05 pts/0    00:00:00 -bash
    mongodb   3098     1  0 11:07 ?        00:00:13 /usr/local/mongodb30/bin/mongod --config /data/mongodb30/mongodb30.conf
    mongodb   3346  3063  0 11:42 pts/0    00:00:00 ps -ef
    mongodb   3347  3063  0 11:42 pts/0    00:00:00 grep mongodb
    [mongodb@test07 ~]$ 
    [mongodb@test07 ~]$ 
    [mongodb@test07 ~]$ /usr/local/mongodb30/bin/mongo --port=38019
    MongoDB shell version: 3.0.2
    connecting to: 127.0.0.1:38019/test
    Server has startup warnings: 
    2015-10-27T11:07:04.640+0800 I CONTROL  ** WARNING: --rest is specified without --httpinterface,
    2015-10-27T11:07:04.640+0800 I CONTROL  **          enabling http interface
    > 
    > show dbs;
    local  0.000GB
    > use admin;
    switched to db admin
    > show collections;
    > 
    > 
    > db.serverStatus()
    {
            "host" : "test07:38019",
            "version" : "3.0.2",
            "process" : "mongod",
            "pid" : NumberLong(3098),
            "uptime" : 2697,
            "uptimeMillis" : NumberLong(2697265),
            "uptimeEstimate" : 2457,
            "localTime" : ISODate("2015-10-27T03:52:01.900Z"),
            "asserts" : {
                    "regular" : 0,
                    "warning" : 0,
                    "msg" : 0,
                    "user" : 0,
                    "rollovers" : 0
            },
            "connections" : {
                    "current" : 1,
                    "available" : 818,
                    "totalCreated" : NumberLong(2)
            },
            "cursors" : {
                    "note" : "deprecated, use server status metrics",
                    "clientCursors_size" : 0,
                    "totalOpen" : 0,
                    "pinned" : 0,
                    "totalNoTimeout" : 0,
                    "timedOut" : 0
            },
            "extra_info" : {
                    "note" : "fields vary by platform",
                    "heap_usage_bytes" : 31212656,
                    "page_faults" : 0
            },
            "globalLock" : {
                    "totalTime" : NumberLong("2697266000"),
                    "currentQueue" : {
                            "total" : 0,
                            "readers" : 0,
                            "writers" : 0
                    },
                    "activeClients" : {
                            "total" : 8,
                            "readers" : 0,
                            "writers" : 0
                    }
            },
            "locks" : {
                    "Global" : {
                            "acquireCount" : {
                                    "r" : NumberLong(724),
                                    "w" : NumberLong(2),
                                    "W" : NumberLong(4)
                            }
                    },
                    "Database" : {
                            "acquireCount" : {
                                    "r" : NumberLong(723),
                                    "R" : NumberLong(1),
                                    "W" : NumberLong(2)
                            }
                    },
                    "Collection" : {
                            "acquireCount" : {
                                    "r" : NumberLong(722)
                            }
                    }
            },
            "network" : {
                    "bytesIn" : 1371,
                    "bytesOut" : 3116,
                    "numRequests" : 18
            },
            "opcounters" : {
                    "insert" : 0,
                    "query" : 1,
                    "update" : 0,
                    "delete" : 0,
                    "getmore" : 0,
                    "command" : 19
            },
            "opcountersRepl" : {
                    "insert" : 0,
                    "query" : 0,
                    "update" : 0,
                    "delete" : 0,
                    "getmore" : 0,
                    "command" : 0
            },
            "storageEngine" : {
                    "name" : "wiredTiger"
            },
            "wiredTiger" : {
                    "uri" : "statistics:",
                    "LSM" : {
                            "sleep for LSM checkpoint throttle" : 0,
                            "sleep for LSM merge throttle" : 0,
                            "rows merged in an LSM tree" : 0,
                            "application work units currently queued" : 0,
                            "merge work units currently queued" : 0,
                            "tree queue hit maximum" : 0,
                            "switch work units currently queued" : 0,
                            "tree maintenance operations scheduled" : 0,
                            "tree maintenance operations discarded" : 0,
                            "tree maintenance operations executed" : 0
                    },
                    "async" : {
                            "number of allocation state races" : 0,
                            "number of operation slots viewed for allocation" : 0,
                            "current work queue length" : 0,
                            "number of flush calls" : 0,
                            "number of times operation allocation failed" : 0,
                            "maximum work queue length" : 0,
                            "number of times worker found no work" : 0,
                            "total allocations" : 0,
                            "total compact calls" : 0,
                            "total insert calls" : 0,
                            "total remove calls" : 0,
                            "total search calls" : 0,
                            "total update calls" : 0
                    },
                    "block-manager" : {
                            "mapped bytes read" : 0,
                            "bytes read" : 12288,
                            "bytes written" : 126976,
                            "mapped blocks read" : 0,
                            "blocks pre-loaded" : 0,
                            "blocks read" : 3,
                            "blocks written" : 27
                    },
                    "cache" : {
                            "tracked dirty bytes in the cache" : 0,
                            "tracked bytes belonging to internal pages in the cache" : 1245,
                            "bytes currently in the cache" : 14644,
                            "tracked bytes belonging to leaf pages in the cache" : 2147482403,
                            "maximum bytes configured" : 2147483648,
                            "tracked bytes belonging to overflow pages in the cache" : 0,
                            "bytes read into cache" : 0,
                            "bytes written from cache" : 20178,
                            "pages evicted by application threads" : 0,
                            "checkpoint blocked page eviction" : 0,
                            "unmodified pages evicted" : 0,
                            "page split during eviction deepened the tree" : 0,
                            "modified pages evicted" : 0,
                            "pages selected for eviction unable to be evicted" : 0,
                            "pages evicted because they exceeded the in-memory maximum" : 0,
                            "pages evicted because they had chains of deleted items" : 0,
                            "failed eviction of pages that exceeded the in-memory maximum" : 0,
                            "hazard pointer blocked page eviction" : 0,
                            "internal pages evicted" : 0,
                            "maximum page size at eviction" : 0,
                            "eviction server candidate queue empty when topping up" : 0,
                            "eviction server candidate queue not empty when topping up" : 0,
                            "eviction server evicting pages" : 0,
                            "eviction server populating queue, but not evicting pages" : 0,
                            "eviction server unable to reach eviction goal" : 0,
                            "pages split during eviction" : 0,
                            "pages walked for eviction" : 0,
                            "eviction worker thread evicting pages" : 0,
                            "in-memory page splits" : 0,
                            "percentage overhead" : 8,
                            "tracked dirty pages in the cache" : 0,
                            "pages currently held in the cache" : 10,
                            "pages read into cache" : 0,
                            "pages written from cache" : 16
                    },
                    "connection" : {
                            "pthread mutex condition wait calls" : 59275,
                            "files currently open" : 7,
                            "memory allocations" : 21765,
                            "memory frees" : 19667,
                            "memory re-allocations" : 64,
                            "total read I/Os" : 8,
                            "pthread mutex shared lock read-lock calls" : 1336,
                            "pthread mutex shared lock write-lock calls" : 2958,
                            "total write I/Os" : 141
                    },
                    "cursor" : {
                            "cursor create calls" : 568,
                            "cursor insert calls" : 24,
                            "cursor next calls" : 12,
                            "cursor prev calls" : 3,
                            "cursor remove calls" : 0,
                            "cursor reset calls" : 119,
                            "cursor search calls" : 604,
                            "cursor search near calls" : 3,
                            "cursor update calls" : 0
                    },
                    "data-handle" : {
                            "connection dhandles swept" : 0,
                            "connection candidate referenced" : 0,
                            "connection sweeps" : 269,
                            "connection time-of-death sets" : 1,
                            "session dhandles swept" : 0,
                            "session sweep attempts" : 56
                    },
                    "log" : {
                            "log buffer size increases" : 0,
                            "total log buffer size" : 1048576,
                            "log bytes of payload data" : 8051,
                            "log bytes written" : 18560,
                            "yields waiting for previous log file close" : 0,
                            "total size of compressed records" : 5113,
                            "total in-memory size of compressed records" : 9524,
                            "log records too small to compress" : 91,
                            "log records not compressed" : 5,
                            "log records compressed" : 6,
                            "maximum log file size" : 104857600,
                            "pre-allocated log files prepared" : 1,
                            "number of pre-allocated log files to create" : 1,
                            "pre-allocated log files used" : 0,
                            "log read operations" : 0,
                            "log release advances write LSN" : 102,
                            "records processed by log scan" : 0,
                            "log scan records requiring two reads" : 0,
                            "log scan operations" : 0,
                            "consolidated slot closures" : 0,
                            "logging bytes consolidated" : 0,
                            "consolidated slot joins" : 0,
                            "consolidated slot join races" : 0,
                            "slots selected for switching that were unavailable" : 0,
                            "record size exceeded maximum" : 0,
                            "failed to find a slot large enough for record" : 0,
                            "consolidated slot join transitions" : 0,
                            "log sync operations" : 92,
                            "log sync_dir operations" : 1,
                            "log server thread advances write LSN" : 0,
                            "log write operations" : 102
                    },
                    "reconciliation" : {
                            "page reconciliation calls" : 16,
                            "page reconciliation calls for eviction" : 0,
                            "split bytes currently awaiting free" : 0,
                            "split objects currently awaiting free" : 0
                    },
                    "session" : {
                            "open cursor count" : 54,
                            "open session count" : 59
                    },
                    "thread-yield" : {
                            "page acquire busy blocked" : 0,
                            "page acquire eviction blocked" : 0,
                            "page acquire locked blocked" : 0,
                            "page acquire read blocked" : 0,
                            "page acquire time sleeping (usecs)" : 0
                    },
                    "transaction" : {
                            "transaction begins" : 50,
                            "transaction checkpoints" : 89,
                            "transaction checkpoint generation" : 89,
                            "transaction checkpoint currently running" : 0,
                            "transaction checkpoint max time (msecs)" : 16,
                            "transaction checkpoint min time (msecs)" : 0,
                            "transaction checkpoint most recent time (msecs)" : 0,
                            "transaction checkpoint total time (msecs)" : 52,
                            "transactions committed" : 3,
                            "transaction failures due to cache overflow" : 0,
                            "transaction range of IDs currently pinned by a checkpoint" : 0,
                            "transaction range of IDs currently pinned" : 1,
                            "transactions rolled back" : 46
                    },
                    "concurrentTransactions" : {
                            "write" : {
                                    "out" : 0,
                                    "available" : 128,
                                    "totalTickets" : 128
                            },
                            "read" : {
                                    "out" : 1,
                                    "available" : 127,
                                    "totalTickets" : 128
                            }
                    }
            },
            "writeBacksQueued" : false,
            "mem" : {
                    "bits" : 64,
                    "resident" : 37,
                    "virtual" : 339,
                    "supported" : true,
                    "mapped" : 0,
                    "mappedWithJournal" : 0
            },
            "metrics" : {
                    "commands" : {
                            "getLog" : {
                                    "failed" : NumberLong(0),
                                    "total" : NumberLong(2)
                            },
                            "isMaster" : {
                                    "failed" : NumberLong(0),
                                    "total" : NumberLong(10)
                            },
                            "listCollections" : {
                                    "failed" : NumberLong(0),
                                    "total" : NumberLong(1)
                            },
                            "listDatabases" : {
                                    "failed" : NumberLong(0),
                                    "total" : NumberLong(1)
                            },
                            "replSetGetStatus" : {
                                    "failed" : NumberLong(2),
                                    "total" : NumberLong(2)
                            },
                            "serverStatus" : {
                                    "failed" : NumberLong(0),
                                    "total" : NumberLong(1)
                            },
                            "whatsmyuri" : {
                                    "failed" : NumberLong(0),
                                    "total" : NumberLong(2)
                            }
                    },
                    "cursor" : {
                            "timedOut" : NumberLong(0),
                            "open" : {
                                    "noTimeout" : NumberLong(0),
                                    "pinned" : NumberLong(0),
                                    "total" : NumberLong(0)
                            }
                    },
                    "document" : {
                            "deleted" : NumberLong(0),
                            "inserted" : NumberLong(0),
                            "returned" : NumberLong(0),
                            "updated" : NumberLong(0)
                    },
                    "getLastError" : {
                            "wtime" : {
                                    "num" : 0,
                                    "totalMillis" : 0
                            },
                            "wtimeouts" : NumberLong(0)
                    },
                    "operation" : {
                            "fastmod" : NumberLong(0),
                            "idhack" : NumberLong(0),
                            "scanAndOrder" : NumberLong(0),
                            "writeConflicts" : NumberLong(0)
                    },
                    "queryExecutor" : {
                            "scanned" : NumberLong(0),
                            "scannedObjects" : NumberLong(0)
                    },
                    "record" : {
                            "moves" : NumberLong(0)
                    },
                    "repl" : {
                            "apply" : {
                                    "batches" : {
                                            "num" : 0,
                                            "totalMillis" : 0
                                    },
                                    "ops" : NumberLong(0)
                            },
                            "buffer" : {
                                    "count" : NumberLong(0),
                                    "maxSizeBytes" : 268435456,
                                    "sizeBytes" : NumberLong(0)
                            },
                            "network" : {
                                    "bytes" : NumberLong(0),
                                    "getmores" : {
                                            "num" : 0,
                                            "totalMillis" : 0
                                    },
                                    "ops" : NumberLong(0),
                                    "readersCreated" : NumberLong(0)
                            },
                            "preload" : {
                                    "docs" : {
                                            "num" : 0,
                                            "totalMillis" : 0
                                    },
                                    "indexes" : {
                                            "num" : 0,
                                            "totalMillis" : 0
                                    }
                            }
                    },
                    "storage" : {
                            "freelist" : {
                                    "search" : {
                                            "bucketExhausted" : NumberLong(0),
                                            "requests" : NumberLong(0),
                                            "scanned" : NumberLong(0)
                                    }
                            }
                    },
                    "ttl" : {
                            "deletedDocuments" : NumberLong(0),
                            "passes" : NumberLong(44)
                    }
            },
            "ok" : 1
    }
    >