HBase入门

分手后的思念是犯贱 2022-05-08 05:58 237阅读 0赞

overview：

*  1.  特点
    2.  安装
    3.  操作
 *  特点
    
    1.  非关系型数据库
    2.  存储k,v
    3.  扩展性强
    4.  在物理存储上是根据列簇来分割的
    5.  对事务支持性差
 *  安装
    
    1.  从官网上下载包
    2.  解压
    3.  角色分配如下：

Hdp01:  namenode  datanode  regionserver  hmaster  zookeeper

Hdp02:  datanode   regionserver  zookeeper

Hdp03:  datanode   regionserver  zookeeper

*  1.  配置文件
        
        1.  修改hbase-env.sh

<table> 
 <tbody> 
 <tr> 
 <td> export JAVA_HOME=/root/apps/jdk1.7.0_67 export HBASE_MANAGES_ZK=false </td> 
 </tr> 
 </tbody> 
</table>

*   *  1.  修改hbase-site.xml

<table> 
 <tbody> 
 <tr> 
 <td> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;configuration&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;!-- 指定hbase在HDFS上存储的路径 --&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;property&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;name&gt;hbase.rootdir&lt;/name&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;value&gt;hdfs://hdp01:9000/hbase&lt;/value&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;/property&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;!-- 指定hbase是分布式的 --&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&lt;property&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;name&gt;hbase.cluster.distributed&lt;/name&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;value&gt;true&lt;/value&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;/property&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;!-- 指定zk的地址，多个用“,”分割 --&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;property&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;name&gt;hbase.zookeeper.quorum&lt;/name&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;value&gt;hdp01:2181,hdp02:2181,hdp03:2181&lt;/value&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;/property&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;/configuration&gt; </td> 
 </tr> 
 </tbody> 
</table>

*   *  1.  修改 regionservers

安装完成：start-hbase.sh启动

客户端：bin/hbase shell

1.  客户端操作省。。。
2.  Api应用

首先创建连接：

<table> 
 <tbody> 
 <tr> 
 <td> Connection conn = null; @Before public void getConn() throws Exception{ // 构建一个连接对象 Configuration conf = HBaseConfiguration.create(); // 会自动加载hbase-site.xml conf.set("hbase.zookeeper.quorum", "hdp-01:2181,hdp-02:2181,hdp-03:2181"); conn = ConnectionFactory.createConnection(conf); } </td> 
 </tr> 
 </tbody> 
</table>

*  1.  DDL操作
        
        1.  创建表

<table> 
 <tbody> 
 <tr> 
 <td> // 从连接中构造一个DDL操作器 Admin admin = conn.getAdmin(); // 创建一个表定义描述对象 HTableDescriptor hTableDescriptor = new HTableDescriptor(TableName.valueOf("user_info")); // 创建列族定义描述对象 HColumnDescriptor hColumnDescriptor_1 = new HColumnDescriptor("base_info"); hColumnDescriptor_1.setMaxVersions(3); // 设置该列族中存储数据的最大版本数,默认是1 HColumnDescriptor hColumnDescriptor_2 = new HColumnDescriptor("extra_info"); // 将列族定义信息对象放入表定义对象中 hTableDescriptor.addFamily(hColumnDescriptor_1); hTableDescriptor.addFamily(hColumnDescriptor_2); // 用ddl操作器对象：admin 来建表 admin.createTable(hTableDescriptor); // 关闭连接 admin.close(); conn.close(); </td> 
 </tr> 
 </tbody> 
</table>

*   *  1.  删除表

<table> 
 <tbody> 
 <tr> 
 <td> Admin admin = conn.getAdmin(); // 停用表 admin.disableTable(TableName.valueOf("user_info")); // 删除表 admin.deleteTable(TableName.valueOf("user_info")); admin.close(); conn.close(); </td> 
 </tr> 
 </tbody> 
</table>

*   *  1.  修改表

<table> 
 <tbody> 
 <tr> 
 <td> Admin admin = conn.getAdmin(); // 取出旧的表定义信息 HTableDescriptor tableDescriptor = admin.getTableDescriptor(TableName.valueOf("user_info")); // 新构造一个列族定义 HColumnDescriptor hColumnDescriptor = new HColumnDescriptor("other_info"); hColumnDescriptor.setBloomFilterType(BloomType.ROWCOL); // 设置该列族的布隆过滤器类型 // 将列族定义添加到表定义对象中 tableDescriptor.addFamily(hColumnDescriptor); // 将修改过的表定义交给admin去提交 admin.modifyTable(TableName.valueOf("user_info"), tableDescriptor); admin.close(); conn.close(); </td> 
 </tr> 
 </tbody> 
</table>

DML操作

*  1.  增（改）删查

<table> 
 <tbody> 
 <tr> 
 <td> // 获取一个操作指定表的table对象,进行DML操作 Table table = conn.getTable(TableName.valueOf("user_info")); // 构造要插入的数据为一个Put类型(一个put对象只能对应一个rowkey)的对象 Put put = new Put(Bytes.toBytes("001")); put.addColumn(Bytes.toBytes("base_info"), Bytes.toBytes("username"), Bytes.toBytes("张三")); put.addColumn(Bytes.toBytes("base_info"), Bytes.toBytes("age"), Bytes.toBytes("18")); put.addColumn(Bytes.toBytes("extra_info"), Bytes.toBytes("addr"), Bytes.toBytes("北京")); Put put2 = new Put(Bytes.toBytes("002")); put2.addColumn(Bytes.toBytes("base_info"), Bytes.toBytes("username"), Bytes.toBytes("李四")); put2.addColumn(Bytes.toBytes("base_info"), Bytes.toBytes("age"), Bytes.toBytes("28")); put2.addColumn(Bytes.toBytes("extra_info"), Bytes.toBytes("addr"), Bytes.toBytes("上海")); ArrayList&lt;Put&gt; puts = new ArrayList&lt;&gt;(); puts.add(put); puts.add(put2); // 插进去 table.put(puts); table.close(); conn.close(); </td> 
 </tr> 
 <tr> 
 <td> Table table = conn.getTable(TableName.valueOf("user_info")); ArrayList&lt;Put&gt; puts = new ArrayList&lt;&gt;(); for(int i=0;i&lt;100000;i++){ Put put = new Put(Bytes.toBytes(""+i)); put.addColumn(Bytes.toBytes("base_info"), Bytes.toBytes("username"), Bytes.toBytes("张三"+i)); put.addColumn(Bytes.toBytes("base_info"), Bytes.toBytes("age"), Bytes.toBytes((18+i)+"")); put.addColumn(Bytes.toBytes("extra_info"), Bytes.toBytes("addr"), Bytes.toBytes("北京")); puts.add(put); } table.put(puts); </td> 
 </tr> 
 <tr> 
 <td> Table table = conn.getTable(TableName.valueOf("user_info")); // 构造一个对象封装要删除的数据信息 Delete delete1 = new Delete(Bytes.toBytes("001")); Delete delete2 = new Delete(Bytes.toBytes("002")); delete2.addColumn(Bytes.toBytes("extra_info"), Bytes.toBytes("addr")); ArrayList&lt;Delete&gt; dels = new ArrayList&lt;&gt;(); dels.add(delete1); dels.add(delete2); table.delete(dels); table.close(); conn.close(); </td> 
 </tr> 
 </tbody> 
</table>

<table> 
 <tbody> 
 <tr> 
 <td> Table table = conn.getTable(TableName.valueOf("user_info")); Get get = new Get("002".getBytes()); Result result = table.get(get); // 从结果中取用户指定的某个key的value byte[] value = result.getValue("base_info".getBytes(), "age".getBytes()); System.out.println(new String(value)); System.out.println("-------------------------"); // 遍历整行结果中的所有kv单元格 CellScanner cellScanner = result.cellScanner(); while(cellScanner.advance()){ Cell cell = cellScanner.current(); byte[] rowArray = cell.getRowArray(); //本kv所属的行键的字节数组 byte[] familyArray = cell.getFamilyArray(); //列族名的字节数组 byte[] qualifierArray = cell.getQualifierArray(); //列名的字节数据 byte[] valueArray = cell.getValueArray(); // value的字节数组 System.out.println("行键: "+new String(rowArray,cell.getRowOffset(),cell.getRowLength())); System.out.println("列族名: "+new String(familyArray,cell.getFamilyOffset(),cell.getFamilyLength())); System.out.println("列名: "+new String(qualifierArray,cell.getQualifierOffset(),cell.getQualifierLength())); System.out.println("value: "+new String(valueArray,cell.getValueOffset(),cell.getValueLength())); } table.close(); conn.close(); </td> 
 </tr> 
 <tr> 
 <td> Table table = conn.getTable(TableName.valueOf("user_info")); // 包含起始行键，不包含结束行键,但是如果真的想查询出末尾的那个行键，那么，可以在末尾行键上拼接一个不可见的字节（\000） Scan scan = new Scan("10".getBytes(), "10000\001".getBytes()); ResultScanner scanner = table.getScanner(scan); Iterator&lt;Result&gt; iterator = scanner.iterator(); while(iterator.hasNext()){ Result result = iterator.next(); // 遍历整行结果中的所有kv单元格 CellScanner cellScanner = result.cellScanner(); while(cellScanner.advance()){ Cell cell = cellScanner.current(); byte[] rowArray = cell.getRowArray(); //本kv所属的行键的字节数组 byte[] familyArray = cell.getFamilyArray(); //列族名的字节数组 byte[] qualifierArray = cell.getQualifierArray(); //列名的字节数据 byte[] valueArray = cell.getValueArray(); // value的字节数组 System.out.println("行键: "+new String(rowArray,cell.getRowOffset(),cell.getRowLength())); System.out.println("列族名: "+new String(familyArray,cell.getFamilyOffset(),cell.getFamilyLength())); System.out.println("列名: "+new String(qualifierArray,cell.getQualifierOffset(),cell.getQualifierLength())); System.out.println("value: "+new String(valueArray,cell.getValueOffset(),cell.getValueLength())); } System.out.println("----------------------"); } </td> 
 </tr> 
 </tbody> 
</table>