docker 应用 —— docker 搭建 hbase 单机环境

客官°小女子只卖身不卖艺 2024-04-19 10:44 68阅读 0赞

### 文章目录 ###

*   *  1、系统说明
     *  2、创建hbase容器
     *  3、进入hbase容器中
     *  4、进入 hbase shell 中：
     *  5、在win10 中 访问 hbase
     *  6、hbase java api

## 1、系统说明 ##

我的开发环境是在win10，在 VMware 中使用 centos7的虚拟机。  
下面的操作就是在 centos7 中使用 docker 搭建 hbase 单机模式，用于学习和开发使用 。

说明：  
centos7 ：主机名是 `hbase01`

> 如果让不熟悉大数据的开发人员来 使用zookeeper、hadoop、hbase等组件搭建环境，需要有多台服务器（或者虚拟机）、经历各种的坑，需要几天时间来做， 在实际工作中比较难实现！

## 2、创建hbase容器 ##

**命令：**

docker run -d -h hbase01    \
    -p 2181:2181 -p 8080:8080 -p 8085:8085 -p 9090:9090 -p 9095:9095    \
    -p 16000:16000 -p 16010:16010 -p 16201:16201 -p 16301:16301   \
    --name hbase6     \
    harisekhon/hbase:1.3

**参数说明：**  
`-d` 表示 后台运行  
`-h` 定义 容器主机 ，我的centos7 主机名是 `hbase01`  
`-p` 表示 端口映射  
`--name` 表示 容器别名，随机起名，要求惟一  
`harisekhon/hbase` 是 image镜像

**执行：**

[root@hbase01 ~]# docker run -d -h hbase01 -p 2181:2181 -p 8080:8080 -p 8085:8085 -p 9090:9090 -p 9095:9095 -p 16000:16000 -p 16010:16010 -p 16201:16201 -p 16301:16301 --name hbase6 harisekhon/hbase:1.3
    38dd85911fed3a45c16e23a40058117bf67b8bb98bcec6e01a6df6ec07af8597
    [root@hbase01 ~]# docker ps
    CONTAINER ID        IMAGE                  COMMAND                   CREATED             STATUS              PORTS                                                                                                                                                                                                                            NAMES
    38dd85911fed        harisekhon/hbase:1.3   "/bin/sh -c \"/entryp…"   7 seconds ago       Up 5 seconds        0.0.0.0:2181->2181/tcp, 0.0.0.0:8080->8080/tcp, 0.0.0.0:8085->8085/tcp, 0.0.0.0:9090->9090/tcp, 0.0.0.0:9095->9095/tcp, 0.0.0.0:16000->16000/tcp, 0.0.0.0:16010->16010/tcp, 0.0.0.0:16201->16201/tcp, 0.0.0.0:16301->16301/tcp   hbase6

## 3、进入hbase容器中 ##

[root@hbase01 ~]# docker exec -it 38dd85911fed /bin/bash
    bash-4.4#

## 4、进入 hbase shell 中： ##

bash-4.4# hbase shell
    hbase(main):001:0> list
    TABLE                                                                                                                                                                                     
    0 row(s) in 0.3140 seconds
    => []
    
    hbase(main):002:0> list_namespace
    list_namespace          list_namespace_tables
    
    hbase(main):002:0> list_namespace
    NAMESPACE                                                                                                                                                                                 
    default                                                                                                                                                                                   
    hbase                                                                                                                                                                                     
    2 row(s) in 0.0280 seconds

hbase shell 脚本的入门使用： https://blog.csdn.net/xiaojin21cen/article/details/101528834

## 5、在win10 中 访问 hbase ##

在win10中配置hosts中，增加映射：

176.16.1.181   hbase01

说明： 176.16.1.181 是虚拟机 centos7的IP

**访问 hbase ：**

http://hbase01:16010/master-status

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpYW9qaW4yMWNlbg_size_16_color_FFFFFF_t_70]

## 6、hbase java api ##

依赖：

<dependency> 
       <groupId>org.apache.hbase</groupId> 
        <artifactId>hbase-server</artifactId> 
        <version>1.3.1</version> 
    </dependency> 
    
    <dependency> 
        <groupId>org.apache.hbase</groupId> 
        <artifactId>hbase-client</artifactId> 
        <version>1.3.1</version> 
    </dependency>

下面代码配置中，除了`hbase01` 主机名不一致，可能需求修改外，其他的不变。

package hbase;
     
    import java.io.IOException;
    import java.util.Iterator;
    
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.Cell;
    import org.apache.hadoop.hbase.CellUtil;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.HColumnDescriptor;
    import org.apache.hadoop.hbase.HTableDescriptor;
    import org.apache.hadoop.hbase.KeyValue;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.Admin;
    import org.apache.hadoop.hbase.client.Connection;
    import org.apache.hadoop.hbase.client.ConnectionFactory;
    import org.apache.hadoop.hbase.client.Delete;
    import org.apache.hadoop.hbase.client.Get;
    import org.apache.hadoop.hbase.client.HTable;
    import org.apache.hadoop.hbase.client.Put;
    import org.apache.hadoop.hbase.client.Result;
    import org.apache.hadoop.hbase.client.ResultScanner;
    import org.apache.hadoop.hbase.client.Scan;
    import org.apache.hadoop.hbase.client.Table;
    import org.apache.hadoop.hbase.filter.BinaryComparator;
    import org.apache.hadoop.hbase.filter.CompareFilter;
    import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp;
    import org.apache.hadoop.hbase.filter.DependentColumnFilter;
    import org.apache.hadoop.hbase.filter.Filter;
    import org.apache.hadoop.hbase.filter.FilterList;
    import org.apache.hadoop.hbase.filter.RegexStringComparator;
    import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
    import org.apache.hadoop.hbase.filter.SubstringComparator;
    import org.apache.hadoop.hbase.filter.ValueFilter;
    import org.apache.hadoop.hbase.util.Bytes;
    import org.junit.Test;
     
    public class HbaseUtility_demo { 
     
    	public static Configuration conf=null;
    	public static Connection conn=null;
     
    	/** * 类级别的初始化，只是在类加载的时候做一次 配置zookeeper的端口2181 * 配置zookeeper的仲裁主机名centos，如果有多个机器，主机名间用冒号隔开 配置hbase master * 还有一种方式是new一个configuration对象，然后用addresource方法去添加xml配置文件 但是像这样显式的配置是会覆盖xml里的配置的 */
    	static { 
    		conf = HBaseConfiguration.create();
    		conf.set("hbase.zookeeper.quorum","hbase01");
            conf.set("hbase.zookeeper.property.clientPort","2181");
            conf.set("hbase.master", "hbase01:16000");
            conf.setInt("hbase.regionserver.port", 16201);
            conf.setInt("hbase.rpc.timeout",200);
            conf.setInt("hbase.client.operation.timeout",300);
            conf.setInt("hbase.client.scanner.timeout.period",200);
            
    		try { 
    			
    			conn = ConnectionFactory.createConnection(conf);
    			
    		} catch (IOException e) { 
    			e.printStackTrace();
    		}
    	}
     
    	/** * 建表，建列族 * * @param tablename, * @param ColumnFamilys * NamespaceDescriptor:维护命名空间的信息,但是namespace，一般用shell来建立 * Admin:提供了一个接口来管理 HBase 数据库的表信息 * HTableDescriptor:维护了表的名字及其对应表的列族,通过HTableDescriptor对象设置表的特性 * HColumnDescriptor:维护着关于列族的信息,可以通过HColumnDescriptor对象设置列族的特性 */
    	
    	@Test
    	public  void createtable() throws IOException { 
    		String tablename="ns2:student";
    		String[] ColumnFamilys=new String[]{ "data"};
    		
    		
    		Admin admin = conn.getAdmin();
    		HTableDescriptor table = new HTableDescriptor(TableName.valueOf(tablename));
    		for (String family : ColumnFamilys) { 
    			HColumnDescriptor columnfamily = new HColumnDescriptor(family);
    			table.addFamily(columnfamily);
    		}
     
    		if (admin.tableExists(TableName.valueOf(tablename))) { 
    			System.out.println("Table Exists");
    		} else { 
    			admin.createTable(table);
    			System.out.println("Table Created");
    			admin.close();
    		}
    	}
     
    	/** * 插入数据,当指定rowkey已经存在，则会覆盖掉之前的旧数据 * * @param tablename, * @param rowkey, * @param ColumnFamilys, * @param columns,@values * Table:用于与单个HBase表通信 Put:用来对单个行执行添加操作 */
    	@Test
    	public void insertdata() throws Exception { 
    		
    		String tablename="ns2:student";
    		String rowkey="1004";
    		String ColumnFamilys="data";
    		String[] columns=new String[]{ "name","age","adress"};
    		String[] values=new String[]{ "zhangsan_44","24","guangzhou"};		
    		
    		Table table = conn.getTable(TableName.valueOf(tablename));
    		Put put = new Put(Bytes.toBytes(rowkey));
    		for (int i = 0; i < columns.length; i++) { 
    			put.addColumn(Bytes.toBytes(ColumnFamilys), Bytes.toBytes(columns[i]), Bytes.toBytes(values[i]));
    		}
    		table.put(put);
    		System.out.println("data inserted");
    		table.close();
    	}
     
    	/** * 扫描全表 * * @param tablename */
    	@Test
    	public  void scantable() throws IOException { 
    		String tablename="ns2:student";
    		Scan scan = new Scan();
    		Table table = conn.getTable(TableName.valueOf(tablename));
    		ResultScanner rs = table.getScanner(scan);
    		for (Result result : rs) { 
    			for (Cell cell : result.listCells()) { 
    				System.out.println(Bytes.toString(cell.getRow()) + " " + "column=" + Bytes.toString(cell.getFamily())
    						+ ":" + Bytes.toString(cell.getQualifier())  + " , value="
    						+ Bytes.toString(cell.getValue()));
    			}
    			System.out.println("");
    		}
    		rs.close();
    	}
    	
    
    	/** * 根据rowkey对表进行scan * * @param tablename * @param rowkey * scan 'student',{ROWPREFIXFILTER => '1'} */
    	@Test
    	public void scanrow() throws IOException { 
    		String tablename="ns2:student";
    		String rowkey="1002";
    		
    		
    		Get get = new Get(Bytes.toBytes(rowkey));
    		Table table = conn.getTable(TableName.valueOf(tablename));
    		Result result = table.get(get);		
    		
    		for (Cell kv : result.listCells()) { 
    			System.out.println(
    					rowkey + " column=" + Bytes.toString(kv.getFamily()) + ":" + Bytes.toString(kv.getQualifier())
    					+ ","  + " , value=" + Bytes.toString(kv.getValue()));
    		}
    	}
     
    	
    	
    	/** * 根据rowkey删除整行的所有列族、所有行、所有版本 * * @param tablename * @param rowkey */
    	@Test
    	public void deleterow() throws IOException { 
    		String tablename="ns2:student";
    		String rowkey="1001";
    		
    		Table table = conn.getTable(TableName.valueOf(tablename));
    		Delete delete = new Delete(Bytes.toBytes(rowkey));
     
    		table.delete(delete);
    		table.close();
     
    		System.out.println("row : " + rowkey + " is deleted");
    	}
    	
    	
    	/** * 删除某个row的指定列 * * @param tablename * @param rowkey * @param columnfamily * @param column */
    	@Test
    	public  void deletecol()throws IOException { 
    		
    		String tablename="ns2:student";
    		String rowkey="1002";
    		String columnfamily="data";
    		String column="age";
    		
    		
    		Table table = conn.getTable(TableName.valueOf(tablename));
     
    		Delete delete = new Delete(Bytes.toBytes(rowkey));
    		delete.deleteColumn(Bytes.toBytes(columnfamily), Bytes.toBytes(column));
     
    		table.delete(delete);
    		table.close();
     
    		System.out.println("row : " + rowkey + " is deleted");
    	}
    	
    	
    	
    	/** * 使用过滤器，获取18-20岁之间的学生信息 * * @param tablename * @param age * @throws IOException */
    	@Test
    	public void scanfilterage() throws IOException { 
    		
    		String tablename="ns2:student";
    		String columnfamily="data";
    		String startage="23";
    		String endage="23";
    		
    		System.out.println("scanfilterage()");
    		
     
    		SingleColumnValueFilter ft1 = new SingleColumnValueFilter(//
    				Bytes.toBytes(columnfamily),//
    				Bytes.toBytes("age"), //
    				CompareFilter.CompareOp.GREATER, //
    				Bytes.toBytes(startage)//
    				);
    		
    		FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
    		filterList.addFilter(ft1);
     
    		Scan scan = new Scan();
    		scan.setFilter(filterList);
     
    		Table table = conn.getTable(TableName.valueOf(tablename));
    		ResultScanner rs = table.getScanner(scan);
    		for (Result r : rs) { 
    			for (Cell cell : r.listCells()) { 
    				System.out.println(Bytes.toString(cell.getRow()) +"\t column = "
    						+ Bytes.toString(cell.getFamily()) + ":" + Bytes.toString(cell.getQualifier()) + " value = "
    						+ Bytes.toString(cell.getValue()));
    			}
    			System.out.println("");
    		}
    		table.close();
    	}
    	
        
        
        @Test
        public void testComboFilter() throws IOException { 
    
            Connection conn = ConnectionFactory.createConnection(conf);
            TableName tname = TableName.valueOf("ns2:student");
            Scan scan = new Scan();
    
            //where ... f2:age <= 13
            SingleColumnValueFilter ftl = new SingleColumnValueFilter(
                    Bytes.toBytes("data"),
                    Bytes.toBytes("age"),
                    CompareFilter.CompareOp.EQUAL,
                   // new BinaryComparator(Bytes.toBytes("21"))
                    Bytes.toBytes("22")
            );
    
            FilterList ft = new FilterList(FilterList.Operator.MUST_PASS_ALL);
            ft.addFilter(ftl);
            scan.setFilter(ft);
            
            
            Table t = conn.getTable(tname);
            ResultScanner rs = t.getScanner(scan);
        	for (Result result : rs) { 
    			for (Cell cell : result.listCells()) { 
    				System.out.println(Bytes.toString(cell.getRow()) + " " + "column=" + Bytes.toString(cell.getFamily())
    						+ ":" + Bytes.toString(cell.getQualifier())  + " , value="
    						+ Bytes.toString(cell.getValue()));
    			}
    			System.out.println("");
    		}
        }
    
        /** * 测试ValueFilter(值过滤器) * 过滤value的值，含有指定的字符子串 */
        @Test
        public void testValueFilter() throws IOException { 
    
            Connection conn = ConnectionFactory.createConnection(conf);
            TableName tname = TableName.valueOf("ns2:student");
            Scan scan = new Scan();
            ValueFilter filter = new ValueFilter(CompareFilter.CompareOp.EQUAL, new SubstringComparator("22"));
            scan.setFilter(filter);
            Table t = conn.getTable(tname);
            ResultScanner rs = t.getScanner(scan);
        	for (Result result : rs) { 
    			for (Cell cell : result.listCells()) { 
    				System.out.println(Bytes.toString(cell.getRow()) + " " + "column=" + Bytes.toString(cell.getFamily())
    						+ ":" + Bytes.toString(cell.getQualifier())  + " , value="
    						+ Bytes.toString(cell.getValue()));
    			}
    			System.out.println("");
    		}
        }
    
        
        /** * DependentColumnFilter 依赖列过滤器 自定义一种过滤基数 */
        @Test
        public void testDepFilter() throws IOException { 
            Connection conn = ConnectionFactory.createConnection(conf);        
            
            TableName tname = TableName.valueOf("ns2:student");
            Scan scan = new Scan();
            DependentColumnFilter filter = new DependentColumnFilter(//
            		Bytes.toBytes("data"),//
                    Bytes.toBytes("name"),//
                    true,//
                    CompareFilter.CompareOp.NOT_EQUAL,//
                    new BinaryComparator(Bytes.toBytes("22"))//
                    );
            scan.setFilter(filter);
            Table t = conn.getTable(tname);
            ResultScanner rs = t.getScanner(scan);
            
            for (Result result : rs) { 
    			for (Cell cell : result.listCells()) { 
    				System.out.println(Bytes.toString(cell.getRow()) + " " + "column=" + Bytes.toString(cell.getFamily())
    						+ ":" + Bytes.toString(cell.getQualifier())  + " , value="
    						+ Bytes.toString(cell.getValue()));
    			}
    			System.out.println("");
    		}
        }
        
        
        @Test //扫描数据
        public void getRow(String tableName, String rowKey) throws IOException{  
    
        	TableName tname = TableName.valueOf("ns2:student");
        	 Scan scan = new Scan();
        	Table t = conn.getTable(tname);
        	ResultScanner rs = t.getScanner(scan);
        	 for (Result result : rs) { 
        		 for(Cell cell : result.rawCells()){ 
        			 System.out.println("行键:" + Bytes.toString(result.getRow()));
        			 System.out.println("列族" + Bytes.toString(CellUtil.cloneFamily(cell)));
        			 System.out.println("列:" + Bytes.toString(CellUtil.cloneQualifier(cell)));
        			 System.out.println("值:" + Bytes.toString(CellUtil.cloneValue(cell)));
        			 System.out.println("时间戳:" + cell.getTimestamp());
        		 }
        	 }
        	 }
     
    	public static void main(String[] args) throws IOException { 
    
    // createtable("student", "imformation", "score");
    // 
    // insertdata("student", "1", "imformation", col1, val1);
    // insertdata("student", "1", "imformation", col2, val2);
    // 
    // deleterow("student", "1");
    // deletecol("student", "1", "imformation", "chinese");
    // deleteversion("student", "1", "imformation", 1533482642629L);
    // deletefamily("student", "imformation");
    // droptable("student");
    // 
    // scantable("student");
    // scanrow("student", "1");
    // scanspecifycolumn("student", "1", "imformation", "chinese");
    // scanspecifytimestamp("student", "imformation", 1533482642629L);
    // scanallversion("student", "1");
    // scanfilterage("student", 18, 20);
     
    	}
    }

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpYW9qaW4yMWNlbg_size_16_color_FFFFFF_t_70]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/18/377bdcb4d9c140a889b7da0757f89ec0.png