大数据从入门到实战 - HBase高级特性：过滤器（一）

叁歲伎倆 2023-01-06 05:28 254阅读 0赞

### 大数据从入门到实战 - HBase高级特性：过滤器（一） ###

*  一、关于此次实践
 *   *  1、实战简介
     *  2、全部任务
 *  二、实践详解
 *   *  1、第 1关：使用过滤器查询指定行的数据
     *  2、第 2关：使用正则表达式与子字符串匹配行键
     *  3、第 3关：列族过滤器、值过滤器、列名过滤器

***叮嘟！这里是小啊呜的学习课程资料整理。好记性不如烂笔头，今天也是努力进步的一天。一起加油进阶吧！***  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70_pic_center]

# 一、关于此次实践 #

## 1、实战简介 ##

HBase过滤器（filter）提供了非常强大的特性来帮助用户提高其处理表中数据的效率，用户不仅可以使用HBase中预定义好的过滤器，而且可以实现自定义的过滤器。

本次实训我们就来初步的了解过滤器的基本知识和开发流程。

## 2、全部任务 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70]

# 二、实践详解 #

## 1、第 1关：使用过滤器查询指定行的数据 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 1]

package step1;
    import java.io.IOException;
    import org.apache.hadoop.cli.util.*;
    import org.apache.hadoop.conf.*;
    import org.apache.hadoop.hbase.*;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.*;
    import org.apache.hadoop.hbase.util.*;
    public class Task { 
     public void query(String tName) throws Exception { 
     /********* Begin *********/
     Configuration config = new Configuration();
     Connection conn = ConnectionFactory.createConnection(config);
     TableName tableName = TableName.valueOf(tName);
     Table table = conn.getTable(tableName);
     Scan scan1 = new Scan();
     scan1.addColumn(Bytes.toBytes("basic_info"), Bytes.toBytes("gender"));
     Filter filter1 = new RowFilter(CompareOperator.EQUAL,
     new BinaryComparator(Bytes.toBytes("2018")));
     scan1.setFilter(filter1);
     ResultScanner scanner1 = table.getScanner(scan1);
     System.out.println("row:2018");
     for (Result result : scanner1) { 
     for(Cell cell : result.listCells()){ 
     System.out.println("basic_info:gender " + new String(CellUtil.cloneValue(cell),"utf-8") );
     }
     }
     scanner1.close();
     Scan scan2 = new Scan();
     scan2.addColumn(Bytes.toBytes("school_info"), Bytes.toBytes("college"));
     Filter filter2 = new RowFilter(CompareOperator.GREATER,
     new BinaryComparator(Bytes.toBytes("2018")));
     scan2.setFilter(filter2);
     ResultScanner scanner2 = table.getScanner(scan2);
     for (Result result : scanner2) { 
     System.out.println("row:" + new String(result.getRow(),"utf-8"));
     for(Cell cell : result.listCells()){ 
     System.out.println("school_info:college " + new String(CellUtil.cloneValue(cell),"utf-8") );
     }
     }
     scanner2.close();
     Scan scan3 = new Scan();
     scan3.addColumn(Bytes.toBytes("basic_info"), Bytes.toBytes("name"));
     Filter filter3 = new RowFilter(CompareOperator.LESS_OR_EQUAL,
     new BinaryComparator(Bytes.toBytes("2020")));
     scan3.setFilter(filter3);
     ResultScanner scanner3 = table.getScanner(scan3);
     for (Result result : scanner3) { 
     System.out.println("row:" + new String(result.getRow(),"utf-8"));
     for(Cell cell : result.listCells()){ 
     System.out.println("basic_info:name " + new String(CellUtil.cloneValue(cell),"utf-8") );
     }
     }
     scanner3.close();
     conn.close();
     /********* End *********/
     }
    }

**评测**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 2]

## 2、第 2关：使用正则表达式与子字符串匹配行键 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 3]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 4]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 5]

package step2;
    import java.io.IOException;
    import org.apache.hadoop.cli.util.*;
    import org.apache.hadoop.conf.*;
    import org.apache.hadoop.hbase.*;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.*;
    import org.apache.hadoop.hbase.filter.SubstringComparator;
    import org.apache.hadoop.hbase.util.*;
    public class Task { 
     public void query() throws Exception { 
     /********* Begin *********/
     Configuration config = new Configuration();
     Connection conn = ConnectionFactory.createConnection(config);
     TableName tableName = TableName.valueOf("t2_student_table");
     Table table = conn.getTable(tableName);
     Scan scan1 = new Scan();
     Filter filter1 = new RowFilter(CompareOperator.EQUAL,new RegexStringComparator("1.*9$")); //匹配任意开头，以2结尾。
     scan1.setFilter(filter1);
     ResultScanner scanner1 = table.getScanner(scan1);
     for (Result result : scanner1) { 
     System.out.println("row:" + new String(result.getRow(),"utf-8"));
     for(Cell cell : result.listCells()){ 
     String family = Bytes.toString(CellUtil.cloneFamily(cell));
     String qualifier = Bytes.toString(CellUtil.cloneQualifier(cell));
     String value = Bytes.toString(CellUtil.cloneValue(cell));
     System.out.println(family + ":" + qualifier + " " + value);
     }
     }
     scanner1.close();
     Scan scan2 = new Scan();
     Filter filter2 = new RowFilter(CompareOperator.EQUAL,new SubstringComparator("231"));
     scan2.setFilter(filter2);
     ResultScanner scanner2 = table.getScanner(scan2);
     for (Result result : scanner2) { 
     System.out.println("row:" + new String(result.getRow(),"utf-8"));
     for(Cell cell : result.listCells()){ 
     String family = Bytes.toString(CellUtil.cloneFamily(cell));
     String qualifier = Bytes.toString(CellUtil.cloneQualifier(cell));
     String value = Bytes.toString(CellUtil.cloneValue(cell));
     System.out.println(family + ":" + qualifier + " " + value);
     }
     }
     scanner2.close();
     conn.close();
     /********* End *********/
     }
    }

**评测**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 6]

## 3、第 3关：列族过滤器、值过滤器、列名过滤器 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 7]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 8]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 9]

package step3;
    import java.io.IOException;
    import org.apache.hadoop.cli.util.*;
    import org.apache.hadoop.conf.*;
    import org.apache.hadoop.hbase.*;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.*;
    import org.apache.hadoop.hbase.util.*;
    import org.apache.hadoop.hbase.filter.SubstringComparator;
    
    public class Task { 
     public void query() throws Exception { 
     /********* Begin *********/
     Configuration config = new Configuration();
     Connection conn = ConnectionFactory.createConnection(config);
     TableName tableName = TableName.valueOf(Bytes.toBytes("t3_student_table"));
     Table table = conn.getTable(tableName);
     Filter filter1 = new FamilyFilter(CompareOperator.EQUAL,
     new BinaryComparator(Bytes.toBytes("school_info")));
     Get get1 = new Get(Bytes.toBytes("1019"));
     get1.setFilter(filter1);
     Result result1 = table.get(get1);
     System.out.println("row:" + new String(result1.getRow(),"utf-8"));
     for(Cell cell : result1.listCells()){ 
     String family = Bytes.toString(CellUtil.cloneFamily(cell));
     String qualifier = Bytes.toString(CellUtil.cloneQualifier(cell));
     String value = Bytes.toString(CellUtil.cloneValue(cell));
     System.out.println(family + ":" + qualifier + " " + value);
     }
     Filter filter2 = new QualifierFilter(CompareOperator.EQUAL,
     new SubstringComparator("c"));
     Get get2 = new Get(Bytes.toBytes("2020"));
     get2.setFilter(filter2);
     Result result2 = table.get(get2);
     System.out.println("row:" + new String(result2.getRow(),"utf-8"));
     for(Cell cell : result2.listCells()){ 
     String family = Bytes.toString(CellUtil.cloneFamily(cell));
     String qualifier = Bytes.toString(CellUtil.cloneQualifier(cell));
     String value = Bytes.toString(CellUtil.cloneValue(cell));
     System.out.println(family + ":" + qualifier + " " + value);
     }
     
     Scan scan3 = new Scan();
     Filter filter3 = new ValueFilter(CompareOperator.EQUAL, new SubstringComparator("寮 "));
     scan3.setFilter(filter3);
     ResultScanner scanner3 = table.getScanner(scan3);
     for (Result result : scanner3) { 
     System.out.println("row:" + new String(result.getRow(),"utf-8"));
     for(Cell cell : result.listCells()){ 
     String family = Bytes.toString(CellUtil.cloneFamily(cell));
     String qualifier = Bytes.toString(CellUtil.cloneQualifier(cell));
     String value = Bytes.toString(CellUtil.cloneValue(cell));
     System.out.println(family + ":" + qualifier + " " + value);
     }
     }
     scanner3.close();
     conn.close();
     /********* End *********/
     }
    }

**评测**

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 10]

**Ending！**  
更多课程知识学习记录随后再来吧！

就酱，嘎啦！

![在这里插入图片描述][20200207152559955.png_pic_center]

注：  
人生在勤，不索何获。

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70_pic_center]: /images/20221119/460df7fd39cc47119975d2546bc6457c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70]: /images/20221119/75ad6dbf2be3400e8257a19ba1f235c8.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 1]: /images/20221119/eb8a818fa9614177a107958d75ed6f5b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 2]: /images/20221119/7f7c0dfcbc7647eb96441b14b87e415d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 3]: /images/20221119/c116285cb4e64a39afb3b207dda21a13.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 4]: /images/20221119/4a0e3cab53e844609d4dc676c2a26475.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 5]: /images/20221119/6e66b2ebac2643fe97fd934f1c03a268.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 6]: /images/20221119/920da9b960954d29b7b321441914a424.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 7]: /images/20221119/b2bb4c870d754cd7971e62cbe4ef6890.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 8]: /images/20221119/09e690a6d9fe4ae899049a4cd1d4ed94.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 9]: /images/20221119/e0dd00f8b95e4620912d7ec58cebf693.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNTQzNzg5_size_16_color_FFFFFF_t_70 10]: /images/20221119/8606756d3ad549c69020e9dc4ef0810d.png
[20200207152559955.png_pic_center]: /images/20221119/f8e95a349a634200baf889c9fefc5f99.png