HDFS缓存与DistributedCache

小咪咪 2022-06-14 00:27 127阅读 0赞

转载：http://dongxicheng.org/mapreduce-nextgen/hadoop-distributedcache-details/      http://www.tuicool.com/articles/zErmAzF

HDFS提供了一个高效的缓存加速机制—— Centralized Cache Management ，可以将一些经常被读取的文件（例如Hive中的fact表）pin到内存中。这些DataNode的缓存也是由NameNode所管理的（NameNode所管理的cache依然是以block形式，而DataNode也会定期向NameNode汇报缓存状态），而客户端可以高效得读取被缓存的数据块；为了能锁定内存，该实现依赖于JNI使用libhadoop.so，所以POSIX资源限制也要进行相应的设置（ulimit -l），并确保下面的参数被设置。

![1][]

## 参数设置 ##

`dfs.datanode.max.locked.memory`

该参数用于确定每个DataNode给缓存使用的最大内存量。设置这个参数和ulimit -l时，需要注意内存空间还需要一些内存用于做其他事情，比如，DataNode和应用程序JVM堆内存、以及操作系统的页缓存，以及计算框架的任务。所以不要使用太高的内存百分比。

下面是可选参数，可用于调优：

`dfs.namenode.path.based.cache.refresh.interval.ms`

NameNode会使用该参数作为两次子路径缓存重复扫描的时间间隔，单位为毫秒。使用该参数来计算要缓存的块，以及每个DataNode包含一个应该被缓存的块的副本。默认为300000ms（5分钟）

`dfs.datanode.fsdatasetcache.max.threads.per.volume`

DataNode会使用这个参数作为用于缓存新数据的每个卷的最大线程数。默认为4.

dfs.cachereport.intervalMsec

DataNode会使用该参数作为两次发送给NameNode的缓存状态报告的时间间隔。单位为毫秒。默认为10000ms(10秒）

`dfs.namenode.path.based.cache.block.map.allocation.percent`

分配给缓存块映射占Java堆内存的百分比。缓存块映射是一个使用hash链接的hash映射。太小的映射空间会导致缓存块太大时访问太过缓慢，而大的映射空间会消耗更多内存。默认为0.25(0.25%)

## Caching Concept ##

缓存指令（Cache Directive）：一个指令（directive）是一个缓存的path，它可以是目录（仅包含该目录下的文件，不包含子目录），也可以是文件；指令（Directive）有一个参数replication，表示该path被缓存的副本数；另外，可以对一个文件进行多次缓存。

缓存池(cache pool）是一个管理实体，拥有类UNIX的权限，可以限制哪个用户和组可以访问该缓存池。写权限允许用户向缓存池添加或从中删除缓存指令(directive) 。读权限允许用户列出缓存池内的缓存指令，还有其他元数据。执行权限不能被使用。缓存池也可以添加资源管理。它可以强加一个最大限制值，可以限制写入缓存池中的指令的字节数。通常，缓存池限制值之和约等于为HDFS在集群中做缓存而保留的总内存量。从缓存池中也可以获得许多统计信息，用于集群用户决定应该缓存什么。缓存池也可以强加一个TTL最大值(time to live)。缓存指令由一个唯一的无重复的64位整数ID来标识。即使缓存指令后来被删除了，ID也不会重复使用。而缓存池由一个唯一的字符串名称来标识。

## 缓存管理命令—hdfs cacheadmin ##

#### 1. 缓存命令 ####

(1). addDirective：添加指令

`hdfs cacheadmin -addDirective -path path -pool pool-name [-force] [-replication replication] [-ttl time-to-live]`

\-path 添加的路径

\-pool 加入的缓冲池名称

\-force 不检查缓存池的资源限制

\-replication 要使用的副本数，默认为1

\-ttl 缓存指令可以保持多长时间。可以按照分钟，小时，天来指定，如30m，4h，2d。有效单位为\[smhd\]。“never”表示永不过期的指令。如果未指定该值，那么，缓存指令就不会过期。

(2). removeDirective：通过id删除指令

`hdfs cacheadmin -removeDirective id`

(3). removeDirectives：删除执行path下所有缓存命令

`hdfs cacheadmin -removeDirective path`

(4). listDirectives：显示某一路径下的缓存信息

`hdfs cacheadmin -listDirectives [-stats] [-path path] [-pool pool]`

\-stats 显示 列出基于path的缓存指令统计信息。

2. 缓存池命令

(1). 添加/修改缓存池：addPool/modifyPool

`hdfs cacheadmin -addPool name [-owner owner] [-group group] [-mode mode] [-limit limit] [-maxTtl maxTtl]`  `hdfs cacheadmin -modifyPool name [-owner owner] [-group group] [-mode mode] [-limit limit] [-maxTtl maxTtl]`

其中，

\-owner/group是该pool的属主/组，默认为当前用户

\-mode是POSIX风格权限，默认为0755

\-limit为该pool中可以缓存的最大字节数，默认没有限制

\-maxTtl 最大的生存期，可以是120s, 30m, 4h, 2d等。

(2). 移除缓冲池：removePool

`hdfs cacheadmin -removePool name`

(3). 列出缓冲池：listPools

`hdfs cacheadmin -listPools [-stats] [name]`

\-stats为显示统计信息

#### 3. 命令实战 ####

当前参数设置：

![2][]

Shell

#创建缓存组，默认为cache_data
    hdfs cacheadmin -addPool cache_data -mode 0777
    Successfully added cache pool cache_data.
    #生成一个1GB大小的文件
    dd if=/dev/zero of=/tmp/test.zero bs=1M count=1024
    1024+0 records in
    1024+0 records out
    1073741824 bytes (1.1 GB) copied, 1.25545 s, 855 MB/s
    hdfs dfs -put /tmp/test.zero /data
    #生成缓存指令
    hdfs cacheadmin -addDirective -path /data -pool cache_data -ttl 1d
    Added cache directive 1
    #显示缓存池的信息
    hdfs cacheadmin -listPools -stats cache_data
    Found 1 result.
    NAME        OWNER  GROUP  MODE            LIMIT  MAXTTL  BYTES_NEEDED  BYTES_CACHED  BYTES_OVERLIMIT  FILES_NEEDED  FILES_CACHED
    cache_data  hdfs   hdfs   rwxrwxrwx   unlimited   never    1073741824    1073741824                0             1             1
    #统计信息，显示EXP Date
    hdfs cacheadmin -listDirectives -path /data
    
    Found 1 entry
     ID POOL         REPL EXPIRY                    PATH  
      1 cache_data      1 2014-12-22T15:38:31+0800  /data
    #删除缓存指令和缓存池
    hdfs cacheadmin -removeDirectives -path /data
    Removed cache directive 1
    Removed every cache directive with path /data
    hdfs cacheadmin -removePool cache_data
    Successfully removed cache pool cache_data.

官方文档： [http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html][http_hadoop.apache.org_docs_current_hadoop-project-dist_hadoop-hdfs_CentralizedCacheManagement.html]

网上有位大大的译文： [http://www.cnblogs.com/zhangningbo/p/4146398.html][http_www.cnblogs.com_zhangningbo_p_4146398.html]

性能对比和原理描述，可以参考 Hadoop Summit 2014的slide

[http://www.slideshare.net/Hadoop\_Summit/inmemory-caching-in-hdfs-lower-latency-same-great-taste-33921794][http_www.slideshare.net_Hadoop_Summit_inmemory-caching-in-hdfs-lower-latency-same-great-taste-33921794]

其中wordcount的对比:

![3][]

DistributedCache是Hadoop提供的文件缓存工具，它能够自动将指定的文件分发到各个节点上，缓存到本地，供用户程序读取使用。它具有以下几个特点：缓存的文件是只读的，修改这些文件内容没有意义；用户可以调整文件可见范围（比如只能用户自己使用，所有用户都可以使用等），进而防止重复拷贝现象；按需拷贝，文件是通过HDFS作为共享数据中心分发到各节点的，且只发给任务被调度到的节点。本文将介绍DistributedCache在Hadoop 1.0和2.0中的使用方法及实现原理。

Hadoop DistributedCache有以下几种典型的应用场景：1）分发字典文件，一些情况下Mapper或者Reducer需要用到一些外部字典，比如黑白名单、词表等；2）map-side join：当多表连接时，一种场景是一个表很大，一个表很小，小到足以加载到内存中，这时可以使用DistributedCache将小表分发到各个节点上，以供Mapper加载使用；3）自动化软件部署：有些情况下，MapReduce需依赖于特定版本的库，比如依赖于某个版本的PHP解释器，一种做法是让集群管理员把这个版本的PHP装到各个机器上，这通常比较麻烦，另一种方法是使用DistributedCache分发到各个节点上，程序运行完后，Hadoop自动将其删除。

Hadoop提供了两种DistributedCache使用方式，一种是通过API，在程序中设置文件路径，另外一种是通过命令行（-files，-archives或-libjars）参数告诉Hadoop，个人建议使用第二种方式，该方式可使用以下三个参数设置文件：

（1）-files：将指定的本地/hdfs文件分发到各个Task的工作目录下，不对文件进行任何处理；

（2）-archives：将指定文件分发到各个Task的工作目录下，并对名称后缀为“.jar”、“.zip”，“.tar.gz”、“.tgz”的文件自动解压，默认情况下，解压后的内容存放到工作目录下名称为解压前文件名的目录中，比如压缩包为dict.zip,则解压后内容存放到目录dict.zip中。为此，你可以给文件起个别名/软链接，比如dict.zip\#dict，这样，压缩包会被解压到目录dict中。

（3）-libjars：指定待分发的jar包，Hadoop将这些jar包分发到各个节点上后，会将其自动添加到任务的CLASSPATH环境变量中。

前面提到，DistributedCache分发的文件是有可见范围的，有的文件可以只对当前程序可见，程序运行完后，直接删除；有的文件只对当前用户可见（该用户所有程序都可以访问）；有的文件对所有用户可见。DistributedCache会为每种资源（文件）计算一个唯一ID，以识别每个资源，从而防止资源重复下载，举个例子，如果文件可见范围是所有用户，则在每个节点上，第一个使用该文件的用户负责缓存该文件，之后的用户直接使用即可，无需重复下载。那么，Hadoop是怎样区分文件可见范围的呢？

在Hadoop 1.0版本中，Hadoop是以HDFS文件的属性作为标识判断文件可见性的，需要注意的是，待缓存的文件即使是在Hadoop提交作业的客户端上，也会首先上传到HDFS的某一目录下，再分发到各个节点上的，因此，HDFS是缓存文件的必经之路。对于经常使用的文件或者字典，建议放到HDFS上，这样可以防止每次重复下载，做法如下：

比如将数据保存在HDFS的/dict/public目录下，并将/dict和/dict/public两层目录的可执行权限全部打开（在Hadoop中，可执行权限的含义与linux中的不同，该权限只对目录有意义，表示可以查看该目录中的子目录），这样，里面所有的资源（文件）便是所有用户可用的，并且第一个用到的应用程序会将之缓存到各个节点上，之后所有的应用程序无需重复下载，可以在提交作业时通过以下命令指定：

> \-files hdfs:///dict/public/blacklist.txt, hdfs:///dict/public/whilelist.txt

如果有多个HDFS集群可以指定namenode的对外rpc地址：

> \-files hdfs://host:port/dict/public/blacklist.txt, hdfs://host:port/dict/public/whilelist.txt

DistributedCache会将blacklist.txt和whilelist.txt两个文件缓存到各个节点的一个公共目录下，并在需要时，在任务的工作目录下建立一个指向这两个文件的软连接。

如果可执行权限没有打开，则默认只对该应用程序的拥有者可见，该用户所有应用程序可共享这些文件。

一旦你对/dict/public下的某个文件进行了修改，则下次有作业用到对应文件时，会发现文件被修改过了，进而自动重新缓存文件。

对于一些频繁使用的字典，不建议存放在客户端，每次通过-files指定，这样的文件，每次都要经历以下流程：上传到HDFS上—》缓存到各个节点上—》之后不再使用这些文件，直到被清除，也就是说，这样的文件，只会被这次运行的应用程序使用，如果再次运行同样的应用程序，即使文件没有被修改，也会重新经历以上流程，非常耗费时间，尤其是字典非常多，非常大时。

DistributedCache内置缓存置换算法，一旦缓存（文件数目达到一定上限或者文件总大小超过某一上限）满了之后，会踢除最久没有使用的文件。

在Hadopo 2.0中，自带的MapReduce框架仍支持1.0的这种DistributedCache使用方式，但DistributedCache本身是由YARN实现的，不再集成到MapReduce中。YARN还提供了很多相关编程接口供用户调用，有兴趣的可以阅读源代码。

下面介绍Hadoop 2.0中，DistributedCache通过命令行分发文件的基本使用方式：

（1）运行Hadoop自带的example例子， dict.txt会被缓存到各个Task的工作目录下，因此，直接像读取本地文件一样，在Mapper和Reducer中，读取dict.txt即可：

（2）Hadoop Streaming例子，需要通过-files指定mapper和reducer可执行文件或者脚本文件，这些文件就是通过DistributedCache分发到各个节点上的。

<table style="width:660px; margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important"> 
 <tbody style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important"> 
  <tr style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important"> 
   <td style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; color:rgb(175,175,175)!important"> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      1 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      2 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      3 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      4 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      5 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      6 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      7 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      8 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      9 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      10 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      11 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      12 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      13 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      14 
    </div> 
    <div style="margin:0px!important; padding:0px 0.5em 0px 1em!important; border-width:0px 3px 0px 0px!important; border-top-style:initial!important; border-right-style:solid!important; border-bottom-style:initial!important; border-left-style:initial!important; border-top-color:initial!important; border-right-color:rgb(108,226,108)!important; border-bottom-color:initial!important; border-left-color:initial!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; text-align:right!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
      15 
    </div> </td> 
   <td style="width:625px; margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important"> 
    <div style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:relative!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important"> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-weight:bold!important; font-size:1em!important; direction:ltr!important; display:inline!important; color:gray!important">#!/bin/bash</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">HADOOP_HOME=</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">/opt/yarn-client</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">INPUT_PATH=</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">/test/input/data</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">OUTPUT_PATH=</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">/test/output/data</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important; color:rgb(255,20,147)!important">echo</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important; color:blue!important"> "Clearing output path: $OUTPUT_PATH"</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">$HADOOP_HOME</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">/bin/hadoop</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important"> fs -rmr $OUTPUT_PATH</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important">
       &nbsp; 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">${HADOOP_HOME}</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">/bin/hadoop</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important"> jar\</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">&nbsp;&nbsp;&nbsp;</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">${HADOOP_HOME}</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">/share/hadoop/tools/lib/hadoop-streaming-2</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">.2.0.jar\</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">&nbsp;&nbsp;</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">-D mapred.reduce.tasks=2\</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">&nbsp;&nbsp;</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">-files mapper,reducer\</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">&nbsp;&nbsp;</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">-input $INPUT_PATH\</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">&nbsp;&nbsp;</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">-output $OUTPUT_PATH\</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">&nbsp;&nbsp;</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">-mapper mapper\</code> 
     </div> 
     <div style="margin:0px!important; padding:0px 1em!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-size:1em!important; direction:ltr!important; white-space:pre!important"> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">&nbsp;&nbsp;</code> 
      <code style="margin:0px!important; padding:0px!important; background:none!important; border:0px!important; bottom:auto!important; float:none!important; height:auto!important; left:auto!important; line-height:1.1em!important; outline:0px!important; overflow:visible!important; position:static!important; right:auto!important; top:auto!important; vertical-align:baseline!important; width:auto!important; font-family:Consolas,&quot;Bitstream Vera Sans Mono&quot;,&quot;Courier New&quot;,Courier,monospace!important; font-size:1em!important; direction:ltr!important; display:inline!important">-reducer reducer</code> 
     </div> 
    </div> </td> 
  </tr> 
 </tbody> 
</table>

（3）接下给出一个缓存压缩文件的例子，假设压缩文件为dict.zip，里面存的数据为：

通过-archives参数指定dict.zip后，该文件被解压后，将被缓存（实际上是软连接）到各个Task的工作目录下的dict.zip目录下，组织结构如下：

你可以在Mapper或Reducer程序中，使用类似下面的代码读取解压后的文件：

如果你想直接将内容解压到Task工作目录下，而不是子目录dict.zip中，可以用“-files”（注意，不要使用-archives，“-files”指定的文件不会被解压）指定dict.zip，并自己在程序中实现解压缩：

总之，Hadoop DistributedCache是一个非常好用的工具，合理的使用它能够解决很多非常困难的问题。

**hdfs缓存可以理解为hdfs一般的数据块datablock ，只不过是在缓存中存在，而DistributedCache是在运行mapreduce时用到的。**

[1]: http://img0.tuicool.com/I7jIVb.png!web
[2]: http://img2.tuicool.com/eQB3Ub.png!web
[http_hadoop.apache.org_docs_current_hadoop-project-dist_hadoop-hdfs_CentralizedCacheManagement.html]: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html
[http_www.cnblogs.com_zhangningbo_p_4146398.html]: http://www.cnblogs.com/zhangningbo/p/4146398.html
[http_www.slideshare.net_Hadoop_Summit_inmemory-caching-in-hdfs-lower-latency-same-great-taste-33921794]: http://www.slideshare.net/Hadoop_Summit/inmemory-caching-in-hdfs-lower-latency-same-great-taste-33921794
[3]: http://img1.tuicool.com/2Iv6bm.png!web