Apache Atlas 安装

拼搏现实的明天。 2022-09-04 11:56 173阅读 0赞

### Apache Atlas 安装 ###

*   *  安装Solr(单节点)
     *  kafka安装
     *  安装Maven并配置MAVEN\_HOME
     *  下载Atlas源码文件并编译(独立Hbase、Solr)
     *  配置Atlas
     *  配置Hive Hook

## 安装Solr(单节点) ##

1.  下载地址：[https://solr.apache.org/downloads.html][https_solr.apache.org_downloads.html]
2.  解压后直接启动
    
        bin/solr start -c -z <zookeeper_host:port>  -p 8983

## kafka安装 ##

[kafka安装连接][kafka]

## 安装Maven并配置MAVEN\_HOME ##

（配置maven镜像）

## 下载Atlas源码文件并编译(独立Hbase、Solr) ##

1.  [http://atlas.apache.org/\#/Downloads][http_atlas.apache.org_Downloads]
2.  解压
    
        tar -zxvf apache-atlas-2.1.0-sources.tar.gz
3.  进入Atlas文件夹编译
    
        # 配置maven jvm
        export MAVEN_OPTS="-Xms2g -Xmx2g"
        # 编译
        mvn clean -DskipTests install
        # 打包
        mvn clean -DskipTests package -Pdist
    
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L00yODM1OTIzMzg_size_16_color_FFFFFF_t_70]
4.  将Atlas文件夹cp至自定义文件夹下
    
        cp -r apache-atlas-sources-2.1.0/distro/target/apache-atlas-2.1.0-bin /usr/local/service/

## 配置Atlas ##

1.  配置atlas-env
    
        # 修改为false
        export MANAGE_EMBEDDED_CASSANDRA=false
        
        # 修改为false
        # indicates whether or not a local instance of Elasticsearch should be started for Atlas
        export MANAGE_LOCAL_ELASTICSEARCH=false
        
        # 添加hbase配置目录
        export HBASE_CONF_DIR=/usr/local/service/hbase/conf
2.  修改atlas-application.properties
    
        atlas.graph.storage.backend=hbase
        atlas.graph.storage.hbase.table=atlas
        
        atlas.graph.storage.hostname=namenode:2181
        atlas.graph.storage.hbase.regions-per-server=1
        atlas.graph.storage.lock.wait-time=10000
        
        #Solr
        #Solr cloud mode properties
        atlas.graph.index.search.solr.mode=cloud
        atlas.graph.index.search.solr.zookeeper-url=namenode:2181
        atlas.graph.index.search.solr.zookeeper-connect-timeout=60000
        atlas.graph.index.search.solr.zookeeper-session-timeout=60000
        atlas.graph.index.search.solr.wait-searcher=true
        
        #########  Notification Configs  #########
        atlas.notification.embedded=false
        atlas.kafka.data=${sys:atlas.home}/data/kafka
        atlas.kafka.zookeeper.connect=namenode:2181
        atlas.kafka.bootstrap.servers=namenode:9092
        atlas.kafka.zookeeper.session.timeout.ms=400
        atlas.kafka.zookeeper.connection.timeout.ms=200
        atlas.kafka.zookeeper.sync.time.ms=20
        atlas.kafka.auto.commit.interval.ms=1000
        atlas.kafka.hook.group.id=atlas
        
        atlas.kafka.enable.auto.commit=false
        atlas.kafka.auto.offset.reset=earliest
        atlas.kafka.session.timeout.ms=30000
        atlas.kafka.offsets.topic.replication.factor=1
        atlas.kafka.poll.timeout.ms=1000
        
        atlas.notification.create.topics=true
        atlas.notification.replicas=1
        atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
        atlas.notification.log.failed.messages=true
        atlas.notification.consumer.retry.interval=500
        atlas.notification.hook.retry.interval=1000
        
        #########  Entity Audit Configs  #########
        atlas.audit.hbase.tablename=apache_atlas_entity_audit
        atlas.audit.zookeeper.session.timeout.ms=5000
        atlas.audit.hbase.zookeeper.quorum=namenode:2181
3.  启动
    
        bin/atlas_start.py

## 配置Hive Hook ##

1.  修改hive-env
    
        export HIVE_AUX_JARS_PATH=/usr/local/service/apache-atlas-2.1.0/hook/hive
2.  修改hive-site.xml
    
        <property>
            <name>hive.exec.post.hooks</name>
            <value>org.apache.atlas.hive.hook.HiveHook</value>
        </property>
3.  修改Atlas的 atlas-application.properties
    
        # 新增
        ######## Hive Hook ########
        atlas.hook.hive.synchronous=true
        atlas.hook.hive.numRetries=3
        atlas.hook.hive.queueSize=10000
        atlas.cluster.name=atlas
        atlas.rest.address=http://namenode:21000
4.  将Atlas的 atlas-application.properties文件分发至所有节点HIVE\_HOME/conf
5.  重启Hive、Atlas（有Hue重启Hue）
6.  在atlas目录下执行倒入命令
    
        bin/import-hive.sh

[https_solr.apache.org_downloads.html]: https://solr.apache.org/downloads.html
[kafka]: https://blog.csdn.net/M283592338/article/details/97099540?ops_request_misc=%257B%2522request%255Fid%2522%253A%2522162884912816780261927812%2522%252C%2522scm%2522%253A%252220140713.130102334.pc%255Fblog.%2522%257D&request_id=162884912816780261927812&biz_id=0&utm_medium=distribute.pc_search_result.none-task-blog-2~blog~first_rank_v2~rank_v29-4-97099540.pc_v2_rank_blog_default&utm_term=kafka&spm=1018.2226.3001.4450
[http_atlas.apache.org_Downloads]: http://atlas.apache.org/#/Downloads
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L00yODM1OTIzMzg_size_16_color_FFFFFF_t_70]: /images/20220829/d20106d3384e46208953a8feaa4d9429.png