mypipe部署

港控/mmm° 2022-09-27 04:00 97阅读 0赞

git clone [https://github.com/mardambey/mypipe.git][https_github.com_mardambey_mypipe.git]  
vi mypipe-runner/src/main/scala/mypipe/runner/PipeRunner.scala  
55行 //shutdown() 注释  
使用的主配置文件  
mypipe-runner/src/main/resources/application.conf ，修改kafka和zookper地址  
mypipe-runner/target/scala-2.11/classes/application.conf (编译后)

数据库 过滤表 配置文件  
mypipe-api/src/main/resources/reference.conf  
mypipe-api/target/scala-2.11/classes/reference.conf(编译后)  
quit-on-empty-mutation-commit-failure = false  
flush-interval-seconds = 1  
include-event-condition = “”” (db == “newtable” && table == “lip”) “”“

创建用于订阅数据库的账户：  
GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO ‘mypipe’@’%’ IDENTIFIED BY ‘mypipe’;  
GRANT ALL PRIVILEGES ON *.* TO ‘mypipe’@’%’;  
GRANT RELOAD ON *.* TO ‘mypipe’@’%’;

在用户目录下新增文件 用阿里云的源  
vi .sbt/repositories  
\[repositories\]  
local  
osc: [http://maven.aliyun.com/nexus/content/groups/public/][http_maven.aliyun.com_nexus_content_groups_public]  
typesafe: [http://repo.typesafe.com/typesafe/ivy-releases/][http_repo.typesafe.com_typesafe_ivy-releases], \[organization\]/\[module\]/(scala\_\[scalaVersion\]/)(sbt\_\[sbtVersion\]/)\[revision\]/\[type\]s/artifact.\[ext\], bootOnly  
sonatype-oss-releases  
maven-central  
sonatype-oss-snapshots

修改编译前文件后 编译  
./sbt clean package  
启动命令  
./sbt “project runner” “runMain mypipe.runner.PipeRunner”

kafka配置

kafka的配置文件编辑  
vi consumer.properties  
zookeeper.connect=172.18.0.1:2181  
vi server.properties  
zookeeper.connect=172.18.0.1:2181

启动 kafka  
bin/kafka-server-start.sh config/server.properties  
或者 bin/kafka-server-start.sh -daemon config/server.properties 后台启动

mypipe后台方法  
crtl + z  
jobs  
bg n

[https_github.com_mardambey_mypipe.git]: https://github.com/mardambey/mypipe.git
[http_maven.aliyun.com_nexus_content_groups_public]: http://maven.aliyun.com/nexus/content/groups/public/
[http_repo.typesafe.com_typesafe_ivy-releases]: http://repo.typesafe.com/typesafe/ivy-releases/