Using pbuilder and sbuild and debuild to build debian package (by quqi99)

我不是女神ヾ 2022-09-05 14:58 140阅读 0赞

作者：张华 发表于：2021-08-18
版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明

问题

在ussuri (16.3.0)里有一个bug, 但在steain(15.3.4)里没有，需要二分，先通过’git tag’找到其中的16.0.0, 16.1.0, 16.3.0三个tag

  1. 15.3.4
  2. 16.0.0 https://launchpad.net/ubuntu/+source/neutron/2:16.0.0-0ubuntu2
  3. 16.0.0.0b1
  4. 16.0.0.0rc1
  5. 16.0.0.0rc2
  6. 16.1.0 https://launchpad.net/ubuntu/+source/neutron/2:16.1.0-0ubuntu2
  7. 16.2.0 https://launchpad.net/ubuntu/+source/neutron/2:16.2.0-0ubuntu2
  8. 16.3.0

前一篇关于debian的文章是：　https://blog.csdn.net/quqi99/article/details/50511173

一些感受:

  • 在bionic中运行git-buildpackage的pristine-tar checkout这步会报一个XD3_INVALID_INPUT错误，在focal上无此问题
  • 在focal上运行git-buildpackage + git-pbuiler(cowpbuilder)时会报一个dbus相关的错误。运行git-buildpackage + sbuild或git-buildpackage + debuild都无此问题。直接运行pdebuild也无此问题(git-pbuilder用的是cowpbuilder是cow格式) .git-pbuilder创建出来的镜像里没有dbus包和环境变量（DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus），暂不清楚是否和这相关. 后来，使用git-pubilder时又提示输入什么密码。看来git-pbuilder确实有问题不能用.
  • git-buildpackage从git源码编译时都要用到tarball，但tarball有通过prinstine-tar命令人工准备和它从neutron git源码中自动tarball两种方式
  • 使用git-buildpackage + sbuild即可
  • 源码包看debian/changelog就能确实是适合在bionic上还是bionic-steain上还是focal上编译，这样能避免包依赖问题

准备测试机器

  1. juju deploy ubuntu debbuild --series=bionic --config hostname=debbuild --constraints "mem=8G cores=2 root-disk=80G"
  2. #  juju add-machine --series bionic
  3. sudo cp /etc/apt/sources.list /etc/apt/sources.list_bak
  4. cat <<EOF | sudo tee /etc/apt/sources.list
  5. deb http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse
  6. deb http://archive.ubuntu.com/ubuntu bionic-security main restricted universe multiverse
  7. deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse
  8. #deb http://archive.ubuntu.com/ubuntu bionic-proposed main restricted universe multiverse
  9. #deb http://archive.ubuntu.com/ubuntu bionic-backports main restricted universe multiverse
  10. deb-src http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse
  11. deb-src http://archive.ubuntu.com/ubuntu bionic-security main restricted universe multiverse
  12. deb-src http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse
  13. #deb-src http://archive.ubuntu.com/ubuntu bionic-proposed main restricted universe multiverse
  14. #deb-src http://archive.ubuntu.com/ubuntu bionic-backports main restricted universe multiverse
  15. EOF
  16. sudo apt clean && sudo apt update

使用pbuilder

会报一些包依赖的错（openstack-pkg-tools (>= 85ubuntu3~) ），见最后，后来证明和pbuilder无关，是该dsc文件就不是针对bionic的所以一堆问题。

  1. sudo apt install pbuilder debootstrap devscripts -y
  2. sudo apt install gnupg ubuntu-dev-tools bzr-builddeb apt-file debhelper dh-systemd openstack-pkg-tools -y
  3. sudo apt install build-essential quilt dh-autoreconf fakeroot dpkg-dev python-sphinx dh-make -y
  4. #sudo apt install apt-cacher-ng -y
  5. #echo 'Acquire::http::Proxy "http://127.0.0.1:3142";' | sudo tee /etc/apt/apt.conf.d/01acng
  6. # sometimes base.tgz can't be generated when using pbuilderrc
  7. sudo pbuilder --create --distribution bionic --architecture amd64 --debootstrapopts --variant=buildd
  8. sudo cp /var/cache/pbuilder/base.tgz /var/cache/pbuilder/bionic-train-base.tgz
  9. # need to use '--basetgz' when using no pbuilderrc
  10. sudo pbuilder update --basetgz /var/cache/pbuilder/bionic-train-base.tgz
  11. #chroot to image to ajust what you want
  12. sudo pbuilder login --save-after-login --basetgz /var/cache/pbuilder/bionic-train-base.tgz
  13.   echo 'precedence ::ffff:0:0/96 100' >> /etc/gai.conf
  14.   apt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -y
  15.   add-apt-repository cloud-archive:train
  16.   #deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main
  17.   #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EA
  18.   sed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.list
  19.   apt update
  20. # then exit chroot, press ctrl-d to save
  22. # but pdebuild needs pbuilderrc
  23. cat <<EOF | sudo tee /etc/pbuilderrc
  24. MIRRORSITE=http://archive.ubuntu.com/ubuntu/
  25. DEBFULLNAME='Zhang Hua'
  26. DEBEMAIL='xxx@xxx.com'
  27. DISTRIBUTION='bionic'
  28. EXTRAPACKAGES=''
  29. #EXTRAPACKAGES+=' vim sudo bash-completion wget git build-essential apt-transport-https ca-certificates ubuntu-cloud-keyring'
  30. # Cloud Archive
  31. #OS_RELEASE='train'
  32. if [ -n "\$OS_RELEASE" ]; then
  33.     BASETGZ="/var/cache/pbuilder/\$DISTRIBUTION-\$OS_RELEASE-base.tgz"
  34. else
  35.     BASETGZ="/var/cache/pbuilder/\$DISTRIBUTION-base.tgz"
  36. fi
  37. EOF
  38. mkdir -p /bak/work/debian/neutron/16.1.0 && cd /bak/work/debian/neutron/16.1.0
  39. dget https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/neutron/2:16.1.0-0ubuntu2/neutron_16.1.0-0ubuntu2.dsc
  40. dpkg-source -x neutron_16.1.0-0ubuntu2.dsc
  41. cd /bak/work/debian/neutron/16.1.0/neutron-16.1.0
  42. # install dependency, use mk-build-deps instead of 'apt build-dep neutron'
  43. add-apt-repository cloud-archive:train  #also add uca
  44. sudo apt install devscripts equivs -y
  45. sudo mk-build-deps --install debian/control
  46. # dpkg-source: error: aborting due to unexpected upstream changes
  47. rm -rf debian/source/fomat
  48. sudo pdebuild --debug
  49. #sudo pdebuild --debbuildopts -sa #when hitting 'Unable to find xxx.orig.tar.gz in upload or distribution'
  50. #DEB_BUILD_OPTIONS=nocheck debuild -us -uc

报这个错：

  1. ubuntu@debbuild:/bak/work/debian/neutron/16.1.0/neutron-16.1.0$ sudo pdebuild 
  2. W: /home/ubuntu/.pbuilderrc does not exist
  3. dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper-compat (= 12) openstack-pkg-tools (>= 85ubuntu3~) python3-pbr (>= 4.0.0)
  4. W: Unmet build-dependency in source
  5. dpkg-source: info: using options from neutron-16.1.0/debian/source/options: --extend-diff-ignore=^[^/]*[.]egg-info/
  6. pyversions: missing X(S)-Python-Version in control file, fall back to debian/pyversions
  7. pyversions: missing debian/pyversions file, fall back to supported versions
  8. py3versions: no X-Python3-Version in control file, using supported versions
  9. debian/rules:42: warning: overriding recipe for target 'override_dh_installinit'
  10. /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit'
  11. dh clean  --with python3 --buildsystem=pybuild
  12. dh: Compatibility levels before 5 are no longer supported (level 1 requested)
  13. debian/rules:17: recipe for target 'clean' failed
  14. make: *** [clean] Error 25

原因是运行pbuilder的机器上也得添加train uca并安装(pbuilder镜像里也记得添加train uca哦)，也别记了将pbuilderrc中的RELEASE放开来构建正确的BASETGZ变量：

  1. apt install debhelper-compat openstack-pkg-tools python3-pbr -y

但是接着会报下列错，这是debian包的问题，因为我们编译的neutron-16.1.0是位于train和ussuri之间的版本，并不是严格的train (train中的debain/control中的确实是python3-neutron-lib (>= 1.29.1),)

  1. pbuilder-satisfydepends-dummy : Depends: python3-neutron-lib (>= 2.2.0) but 1.29.1-0ubuntu1~cloud0 is to be installed
  2.                                  Depends: python3-ovsdbapp (>= 1.0.0) but 0.17.0-0ubuntu1~cloud0 is to be installed
  3.                                  Depends: python3-pyroute2 (>= 0.5.7) but 0.5.6-0ubuntu1~cloud0 is to be installed
  4.  python3-cryptography : Depends: libssl1.1 (>= 1.1.1) but 1.1.0g-2ubuntu4 is installed

后来又这样：

  1. sudo mv /etc/pbuilderrc /tmp/
  2. sudo pbuilder --create --distribution focal --architecture amd64 --debootstrapopts --variant=buildd
  3. sudo mv /tmp/pbuilderrc /etc/
  4. sudo cp /var/cache/pbuilder/base.tgz /var/cache/pbuilder/focal-base.tgz
  5. sudo pbuilder --login --basetgz /var/cache/pbuilder/focal-base.tgz --save-after-login
  6. #/etc/pbuilderrc will point out 'BASETGZ='
  7. gbp buildpackage --git-pbuilder --git-dist=focal --git-arch=amd64 --git-ignore-branch --git-ignore-new -uc -us -j8

结果又提示输出密码：

  1. [sudo] password for pbuilder:

看来gbp与pbuilder结合这一块确实有问题

使用sbuild

最后遇到错误”Not removing build depends: cloned chroot in use”

  1. #using sbuild - https://wiki.ubuntu.com/SecurityTeam/BuildEnvironment
  2. sudo apt install sbuild debhelper ubuntu-dev-tools piuparts moreutils schroot gnupg-agent -y
  3. sudo adduser $USER sbuild
  4. mkdir -p $HOME/ubuntu/scratch
  5. cat <<EOF | sudo tee -a  /etc/schroot/sbuild/fstab
  6. /home/$USER/ubuntu/scratch  /scratch          none  rw,bind  0  0
  7. EOF
  8. cat <<EOF | tee ~/.sbuildrc
  9. # Name to use as override in .changes files for the Maintainer: field
  10. # (mandatory, no default!).
  11. \$maintainer_name='Zhang Hua <xxx.zhang@xxx.com>';
  12. # Default distribution to build.
  13. \$distribution = "bionic";
  14. # Build arch-all by default.
  15. \$build_arch_all = 0;
  17. # When to purge the build directory afterwards; possible values are "never",
  18. # "successful", and "always".  "always" is the default. It can be helpful
  19. # to preserve failing builds for debugging purposes.  Switch these comments
  20. # if you want to preserve even successful builds, and then use
  21. # "schroot -e --all-sessions" to clean them up manually.
  22. \$purge_build_directory = 'successful';
  23. \$purge_session = 'successful';
  24. \$purge_build_deps = 'successful';
  25. # $purge_build_directory = 'never';
  26. # $purge_session = 'never';
  27. # $purge_build_deps = 'never';
  29. # Directory for writing build logs to
  30. \$log_dir=\$ENV{HOME}."/ubuntu/logs";
  32. # don't remove this, Perl needs it:
  33. 1;
  34. EOF
  35. mkdir -p $HOME/ubuntu/{build,logs}
  36. cat <<EOF | tee ~/.mk-sbuild.rc
  37. SCHROOT_CONF_SUFFIX="source-root-users=root,sbuild,admin
  38. source-root-groups=root,sbuild,admin
  39. preserve-environment=true"
  40. SKIP_UPDATES="1"
  41. SKIP_PROPOSED="1"
  42. EOF
  43. #mk-sbuild bionic --arch=amd64 --skip-updates --debootstrap-mirror=http://<mirror>/ubuntu --distro=ubuntu
  44. # sg sbuild  #do not need this line if you reenter ssh
  45. mk-sbuild bionic --arch=amd64 --skip-updates
  47. # install uca
  48. schroot -l
  49. schroot -c source:bionic-amd64 -u root
  50.   echo 'precedence ::ffff:0:0/96 100' >> /etc/gai.conf
  51.   apt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -y
  52.   add-apt-repository cloud-archive:train
  53.   #deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main
  54.   #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EA
  55.   apt update
  56.   chmod 1777 /tmp && apt update
  57. exit
  59.  To CHANGE the golden image: sudo schroot -c source:bionic-amd64 -u root
  60.  To ENTER an image snapshot: schroot -c bionic-amd64
  61.  To BUILD within a snapshot: sbuild -A -d bionic-amd64 PACKAGE*.dsc
  62.  To BUILD for : sbuild -A -d bionic-amd64 --host  PACKAGE*.dsc
  64. mkdir -p /bak/work/debian/neutron/16.1.0 && cd /bak/work/debian/neutron/16.1.0
  65. dget https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/neutron/2:16.1.0-0ubuntu2/neutron_16.1.0-0ubuntu2.dsc
  66. dpkg-source -x neutron_16.1.0-0ubuntu2.dsc
  67. sbuild-update -udc bionic-amd64 #it equals: apt update && apt upgrade
  68. cd /bak/work/debian/neutron/16.1.0/neutron-16.1.0
  69. sbuild --verbose -d bionic-amd64 ../neutron_*.dsc  #use .. to point out the parent directory
  70. #dsc: amd64 not in arch list or does not match any arch wildcards: all -- skipping
  71. # ../neutron_*.dsc is using arch=all so need to use --arch-all for sbuild
  72. sbuild --verbose -d bionic-amd64 --arch-all ../neutron_*.dsc
  73. # Not removing build depends: cloned chroot in use
  74. # E: Unable to correct problems, you have held broken packages.

镜像里添加了UCA后，报的错也和pbuilder是一样的，当然，这种错误和publider和sbuild本身都无关,所以sbuild也运行正确。

  1. sbuild-build-depends-neutron-dummy : Depends: python3-neutron-lib (>= 2.2.0) but 1.29.1-0ubuntu1~cloud0 is to be installed
  2.                                       Depends: python3-ovsdbapp (>= 1.0.0) but 0.17.0-0ubuntu1~cloud0 is to be installed
  3.                                       Depends: python3-pyroute2 (>= 0.5.7) but 0.5.6-0ubuntu1~cloud0 is to be installed

注意，要改为使用focal的话，或者修改默认值（sed -i -e “s/bionic/focal/g” ~/.sbuildrc），或者使用（’–git-builder=sbuild -As -d focal-amd64’）

  1. gbp buildpackage --git-debian-branch=debian/2%16.1.0-0ubuntu2 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As -d focal-amd64' --git-dist=focal --git-arch=amd64 -j8

使用debuild

先使用sbuild快速弄一个bionic环境出来：

  1. sudo cp /etc/schroot/chroot.d/sbuild-bionic-amd64 /etc/schroot/chroot.d/sbuild-bionic-amd64-tmp
  2. sudo cp -r /var/lib/schroot/chroots/bionic-amd64 /var/lib/schroot/chroots/bionic-amd64-tmp
  3. sudo sed -i -e "s/bionic-amd64/bionic-amd64-tmp/g" /etc/schroot/chroot.d/sbuild-bionic-amd64-tmp
  4. #sudo rm /etc/schroot/chroot.d/sbuild-bionic-amd64-tmp
  5. #sudo rm -rf /var/lib/schroot/chroots/bionic-amd64-tmp
  6. schroot -l
  7. schroot -c source:bionic-amd64-tmp -u root

再试debuild

  1. apt install devscripts curl ca-certificates bash-completion -y
  2. dget https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/neutron/2:16.0.0-0ubuntu2/neutron_16.0.0-0ubuntu2.dsc
  3. dpkg-source -x neutron_16.0.0-0ubuntu2.dsc
  4. cd neutron-16.0.0/
  5. # install dependency, use mk-build-deps instead of 'apt build-dep neutron'
  6. apt install devscripts equivs -y
  7. mk-build-deps --install debian/control
  8. apt install debhelper-compat openstack-pkg-tools python3-pbr -y
  9. # install some dependency according to the output of 'debuild -i -us -uc -S'
  10. apt install debhelper-compat dh-python openstack-pkg-tools python3-all python3-pbr -y
  11. #build unsigned source package
  12. debuild -i -us -uc -S

debuild的分解动作如下：

  • 清理源代码树(debian/rules clean)
  • 构建源代码包(dpkg-source -b)
  • 构建程序(debian/rules build)
  • 构建二进制包(fakeroot debian/rules binary)
  • 使用 gpg 签署 .dsc 文件 使用
  • dpkg-genchanges 和 gpg 创建并签署上传用的.changes 文件

报这个错：

  1. dpkg-buildpackage: info: binary and diff upload (original source NOT included)

那就加-sa参数。

  1. debuild -i -us -uc -S -sa

上面测16.0.1因为源码有问题不典型，让我们改测train(15.3.4)来只证明debuild可以即可(注意：下列参数-S也改成了-b):

  1. apt-get source neutron
  2. mk-build-deps --install debian/control
  3. debuild -i -us -uc -b

通过自己从头构建熟悉git-buildpackage原理

https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.upstream-git.html
git-buildpackage的原理是：

  • 准备git工程. 注意：这里初始添加了README.md之后会报：will cause dpkg-source: error: aborting due to unexpected upstream changes，但初始不加，在git branch时看不到master与upstream分支，还是先添加，呆会再删除

    mkdir neutron-git-buildpackage && cd neutron-git-buildpackage
    echo “# neutron-git-buildpackage” >> README.md
    git init
    git add README.md
    git commit -m “first commit”

    git@github.com:zhhuabj/neutron-git-buildpackage.git" class="reference-link">git remote add origin git@github.com:zhhuabj/neutron-git-buildpackage.git

    git push origin master

  • 会将neutron source code (not debian/ directory)提交到upstream branch，故先创建它：

    git checkout -b upstream

  • 回到master分支(此时好像不能创建新分支，是都测试好之后再创建分支）并准备neutron source code, ‘gbp import-dsc’会自动将neutron source code下载下来，并提交到upstream branch

    git checkout master

    gbp import-orig /tmp/tarball/new-upstream.tar.gz -u 15.3.4

    gbp import-dsc https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc

  • 也在master分支准备debianized repository, 这时master中同时有upstream中的neutron源码和debian目录（Imported the debian/ directory into master as well as upstream’s files）

    git checkout master
    wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    git add *
    git commit -m ‘init master’

  • 此时状态是：

    ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ git branch

    • master
      upstream
      ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ ls
      AUTHORS LICENSE api-ref debian neutron releasenotes setup.py
      CONTRIBUTING.rst PKG-INFO babel.cfg devstack neutron.egg-info requirements.txt test-requirements.txt
      ChangeLog README.rst bin doc playbooks roles tools
      HACKING.rst TESTING.rst bindep.txt etc rally-jobs setup.cfg tox.ini
      ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ git checkout upstream
      Switched to branch ‘upstream’
      ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ ls
      AUTHORS HACKING.rst README.rst babel.cfg devstack neutron rally-jobs roles test-requirements.txt
      CONTRIBUTING.rst LICENSE TESTING.rst bin doc neutron.egg-info releasenotes setup.cfg tools
      ChangeLog PKG-INFO api-ref bindep.txt etc playbooks requirements.txt setup.py tox.ini
      ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ git tag
      debian/2%15.3.4-0ubuntu1_cloud1
      upstream/15.3.4

  • pristine-tar分支中准备tarball

    git checkout -b pristine-tar
    wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4.orig.tar.gz
    chmod +x neutron_15.3.4.orig.tar.gz
    git add neutron_15.3.4.orig.tar.gz
    git commit -m ‘neutron_15.3.4.orig.tar.gz’

    pristine-tar: committed neutron_15.3.4.orig.tar.gz.delta to branch pristine-tar

    pristine-tar commit ./neutron_15.3.4.orig.tar.gz 15.3.4
    pristine-tar list |grep 15

    Remember to tell git to push both the pristine-tar branch, and your tag:

    git push —all —tags

    once your local tarball has lost - http://manpages.ubuntu.com/manpages/trusty/man1/pristine-tar.1.html

    pristine-tar checkout ../hello-1.0.tar.gz

或者直接从上游git中获取

  1. # generate tarball from upstream
  2. cd /bak/openstack/neutron
  3. git checkout master
  4. git checkout -b 15.3.4 15.3.4
  5. git archive --output=../neutron_15.3.4.tar.gz --format=tar HEAD
  7. # commit tarball into this git repo
  8. pristine-tar commit /bak/openstack/neutron_15.3.4.orig.tar.gz 15.3.4

  • 之前就可以回master分支，安装依赖并运行gbp buildpackage了，它会调用’dpkg-buildpackage -rfakeroot -us -uc -ui -i -I’。都测试完之后，就可以将master分支打tag了

    git checkout master

    also need to enable UCA

    sudo mk-build-deps —install debian/control #install dependency
    gbp buildpackage —git-ignore-branch —git-ignore-new

总结，这个重点，即要安装UCA，也要安装依赖，git有改还得添加–git-ignore-new来忽略改动。
但是它会报这个错：#neutron_15.3.4.orig.tar.gz does not match stored hash，用下列workaround就OK啦：

  1. #neutron_15.3.4.orig.tar.gz does not match stored hash
  2. pristine-tar checkout ../neutron_15.3.4.orig.tar.gz
  3. gbp buildpackage --git-tarball-dir=/bak/work --git-ignore-branch --git-ignore-new

解决运行gbp import-dsc过程中遇到的问题

https://wiki.debian.org/PackagingWithGit
得将neutron source code 导入到debianized repository中去，
1, 先看通过”gbp import-dsc”从dsc中导入neutron source code
可以直接从neutron的上游下载source code:

  1. https://releases.openstack.org/train/#train-neutron
  2. https://tarballs.openstack.org/neutron/neutron-15.3.4.tar.gz

也可以用’gbp import-dsc’从dsc中导入neutron source code

  1. sudo apt install git-buildpackage -y
  2. mkdir -p /tmp/deb
  3. https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/
  4. wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc
  5. sudo chown -R $USER ~/.gnupg/
  6. # bakup gpg keys
  7. gpg --output mygpgkey_pub.gpg --armor --export $GPGKEY
  8. gpg --output mygpgkey_sec.gpg --armor --export-secret-key $GPGKEY
  9. # import gpg keys from other machine
  10. sudo apt-get install rng-tools gnupg-agent -y && sudo rngd -r /dev/urandom && killall -q gpg-agent && eval $(gpg-agent --daemon)
  11. gpg --import mygpgkey_pub.gpg
  12. gpg --allow-secret-key-import --import mygpgkey_sec.gpg
  13. gpg --list-public
  14. gpg --list-sig
  15. export GPGKEY=C7100A4CD0D8F3AE44212746E6A84FFFCF31A67F
  16. gpg --send-keys --keyserver keyserver.ubuntu.com $GPGKEY 
  18. # import dsc. the dsc provider signs dsc, and dsc user verifies dsc
  19. gpg --verify ./neutron_15.3.4-0ubuntu1~cloud1.dsc
  20. gpg --search-keys 3516D1A5BF0077E71414E19715B9B3EE0DCDF806
  21. gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3516D1A5BF0077E71414E19715B9B3EE0DCDF806
  22. # you can ignore 'There is no indication that the signature belongs to the owner'
  23. # as it simply means you have not established a web of trust with other GPG users
  24. gpg --verify ./neutron_15.3.4-0ubuntu1~cloud1.dsc
  25. rm -rf ./neutron_15.3.4-0ubuntu1~cloud1.dsc
  26. git config --global user.email "xxx.zhang@xxx.com"
  27. git config --global user.name "Zhang Hua"
  28. ＃　Then export the key to your local trustedkeys to make it trusted:
  29. gpg --no-default-keyring -a --export 3516D1A5BF0077E71414E19715B9B3EE0DCDF806 | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -
  30. # but it still says: gpg: Can't check signature: No public key
  31. gbp import-dsc https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc

尽管已经导入key, 但上面使用’gbp import-dsc’仍然报“Can’t check signature: No public key”。是因为还要将dsc provider的公钥 export the key to your local trustedkeys to make it trusted

  1. ＃　Then export the key to your local trustedkeys to make it trusted:
  2. gpg --no-default-keyring -a --export 3516D1A5BF0077E71414E19715B9B3EE0DCDF806 | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -

如果还是不行，可以使用下列dget命令作为workaround，dget即使没有证书也会下载文件：

  1. sudo apt-get install debian-keyring　-y
  2. dget --download-only -q https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc
  3. dpkg-source -x neutron_15.3.4-0ubuntu1~cloud1.dsc
  4. gpg --keyserver keyserver.ubuntu.com --recv-keys 3516D1A5BF0077E71414E19715B9B3EE0DCDF806
  5. ＃　Then export the key to your local trustedkeys to make it trusted:
  6. gpg --no-default-keyring -a --export 3516D1A5BF0077E71414E19715B9B3EE0DCDF806 | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -

使用ubuntu-openstack-dev debian源码构建

上面熟悉了从零构建git-buildpackage工程，实际上ubuntu-openstack-dev中已经有了这个工程了。

  • master分支应该同时有debian目录和neutron源码
  • upstream分支只有neutron源码，没有debain目录
  • pristine-tar分支只有tarball，并且使用pristine-tar list命令能看到，也要能运行pristine-tar checkout …/xxx.tar.gz
  • 编译时是在master分支运行

  • 暂不清楚如果编译15.3.4分支，因为15.3.4下没有debain目录，有debian目录的是这个：

    ubuntu@debbuild:/bak/work/debsource/neutron$ git tag |grep debian |grep 15.3.4
    debian/2%15.3.4-0ubuntu1_cloud0
    debian/2%15.3.4-0ubuntu1_cloud1

git-buildpackage命令运行在neutron代码分支之后，会先使用’pristine-tar checkout …/xxx.orig.tar.gz’先tarball弄到neutron代码分支（tarball存在pristine-tar分支中），然后再build.

  1. # https://wiki.ubuntu.com/OpenStack/CorePackages
  2. sudo apt install git-buildpackage -y
  3. sudo mkdir -p /bak/work/debsource && sudo chown -R $USER /bak && cd /bak/work/debsource
  4. git clone https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron
  5. cd neutron
  6. git checkout pristine-tar
  7. git checkout upstream
  8. git checkout master
  10. # 仅编译15.3.4, 这个tag (debian/2%15.3.4-0ubuntu1_cloud0)有debian目录，应该用它
  11. git checkout -b debian/2%15.3.4-0ubuntu1_cloud0 debian/2%15.3.4-0ubuntu1_cloud0
  12. git checkout debian/2%15.3.4-0ubuntu1_cloud0
  13. gbp buildpackage  --git-ignore-branch --git-ignore-new -us -uc

但是上面命令报下面这个错：

  1. xdelta3: target window checksum mismatch: XD3_INVALID_INPUT

所以本想用下列workaround (–git-tarball-dir=/bak/openstack/tarball)来避开上面问题，但是又报不匹配的错

  1. # generate tarball from upstream
  2. cd /bak/openstack/neutron
  3. git checkout master
  4. git checkout -b 15.3.4 15.3.4
  5. git archive --output=/bak/openstack/tarball/neutron_15.3.4.orig.tar.gz --format=tar HEAD
  7. gbp buildpackage --git-tarball-dir=/bak/openstack/tarball --git-ignore-branch --git-ignore-new -us -uc

看来还是得想办法解决XD3_INVALID_INPUT的问题。得想办法运行下列命令成功：

  1. pristine-tar checkout ../neutron_15.3.4.orig.tar.gz

但这个命令在focal上是能运行成功的，可能是bionic的pristine-tar版本有问题吧，可以在focal导入tarball后再拷到bionic中，再使用’–git-tarball-dir=’来workaroud.
或者我们可以直接在focal中使用pbuilder + git-buildpackage 来规避这个问题。

使用pbuilder + ubuntu-openstack-dev debian源码构建

第一步，创建pbuilder环境。这里的pbuilder实际上是cowpbuilder, 镜像格式是cow格式。

  1. sudo apt-get install git-buildpackage cowbuilder debian-archive-keyring -y
  2. #it will delete /var/cache/pbuilder/aptcache
  3. #sudo pbuilder clean
  5. #sudo pbuilder --create --distribution bionic --architecture amd64 --debootstrapopts --variant=buildd
  6. rm -rf sudo rm -rf /var/cache/pbuilder/base-bionic-amd64.cow
  7. sudo mv /etc/pbuilderrc /tmp/
  8. DIST=bionic ARCH=amd64 git-pbuilder create
  9. sudo mv /tmp/pbuilderrc /etc/
  11. #sudo pbuilder update --basetgz /var/cache/pbuilder/bionic-train-base.tgz
  12. DIST=bionic ARCH=amd64 git-pbuilder update
  13. #sudo pbuilder login --save-after-login --basetgz /var/cache/pbuilder/bionic-train-base.tgz
  14. DIST=bionic ARCH=amd64 git-pbuilder login --save-after-login
  15.   echo 'precedence ::ffff:0:0/96 100' >> /etc/gai.conf
  16.   apt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -y
  17.   add-apt-repository cloud-archive:train
  18.   #deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main
  19.   #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EA
  20.   sed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.list
  21.   apt update
  22.   exit

第二步，使用pubilder (在gbp buuildpackage中添加–git-pbuilder —git-dist=bionic —git-arch=amd64）编译

  1. sudo apt install devscripts equivs -y
  2. sudo mkdir -p /bak/work/debsource && sudo chown -R $USER /bak && cd /bak/work/debsource
  3. git clone https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron
  4. cd neutron
  5. git checkout pristine-tar
  6. git checkout upstream
  7. git checkout master
  8. git checkout -b debian/2%15.3.4-0ubuntu1_cloud0 debian/2%15.3.4-0ubuntu1_cloud0
  9. git checkout debian/2%15.3.4-0ubuntu1_cloud0
  11. #sudo mk-build-deps --install debian/control
  12. #gbp buildpackage  --git-ignore-branch --git-ignore-new -us -uc
  13. gbp buildpackage --git-pbuilder --git-dist=bionic --git-arch=amd64 --git-ignore-branch --git-ignore-new -uc -us -j16
  14. #debuild clean

在neutron源码中自动导入tarball构建

下面直接从neutron源码中使用gbp来编译:
https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.upstream-git.html
1, 首先，neutron source code必须从orgin更名为upstream避免和debian repository混淆

  1. git clone --no-checkout -o upstream https://github.com/openstack/neutron

2, 将15.3.4的源码切换到debian/15.3.4分支，并且将debian目录加入到debian分支

  1. cd neutron
  2. git checkout -b debian/15.3.4 15.3.4
  3. wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  4. tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  5. rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  6. git add debian/
  7. git commit -m 'add debian'
  8. #gbp:error: Couldn't commit to 'pristine-tar' with upstream '15.3.4': pristine-tar: more than one ref matches "15.3.4":

3, 因为debian/15.3.4分支有debian目录，应该是在这个分支下运行编译命令，这们这里没有指定tarball包因为’–git-pristine-tar —git-pristine-tar-commit ‘会自动从源码中准备tarball

  1. gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 -us -uc

若更新了源文件的话, ‘gbp dch’会在changelog中添加一记录（15.3.4-0ubuntu1cloud1ubuntu11.gbp5d92a7），’–git-ignore-new’用于告诉　‘gbp buildpackage’工作目录不干净。

  1. git fetch upstream
  2. git merge 15.3.4
  3. # gbp:info: Changelog 2:15.3.4-0ubuntu1~cloud1ubuntu1~1.gbp5d92a7 (snapshot #1) prepared up to 5d92a7b
  4. gbp dch --debian-branch=debian/15.3.4 --snapshot --auto debian/
  5. gbp buildpackage --git-ignore-new --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' -us -uc

测试没问题之后，修改changelog和build一个新版本

  1. gbp dch --release --auto --debian-branch=debian/15.3.4
  2. git commit -m"Release 1.1-1" debian/changelog
  3. gbp buildpackage --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 -us -uc

再总结 - 通过自己从头构建熟悉git-buildpackage原理

这个是对”通过自己从头构建熟悉git-buildpackage原理“这一节的总结，这里还是要自己准备tarball麻烦，最好参考”通过neutron git来使用gbp“这一节由git来自动处理tarball. 还最好考虑git-buildpackage怎么与pbuilder结合(git-pbuilder)。
1, 准备git工程, 注意：这里初始添加了README.md之后会报：will cause dpkg-source: error: aborting due to unexpected upstream changes，但初始不加，在git branch时看不到master与upstream分支，还是先添加，呆会再删除 (注意：最后一次试就又遇到这个问题，不需要删除）.

  1. mkdir neutron-git-buildpackage2 && cd neutron-git-buildpackage2
  2. echo "# neutron-git-buildpackage" >> README.md
  3. git init
  4. git add README.md
  5. git commit -m "first commit"
  6. #git remote add origin git@github.com:zhhuabj/neutron-git-buildpackage.git
  7. #git push origin master

2, neutron源码工程下准备 tarball

  1. mkdir -p /bak/openstack/tarball/ && cd /bak/openstack
  2. git clone https://github.com/openstack/neutron
  3. cd neutron
  4. git checkout master
  5. git checkout -b 15.3.4 15.3.4
  6. mkdir -p /tmp/tarball
  7. git archive --output=/tmp/tarball/neutron_15.3.4.orig.tar.gz --format=tar.gz HEAD

3, 创建upstream分支

  1. git checkout -b upstream

4, 回到master分支（这时不能使用15.3.4，但master测试没问题后再加tag为15.3.4)导入tarball

  1. git checkout master
  2. gbp import-orig /tmp/tarball/neutron_15.3.4.orig.tar.gz -u 15.3.4

只有使用’gbp import-orig’导入tarball包才会提交到upstream分支，并merge到master分支（也就是upstream与master分支都有）, 此时状态为：

  1. ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git branch
  2. * master
  3.   upstream
  4. ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ ls
  5. CONTRIBUTING.rst  LICENSE     TESTING.rst  babel.cfg  bindep.txt  doc  neutron    rally-jobs    requirements.txt  setup.cfg  test-requirements.txt  tox.ini
  6. HACKING.rst       README.rst  api-ref      bin        devstack    etc  playbooks  releasenotes  roles             setup.py   tools
  7. ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git checkout upstream
  8. Switched to branch 'upstream'
  9. ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ ls
  10. CONTRIBUTING.rst  LICENSE     TESTING.rst  babel.cfg  bindep.txt  doc  neutron    rally-jobs    requirements.txt  setup.cfg  test-requirements.txt  tox.ini
  11. HACKING.rst       README.rst  api-ref      bin        devstack    etc  playbooks  releasenotes  roles             setup.py   tools
  12. ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git tag
  13. upstream/15.3.4

5, master分支中添加debian目录(此时只能用master，不能创建tag, tag可在master中测试好了再打)

  1. git checkout master
  2. wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  3. tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  4. rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  5. git add debian
  6. git commit -m 'add debain'

6, pristine-tar commit tarball, 总之，通过’pristine-tar list’ 得能看到neutron_15.3.4.orig.tar.gz 15.3.4, 这样编译系统在编译的时候才能通过’pristine-tar checkout …/neutron_15.3.4.orig.tar.gz’获取它。

  1. pristine-tar commit /tmp/tarball/neutron_15.3.4.orig.tar.gz 15.3.4
  2. pristine-tar list

pristine-tar commit会自动添加pristine-tar分支：

  1. ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git checkout pristine-tar
  2. Switched to branch 'pristine-tar'
  3. ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ ls
  4. neutron_15.3.4.orig.tar.gz.delta  neutron_15.3.4.orig.tar.gz.id

７，回到master分支编译. 它是使用debuild来编译的，应该也可以和pbuilder等集成。

  1. git checkout master
  2. gbp buildpackage --git-ignore-branch --git-ignore-new

上面会遇到这个错误”pristine-tar: /bak/work/build-area/neutron_15.3.4.orig.tar.gz does not match stored hash“，下面使用pristine-tar checkout时也会报这个错，直接用–git-tarall-dir来指定之前使用git archive创建的tarball来workaround就好了（-us -uc意为不签名)：

  1. #pristine-tar checkout /tmp/tarball/neutron_15.3.4.orig.tar.gz
  2. #gbp buildpackage --git-tarball-dir=/tmp/tarball --git-ignore-branch --git-ignore-new -S -us -uc -sa
  3. gbp buildpackage --git-tarball-dir=/tmp/tarball --git-ignore-branch --git-ignore-new -us -uc

8，编译后的包在/bak/work/build-area目录中
9, 都没问题之后，打tag归档，今后可以在该tag下运行编译命令了：

  1. git checkout master
  2. git tag '15.3.4'
  3. git checkout -b 15.3.4 15.3.4
  5. git checkout 15.3.4
  6. gbp buildpackage --git-tarball-dir=/tmp/tarball --git-ignore-branch --git-ignore-new -us -uc

再总结 - 在neutron源码中自动导入tarball构建

这是针对”在neutron源码中自动导入tarball构建”一节更快速的总结.

  1. sudo apt install git-buildpackage pristine-tar -y
  2. # git clone --no-checkout -o upstream https://github.com/openstack/neutron
  3. git clone https://github.com/openstack/neutron
  4. cd neutron
  5. # create upstream branch
  6. git checkout master && git checkout -b upstream
  7. # create pristine-tar branch, but we don't need to prepare tarball because '--git-pristine-tar --git-pristine-tar-commit' will help us create it automatically
  8. git checkout master && git checkout -b pristine-tar
  9. # create debain/15.3.4 branch with upstream neutron=15.3.4
  10. git checkout master && git checkout -b debian/15.3.4 15.3.4
  11. # add debian package into debian/15.3.4
  12. git checkout debian/15.3.4
  13. wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  14. tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  15. rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  16. git add debian
  17. git commit -m 'add debian for 15.3.4'
  18. # run 'gbp buildpackage' command under the directory which has debian
  19. # --git-pristine-tar and --git-pristine-tar-commit will create tarball automatically
  20. git checkout debian/15.3.4
  21. gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc

报下列错，那是因为要检出15.3.4分支(git checkout master && git checkout -b 15.3.4 15.3.4)：

  1. ubuntu@debbuild:/bak/openstack/neutron$ gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc
  2. gbp:error: Couldn't commit to 'pristine-tar' with upstream '15.3.4': pristine-tar: more than one ref matches "15.3.4":
  3. 480f4b7479bc0b19752edde46fe90660a94cc654 refs/heads/debian/15.3.4
  4. 729f6aefe103a0ba10b71888f84b6247d694bc04 refs/tags/15.3.4

附录 - 如何检查包依赖问题

上面种种方法都有这么一个问题（openstack-pkg-tools (>= 85ubuntu3~)），所以我们有必要检查这个源码包究竟有没有问题。
原来，错误原因是必须添加：sudo add-apt-repository cloud-archive:stein

  1. sudo add-apt-repository cloud-archive:stein
  2. sudo sed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.list
  3. sudo apt update
  4. sudo apt-cache policy neutron-l3-agent
  5. sudo apt-get source neutron=2:15.3.4-0ubuntu1~cloud1
  6. cd neutron-15.3.4/
  7. $ grep -r 'openstack-pkg-tools' debian/control 
  8.  openstack-pkg-tools (>= 85ubuntu3~),
  9. $ sudo apt-cache policy openstack-pkg-tools |grep Candidate
  10.   Candidate: 99ubuntu1~cloud0

附录 - 其他

1, gbp buildpackage之后无问题，可以加tag，会自动将neutron source code和debian source code加tag: debian/2%15.3.4-0ubuntu1_cloud1

  1. gbp buildpackage --git-tag-only --git-sign-tags --git-ignore-new
  2. #git checkout -b debian/2%15.3.4-0ubuntu1_cloud1 debian/2%15.3.4-0ubuntu1_cloud1

2, 然后可以推入远程repo

  1. git push origin --all --follow-tags

最佳实践

  • 使用pbuilder避免在物理机上安装依赖
  • 直接使用neutron源码，并在neutron源码自动产生tarball的方式

第一步，创建pbuilder环境

  1. #sudo pbuilder --create --distribution bionic --architecture amd64 --debootstrapopts --variant=buildd
  2. rm -rf sudo rm -rf /var/cache/pbuilder/base-bionic-amd64.cow
  3. sudo mv /etc/pbuilderrc /tmp/
  4. DIST=bionic ARCH=amd64 git-pbuilder create
  5. sudo mv /tmp/pbuilderrc /etc/
  7. #sudo pbuilder update --basetgz /var/cache/pbuilder/bionic-train-base.tgz
  8. DIST=bionic ARCH=amd64 git-pbuilder update
  9. #sudo pbuilder login --save-after-login --basetgz /var/cache/pbuilder/bionic-train-base.tgz
  10. DIST=bionic ARCH=amd64 git-pbuilder login --save-after-login
  11.   echo 'precedence ::ffff:0:0/96 100' >> /etc/gai.conf
  12.   apt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -y
  13.   # Failed to connect to socket /run/user/1000/bus: No such file or directory
  14.   apt install dbus -y && /etc/init.d/dbus start
  15.   add-apt-repository cloud-archive:train
  16.   #deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main
  17.   #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EA
  18.   sed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.list
  19.   apt update
  20.   exit

第二步，准备neutron源码环境，还要创建upstream与prinstine-tar两个分支，以及要编译的15.3.4分支，以及存放15.3.4源码及debian目录的debian/15.3.4分支

  1. sudo apt install git-buildpackage pristine-tar -y
  2. git clone https://github.com/openstack/neutron
  3. cd neutron
  4. git checkout master && git checkout -b upstream
  5. git checkout master && git checkout -b pristine-tar
  6. git checkout master && git checkout -b debian/15.3.4 15.3.4
  7. git checkout master && git checkout -b 15.3.4 15.3.4
  9. git checkout debian/15.3.4
  10. wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  11. tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  12. rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  13. git add debian
  14. git commit -m 'add debian for 15.3.4'

第三步，使用了git-builder来编译。并且通过“–git-pristine-tar —git-pristine-tar-commit ”来配置自动导出tarball

  1. git checkout debian/15.3.4
  2. # dont's use git-builder
  3. #gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc
  5. # use git-builder
  6. gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc --git-pbuilder --git-dist=bionic --git-arch=amd64 -j8

但是会报下列dbus的错：

  1. dbus.exceptions.DBusException: org.freedesktop.DBus.Error.FileNotFound: Failed to connect to socket /run/user/1000/bus: No such file or directory
  2. debian/rules:16: recipe for target 'override_dh_install' failed

试了下面的还是不行。

  1. DIST=bionic ARCH=amd64 git-pbuilder login --save-after-login
  2. apt install dbus
  3. /etc/init.d/dbus start
  4. exit

git-pbuilder创建出来的镜像里没有dbus包和环境变量（DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus），暂不清楚是否和这相关。
如果不使用git-builder，直接运行’gpb buildpackage’ + debuild模式，it works. 注意：下列这个测试没用pbuilder只能在bionic上进行。

  1. # NOTE: must run on bionic
  2. sudo mk-build-deps --install debian/control
  3. gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc -j8

将pbuilder换成sbuild也成功了（换sbuild时要去掉-us -uc, 见：https://www.pseudorandom.co.uk/2008/sbuild-dm/ , 另外，使用的是–git-builder=‘sbuild -A’， -A代表all-arch，构建源码包时用: —git-builder=‘sbuild -As’

  1. gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new --git-builder='sbuild -A' --git-dist=bionic --git-arch=amd64 -j8

在neutron debian source code(https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron)中运行pbuilder也出现了dbus的错误，现在确认是这个dbus错误只是在pbuilder中才出现。

示例：ubuntu debian源码上编译16.1.0

neutron debian source 上运行：

  1. git checkout -b debian/2%16.1.0-0ubuntu2 debian/2%16.1.0-0ubuntu2
  2. gbp buildpackage --git-debian-branch=debian/2%16.1.0-0ubuntu2 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As' --git-dist=bionic --git-arch=amd64 -j8

但会报错：

  1. sbuild-build-depends-main-dummy : Depends: python3-neutron-lib (>= 2.2.0) but 1.29.1-0ubuntu1~cloud0 is to be installed
  2.                                    Depends: python3-ovsdbapp (>= 1.0.0) but 0.17.0-0ubuntu1~cloud0 is to be installed
  3.                                    Depends: python3-pyroute2 (>= 0.5.7) but 0.5.6-0ubuntu1~cloud0 is to be installed

那是因为镜像中使用的是train UCA(python3-neutron-lib=1.29.1)，train对应的是15.3.4，并不是16.1.0，所以有问题。vimdiff一下：

  1. git checkout debian/2%16.1.0-0ubuntu2
  2. mkdir -p /tmp/16.1.0 && cp -r debian /tmp/16.1.0/
  3. git checkout debian/2%15.3.4-0ubuntu1_cloud0
  4. mkdir -p /tmp/15.3.4 && cp -r debian /tmp/15.3.4/
  5. vimdiff /tmp/15.3.4/debian/control /tmp/16.1.0/debian/control
  7. 15.3.4 -> python3-neutron-lib (>= 1.29.1), 
  8. 16.1.0 -> python3-neutron-lib (>= 2.2.0),

python3-neutron-lib 2.2.0来自focal前的一个版本也就是19.10

  1. apt install devscripts liburi-perl wget -y
  2. root@node1:/# rmadison python3-neutron-lib
  3.  python3-neutron-lib | 0.0.2-2         | xenial/universe | all
  4.  python3-neutron-lib | 1.13.0-0ubuntu1 | bionic/universe | all
  5.  python3-neutron-lib | 2.3.0-0ubuntu1  | focal           | all
  6.  python3-neutron-lib | 2.10.1-0ubuntu1 | hirsute         | all
  7.  python3-neutron-lib | 2.12.0-0ubuntu1 | impish          | all
  9. $ cmadison python3-neutron-lib
  10.  python3-neutron-lib | 0.0.2-2~cloud0         | mitaka            | trusty-updates  | all
  11.  python3-neutron-lib | 0.0.2-2~cloud0         | mitaka-proposed   | trusty-proposed | all
  12.  python3-neutron-lib | 1.13.0-0ubuntu1~cloud0 | queens            | xenial-updates  | all
  13.  python3-neutron-lib | 1.13.0-0ubuntu1~cloud0 | queens-proposed   | xenial-proposed | all
  14.  python3-neutron-lib | 1.25.0-0ubuntu1~cloud0 | stein             | bionic-updates  | all
  15.  python3-neutron-lib | 1.25.0-0ubuntu1~cloud0 | stein-proposed    | bionic-proposed | all
  16.  python3-neutron-lib | 1.29.1-0ubuntu1~cloud0 | train             | bionic-updates  | all
  17.  python3-neutron-lib | 1.29.1-0ubuntu1~cloud0 | train-proposed    | bionic-proposed | all
  18.  python3-neutron-lib | 2.3.0-0ubuntu1~cloud0  | ussuri            | bionic-updates  | all
  19.  python3-neutron-lib | 2.3.0-0ubuntu1~cloud0  | ussuri-proposed   | bionic-proposed | all
  20.  python3-neutron-lib | 2.6.1-0ubuntu1~cloud0  | victoria          | focal-updates   | all
  21.  python3-neutron-lib | 2.6.1-0ubuntu1~cloud0  | victoria-proposed | focal-proposed  | all
  22.  python3-neutron-lib | 2.10.1-0ubuntu1~cloud0 | wallaby           | focal-updates   | all
  23.  python3-neutron-lib | 2.10.1-0ubuntu1~cloud0 | wallaby-proposed  | focal-proposed  | all
  24.  python3-neutron-lib | 2.12.0-0ubuntu1~cloud0 | xena              | focal-updates   | all
  25.  python3-neutron-lib | 2.12.0-0ubuntu1~cloud0 | xena-proposed     | focal-proposed  | all

所以上面的16.1.0的源码显然是针对focal来做的（这里也有显示：https://launchpad.net/ubuntu/+source/neutron/2%3A16.1.0-0ubuntu2/+publishinghistory），所以：

  • 目前bionic-stein的版本是15.3.4, focal-ussuri的版本是16.4.0, 现在想要测的是16.1.0
  • 如果在bionic里测16.1.0的话，得修改debian/control与requirment.txt中的发生问题的pip版本号，但这也不一定行，因为neutron代码也变了既然requirement.txt说依赖某个包的高版本可能就真是需要高版本

  • 那样就只能在focal里测16.1.0了。将focal现有的16.4.0改成16.1.0来测，sbuild的镜像什么的都得由bionic改成16.1.0

    create focal sbuid image

    mk-sbuild focal —arch=amd64
    schroot -l
    schroot -c source:focal-amd64 -u root

    clone neutron debian source code

    cd /bak/work/debsource
    git clone https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron
    cd neutron
    git checkout pristine-tar
    git checkout upstream
    git checkout master

    build with focal image

    git checkout -b debian/2%16.1.0-0ubuntu2 debian/2%16.1.0-0ubuntu2
    git checkout debian/2%16.1.0-0ubuntu2

    注意：下面使用了—git-dist=focal —git-arch=amd64来指定用focal但实际用的还是bionic，

    原来git-dist只是针对pbuilder，这样使用了~/.sbuildrc的默认项：sed -i -e “s/bionic/focal/g” ~/.sbuildrc

    gbp buildpackage —git-debian-branch=debian/2%16.1.0-0ubuntu2 —git-ignore-branch —git-ignore-new —git-builder=’sbuild -As’ —git-dist=focal —git-arch=amd64 -j8

    所以或者使用（sed -i -e “s/bionic/focal/g” ~/.sbuildrc），或者使用（’—git-builder=sbuild -As -d focal-amd64’）

    gbp buildpackage —git-debian-branch=debian/2%16.1.0-0ubuntu2 —git-ignore-branch —git-ignore-new —git-builder=’sbuild -As -d focal-amd64’ —git-dist=focal —git-arch=amd64 -j8

示例：neutron upsteam源码上编译15.3.4

  1. # prepare bionic-train sbuild image
  2. sudo cp /etc/schroot/chroot.d/sbuild-bionic-amd64 /etc/schroot/chroot.d/sbuild-bionic-train-amd64
  3. sudo cp -r /var/lib/schroot/chroots/bionic-amd64 /var/lib/schroot/chroots/bionic-train-amd64
  4. sudo sed -i -e "s/bionic-amd64/bionic-train-amd64/g" /etc/schroot/chroot.d/sbuild-bionic-train-amd64
  5. schroot -l
  6. schroot -c source:bionic-train-amd64 -u root
  7.   apt install devscripts curl ca-certificates bash-completion software-properties-common vim -y
  8.   # Couldn't create temporary file /tmp/apt.conf.5gKxoB for passing config to apt-key
  9.   chmod 1777 /tmp
  10.   # /usr/bin/apt-key: cannot create /dev/null: Permission denied
  11.   # https://askubuntu.com/questions/764719/fix-permissions-after-upgrade-to-16-04
  12.   chmod go+rw /dev/null
  13.   add-apt-repository cloud-archive:train
  15. git clone https://github.com/openstack/neutron
  16. cd neutron
  17. #upstream与pristine-tar不是必须的吧
  18. #git checkout master && git checkout -b upstream
  19. #git checkout master && git checkout -b pristine-tar
  20. git checkout master && git checkout -b debian/15.3.4 15.3.4
  21. git checkout master && git checkout -b 15.3.4 15.3.4
  23. git checkout debian/15.3.4
  24. wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  25. tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  26. rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  27. git add debian
  28. git commit -m 'add debian for 15.3.4'
  30. # 已经在15.3.4分支下，--git-debian-branch=debian/15.3.4也不是必须的吧
  31. git checkout debian/15.3.4
  32. #gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc
  33. gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As -d bionic-train-amd64' -j8

示例 - 在15.3.4与16.0.0之间二分准备15.3.5的测试包

gbp buildpackage要想不自己准备tarball包的话，只能根据tag来做，不能根据commit来做，所以需自己准备如15.3.5与debian/15.3.5 (不自己准备tag的话似乎tarball包不更新使用15.3.4).
当然，想自己准备tarball的话，就是另外一种方式。这里讲的是不自己准备tarball包的模式。

  1. e562ecec2669c39b3628f7744011467c7394088e (在15.3.4基础上提交debian目录的commit id)
  2. 15.3.4 - 0f18d406a4134cf769fdf6f2a004c98cbe409215
  3. 16.0.0 - c06cb95b6f46574e74244fd738873290fd1773cb
  5. # 下载代码
  6. # first fork https://github.com/openstack/neutron
  7. git clone https://github.com/zhhuabj/neutron.git
  8. git config user.name
  9. git config user.email
  10. git remote add upstream https://github.com/openstack/neutron
  11. git remote set-url --push upstream no_push
  12. # update
  13. git fetch upstream
  14. git checkout master
  15. git merge upstream/master
  16. git push origin master
  17. #git rebase upstream/master
  18. #git push origin master -f
  20. # http://www.360doc.com/content/18/0208/09/19862658_728539054.shtml
  21. # git二分，先找到15.3.4与16.0.0之间的中间版本1c2e10f859
  22. cd neutron
  23. git checkout master
  24. git bisect start
  25. git bisect good 15.3.4
  26. $ git bisect bad 16.0.0
  27. Bisecting: a merge base must be tested
  28. [1c2e10f8595d2286bd9bec513bc5a346a84a6f7c] Merge "Remove get_external_network_id for router"
  29. $ git status
  30. HEAD detached at 1c2e10f859
  32. # 在上面1c2e10f859的基础上创建几个分支
  33. git checkout -b 15.3.5
  34. git checkout -b debian/15.3.5
  35. # 往debian/15.3.5中添加15.3.4的debian目录
  36. cp -r ../debian .
  37. git add debian/ && git commit -m 'add debian'
  39. # 修改debian/changelog
  40. dch -i
  41. neutron (2:15.3.5-0ubuntu1~cloud0) bionic-train; urgency=medium
  42. git commit -m "releasing package 2:15.3.5-0ubuntu1~cloud0" debian/changelog
  44. # 视情况清空debian/patches/避免一些错误, 并记得用debcommit提交
  45. cp -r debian/patches/ /tmp/
  46. rm -rf debian/patches/* && touch debian/patches/series
  47. git add -u && debcommit -m 'delete series'
  48. # 然后git log会看到：commit f1ed620b2abd4a04013ed9549fcafff5a290ff5b (HEAD -> debian/15.3.5)
  50. # 编译
  51. git checkout 15.3.5 && git tag --no-sign 15.3.5 -m "15.3.5"
  52. git checkout debian/15.3.5 && git tag --no-sign debian/15.3.5 -m "debian/15.3.5"
  53. git checkout debian/15.3.5
  54. #git push --all
  55. #git push --tags
  57. # 测试之后可以使用'git bisect bad'或'git bisect good'来标记, 使用’git bisect replay‘可以看到曾经测试过的内容
  58. git bisect log > bisectlog
  59. git bisect bad | git bisect good
  61. # 下一个二分可以先回到15.3.4基准，然后再git merge下一个二分点
  62. git reset --hard e562ecec2669c39b3628f7744011467c7394088e
  63. git merge <下一个二分点>
  65. # 二分完成之后
  66. git bisect reset

一个错误：

  1. # 错误 - https://paste.ubuntu.com/p/DtfMXg3Yq2/
  2. # The 'weakrefmethod>=1.0.2' distribution was not found and is required by the application
  3. 1, 15.3.4与1c2e10f859源码中有对weakrefmethod的定义
  4. # 15.3.4
  5. $ grep -r 'weakrefmethod' requirements.txt
  6. weakrefmethod>=1.0.2;python_version=='2.7' # PSF
  7. # git checkout -b 1c2e10f859 1c2e10f859
  8. $ grep -r 'weakrefmethod' requirements.txt 
  9. weakrefmethod>=1.0.2;python_version=='2.7' # PSF
  11. 2, 在tarball中也有啊
  12. hua@node1:/bak/work/tmp/build-area/neutron-15.3.5$ grep -r 'weakrefmethod' requirements.txt 
  13. weakrefmethod>=1.0.2;python_version=='2.7' # PSF
  15. 3, 难道是要在/bak/work/tmp/neutron/debian/control中加吗？
  17. git add -u && debcommit -m 'add python-weakrefmethod'
  19. hua@node1:/bak/work/tmp/neutron$ git diff
  20. diff --git a/debian/control b/debian/control
  21. index f25fd6f9ea..3f65330fd2 100644
  22. --- a/debian/control
  23. +++ b/debian/control
  24. @@ -13,6 +13,7 @@ Build-Depends:
  25.   python3-all,
  26.   python3-pbr (>= 4.0.0),
  27.   python3-setuptools,
  28. + python-weakrefmethod,
  29.  Build-Depends-Indep:
  30.   crudini,
  31.   python3-alembic (>= 0.8.10),

通过debuild调试上速问题的测试环境快速搭建：

  1. sudo cp /etc/schroot/chroot.d/sbuild-bionic-train-amd64 /etc/schroot/chroot.d/sbuild-bionic-train-amd64-tmp
  2. sudo cp -r /var/lib/schroot/chroots/bionic-train-amd64 /var/lib/schroot/chroots/bionic-train-amd64-tmp
  3. sudo sed -i -e "s/bionic-train-amd64/bionic-train-amd64-tmp/g" /etc/schroot/chroot.d/sbuild-bionic-train-amd64-tmp
  4. #sudo rm /etc/schroot/chroot.d/sbuild-bionic-train-amd64-tmp
  5. #sudo rm -rf /var/lib/schroot/chroots/bionic-train-amd64-tmp
  6. schroot -l
  7. schroot -c source:bionic-train-amd64-tmp -u root
  8. chmod 1777 /tmp
  9. apt install git vim wget devscripts equivs -y
  10. git clone https://github.com/zhhuabj/neutron.git && cd neutron
  11. git bisect start
  12. git bisect good 15.3.4
  13. git bisect bad 16.0.0
  14. git checkout -b 15.3.5
  15. git checkout -b debian/15.3.5
  16. wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  17. tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  18. rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
  19. # mkdir /home/hua
  20. #git config --global user.email "you@example.com"
  21. #git config --global user.name "Your Name"
  22. git add debian/ && git commit -m 'add debian'
  23. dch -i
  24. #neutron (2:15.3.5-0ubuntu1~cloud0) bionic-train; urgency=medium
  25. git commit -m "releasing package 2:15.3.5-0ubuntu1~cloud0" debian/changelog
  27. cp -r debian/patches/ /tmp/
  28. rm -rf debian/patches/* && touch debian/patches/series
  29. git add -u && debcommit -m 'delete series'
  31. git checkout 15.3.5 && git tag --no-sign 15.3.5 -m "15.3.5"
  32. git checkout debian/15.3.5 && git tag --no-sign debian/15.3.5 -m "debian/15.3.5"
  34. git checkout 15.3.5
  35. git archive --format=tar HEAD |gzip > ../neutron_15.3.5.orig.tar.gz
  36. git checkout debian/15.3.5
  37. mk-build-deps --install debian/control
  38. rm -rf ../neutron_15.3.5-0ubuntu1~cloud0* && rm -rf .stestr
  39. debuild -i -us -uc

有很多种方式有可以很轻易重现上面问题：
一种是直接运行’stestr run —subunit’，不清楚要不要运行’tox -e cover’（运行这句也hang在那）

  1. stestr取代了testrepository - https://storyboard.openstack.org/#!/story/1666232
  2. # https://github.com/mtreinish/stestr/
  3. apt install python-tox python3-stestr python3-mock -y
  4. #tox -e cover
  5. stestr run --subunit

二是直接运行debian/rules build：

  1. debian/rules clean && debian/rules build

三是运行’pkgos-dh_auto_test —no-py2’：

  1. ls build/lib.linux-x86_64-2.7/neutron/
  2. cat /usr/share/openstack-pkg-tools/pkgos.make
  3. pkgos-dh_auto_test --no-py2
  5. vim debian/rules
  6. ifeq (,$(findstring nocheck, $(DEB_BUILD_OPTIONS)))
  7. override_dh_auto_test:
  8.         pkgos-dh_auto_test --no-py2
  9. endif

所以最后通过‘DEB_BUILD_OPTIONS=nocheck’（不再运行unit test)来bypass了上述错误

  1. echo 'DEBOOTSTRAP_PROXY=http://t440p:3142/' >> ~/.mk-sbuild.rc
  2. gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.5 --git-ignore-branch --git-ignore-new --git-builder='DEB_BUILD_OPTIONS=nocheck sbuild -As -d bionic-train-amd64' -j8

Reference

[1] https://www.debian.org/doc/manuals/maint-guide/build.zh-cn.html
[2] https://wiki.softwareheritage.org/wiki/Debian\_packaging

发表评论

表情：
评论列表 （有 0 条评论，140人围观）

还没有评论，来说两句吧...

相关阅读

    相关 debug glance(by quqi99)

    作者：张华 发表于：2021-05-19 版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明 glance与nova都使用rbd的话使

    忘是亡心i忘是亡心i/ 0 赞/ 298 阅读

    相关 Use TSC for KVM Guest (by quqi99)

    作者：张华 发表于：2020-12-18 版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明 要想在KVM Guest中使用TSC,

    亦凉亦凉/ 0 赞/ 219 阅读