目的
CockroachDB自 23.1开始,编译模式逐步放弃了Makefile,改用Bazel构建。实际上在23.1代码执行make buildshort 能够成功,但make build应该就会因为生成js文件有问题而失败。可惜Bazel构建也不是官网说的那样简单,本文详细描述编译CockroachDB 23.1.x的编译过程,供对CockroachDB感兴趣的同仁参考。
回到顶部
编译环境
Rocky Linux 9.2,原因是Rocky Linux 9.2 自带各种模块版本较高,符合cockroachdb编译要求,并且外围模块趋于稳定,不会出现9.0时因为openssl版本问题导致nodejs需要降级的异常操作
安装RockyLinux 9.2 时,仅需选择Server环境,然后选择开发工具和效率工具即可。
CockroachDB版本:官网的release-23.1分支,今年国内访问github的情况有所改善,虽然不稳定,但是多试几次,尤其是ssh模式还是可以下载代码的
回到顶部
官方参考资料
-
1、Getting and building CockroachDB from source
-
2、Developing with Bazel
回到顶部
准备和检查编译环境
-
1、克隆代码库
采用ssh模式克隆cockroachdb的release-23.1(本文假设您已经有了一个github账号)
git clone -b release-23.1 git@github.com:cockroachdb/cockroach.git
不要嫌慢,能下载就是好事儿,我的网络是天津联通,github下载速度再10K~2M之间
-
2、下载的同时,可以按官网逐项对比Linux x86平台的环境准备
-
2.1、A C++ compiler that supports C++11. Note that GCC prior to 6.0 doesn't work due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48891。缺省已经安装的gcc版本是11.3.1,是支持C++11标准的
g++ -v
gcc version 11.3.1 20221121 (Red Hat 11.3.1-4) (GCC)
-
2.2、The standard C/C++ development headers on your system. 同2.1 缺省已经安装头文件
-
2.3、On GNU/Linux, the terminfo development libraries, which may be part of a ncurses development package (e.g.
libncurses-dev on Debian/Ubuntu, but ncurses-devel on CentOS). You also need the libresolv-wrapper package on Ubuntu < 22.04. rocky linux缺省没有安装ncurses-devel,需要执行dnf安装
dnf install ncurses-devel
-
2.4、Git 1.9+, we recommend v2.20+,缺省已经安装
git -v
git version 2.39.1
-
2.5、Bash (4+ is preferred) ,缺省已经安装
bash --version
GNU bash, version 5.1.8(1)-release (x86_64-redhat-linux-gnu)
-
2.6、GNU Make (4.2+ is known to work),缺省已经安装
make -v
GNU Make 4.3
-
2.7、CMake 3.20.* (on Macs 3.21+ are also known to work)。缺省没有安装cmake,需要执行dnf安装
dnf install cmake
cmake --version
cmake version 3.20.2
autoconf --version
autoconf (GNU Autoconf) 2.69
bison --version
bison (GNU Bison) 3.7.4
patch -v
GNU patch 2.7.6
-
2.11、Bazelisk (you can use Bazel directly instead of Bazelisk, although Bazelisk makes sure you are using the same version that we use in CI)
安装Bazelisk有多种方式,我们采用安装nodejs,然后用npm安装Bazelisk
dnf install nodejs
node -v
v16.19.1
npm -v
8.19.3
设置国内代理并验证
npm config set registry http://registry.npm.taobao.org/
npm get registry
http://registry.npm.taobao.org/
安装Bazelisk
npm install -g @bazel/bazelisk
安装完成后检查
whereis bazel
bazel: /usr/local/bin/bazel
whereis bazelisk
bazelisk: /usr/local/bin/bazelisk
-
3 开始构建
等待clone代码完成,就开始构建
-
3.1 首先运行 dev doctor 检查代码环境
cd cockroach
./dev doctor
/home/ansible/works/cockroach/bin/dev-versions/dev.84 not found, building...
2023/09/06 10:29:45 Downloading https://github.com/cockroachdb/bazel/releases/download/6.2.1/bazel-6.2.1-linux-x86_64...
长时间等待后
Starting local Bazel server and connecting to it...
INFO: Analyzed target //pkg/cmd/dev:dev (86 packages loaded, 10730 targets configured).
INFO: Found 1 target...
Target //pkg/cmd/dev:dev up-to-date:
_bazel/bin/pkg/cmd/dev/dev_/dev
INFO: Elapsed time: 254.017s, Critical Path: 12.70s
INFO: 20 processes: 5 internal, 15 linux-sandbox.
INFO: Build completed successfully, 20 total actions
INFO: Invocation ID: 5039d549-7ae7-4c4f-86b6-8695625c41fc
=============================
=== RUNNING DOCTOR CHECKS ===
=============================
INFO: Invocation ID: 27237479-0160-45db-92af-83445ca320d0
INFO: Build option --//build/toolchains:nogo_disable_flag has changed, discarding analysis cache.
INFO: Analyzed target //build/bazelutil:test_nogo_configured (1 packages loaded, 29 targets configured).
INFO: Found 1 target...
Target //build/bazelutil:test_nogo_configured up-to-date:
_bazel/bin/build/bazelutil/test_nogo_configured.sh
INFO: Elapsed time: 0.465s, Critical Path: 0.01s
INFO: 4 processes: 4 internal.
INFO: Build completed successfully, 4 total actions
Configuring cache...
INFO: Build option --//build/toolchains:nogo_disable_flag has changed, discarding analysis cache.
INFO: Analyzed target @com_github_buchgr_bazel_remote//:bazel-remote (195 packages loaded, 12422 targets configured).
INFO: Found 1 target...
Target @com_github_buchgr_bazel_remote//:bazel-remote up-to-date:
_bazel/bin/external/com_github_buchgr_bazel_remote/bazel-remote_/bazel-remote
INFO: Elapsed time: 63.612s, Critical Path: 29.20s
INFO: 206 processes: 7 internal, 199 linux-sandbox.
INFO: Build completed successfully, 206 total actions
Using cache configuration file at /home/ansible/.cache/dev-bazel-remote/config.yml
报告成功
我在一台虚拟机上针对不同提交版本多次执行检查,有可能报告一些错误,但进行交互后都能成功,遇到过如下情况
-
DOCTOR >> Which config you want to use (dev,crosslinux)? [dev] > 输入dev
-
DOCTOR >> Do you want to use the
lintonbuild configuration? [y] > 输入 y
-
DOCTOR >> Please set a tmpdir that will be used for storing test artifacts [/tmp/cockroach] > 直接回车
-
DOCTOR >> Do you want me to update your ~/.bazelrc for you to configure the loopback cache? [y] > 直接回车
-
3.2 编译主程序
./dev build
$ bazel build //pkg/cmd/cockroach:cockroach
INFO: Invocation ID: c009abbc-9b9f-493a-b491-a86df5a3a2c5
INFO: Build options
INFO: Analyzed target //pkg/cmd/cockroach:cockroach (4313 packages loaded, 48317 targets configured).
INFO: Found 1 target...
ERROR: /home/ansible/works/cockroach/pkg/ccl/gssapiccl/BUILD.bazel:4:11: GoCompilePkg pkg/ccl/gssapiccl/gssapiccl.a failed: (Exit 1): builder failed: error executing command (from target //pkg/ccl/gssapiccl:gssapiccl) bazel-out/k8-opt-exec-2B5CBBC6/bin/external/go_sdk/builder_reset/builder compilepkg -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,bazel,gss -src pkg/ccl/gssapiccl/gssapi.go -src ... (remaining 53 arguments skipped)
Use
/usr/bin/ld.gold: error: cannot find -lresolv_wrapper
external/archived_cdep_libkrb5_linux/lib/libkrb5.a(dnsglue.o):dnsglue.c:function krb5int_dns_init: error: undefined reference to '__res_nsearch'
collect2: error: ld returned 1 exit status
compilepkg: error running subcommand /usr/bin/gcc: exit status 1
Target //pkg/cmd/cockroach:cockroach failed to build
Use
INFO: Elapsed time: 282.874s, Critical Path: 177.14s
INFO: 2913 processes: 38 internal, 2875 linux-sandbox.
INFO: Build Event Protocol files produced successfully.
FAILED: Build did NOT complete successfully
ERROR: exit status 1
可惜,编译失败了,报告libkrb5.a编译失败,原因是rockylinux 9.2 上的makefile对krb5不友好,一些宏的判断有问题,此处问题解决有两个办法:
第一种修改方式
vi .bazelrc
删除gss编译项
第二种修改方式
因为我们仅仅是为了研究cockroachdb的核心技术,也可以简单删除krb5涉及的代码,具体执行如下
rm pkg/ccl/gssapiccl/ -rf
vi pkg/ccl/BUILD.bazel
vi pkg/ccl/ccl_init.go
然后重新执行编译程序
[ansible@xtp2 cockroach]$ ./dev build
$ bazel build //pkg/cmd/cockroach:cockroach
INFO: Invocation ID: 1eb1ac48-7ac9-4d50-8b56-b1b8911c4e8b
INFO: Analyzed target //pkg/cmd/cockroach:cockroach (1 packages loaded, 4 targets configured).
INFO: Found 1 target...
Target //pkg/cmd/cockroach:cockroach up-to-date:
_bazel/bin/pkg/cmd/cockroach/cockroach_/cockroach
INFO: Elapsed time: 40.259s, Critical Path: 38.21s
INFO: 4 processes: 1 internal, 3 linux-sandbox.
INFO: Build Event Protocol files produced successfully.
INFO: Build completed successfully, 4 total actions
Successfully built binary for target //pkg/cmd/cockroach:cockroach at cockroach
然后验证
./cockroach --version
cockroach version details:
Build Tag: v23.1.10-dev
Build Time:
Distribution: CCL
Platform: linux amd64
Go Version: go1.19.10
C Compiler: gcc 11.3.1 20221121 (Red Hat 11.3.1-4)
Build Commit ID:
Build Type:
Enabled Assertions: false
(use './cockroach version --build-tag' to display only the build tag)
再然后就按手册测试更多的功能了
按版本验证的角度,我们应该运行 ./dev test (对应之前的make test)进行完整验证,可惜 ./dev test 运行过程中占满了我的100G虚拟机磁盘,最后执行失败。
回到顶部
总结
- 1、在Rockylinux9.2上,编译libkrb5有问题,需要修改bazelrc文件
- 2、执行单元测试 ./dev test 失败,原因是生成了.cache/bazel 80G以上的文件,导致没有磁盘剩余空间执行终止
- 3、建议,如果不关注cockroach ui,仅仅学习研究cockroach的主程序,还是采用传统的make buildshort, make test 吧
(责任编辑:IT) |