Golang调用Python3

落日映苍穹つ 2022-09-15 12:42 270阅读 0赞

python2 Golang调用Python请参考[Golang调用Python-阿里云开发者社区][Golang_Python-] [python3 请参考 https://github.com/DataDog/go-python3][python3 _ https_github.com_DataDog_go-python3]。

Python是时髦的机器学习御用开发语言，Golang是大红大紫的新时代后端开发语言。Python很适合让搞算法的写写模型，而Golang很适合提供API服务，两位同志都红的发紫，这里就介绍一下正确搅基的办法。

## 原理 ##

Python提供了丰富的[C-API][]。而C和Go又可以通过cgo无缝集成。所以，直接通过Golang调用libpython，就可以实现Go调Python的功能了。确实没啥神奇，只要会用C调Python，马上就知道怎么用了。但问题是，如果有的选择，这个年代还有多少人愿意去裸写C和C++呢？诚心默念Golang大法好。

## 准备工作 ##

*  Python3 ：确保Python正确安装，所谓正确安装，就是在系统中能找到`libpython.so(dylib)`，找到`Python.h`。一般linux直接安装`python-devel（`[Python Source Releases | Python.org][Python Source Releases _ Python.org]`）`，mac直接用homebrew安装就可以（目前只支持python3.7[https://www.python.org/downloads/macos/][https_www.python.org_downloads_macos]）。
    
    <table style="width:500px;"> 
     <tbody> 
      <tr> 
       <td>系统</td> 
       <td>安装细节</td> 
      </tr> 
      <tr> 
       <td>linux</td> 
       <td> <p>wget <a href="https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz" title="https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz" rel="nofollow">https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz</a></p> <p>tar -xvJf&nbsp; Python-3.7.0.tar.xz</p> <p>cd python3</p> <p>./configure --prefix=/usr/local/python3</p> <p>make &amp;&amp; make install</p> <p>python3 -V</p> <p>export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig/</p> </td> 
      </tr> 
      <tr> 
       <td>mac</td> 
       <td> <p>直接下载执行安装python3.7。and 执行以下语句</p> <p>export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/Library/Frameworks/Python.framework/Versions/3.7/lib/pkgconfig/</p> </td> 
      </tr> 
     </tbody> 
    </table>
 *  Golang安装：Golang不需要什么特殊的处理，能找到`go`即可。

## 安装libpython-go-binding ##

虽然直接用cgo调用libpython也不是不可以，但是有native-binding用起来肯定要爽的多。Github上有一个现成的Binding库[go-python3][]。

go clean github.com/datadog/go-python3
    go clean -cache
    
    export PKG_CONFIG_PATH=/Library/Frameworks/Python.framework/Versions/3.7/lib/pkgconfig
    go get github.com/datadog/go-python3

## Have a try ##

首先写一个测试Python3 hi在/Users/zlw/App/go/src/sbos-algorithm/pyoutliers目录下的脚本

import numpy
    import sklearn
    
    a = 10
    
    def b(xixi):
        return xixi + "haha"

然后写一个Go脚本：

package main
    
    import (
        "github.com/DataDog/go-python3"
        "fmt"
    )
    
    func init() {
        err := python3.Py_Initialize()
    	if !python3.Py_IsInitialized() {
    		fmt.Println("Error initializing the python interpreter")
            panic(err.Error())
    		os.Exit(1)
    	}
    }
    
    var PyStr = python3.PyUnicode_FromString
    var GoStr = python3.PyUnicode_AsUTF8
    
    func main() {
        // import hello
         InsertBeforeSysPath("/Library/Frameworks/Python.framework/
    Versions/3.7/lib/python3.7/site-packages/")
    	hello := ImportModule("/Users/zlw/App/go/src/sbos-algorithm/pyoutliers", "hello")
    	fmt.Printf("[MODULE] repr(hello) = %s\n", GoStr(hello.Repr()))
    
    	// print(hello.a)
    	a := hello.GetAttrString("a")
    	fmt.Printf("[VARS] a = %#v\n", python3.PyLong_AsLong(a))
    
        // print(hello.hi)
    	hi := hello.GetAttrString("hi")
    	fmt.Printf("[FUNC] hi = %#v\n", hi)
    
    	// args = tuple("xixi",)
    	bArgs := python3.PyTuple_New(1)
    	python3.PyTuple_SetItem(bArgs, 0, PyStr("xixi"))
    
    	// hi(*args)
    	res := hi.Call(bArgs, python3.Py_None)
    	fmt.Printf("[CALL] hi('xixi') = %s\n", GoStr(res))
    
    	// sklearn
    	sklearn := hello.GetAttrString("sklearn")
    	skVersion := sklearn.GetAttrString("__version__")
    	fmt.Printf("[IMPORT] sklearn = %s\n", GoStr(sklearn.Repr()))
    	fmt.Printf("[IMPORT] sklearn version =  %s\n", GoStr(skVersion.Repr()))
    }
    
    // InsertBeforeSysPath will add given dir to python import path
    func InsertBeforeSysPath(p string) string {
    	sysModule := python3.PyImport_ImportModule("sys")
    	path := sysModule.GetAttrString("path")
    	python3.PyList_Insert(path, 0, PyStr(p))
    	return GoStr(path.Repr())
    }
    
    // ImportModule will import python module from given directory
    func ImportModule(dir, name string) *python3.PyObject {
    	sysModule := python3.PyImport_ImportModule("sys")
    	path := sysModule.GetAttrString("path")
    	python3.PyList_Insert(path, 0, python3.PyUnicode_FromString(dir))
    	return python3.PyImport_ImportModule(name)
    }

打印输出为：

[MODULE] repr(hello) = <module 'hello' from '/Users/zlw/App/go/src/sbos-algorithm/pyoutliers/hello.py'>
    [VARS] a = 10
    [FUNC] hi = &python3.PyObject{ob_refcnt:2, ob_type:(*python3._Ctype_struct__typeobject)(0x4f3aed0)}
    [CALL] hi('xixi') = xixi,hello
    [IMPORT] sklearn = <module 'sklearn' from '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/sklearn/__init__.py'>
    [IMPORT] sklearn version =  '1.0'

这里简单解释一下。首先将这个脚本的路径添加到`sys.path`中。然后调用`PyImport_ImportModule`导入包

使用`GetAttrString`可以根据属性名获取对象的属性，相当于python3中的`.`操作。调用Python3函数可以采用`Object.Call`方法，，列表参数使用Tuple来构建。返回值用`PyUnicode_AsUTF8`从Python3字符串转换为C或Go的字符串。

更多用法可以参考[Python-C API文档][Python-C API]。

但是只要有这几个API，就足够 Make python module rock & roll。充分利用Golang和Python各自的特性，构建灵活而强大的应用了。

[Golang_Python-]: https://developer.aliyun.com/article/117329
[python3 _ https_github.com_DataDog_go-python3]: https://github.com/DataDog/go-python3
[C-API]: https://docs.python.org/2/c-api/
[Python Source Releases _ Python.org]: https://www.python.org/downloads/source/
[https_www.python.org_downloads_macos]: https://www.python.org/downloads/macos/
[go-python3]: https://github.com/sbinet/go-python
[Python-C API]: https://docs.python.org/2/c-api/object.html