docker中pyppeteer使用及解决方法

我不是女神ヾ 2023-02-19 05:24 9阅读 0赞

pyppeteer使用及docker中产生大量僵尸进程的解决方法  
[https://juejin.im/post/5e0db9295188253a8b42e63c][https_juejin.im_post_5e0db9295188253a8b42e63c]  
https://github.com/miyakogi/pyppeteer

前期配置  
安装

python3 -m pip install pyppeteer

pyppeteer 第一次启动时会自动下载chromium，  
手动下载chromium方法,需要科学上网

pyppeteer-install

修复小bug

找到pyppeteer包下的connection.py模块，在其43行和44行改为下面这样：

self._ws = websockets.client.connect(self._url, max_size=None, loop=self._loop)
    
    self._ws = websockets.client.connect(self._url, max_size=None, loop=self._loop, ping_interval=None, ping_timeout=None)

使用 docker browserless 镜像建立chrome连接池  
参考 https://zhuanlan.zhihu.com/p/66296309  
[https://docs.browserless.io/docs/docker-quickstart.html][https_docs.browserless.io_docs_docker-quickstart.html]  
[https://hub.docker.com/r/browserless/chrome/dockerfile][https_hub.docker.com_r_browserless_chrome_dockerfile]  
docker 配置browserless服务（win10上没有配置成功，ubuntu服务器上没有问题）

```bash
    docker pull browserless/chrome:latest

docker run  -d  -p 3000:3000  --shm-size 2gb    --restart always  -e "CONNECTION_TIMEOUT=-1" -e "DEBUG=browserless/chrome"  -e "MAX_CONCURRENT_SESSIONS=10"  -e "EXIT_ON_HEALTH_FAILURE=true" -e "PREBOOT_CHROME=true" -e "KEEP_ALIVE=true"  --name browserless  browserless/chrome:latest

[https://docs.browserless.io/docs/docker.html\#connection-timeout][https_docs.browserless.io_docs_docker.html_connection-timeout] browserless/chrome 文档

import asyncio
    import time
    import pyppeteer
    
    async def main():
        browser = await pyppeteer.connect({ 'browserWSEndpoint': 'ws://172.20.1.187:3000'})   # 修改为自己的ip
        page = await browser.newPage()
        await page.goto('https://www.baidu.com')
        await page.screenshot({ 'path': 'example.png'})
        print(await browser.version())
        print(await browser.userAgent())
        time.sleep(3)
        await browser.disconnect()
        
    asyncio.get_event_loop().run_until_complete(main())

[https_juejin.im_post_5e0db9295188253a8b42e63c]: https://juejin.im/post/5e0db9295188253a8b42e63c
[https_docs.browserless.io_docs_docker-quickstart.html]: https://docs.browserless.io/docs/docker-quickstart.html
[https_hub.docker.com_r_browserless_chrome_dockerfile]: https://hub.docker.com/r/browserless/chrome/dockerfile
[https_docs.browserless.io_docs_docker.html_connection-timeout]: https://docs.browserless.io/docs/docker.html#connection-timeout