【python初级】使用bs4. BeautifulSoup解析网页介绍

灰太狼 2022-11-21 11:46 282阅读 0赞

### 【python初级】使用bs4. BeautifulSoup解析网页 ###

*  1.背景
 *  2.安装
 *  3.简单示例

# 1.背景 #

爬虫模拟浏览器获取网页的数据。  
网页有唯一的URL地址，页面信息由HTML来描述，并使用HTTP/HTTPS协议来传输HTML网页数据；  
关于：HTML（HyperText Markup Language）超文本标记语言是一种用于创建网页的标准标记语言。

# 2.安装 #

pip install bs4

C:\Users\G7>python -V
    Python 3.6.8
    
    C:\Users\G7>pip install bs4
    Collecting bs4
      Downloading bs4-0.0.1.tar.gz (1.1 kB)
    Collecting beautifulsoup4
      Downloading beautifulsoup4-4.9.3-py3-none-any.whl (115 kB)
         |████████████████████████████████| 115 kB 285 kB/s
    Collecting soupsieve>1.2; python_version >= "3.0"
      Downloading soupsieve-2.0.1-py3-none-any.whl (32 kB)
    Building wheels for collected packages: bs4
      Building wheel for bs4 (setup.py) ... done
      Created wheel for bs4: filename=bs4-0.0.1-py3-none-any.whl size=1277 sha256=5fd8e30825a0170b1f16f78e84fba037685af463843685dae9c0593bc7a8c926
      Stored in directory: c:\users\g7\appdata\local\pip\cache\wheels\19\f5\6d\a97dd4f22376d4472d5f4c76c7646876052ff3166b3cf71050
    Successfully built bs4
    Installing collected packages: soupsieve, beautifulsoup4, bs4
    Successfully installed beautifulsoup4-4.9.3 bs4-0.0.1 soupsieve-2.0.1

# 3.简单示例 #

import requests
    import bs4
    if __name__=="__main__":
        url="http://www.cwl.gov.cn/"
        #构造一个向服务器请求资源的url对象
        req=requests.get(url=url)
    
        #设置请求网页的编码格式
        req.encoding="utf-8"
        #获得网页的源代码
        html=req.text
    
        #html parser解析器
        #BeautifulSoup 是一个可以从HTML或XML文件中提取数据的Python库
        bf=bs4.BeautifulSoup(html, "lxml")
    
        #遍历文档进行搜索
        img_url=bf.find_all(class_="n_item")[0]
        print(img_url)
        #<div class="n_item"><a href="/zzjg/" target="_blank">组织机构</a></div>