python简单的爬取网页上的图片

淩亂°似流年 2023-01-05 01:20 209阅读 0赞

如果需要大量的图片，在网页中一个一个下载，耗费时间太长了，如果可以有程序自动帮我们下载图片，这样就最好了，既快速又方便。

接下来，我将把学习笔记记录如下，以下内容仅仅是突击学习、尽快使用的效果，而对于python爬取还是有很多很多不了解的地方。

**参考链接：[https://www.cnblogs.com/dearvee/p/6558571.html][https_www.cnblogs.com_dearvee_p_6558571.html]**

**参考链接：[https://blog.csdn.net/zjy18886018024/article/details/80001097][https_blog.csdn.net_zjy18886018024_article_details_80001097]**

**参考链接：[https://www.cnblogs.com/roboot/p/11410323.html][https_www.cnblogs.com_roboot_p_11410323.html]**

**参考链接：[https://www.jb51.net/article/168484.htm][https_www.jb51.net_article_168484.htm]**

**参考链接：[https://blog.csdn.net/qq\_38412868/article/details/82080260][https_blog.csdn.net_qq_38412868_article_details_82080260]**

# 源码示例 #

**参考链接：[https://www.jb51.net/article/168484.htm][https_www.jb51.net_article_168484.htm]**

给出一个链接，[https://tieba.baidu.com/p/2460150866?red\_tag=2171084898][https_tieba.baidu.com_p_2460150866_red_tag_2171084898]，里面有很多图片，需要把它们全部下载到一个本地文件夹内。

import urllib.request
    import re
    import os
    import urllib
    
    #根据给定的网址来获取网页详细信息，得到的html就是网页的源代码 
    def getHtml(url):
      page = urllib.request.urlopen(url)
      html = page.read()
      return html.decode('UTF-8')
     
    def getImg(html):
      reg = r'src="(.+?\.jpg)" pic_ext'
      imgre = re.compile(reg)
      imglist = imgre.findall(html)#表示在整个网页中过滤出所有图片的地址，放在imglist中
      x = 0
      path = 'E:/Python/get_picture/picture'
      # 将图片保存
      if not os.path.isdir(path): 
        os.makedirs(path) 
      paths = path+'/'   #保存在test路径下 
     
      for imgurl in imglist: 
        urllib.request.urlretrieve(imgurl,'{0}{1}.jpg'.format(paths,x)) #打开imglist中保存的图片网址，并下载图片保存在本地，format格式化字符串 
        x = x + 1
      return imglist
    html = getHtml("http://tieba.baidu.com/p/2460150866")#获取该网址网页详细信息，得到的html就是网页的源代码 
    print (getImg(html)) #从网页源代码中分析并下载保存图片

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3lxbF82MTc1NDAyOTg_size_16_color_FFFFFF_t_70][]

#  源码示例 #

**参考链接：[https://blog.csdn.net/qq\_38412868/article/details/82080260][https_blog.csdn.net_qq_38412868_article_details_82080260]**

需要下载图片的链接：[http://desk.zol.com.cn/dongman/longmao/][http_desk.zol.com.cn_dongman_longmao]

查看网页源代码：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3lxbF82MTc1NDAyOTg_size_16_color_FFFFFF_t_70 1][]

from bs4 import BeautifulSoup
    import requests
     
    def download(img_url,headers,n):
        req = requests.get(img_url, headers=headers)
        name = '%s'%n+'='+img_url[-15:]
        path = r'E:/Python/get_picture/picture'
        file_name = path + '\\' + name
        f = open(file_name, 'wb')
        f.write(req.content)
        f.close
     
    def parses_picture(url,headers,n):
        url = r'http://desk.zol.com.cn/' + url
        img_req = requests.get(url, headers=headers)
        img_req.encoding = 'gb2312'
        html = img_req.text
        bf = BeautifulSoup(html, 'lxml')
        try:
            img_url = bf.find('div', class_='photo').find('img').get('src')
            download(img_url,headers,n)
            url1 = bf.find('div',id='photo-next').a.get('href')
            parses_picture(url1,headers,n)
        except:
            print(u'第%s图片下载完成'%n)
     
    if __name__=='__main__':
        url='http://desk.zol.com.cn/dongman/longmao/'
        headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36"}
        req = requests.get(url=url, headers=headers)
        req=requests.get(url=url,headers=headers)
        req.encoding = 'gb2312'
        html=req.text
        bf=BeautifulSoup(html,'lxml')
        targets_url=bf.find_all('li',class_='photo-list-padding')
        n=1
        for each in targets_url:
            url = each.a.get('href')
            parses_picture(url,headers,n)
            n=n+1

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3lxbF82MTc1NDAyOTg_size_16_color_FFFFFF_t_70 2][]

[https_www.cnblogs.com_dearvee_p_6558571.html]: https://www.cnblogs.com/dearvee/p/6558571.html
[https_blog.csdn.net_zjy18886018024_article_details_80001097]: https://blog.csdn.net/zjy18886018024/article/details/80001097
[https_www.cnblogs.com_roboot_p_11410323.html]: https://www.cnblogs.com/roboot/p/11410323.html
[https_www.jb51.net_article_168484.htm]: https://www.jb51.net/article/168484.htm
[https_blog.csdn.net_qq_38412868_article_details_82080260]: https://blog.csdn.net/qq_38412868/article/details/82080260
[https_tieba.baidu.com_p_2460150866_red_tag_2171084898]: https://tieba.baidu.com/p/2460150866?red_tag=2171084898
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3lxbF82MTc1NDAyOTg_size_16_color_FFFFFF_t_70]: /images/20221119/995eb9bb7c4645148e03bedb9f4d1973.png
[http_desk.zol.com.cn_dongman_longmao]: http://desk.zol.com.cn/dongman/longmao/
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3lxbF82MTc1NDAyOTg_size_16_color_FFFFFF_t_70 1]: /images/20221119/521c5b73527245adb6fd9f9750e52809.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3lxbF82MTc1NDAyOTg_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20210111123920599.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3lxbF82MTc1NDAyOTg=,size_16,color_FFFFFF,t_70