如何让你在众多二手车中挑中满意的？python帮你实现（附源码）

以你之姓@ 2023-02-13 00:24 21阅读 0赞

## 前言 ##

老司机带你去看车，网上的几千条的二手车数据，只需几十行代码，就可以统统获取，保存数据到我们本地电脑上

## 知识点： ##

1.python基础知识  
2.函数  
3.requests库  
4.xpath适合零基础的同学

## 环境： ##

windows + pycharm + python3

## 爬虫流程： ##

1.目标网址  
2. 发送请求，获取响应  
3. 解析网页 提取数据  
4. 保存数据

## 步骤： ##

**1.导入工具**

import io
    import sys
    import requests   # pip install requests
    from lxml import etree   # pip

**2.获取汽车详情页面的url，解析网站**

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0_size_16_color_FFFFFF_t_70]

def get_detail_urls(url):
        # 目标网址
        # url = 'https://www.guazi.com/cs/buy/o3/'
        # 发送请求，获取响应
        resp = requests.get(url,headers=headers)
        text = resp.content.decode('utf-8')
        # 解析网页
        html = etree.HTML(text)
        ul = html.xpath('//ul[@class="carlist clearfix js-top"]')[0]
        # print(ul)
        lis = ul.xpath('./li')
        detail_urls = []
        for li in lis:
            detail_url = li.xpath('./a/@href')
            # print(detail_url)
            detail_url = 'https://www.guazi.com' + detail_url[0]
            # print(detail_url)
            detail_urls.append(detail_url)
    
        return detail_urls

**3.添加请求头**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0_size_16_color_FFFFFF_t_70_pic_center]

headers = { 
        'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36',
        'Cookie':'uuid=5a823c6f-3504-47a9-8360-f9a5040e5f23; ganji_uuid=4238534742401031078259; lg=1; Hm_lvt_936a6d5df3f3d309bda39e92da3dd52f=1590045325; track_id=79952087417704448; antipas=q7222002m3213k0641719; cityDomain=cs; clueSourceCode=%2A%2300; user_city_id=204; sessionid=38afa34e-f972-431b-ce65-010f82a03571; close_finance_popup=2020-05-23; cainfo=%7B%22ca_a%22%3A%22-%22%2C%22ca_b%22%3A%22-%22%2C%22ca_s%22%3A%22pz_baidu%22%2C%22ca_n%22%3A%22pcbiaoti%22%2C%22ca_medium%22%3A%22-%22%2C%22ca_term%22%3A%22-%22%2C%22ca_content%22%3A%22%22%2C%22ca_campaign%22%3A%22%22%2C%22ca_kw%22%3A%22-%22%2C%22ca_i%22%3A%22-%22%2C%22scode%22%3A%22-%22%2C%22keyword%22%3A%22-%22%2C%22ca_keywordid%22%3A%22-%22%2C%22ca_transid%22%3A%22%22%2C%22platform%22%3A%221%22%2C%22version%22%3A1%2C%22track_id%22%3A%2279952087417704448%22%2C%22display_finance_flag%22%3A%22-%22%2C%22client_ab%22%3A%22-%22%2C%22guid%22%3A%225a823c6f-3504-47a9-8360-f9a5040e5f23%22%2C%22ca_city%22%3A%22cs%22%2C%22sessionid%22%3A%2238afa34e-f972-431b-ce65-010f82a03571%22%7D; preTime=%7B%22last%22%3A1590217273%2C%22this%22%3A1586866452%2C%22pre%22%3A1586866452%7D',
    }

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0_size_16_color_FFFFFF_t_70_pic_center 1]

**4.提取每辆汽车详情页面的数据**

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0_size_16_color_FFFFFF_t_70_pic_center 2]

def parse_detail_page(url):
        resp = requests.get(url,headers=headers)
        text = resp.content.decode('utf-8')
        html = etree.HTML(text)
        # 标题
        title = html.xpath('//div[@class="product-textbox"]/h2/text()')[0]
        title = title.strip()
        print(title)
        # 信息
        info = html.xpath('//div[@class="product-textbox"]/ul/li/span/text()')
        # print(info)
    
        infos = { }
        cardtime = info[0]
        km = info[1]
        displacement = info[2]
        speedbox = info[3]
    
        infos['title'] = title
        infos['cardtime'] = cardtime
        infos['km'] = km
        infos['displacement'] = displacement
        infos['speedbox'] = speedbox
        print(infos)
        return infos

**5.保存数据**

def save_data(infos, f):
        f.write('{},{},{},{},{}\n'.format(infos['title'],infos['cardtime'],infos['km'],infos['displacement'],infos['speedbox']))
    
    
    
    if __name__ == '__main__':
        
        base_url = 'https://www.guazi.com/cs/buy/o{}/'
        with open('guazi.csv','a',encoding='utf-8') as f:
            for x in range(1,51):
                url = base_url.format(x)
                detail_urls = get_detail_urls(url)
                for detail_url in detail_urls:
                    infos = parse_detail_page(detail_url)
                    save_data(infos, f)

最后运行代码，效果如下图

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0_size_16_color_FFFFFF_t_70_pic_center 3]

欢迎点击左上角关注小编，除了分享技术文章之外还有很多福利，私信学习资料可以领取包括不限于Python实战演练、PDF电子文档、面试集锦、学习资料等。  
![在这里插入图片描述][2020052619101928.png_pic_center]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/2020052616472498.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0_size_16_color_FFFFFF_t_70_pic_center]: https://img-blog.csdnimg.cn/20200526165825501.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0_size_16_color_FFFFFF_t_70_pic_center 1]: https://img-blog.csdnimg.cn/20200526185653432.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0_size_16_color_FFFFFF_t_70_pic_center 2]: https://img-blog.csdnimg.cn/20200526185757939.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0_size_16_color_FFFFFF_t_70_pic_center 3]: https://img-blog.csdnimg.cn/20200526190009599.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ2NjE0MTU0,size_16,color_FFFFFF,t_70#pic_center
[2020052619101928.png_pic_center]: https://img-blog.csdnimg.cn/2020052619101928.png#pic_center