Python Selenium爬取百度百科旅游景点的消息盒

梦里梦外; 2022-05-24 10:50 223阅读 0赞

# 一、编程环境 #

操作系统：Win 10  
编程语言：Python 3.6

# 二、什么是消息盒 #

Infobox（消息盒）是一种模板，包含了一系列的信息框，通常是由  
成对的label（标签）和data（数据）组成。  
下图是百度百科词条“北京故宫”的消息盒：

![1.png][]

# 三、景点名单准备 #

在与python文件同级的目录下创建scenic\_spots\_5A\_namelist.txt，内容为

北京故宫
    颐和园
    天坛公园

![2.png][]

# 四、程序 #

import os
    import time
    import codecs
    from selenium import webdriver
    from selenium.webdriver.common.keys import Keys
    
    
    global info  # 全局变量
    driver = webdriver.Chrome(executable_path="C:\Program Files (x86)\Google\Chrome\Application\chromedriver.exe")
    
    
    def getInfo(name):
        try:
            # 创建路径和文件
            global info
            basePath = "scenic_spots_5A"  
            if not os.path.exists(basePath):  
                os.makedirs(basePath)  
            scenicFile = os.path.join(basePath,"scenic_info.txt")  
            if not os.path.exists(scenicFile):  
                info = codecs.open(scenicFile,'w','utf-8')  
            else:  
                info = codecs.open(scenicFile,'a','utf-8')
    
            driver.get("http://baike.baidu.com/")
            elem_input = driver.find_element_by_xpath("//form[@id='searchForm']/input")
            name = name.rstrip('\n')        # 景点名称是从文件中读取的，含有换行符（最后一行的景点名称可能不含护身符）
            elem_input.send_keys(name)
            elem_input.send_keys(Keys.RETURN)
            info.write(name + '\r\n')       # codecs不支持'\n'换行
            time.sleep(2)
            print (driver.current_url)
            print (driver.title)
    
            elem_name = driver.find_elements_by_xpath("//div[@class='basic-info cmn-clearfix']/dl/dt")    
            elem_value = driver.find_elements_by_xpath("//div[@class='basic-info cmn-clearfix']/dl/dd")  
    
            #create dictionary key-value 
            #字典是一种散列表结构,数据输入后按特征被散列,不记录原来的数据,顺序建议元组 
            elem_dic = dict(zip(elem_name,elem_value))   
            for key in elem_dic:    
                print (key.text,elem_dic[key].text)    
                info.writelines(key.text+" "+elem_dic[key].text+'\r\n')    
            time.sleep(5)       # 让程序停5秒，以便把信息写入文件
        except Exception as e:  
            print ("Error: ", e)
        finally:
            print('\n')
            info.write('\r\n')  # 每篇信息之后，多一行空行
    
    
    def main():
        global info
        source = open("scenic_spots_5A_namelist.txt", 'r')
        for name in source:
            getInfo(name)
        print ('End Read Files!')
        time.sleep(5)
        source.close()
        info.close()
        driver.close()
    
    
    main()

# 五、运行结果 #

生成了文件夹scenic\_spots\_5A，此文件夹中包含一个名为scenic\_info.txt的文件

![3.png][]

!\[4.png\]([https://upload-images.jianshu.io/upload\_images/6946981-2da7da6e5e885cfd.png][https_upload-images.jianshu.io_upload_images_6946981-2da7da6e5e885cfd.png]? imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

scenice\_info.txt中的内容为

![5.png][]

# 六、参考 #

[https://blog.csdn.net/eastmount/article/details/48234733][https_blog.csdn.net_eastmount_article_details_48234733]

> TopCoder & Codeforces & AtCoder交流QQ群：648202993  
> 更多内容请关注微信公众号  
> ![wechat\_public\_header.jpg][wechat_public_header.jpg]

[1.png]: /images/20220524/32c3b032e5c0441cbb1560aee3f4ab60.png
[2.png]: /images/20220524/0c9045cf29594bd2ba6a71f58ddc9df2.png
[3.png]: /images/20220524/f520bded504d43308f773354821e7d27.png
[https_upload-images.jianshu.io_upload_images_6946981-2da7da6e5e885cfd.png]: https://upload-images.jianshu.io/upload_images/6946981-2da7da6e5e885cfd.png
[5.png]: /images/20220524/052c465778fb4feb996b0c929643ee83.png
[https_blog.csdn.net_eastmount_article_details_48234733]: https://blog.csdn.net/eastmount/article/details/48234733
[wechat_public_header.jpg]: /images/20220524/594985048fb343efadf7c775afe88e2a.png