Scrapy框架之selecter.xpath()函数的语法总结

朴灿烈づ我的快乐病毒、 2021-09-18 03:14 333阅读 0赞

### **Xpath（XML的语法规则）语法规则官方手册：[http://www.w3school.com.cn/xpath/xpath\_syntax.asp][http_www.w3school.com.cn_xpath_xpath_syntax.asp]** ###

### **具体用法总结：（函数部分参考xml手册）** ###

html = '''
            <div>
                <ul>
                <li class="toctree-l1"><a class="reference internal" href="intro/overview.html">Scrapy at a glance</a></li>
                <li class="toctree-l1"><a class="reference internal" href="intro/install.html">Installation guide</a></li>
                <li class="toctree-l1"><a class="reference internal" href="intro/tutorial.html">Scrapy Tutorial</a></li>
                <li class="toctree-l1"><a class="reference internal" href="intro/examples.html">Examples</a></li>
                </ul>
                <p class="caption"><span class="caption-text">Basic concepts</span></p>
                <ul>
                <li class="toctree-l1"><a class="reference internal" href="topics/commands.html">Command line tool</a></li>
                <li class="toctree-l1"><a class="reference internal" href="topics/spiders.html">Spiders</a></li>
                <li class="toctree-l1"><a class="reference internal" href="topics/selectors.html">Selectors</a></li>
                <li class="toctree-l1"><a class="reference internal" href="topics/items.html">Items</a></li>
                <li class="toctree-l1"><a class="reference internal" href="topics/loaders.html">Item Loaders</a></li>
                <li class="toctree-l1"><a class="reference internal" href="topics/shell.html">Scrapy shell</a></li>
                <li class="toctree-l1"><a class="reference internal" href="topics/item-pipeline.html">Item Pipeline</a></li>
                <li class="toctree-l1"><a class="reference internal" href="topics/feed-exports.html">Feed exports</a></li>
                <li class="toctree-l1"><a class="reference internal" href="topics/request-response.html">Requests and Responses</a></li>
                <li class="toctree-l1"><a class="reference internal" href="topics/link-extractors.html">Link Extractors</a></li>
                <li class="toctree-l1"><a class="reference internal" href="topics/settings.html">Settings</a></li>
                <li class="toctree-l1"><a class="reference internal" href="topics/exceptions.html">Exceptions</a></li>
                </ul>
            </div>
    
            '''
    
            sel = Selector(text=html)
    
            xpath是基于XML的路径寻找，"/"代表html根路径
            path001 = sel.xpath("/")
    
            xpath是基于XML的路径寻找，"//"代表html根后代的子元素(html后代所以ul元素)
            path002 = sel.xpath("//ul")
    
            xpath是基于XML的路径寻找，".//"代表当前目录下所以后代的子元素(当前div目录后代所以ul元素)
            path003 = sel.xpath(".//ul")
    
            当前节点元素(div)
            path004 = sel.xpath(".")
    
            ul的父亲：div
            path005 = sel.xpath(".//ul/..")
    
            xpath获取文本值text()函数
            path005 = sel.xpath(".//li/a/text()")
    
            xpath获取属性值函数@属性名
            path005 = sel.xpath(".//li/a/@href")
    
    
            最后:[谓语]的用法,[1]、[last()]、()[1]、((1)[])
    
            xpath[谓语]的用法之一: 不带()的用法
            path = sel.xpath("//ul[2]/li[3]/a/text()")
    
            xpath[谓语]的用法之一: 带()的用法
            特别注意：(//li)[2]如何后面还要查询子元素,必须加()
            node = sel.xpath("( (//li)[2] )/a/text()")
    
            [谓语]运法之last()函数变量运算
            path = sel.xpath("(//li)[last()-2]/a/text()")
    
            [谓语]运法之position()函数变量运算
            path = sel.xpath("(//li)[position() <= 3]/a/text()")

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzM0MzE0NA_size_16_color_FFFFFF_t_70][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzM0MzE0NA_size_16_color_FFFFFF_t_70 1][]

[http_www.w3school.com.cn_xpath_xpath_syntax.asp]: http://www.w3school.com.cn/xpath/xpath_syntax.asp
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzM0MzE0NA_size_16_color_FFFFFF_t_70]: /images/20210812/c338d4621da44429a76a9fd90aaa039b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzM0MzE0NA_size_16_color_FFFFFF_t_70 1]: /images/20210812/bdb85e0e542f478f92a8b21417e7d4bd.png