爬虫入门小程序-java

水深无声 2022-11-29 11:26 158阅读 0赞

来个小程序，先小爬一下。。。

## 1.pom文件 ##

<dependencies>
            <dependency>
                <groupId>org.apache.httpcomponents</groupId>
                <artifactId>httpclient</artifactId>
                <version>4.5.2</version>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
                <version>1.7.25</version>
            </dependency>
        </dependencies>

## 2.properties配置文件 ##

log4j.rootLogger=DEBUG,Console
    log4j.logger.cn.itcast=DEBUG
    #向控制台打印信息
    log4j.appender.Console=org.apache.log4j.ConsoleAppender
    #信息的样式设置
    log4j.appender.logfile.layout=org.apache.log4j.PatternLayout
    log4j.appender.logfile.layout.ConversionPattern=%p %d %F %M  %m%n

## 3.test类 ##

public class CrawlerFirst {
        public static void main(String[] args) throws Exception{
            //1.打开浏览器，HTTPClient对象
            CloseableHttpClient httpClient = HttpClients.createDefault();
            //2.输入网址,发起get请求创建HTTPGet对象
            HttpGet httpGet=new HttpGet("https://blog.csdn.net/luckystar_99");
            //3.按回车，发起请求，返回响应
            CloseableHttpResponse response = httpClient.execute(httpGet);
            //4.解析响应，获取数据，
            //判断状态码是否是200
            if(response.getStatusLine().getStatusCode()==200){
                HttpEntity httpEntity=response.getEntity();
                String content=EntityUtils.toString(httpEntity,"utf8");
                System.out.println(content);
            }
        }
    }

## 4.OK啦！ ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x1Y2t5c3Rhcl85OQ_size_16_color_FFFFFF_t_70_pic_center]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x1Y2t5c3Rhcl85OQ_size_16_color_FFFFFF_t_70_pic_center]: /images/20221124/7484dca75559436cb7ba81527bf724da.png