CSDN 数据访问可视化，写给CSDN 群友们用用

左手的ㄟ右手 2023-06-02 09:28 6阅读 0赞

### 文章目录 ###

*  CSDN 数据访问可视化，写给CSDN 群友们用用
 *   *   *   *  1、大致界面
             *  2、GitHub 项目地址
             *  3、如何使用
             *  4、具体代码

# CSDN 数据访问可视化，写给CSDN 群友们用用 #

--------------------

#### 1、大致界面 ####

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly93cmV0Y2hhbnQuYmxvZy5jc2RuLm5ldA_size_16_color_FFFFFF_t_70]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly93cmV0Y2hhbnQuYmxvZy5jc2RuLm5ldA_size_16_color_FFFFFF_t_70 1]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly93cmV0Y2hhbnQuYmxvZy5jc2RuLm5ldA_size_16_color_FFFFFF_t_70 2]

内页就不贴图了

#### 2、GitHub 项目地址 ####

[这里是传送门：https://github.com/FrankZuozuo/csdn-chart][https_github.com_FrankZuozuo_csdn-chart]

有兴趣的话点个小星星吧，虽然没啥用

#### 3、如何使用 ####

数据库脚本先跑一下，包括建立数据库，建立表

## 创建数据库
    CREATE DATABASE `csdn` /*!40100 COLLATE 'utf8mb4_general_ci' */;
    
    
    ## 创建表
    CREATE TABLE `archival_storage`
    (
        `pk_id`          BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
        `gmt_create`     DATETIME            NOT NULL DEFAULT CURRENT_TIMESTAMP,
        `gmt_modified`   DATETIME            NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
        `remark`         VARCHAR(300)        NOT NULL DEFAULT '',
        `sort`           INT(11) UNSIGNED    NOT NULL DEFAULT '0',
        `deleted`        TINYINT(1) UNSIGNED NOT NULL DEFAULT '0',
        `article_number` INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        `year`           INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        `month`          INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        PRIMARY KEY (`pk_id`),
        UNIQUE INDEX `year_month` (`year`, `month`)
    )
        COLLATE = 'utf8mb4_general_ci'
        ENGINE = InnoDB
        AUTO_INCREMENT = 163
    ;
    
    
    CREATE TABLE `article_info`
    (
        `pk_id`          BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
        `gmt_create`     DATETIME            NOT NULL DEFAULT CURRENT_TIMESTAMP,
        `gmt_modified`   DATETIME            NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
        `remark`         VARCHAR(300)        NOT NULL DEFAULT '',
        `sort`           INT(11) UNSIGNED    NOT NULL DEFAULT '0',
        `deleted`        TINYINT(1) UNSIGNED NOT NULL DEFAULT '0',
        `url`            VARCHAR(500)        NOT NULL DEFAULT '',
        `article_name`   VARCHAR(500)        NOT NULL DEFAULT '',
        `read_number`    INT(11)             NOT NULL DEFAULT '0',
        `comment_number` INT(11)             NOT NULL DEFAULT '0',
        PRIMARY KEY (`pk_id`),
        UNIQUE INDEX `url` (`url`)
    )
        COLLATE = 'utf8mb4_general_ci'
        ENGINE = InnoDB
        AUTO_INCREMENT = 1325
    ;
    
    
    CREATE TABLE `data`
    (
        `pk_id`          BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
        `gmt_create`     DATETIME            NOT NULL DEFAULT CURRENT_TIMESTAMP,
        `gmt_modified`   DATETIME            NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
        `remark`         VARCHAR(300)        NOT NULL DEFAULT '',
        `sort`           INT(11) UNSIGNED    NOT NULL DEFAULT '0',
        `deleted`        TINYINT(1) UNSIGNED NOT NULL DEFAULT '0',
        `level`          INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        `visit_number`   INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        `integral`       INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        `top`            INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        `article_number` INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        `fans`           INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        `like_number`    INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        `comment_number` INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        `time_point`     INT(10) UNSIGNED    NOT NULL DEFAULT '0',
        PRIMARY KEY (`pk_id`)
    )
        COLLATE = 'utf8mb4_general_ci'
        ENGINE = InnoDB
        AUTO_INCREMENT = 197
    ;
    
    
    CREATE TABLE `user_info`
    (
        `pk_id`        BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
        `gmt_create`   DATETIME            NOT NULL DEFAULT CURRENT_TIMESTAMP,
        `gmt_modified` DATETIME            NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
        `remark`       VARCHAR(300)        NOT NULL DEFAULT '',
        `sort`         INT(11) UNSIGNED    NOT NULL DEFAULT '0',
        `deleted`      TINYINT(1) UNSIGNED NOT NULL DEFAULT '0',
        `nickname`     VARCHAR(100)        NOT NULL DEFAULT '',
        `head_img_url` VARCHAR(200)        NOT NULL DEFAULT '',
        `motto`        VARCHAR(300)        NOT NULL DEFAULT '',
        `version`      BIGINT(20)          NOT NULL DEFAULT '0',
        PRIMARY KEY (`pk_id`)
    )
        COLLATE = 'utf8mb4_general_ci'
        ENGINE = InnoDB
        AUTO_INCREMENT = 555
    ;

然后把项目里面的`application.yml`数据库连接信息改成你自己的，博客地址`csdn.csdn-url`改成你自己的就可以了。

spring:
      application:
        name: csdn-chart
      datasource:
        driver-class-name: com.mysql.cj.jdbc.Driver
        url: jdbc:mysql://localhost:3306/csdn?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=GMT%2B8
        username: root
        password: root
      thymeleaf:
        cache: false
      execute:
        core-pool-size: 10
        max-pool-size: 200
        queue-capacity: 10
    
    server:
      port: 9253
    
    
    csdn:
      csdn-url: https://wretchant.blog.csdn.net

报表的数据是每5分钟抓取一次，你可以根据自己的实际情况修改  
50万访问以下的，建议1-2个小时抓取一次，50-500万的，建议半个小时到1个小时抓取一次，500万以上的，建议10-20分钟左右抓取一次

修改一下`DataTimer` 类的 `fixedRate` 即可

#### 4、具体代码 ####

package com.wretchant.csdnchart.annotation;
    
    import java.lang.annotation.*;
    
    @Retention(RetentionPolicy.RUNTIME)
    @Target({ ElementType.METHOD, ElementType.TYPE, ElementType.FIELD})
    @Documented
    public @interface InfoLog { 
    
      String value();
    }

package com.wretchant.csdnchart.configuration;
    
    import org.springframework.context.annotation.Configuration;
    import org.springframework.context.annotation.EnableAspectJAutoProxy;
    
    @EnableAspectJAutoProxy
    @Configuration
    public class AspectConfiguration { }

package com.wretchant.csdnchart.configuration;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import lombok.Setter;
    import org.springframework.boot.context.properties.ConfigurationProperties;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.scheduling.annotation.EnableAsync;
    import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
    
    import java.util.concurrent.Executor;
    
    @ConfigurationProperties("spring.execute")
    @EnableAsync
    @Configuration
    @Setter
    public class ThreadConfiguration { 
    
      private int corePoolSize;
      private int maxPoolSize;
      private int queueCapacity;
    
      @Bean
      @InfoLog("开始设置线程池")
      Executor executor() { 
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(corePoolSize);
        executor.setMaxPoolSize(maxPoolSize);
        executor.setQueueCapacity(queueCapacity);
        return executor;
      }
    }

package com.wretchant.csdnchart.configuration;
    
    import org.springframework.context.annotation.Configuration;
    import org.springframework.scheduling.annotation.EnableScheduling;
    
    @EnableScheduling
    @Configuration
    public class TimerConfiguration { }

package com.wretchant.csdnchart.core;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    
    
    public interface Converter<IN, OUT> { 
    
      @InfoLog("进行对象互转")
      OUT converter(IN in);
    }

package com.wretchant.csdnchart.core;
    
    import lombok.AllArgsConstructor;
    import lombok.Data;
    
    
    @AllArgsConstructor
    @Data
    public class R<T> { 
    
      private T t;
      private String msg;
      private int code;
    
      public static <T> R<T> success(T t) { 
        return new R<T>(t, "success", 200);
      }
    
      public static R<String> success() { 
        return new R<String>("", "success", 200);
      }
    
      public static R<String> fail() { 
        return new R<String>("", "fail", 500);
      }
    
      public static R<String> fail(String msg) { 
        return new R<String>("", msg, 500);
      }
    }

package com.wretchant.csdnchart.ctrl.base;
    
    public abstract class BaseCtrl { }

package com.wretchant.csdnchart.ctrl.business.api;
    
    import com.google.common.collect.Lists;
    import com.google.common.collect.Maps;
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.core.R;
    import com.wretchant.csdnchart.entity.DataTable;
    import com.wretchant.csdnchart.entity.DataTableEnum;
    import com.wretchant.csdnchart.service.DataTableService;
    import org.springframework.web.bind.annotation.GetMapping;
    import org.springframework.web.bind.annotation.PathVariable;
    import org.springframework.web.bind.annotation.RequestMapping;
    import org.springframework.web.bind.annotation.RestController;
    
    import java.time.format.DateTimeFormatter;
    import java.util.Collections;
    import java.util.List;
    import java.util.Map;
    
    @RestController
    @RequestMapping("/api/data")
    public class DataCtrl { 
    
      private final DataTableService dataTableService;
    
      public DataCtrl(DataTableService dataTableService) { 
        this.dataTableService = dataTableService;
      }
    
      @InfoLog("前端获取统计数据")
      @GetMapping("/t/{size}")
      public R t(DataTableEnum dataTableEnum, @PathVariable(name = "size") Integer size) { 
    
        List<DataTable> list = dataTableService.list(size);
        Map<String, Object> map = Maps.newHashMapWithExpectedSize(2);
        List<String> datetime = Lists.newArrayListWithExpectedSize(size);
        List<Integer> data = Lists.newArrayListWithExpectedSize(size);
    
        list.forEach(
            dataTable -> { 
              String pattern = "yyyy-MM-dd HH:mm:ss";
              String format = DateTimeFormatter.ofPattern(pattern).format(dataTable.getGmtCreate());
              datetime.add(format);
    
              switch (dataTableEnum) { 
                case TOP:
                  data.add(dataTable.getTop());
                  break;
                case FANS:
                  data.add(dataTable.getFans());
                  break;
                case LEVEL:
                  data.add(dataTable.getLevel());
                  break;
                case INTEGRAL:
                  data.add(dataTable.getIntegral());
                  break;
                case LIKE_NUMBER:
                  data.add(dataTable.getLikeNumber());
                  break;
                case VISIT_NUMBER:
                  data.add(dataTable.getVisitNumber());
                  break;
                case ARTICLE_NUMBER:
                  data.add(dataTable.getArticleNumber());
                  break;
                case COMMENT_NUMBER:
                  data.add(dataTable.getCommentNumber());
                  break;
                default:
              }
            });
    
        Integer max = Collections.max(data);
        Integer min = Collections.min(data);
        map.put("max", max);
        map.put("min", min);
        map.put("datetime", Lists.reverse(datetime));
        map.put("data", Lists.reverse(data));
        return R.success(map);
      }
    }

package com.wretchant.csdnchart.ctrl.business.web;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.service.DataTableService;
    import org.springframework.stereotype.Controller;
    import org.springframework.ui.Model;
    import org.springframework.web.bind.annotation.GetMapping;
    import org.springframework.web.bind.annotation.PathVariable;
    import org.springframework.web.bind.annotation.RequestMapping;
    
    @RequestMapping("/web/add")
    @Controller
    public class WebAddCtrl { 
    
      private final DataTableService dataTableService;
    
      public WebAddCtrl(DataTableService dataTableService) { 
        this.dataTableService = dataTableService;
      }
    
      @InfoLog("查看数据增量")
      @GetMapping("/{size}")
      public String add(Model model, @PathVariable(name = "size") Integer size) { 
        model.addAttribute("list", dataTableService.count(size));
        return "add";
      }
    }

package com.wretchant.csdnchart.ctrl.business.web;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.service.ArticleInfoService;
    import org.springframework.stereotype.Controller;
    import org.springframework.ui.Model;
    import org.springframework.web.bind.annotation.GetMapping;
    import org.springframework.web.bind.annotation.RequestMapping;
    
    @RequestMapping("/web/article")
    @Controller
    public class WebArticleCtrl { 
    
      private final ArticleInfoService articleInfoService;
    
      public WebArticleCtrl(ArticleInfoService articleInfoService) { 
        this.articleInfoService = articleInfoService;
      }
    
      @GetMapping
      @InfoLog("进入报表页面")
      public String chart(Model model) { 
        model.addAttribute("list", articleInfoService.list());
        return "article";
      }
    }

package com.wretchant.csdnchart.ctrl.business.web;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import org.springframework.stereotype.Controller;
    import org.springframework.ui.Model;
    import org.springframework.web.bind.annotation.GetMapping;
    import org.springframework.web.bind.annotation.PathVariable;
    import org.springframework.web.bind.annotation.RequestMapping;
    
    @RequestMapping("/web/chart")
    @Controller
    public class WebChartCtrl { 
    
      @GetMapping("/{size}")
      @InfoLog("进入报表页面")
      public String chart(Model model, @PathVariable(name = "size") Integer size) { 
        model.addAttribute("size", size);
        return "chart";
      }
    }

package com.wretchant.csdnchart.ctrl.business.web;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.entity.ConduitEnum;
    import com.wretchant.csdnchart.service.ArticleInfoService;
    import org.springframework.stereotype.Controller;
    import org.springframework.ui.Model;
    import org.springframework.web.bind.annotation.GetMapping;
    import org.springframework.web.bind.annotation.PathVariable;
    import org.springframework.web.bind.annotation.RequestMapping;
    
    @RequestMapping("/web/conduit")
    @Controller
    public class WebConduitCtrl { 
    
      private final ArticleInfoService articleInfoService;
    
      public WebConduitCtrl(ArticleInfoService articleInfoService) { 
        this.articleInfoService = articleInfoService;
      }
    
      @InfoLog("查看Conduit 文章列表")
      @GetMapping("/{conduit}")
      public String add(Model model, @PathVariable(name = "conduit") ConduitEnum conduit) { 
        model.addAttribute("list", articleInfoService.list(conduit));
        return "article";
      }
    }

package com.wretchant.csdnchart.ctrl.business.web;
    
    import cn.hutool.core.bean.BeanUtil;
    import com.google.common.collect.Lists;
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.entity.DataTable;
    import com.wretchant.csdnchart.entity.DataTableVo;
    import com.wretchant.csdnchart.service.DataTableService;
    import lombok.extern.slf4j.Slf4j;
    import org.springframework.ui.Model;
    import org.springframework.web.bind.annotation.GetMapping;
    import org.springframework.web.bind.annotation.RequestMapping;
    import org.springframework.web.bind.annotation.RestController;
    
    import java.util.List;
    
    @RequestMapping("/web/data")
    @Slf4j
    @RestController
    public class WebDataCtrl { 
    
      private final DataTableService dataTableService;
    
      public WebDataCtrl(DataTableService dataTableService) { 
        this.dataTableService = dataTableService;
      }
    
      @InfoLog("前端查看全部的统计数据")
      @GetMapping
      public String index(Model model) { 
    
        List<DataTableVo> vos = Lists.newArrayList();
        List<DataTable> list = dataTableService.list();
    
        for (int i = 0; i < list.size(); i++) { 
          DataTable dataTable = list.get(i);
          DataTableVo dataTableVo = new DataTableVo();
    
          BeanUtil.copyProperties(dataTable, dataTableVo);
          if (i != 0) { 
            DataTable old = list.get(i - 1);
            dataTableVo.setLevelAdd(dataTable.getLevel() - old.getLevel());
            dataTableVo.setVisitNumberAdd(dataTable.getVisitNumber() - old.getVisitNumber());
            dataTableVo.setIntegralAdd(dataTable.getIntegral() - old.getIntegral());
            dataTableVo.setTopAdd(dataTable.getTop() - old.getTop());
    
            dataTableVo.setArticleNumberAdd(dataTable.getArticleNumber() - old.getArticleNumber());
            dataTableVo.setFansAdd(dataTable.getFans() - old.getFans());
            dataTableVo.setLikeNumberAdd(dataTable.getLikeNumber() - old.getLikeNumber());
            dataTableVo.setCommentNumberAdd(dataTable.getCommentNumber() - old.getCommentNumber());
          }
    
          vos.add(dataTableVo);
        }
    
        model.addAttribute("list", vos);
        return "data";
      }
    }

package com.wretchant.csdnchart.ctrl.business;
    
    import com.google.common.collect.Lists;
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.entity.ArchivalStorage;
    import com.wretchant.csdnchart.service.ArchivalStorageService;
    import com.wretchant.csdnchart.service.ArticleInfoService;
    import com.wretchant.csdnchart.service.DataTableService;
    import com.wretchant.csdnchart.service.UserInfoService;
    import org.springframework.stereotype.Controller;
    import org.springframework.ui.Model;
    import org.springframework.web.bind.annotation.GetMapping;
    import org.springframework.web.bind.annotation.RequestMapping;
    
    import java.util.Collections;
    import java.util.List;
    
    @RequestMapping("/")
    @Controller
    public class IndexCtrl { 
    
      private final ArchivalStorageService archivalStorageService;
      private final ArticleInfoService articleInfoService;
      private final DataTableService dataTableService;
      private final UserInfoService userInfoService;
    
      public IndexCtrl(
          UserInfoService userInfoService,
          ArchivalStorageService archivalStorageService,
          ArticleInfoService articleInfoService,
          DataTableService dataTableService) { 
        this.userInfoService = userInfoService;
        this.archivalStorageService = archivalStorageService;
        this.articleInfoService = articleInfoService;
        this.dataTableService = dataTableService;
      }
    
      @InfoLog("进入首页")
      @GetMapping
      public String index(Model model) { 
    
        model.addAttribute("userInfo", userInfoService.findTop());
    
        List<ArchivalStorage> list = archivalStorageService.list();
        List<String> dates = Lists.newArrayListWithExpectedSize(list.size());
        List<Integer> datas = Lists.newArrayListWithExpectedSize(list.size());
        list = Lists.reverse(list);
        list.forEach(
            archivalStorage -> { 
              datas.add(archivalStorage.getArticleNumber());
              dates.add(archivalStorage.getYear() + "." + archivalStorage.getMonth());
            });
    
        model.addAttribute("userInfo", userInfoService.findTop());
        model.addAttribute("dates", dates);
        model.addAttribute("datas", datas);
        model.addAttribute("max", Collections.max(datas));
        model.addAttribute("min", Collections.min(datas));
    
        model.addAttribute("last", dataTableService.last());
        model.addAttribute("count", articleInfoService.count());
        return "index";
      }
    }

package com.wretchant.csdnchart.entity;
    
    import lombok.Data;
    import org.hibernate.annotations.DynamicInsert;
    import org.hibernate.annotations.DynamicUpdate;
    
    import javax.persistence.*;
    import java.io.Serializable;
    import java.time.LocalDateTime;
    
    @DynamicUpdate
    @DynamicInsert
    @Table(name = "archival_storage")
    @Entity
    @Data
    public class ArchivalStorage implements Serializable { 
    
      private static final long serialVersionUID = -5445657205714414658L;
    
      @Id
      @GeneratedValue(strategy = GenerationType.IDENTITY)
      private Long pkId;
    
      private LocalDateTime gmtCreate;
      private LocalDateTime gmtModified;
      private Integer sort;
      private String remark;
      private Boolean deleted;
    
      /** 文章数量 */
      private Integer articleNumber;
    
      private Integer year;
      private Integer month;
    }

package com.wretchant.csdnchart.entity;
    
    import lombok.Data;
    import org.hibernate.annotations.DynamicInsert;
    import org.hibernate.annotations.DynamicUpdate;
    
    import javax.persistence.*;
    import java.io.Serializable;
    import java.time.LocalDateTime;
    
    @DynamicUpdate
    @DynamicInsert
    @Table(name = "article_info")
    @Entity
    @Data
    public class ArticleInfo implements Serializable { 
    
      private static final long serialVersionUID = -8837487575308630301L;
    
      @Id
      @GeneratedValue(strategy = GenerationType.IDENTITY)
      private Long pkId;
    
      private LocalDateTime gmtCreate;
      private LocalDateTime gmtModified;
      private Integer sort;
      private String remark;
      private Boolean deleted;
    
      /** 文章名称 */
      private String articleName;
      /** 文章地址 */
      private String url;
      /** 阅读数 */
      private Integer readNumber;
      /** 评论数 */
      private Integer commentNumber;
    }

package com.wretchant.csdnchart.entity;
    
    public enum ConduitEnum { 
      VISIT_MOST,
      COMMENT_MOST,
      NEW_MOST
    }

package com.wretchant.csdnchart.entity;
    
    import lombok.Data;
    import org.springframework.boot.context.properties.ConfigurationProperties;
    import org.springframework.context.annotation.Configuration;
    
    @Configuration
    @Data
    @ConfigurationProperties("csdn")
    public class CsdnConfig { 
    
      /** csdn 博客主页的链接 */
      private String csdnUrl;
    
      /** 爬取文章数据时，需要的Restful url 路径 */
      public static final String ARTICLE_LIST = "/article/list/";
    }

package com.wretchant.csdnchart.entity;
    
    import com.google.common.collect.Lists;
    import lombok.Data;
    import org.hibernate.annotations.DynamicInsert;
    import org.hibernate.annotations.DynamicUpdate;
    
    import javax.persistence.*;
    import java.io.Serializable;
    import java.lang.reflect.Field;
    import java.time.LocalDateTime;
    import java.util.List;
    
    @DynamicUpdate
    @DynamicInsert
    @Table(name = "data")
    @Entity
    @Data
    public class DataTable implements Serializable { 
    
      private static final long serialVersionUID = -2669109119732793870L;
    
      @Id
      @GeneratedValue(strategy = GenerationType.IDENTITY)
      private Long pkId;
    
      private LocalDateTime gmtCreate;
      private LocalDateTime gmtModified;
      private Integer sort;
      private String remark;
      private Boolean deleted;
    
      /** 时间点 */
      private Integer timePoint;
    
      /** 等级 */
      private Integer level;
      /** 访问量 */
      private Integer visitNumber;
      /** 积分 */
      private Integer integral;
      /** 排名 */
      private Integer top;
      /** 原创文章数量 */
      private Integer articleNumber;
      /** 粉丝数量 */
      private Integer fans;
      /** 喜欢数量 */
      private Integer likeNumber;
      /** 评论数量 */
      private Integer commentNumber;
    
      public static void main(String[] args) { 
        Field[] fields = DataTableViewEnum.class.getDeclaredFields();
        List<Field> fs = Lists.newArrayList(fields);
        fs.stream()
            .filter(field -> !"$VALUES".equals(field.getName()) && !"field".equals(field.getName()))
            .forEach(
                field -> { 
                  try { 
                    DataTableViewEnum dataTableViewEnum =
                        (DataTableViewEnum) field.get(field.getName());
                    System.out.println(dataTableViewEnum.getField());
                  } catch (IllegalAccessException e) { 
                    e.printStackTrace();
                  }
                  System.out.println(field.getName());
                });
      }
    }

package com.wretchant.csdnchart.entity;
    
    import lombok.Data;
    
    @Data
    public class DataTableDto { 
      /** 等级 */
      private Integer level;
      /** 访问量 */
      private Integer visitNumber;
      /** 积分 */
      private Integer integral;
      /** 排名 */
      private Integer top;
      /** 原创文章数量 */
      private Integer articleNumber;
      /** 粉丝数量 */
      private Integer fans;
      /** 喜欢数量 */
      private Integer likeNumber;
      /** 评论数量 */
      private Integer commentNumber;
    }

package com.wretchant.csdnchart.entity;
    
    import cn.hutool.core.bean.BeanUtil;
    import com.wretchant.csdnchart.core.Converter;
    
    import java.time.LocalDateTime;
    
    public class DataTableDto2DataTableConverter implements Converter<DataTableDto, DataTable> { 
      @Override
      public DataTable converter(DataTableDto dataTableDto) { 
        DataTable dataTable = new DataTable();
        BeanUtil.copyProperties(dataTableDto, dataTable);
    
        LocalDateTime now = LocalDateTime.now();
        int hour = now.getHour();
        dataTable.setTimePoint(hour);
    
        return dataTable;
      }
    }

package com.wretchant.csdnchart.entity;
    
    public enum DataTableEnum { 
      LEVEL,
      VISIT_NUMBER,
      INTEGRAL,
      TOP,
      ARTICLE_NUMBER,
      FANS,
      LIKE_NUMBER,
      COMMENT_NUMBER
    }

package com.wretchant.csdnchart.entity;
    
    import lombok.Getter;
    
    @Getter
    public enum DataTableViewEnum { 
      LEVEL("level"),
      VISIT_NUMBER("visitNumber"),
      INTEGRAL("integral"),
      TOP("top"),
      ARTICLE_NUMBER("articleNumber"),
      FANS("fans"),
      LIKE_NUMBER("likeNumber"),
      COMMENT_NUMBER("commentNumber");
    
      private String field;
    
      DataTableViewEnum(String field) { 
        this.field = field;
      }
    }

package com.wretchant.csdnchart.entity;
    
    import lombok.Data;
    
    import java.time.LocalDateTime;
    
    @Data
    public class DataTableVo { 
    
      private LocalDateTime gmtCreate;
    
      /** 等级 */
      private Integer level;
      /** 访问量 */
      private Integer visitNumber;
      /** 积分 */
      private Integer integral;
      /** 排名 */
      private Integer top;
      /** 原创文章数量 */
      private Integer articleNumber;
      /** 粉丝数量 */
      private Integer fans;
      /** 喜欢数量 */
      private Integer likeNumber;
      /** 评论数量 */
      private Integer commentNumber;
    
      // ---- 增量
      /** 等级 */
      private Integer levelAdd = 0;
      /** 访问量 */
      private Integer visitNumberAdd = 0;
      /** 积分 */
      private Integer integralAdd = 0;
      /** 排名 */
      private Integer topAdd = 0;
      /** 原创文章数量 */
      private Integer articleNumberAdd = 0;
      /** 粉丝数量 */
      private Integer fansAdd = 0;
      /** 喜欢数量 */
      private Integer likeNumberAdd = 0;
      /** 评论数量 */
      private Integer commentNumberAdd = 0;
    }

package com.wretchant.csdnchart.entity;
    
    import lombok.Data;
    import org.hibernate.annotations.DynamicInsert;
    import org.hibernate.annotations.DynamicUpdate;
    
    import javax.persistence.*;
    import java.io.Serializable;
    import java.time.LocalDateTime;
    
    @DynamicUpdate
    @DynamicInsert
    @Table(name = "user_info")
    @Entity
    @Data
    public class UserInfo implements Serializable { 
    
      private static final long serialVersionUID = 1118676562861971250L;
    
      @Id
      @GeneratedValue(strategy = GenerationType.IDENTITY)
      private Long pkId;
    
      private LocalDateTime gmtCreate;
      private LocalDateTime gmtModified;
      private Integer sort;
      private String remark;
      private Boolean deleted;
    
      private String nickname;
      private String headImgUrl;
      private String motto;
      private Long version;
    }

package com.wretchant.csdnchart.framework.aop;
    
    import cn.hutool.core.date.DateUtil;
    import cn.hutool.core.date.TimeInterval;
    import com.wretchant.csdnchart.annotation.InfoLog;
    import lombok.extern.slf4j.Slf4j;
    import org.aspectj.lang.ProceedingJoinPoint;
    import org.aspectj.lang.annotation.Around;
    import org.aspectj.lang.annotation.Aspect;
    import org.aspectj.lang.annotation.Pointcut;
    import org.aspectj.lang.reflect.MethodSignature;
    import org.springframework.stereotype.Component;
    
    import java.lang.reflect.Method;
    
    @Slf4j
    @Component
    @Aspect
    public class InfoLogAspect { 
    
      @Pointcut("@annotation(com.wretchant.csdnchart.annotation.InfoLog)")
      public void infoLog() { }
    
      @Around("infoLog()")
      public Object before(ProceedingJoinPoint joinPoint) throws Throwable { 
    
        MethodSignature signature = (MethodSignature) joinPoint.getSignature();
        Method method = signature.getMethod();
        InfoLog annotation = method.getAnnotation(InfoLog.class);
        String value = annotation.value();
    
        TimeInterval timer = DateUtil.timer();
        Object proceed = joinPoint.proceed();
    
        if (log.isInfoEnabled()) { 
          log.info("[{}] : [{}]", timer.interval(), value);
        }
        return proceed;
      }
    }

package com.wretchant.csdnchart.framework;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.entity.ArchivalStorage;
    import com.wretchant.csdnchart.entity.CsdnConfig;
    import com.wretchant.csdnchart.service.ArchivalStorageService;
    import com.wretchant.csdnchart.service.ConnService;
    import lombok.extern.slf4j.Slf4j;
    import org.apache.commons.lang3.StringUtils;
    import org.jsoup.nodes.Document;
    import org.jsoup.select.Elements;
    import org.springframework.boot.ApplicationArguments;
    import org.springframework.boot.ApplicationRunner;
    import org.springframework.stereotype.Component;
    
    @Slf4j
    @Component
    public class ArchivalStorageInitRunner implements ApplicationRunner { 
    
      private final ArchivalStorageService archivalStorageService;
      private final CsdnConfig csdnConfig;
      private final ConnService connService;
    
      public ArchivalStorageInitRunner(
          CsdnConfig csdnConfig,
          ConnService connService,
          ArchivalStorageService archivalStorageService) { 
        this.csdnConfig = csdnConfig;
        this.connService = connService;
        this.archivalStorageService = archivalStorageService;
      }
    
      @InfoLog("抓取文章在每月的归档存储数据")
      @Override
      public void run(ApplicationArguments args) throws Exception { 
    
        Document document = connService.conn(csdnConfig.getCsdnUrl());
        Elements elements = document.body().select(".archive-list");
        Elements select = elements.select("ul a");
        select.forEach(
            element -> { 
              String href = element.attr("href");
              if (StringUtils.isNotBlank(href)) { 
                String datetime = href.substring(href.length() - 7);
    
                String[] split = datetime.split("/");
                Integer year = Integer.valueOf(split[0]);
                Integer month = Integer.valueOf(split[1]);
    
                String span = element.select("span").get(0).html();
                if (StringUtils.isNotBlank(span)) { 
                  String articleNumber = span.substring(0, span.length() - 1);
                  create(Integer.valueOf(articleNumber), year, month);
                }
              }
            });
      }
    
      private void create(Integer articleNumber, Integer year, Integer month) { 
    
        ArchivalStorage archivalStorage = new ArchivalStorage();
        archivalStorage.setArticleNumber(articleNumber);
        archivalStorage.setYear(year);
        archivalStorage.setMonth(month);
    
        archivalStorageService.create(archivalStorage);
      }
    }

package com.wretchant.csdnchart.framework;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.entity.ArticleInfo;
    import com.wretchant.csdnchart.entity.CsdnConfig;
    import com.wretchant.csdnchart.service.ArticleInfoService;
    import com.wretchant.csdnchart.service.ConnService;
    import lombok.extern.slf4j.Slf4j;
    import org.jsoup.nodes.Document;
    import org.jsoup.select.Elements;
    import org.springframework.boot.ApplicationArguments;
    import org.springframework.boot.ApplicationRunner;
    import org.springframework.stereotype.Component;
    
    @Slf4j
    @Component
    public class ArticleInfoInitRunner implements ApplicationRunner { 
    
      /** 在不知道作者有多少篇的文章的时候，设定一个最大值，即20000篇文章 */
      private static final int PAGE_MAX = 1000;
    
      private final ArticleInfoService articleInfoService;
      private final CsdnConfig csdnConfig;
      private final ConnService connService;
    
      public ArticleInfoInitRunner(
          CsdnConfig csdnConfig, ArticleInfoService articleInfoService, ConnService connService) { 
        this.csdnConfig = csdnConfig;
        this.articleInfoService = articleInfoService;
        this.connService = connService;
      }
    
      @InfoLog("抓取全部的文章数据")
      @Override
      public void run(ApplicationArguments args) throws Exception { 
    
        // 页数从1开始
        for (int i = 1; i < PAGE_MAX; i++) { 
    
          String s = csdnConfig.getCsdnUrl() + CsdnConfig.ARTICLE_LIST + i;
          Document document = connService.conn(s);
          Elements elements = document.body().select(".article-list");
          Elements select = elements.select(".article-item-box");
          if (select.isEmpty()) { 
            // 当最后一页结束时，全循环结束
            break;
          }
          select.forEach(
              element -> { 
                Elements h4A = element.select("h4 a");
                String url = h4A.attr("href");
                String articleName = h4A.text();
    
                Elements numbers = element.select(".num");
                String read = numbers.get(0).html();
                String comment = numbers.get(1).html();
    
                create(articleName, url, Integer.valueOf(read), Integer.valueOf(comment));
              });
        }
      }
    
      private void create(String articleName, String url, Integer read, Integer comment) { 
        ArticleInfo articleInfo = new ArticleInfo();
    
        articleInfo.setArticleName(articleName);
        articleInfo.setUrl(url);
        articleInfo.setReadNumber(read);
        articleInfo.setCommentNumber(comment);
        articleInfoService.create(articleInfo);
      }
    }

package com.wretchant.csdnchart.framework;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.entity.CsdnConfig;
    import com.wretchant.csdnchart.entity.UserInfo;
    import com.wretchant.csdnchart.service.ConnService;
    import com.wretchant.csdnchart.service.UserInfoService;
    import org.jsoup.nodes.Document;
    import org.jsoup.select.Elements;
    import org.springframework.boot.ApplicationArguments;
    import org.springframework.boot.ApplicationRunner;
    import org.springframework.stereotype.Component;
    
    @Component
    public class UserInfoInitRunner implements ApplicationRunner { 
    
      private final CsdnConfig csdnConfig;
      private final UserInfoService userInfoService;
      private final ConnService connService;
    
      public UserInfoInitRunner(
          CsdnConfig csdnConfig, UserInfoService userInfoService, ConnService connService) { 
        this.csdnConfig = csdnConfig;
        this.userInfoService = userInfoService;
        this.connService = connService;
      }
    
      @InfoLog("抓取用户的个人信息")
      @Override
      public void run(ApplicationArguments args) throws Exception { 
    
        Document document = connService.conn(csdnConfig.getCsdnUrl());
        Elements elements = document.body().select(".title-box");
        String nickname = elements.select("a").get(0).html();
        String motto = elements.select("p").get(0).html();
    
        Elements select = document.body().select(".avatar_pic");
        String src = select.get(0).attr("src");
    
        // 因为这里抓的头像图片太小 - 只有三分之一大小，不好看，我们把它变成大的
        String headImgUrl = src.replace("3_", "1_");
    
        create(nickname, motto, headImgUrl);
      }
    
      private void create(String nickname, String motto, String headImgUrl) { 
    
        UserInfo userInfo = new UserInfo();
    
        userInfo.setHeadImgUrl(headImgUrl);
        userInfo.setNickname(nickname);
        userInfo.setMotto(motto);
    
        userInfoService.create(userInfo);
      }
    }

package com.wretchant.csdnchart.repo;
    
    import com.wretchant.csdnchart.entity.ArchivalStorage;
    import org.springframework.data.jpa.repository.JpaRepository;
    
    import java.util.Optional;
    
    public interface ArchivalStorageRepo extends JpaRepository<ArchivalStorage, Long> { 
    
      Optional<ArchivalStorage> findByYearAndMonth(Integer year, Integer month);
    }

package com.wretchant.csdnchart.repo;
    
    import com.wretchant.csdnchart.entity.ArticleInfo;
    import org.springframework.data.jpa.repository.JpaRepository;
    import org.springframework.data.jpa.repository.Query;
    
    import java.util.List;
    import java.util.Optional;
    
    public interface ArticleInfoRepo extends JpaRepository<ArticleInfo, Long> { 
    
      Optional<ArticleInfo> findByUrl(String url);
    
      @Query(
          nativeQuery = true,
          value = "select * from article_info a order by a.read_number desc limit ?1")
      List<ArticleInfo> listOrderByReadNumberLimit(Integer limit);
    
      @Query(
          nativeQuery = true,
          value = "select * from article_info a order by a.comment_number desc limit ?1")
      List<ArticleInfo> listOrderByCommentNumberLimit(Integer limit);
    
      @Query(
          nativeQuery = true,
          value = "select * from article_info a order by a.gmt_create desc limit ?1")
      List<ArticleInfo> listOrderByGmtCreateLimit(Integer limit);
    }

package com.wretchant.csdnchart.repo;
    
    import com.wretchant.csdnchart.entity.DataTable;
    import org.springframework.data.jpa.repository.JpaRepository;
    import org.springframework.data.jpa.repository.Query;
    
    import java.util.List;
    
    public interface DataTableRepo extends JpaRepository<DataTable, Long> { 
    
      @Query(nativeQuery = true, value = "select * from data d order by d.gmt_create desc limit ?1")
      List<DataTable> list(Integer limit);
    
      @Query(
          nativeQuery = true,
          value =
              "SELECT ANY_VALUE(DATE_FORMAT(d.gmt_create,'%Y-%m-%d')) AS `gmt_create`,\n"
                  + "ANY_VALUE(d.pk_id) AS `pk_id`,\n"
                  + "ANY_VALUE(d.deleted) AS `deleted`,\n"
                  + "ANY_VALUE(d.gmt_modified) AS `gmt_modified`,\n"
                  + "ANY_VALUE(d.remark) AS `remark`,\n"
                  + "ANY_VALUE(d.sort) AS `sort`,\n"
                  + "d.time_point,\n"
                  + "ANY_VALUE((MAX(d.visit_number) - MIN(d.visit_number))) AS `visit_number`,\n"
                  + "ANY_VALUE((MAX(d.level) - MIN(d.level))) AS `level`,\n"
                  + "ANY_VALUE((MAX(d.integral) - MIN(d.integral))) AS `integral`,\n"
                  + "ANY_VALUE((MAX(d.top) - MIN(d.top))) AS `top`,\n"
                  + "ANY_VALUE((MAX(d.article_number) - MIN(d.article_number))) AS `article_number`,\n"
                  + "ANY_VALUE((MAX(d.fans) - MIN(d.fans))) AS `fans`,\n"
                  + "ANY_VALUE((MAX(d.like_number) - MIN(d.like_number))) AS `like_number`,\n"
                  + "ANY_VALUE((MAX(d.comment_number) - MIN(d.comment_number))) AS `comment_number`\n"
                  + "FROM `data` d\n"
                  + "GROUP BY d.time_point\n"
                  + "ORDER BY gmt_create DESC, d.time_point DESC\n"
                  + "LIMIT ?1")
      List<DataTable> count(Integer limit);
    }

package com.wretchant.csdnchart.repo;
    
    import com.wretchant.csdnchart.entity.UserInfo;
    import org.springframework.data.jpa.repository.JpaRepository;
    import org.springframework.data.jpa.repository.Query;
    
    public interface UserInfoRepo extends JpaRepository<UserInfo, Long> { 
    
      @Query(nativeQuery = true, value = "select * from user_info u order by u.version desc limit 1")
      UserInfo findTop();
    }

package com.wretchant.csdnchart.service;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.entity.ArchivalStorage;
    
    import java.util.List;
    
    public interface ArchivalStorageService { 
    
      @InfoLog("创建文章数据归档统计")
      ArchivalStorage create(ArchivalStorage archivalStorage);
    
      @InfoLog("查询全部的文章归档数据")
      List<ArchivalStorage> list();
    }

package com.wretchant.csdnchart.service;
    
    import com.wretchant.csdnchart.entity.ArchivalStorage;
    import com.wretchant.csdnchart.repo.ArchivalStorageRepo;
    import org.springframework.stereotype.Service;
    
    import java.util.List;
    import java.util.Optional;
    
    @Service
    public class ArchivalStorageServiceImpl implements ArchivalStorageService { 
    
      private final ArchivalStorageRepo archivalStorageRepo;
    
      public ArchivalStorageServiceImpl(ArchivalStorageRepo archivalStorageRepo) { 
        this.archivalStorageRepo = archivalStorageRepo;
      }
    
      @Override
      public ArchivalStorage create(ArchivalStorage archivalStorage) { 
    
        Integer year = archivalStorage.getYear();
        Integer month = archivalStorage.getMonth();
        Optional<ArchivalStorage> dbOne = archivalStorageRepo.findByYearAndMonth(year, month);
    
        // 有则更新，无则创建
        if (dbOne.isPresent()) { 
          ArchivalStorage storage = dbOne.get();
          if (archivalStorage.getArticleNumber().equals(storage.getArticleNumber())) { 
            return storage;
          }
          storage.setArticleNumber(archivalStorage.getArticleNumber());
          return archivalStorageRepo.saveAndFlush(storage);
        }
        return archivalStorageRepo.saveAndFlush(archivalStorage);
      }
    
      @Override
      public List<ArchivalStorage> list() { 
        return archivalStorageRepo.findAll();
      }
    }

package com.wretchant.csdnchart.service;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.entity.ArticleInfo;
    import com.wretchant.csdnchart.entity.ConduitEnum;
    
    import java.util.List;
    
    public interface ArticleInfoService { 
    
      @InfoLog("创建文章信息")
      ArticleInfo create(ArticleInfo articleInfo);
    
      @InfoLog("查询全部的文章")
      List<ArticleInfo> list();
    
      @InfoLog("统计文章一共有多少篇")
      long count();
    
      @InfoLog("service：查看Conduit 文章列表")
      List<ArticleInfo> list(ConduitEnum conduit);
    }

package com.wretchant.csdnchart.service;
    
    import com.google.common.collect.Lists;
    import com.wretchant.csdnchart.entity.ArticleInfo;
    import com.wretchant.csdnchart.entity.ConduitEnum;
    import com.wretchant.csdnchart.repo.ArticleInfoRepo;
    import org.springframework.stereotype.Service;
    
    import java.util.List;
    import java.util.Optional;
    
    @Service
    public class ArticleInfoServiceImpl implements ArticleInfoService { 
    
      private final ArticleInfoRepo articleInfoRepo;
    
      public ArticleInfoServiceImpl(ArticleInfoRepo articleInfoRepo) { 
        this.articleInfoRepo = articleInfoRepo;
      }
    
      @Override
      public ArticleInfo create(ArticleInfo articleInfo) { 
    
        Optional<ArticleInfo> byUrl = articleInfoRepo.findByUrl(articleInfo.getUrl());
        if (byUrl.isPresent()) { 
          ArticleInfo article = byUrl.get();
          boolean b =
              article.getCommentNumber().equals(articleInfo.getCommentNumber())
                  && article.getReadNumber().equals(articleInfo.getReadNumber());
          if (b) { 
            return article;
          }
    
          article.setCommentNumber(articleInfo.getCommentNumber());
          article.setReadNumber(articleInfo.getReadNumber());
          return articleInfoRepo.saveAndFlush(article);
        }
        return articleInfoRepo.saveAndFlush(articleInfo);
      }
    
      @Override
      public List<ArticleInfo> list() { 
        return articleInfoRepo.findAll();
      }
    
      @Override
      public long count() { 
        return articleInfoRepo.count();
      }
    
      @Override
      public List<ArticleInfo> list(ConduitEnum conduit) { 
        int number = 10;
        switch (conduit) { 
          case NEW_MOST:
            return articleInfoRepo.listOrderByGmtCreateLimit(number);
          case VISIT_MOST:
            return articleInfoRepo.listOrderByReadNumberLimit(number);
          case COMMENT_MOST:
            return articleInfoRepo.listOrderByCommentNumberLimit(number);
          default:
        }
        return Lists.newArrayList();
      }
    }

package com.wretchant.csdnchart.service;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import org.jsoup.nodes.Document;
    
    public interface ConnService { 
    
      @InfoLog("开始抓取数据")
      Document conn(String url) throws Exception;
    }

package com.wretchant.csdnchart.service;
    
    import org.jsoup.Connection;
    import org.jsoup.Jsoup;
    import org.jsoup.nodes.Document;
    import org.springframework.stereotype.Service;
    
    @Service
    public class ConnServiceImpl implements ConnService { 
    
      @Override
      public Document conn(String url) throws Exception { 
    
        Connection conn = Jsoup.connect(url);
        conn.header(
            "Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
        conn.header("Accept-Encoding", "gzip, deflate, sdch");
        conn.header("Accept-Language", "zh-CN,zh;q=0.8");
        conn.header(
            "User-Agent",
            "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36");
        return conn.get();
      }
    }

package com.wretchant.csdnchart.service;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.entity.DataTable;
    import com.wretchant.csdnchart.entity.DataTableDto;
    
    import java.util.List;
    
    public interface DataTableService { 
    
      @InfoLog("爬取数据入库")
      DataTable create(DataTableDto dataTableDto);
    
      @InfoLog("查询所有的统计数据")
      List<DataTable> list();
    
      @InfoLog("查询指定数量的统计数据")
      List<DataTable> list(Integer limit);
    
      @InfoLog("查询指标数据增量")
      List<DataTable> count(Integer limit);
    
      @InfoLog("查询最近的一条数据")
      DataTable last();
    }

package com.wretchant.csdnchart.service;
    
    import com.wretchant.csdnchart.core.Converter;
    import com.wretchant.csdnchart.entity.DataTable;
    import com.wretchant.csdnchart.entity.DataTableDto;
    import com.wretchant.csdnchart.entity.DataTableDto2DataTableConverter;
    import com.wretchant.csdnchart.repo.DataTableRepo;
    import org.springframework.stereotype.Service;
    
    import java.util.List;
    
    @Service
    public class DataTableServiceImpl implements DataTableService { 
    
      private final DataTableRepo dataTableRepo;
    
      public DataTableServiceImpl(DataTableRepo dataTableRepo) { 
        this.dataTableRepo = dataTableRepo;
      }
    
      @Override
      public DataTable create(DataTableDto dataTableDto) { 
        Converter<DataTableDto, DataTable> converter = new DataTableDto2DataTableConverter();
        return dataTableRepo.saveAndFlush(converter.converter(dataTableDto));
      }
    
      @Override
      public List<DataTable> list() { 
        return dataTableRepo.findAll();
      }
    
      @Override
      public List<DataTable> list(Integer limit) { 
        return dataTableRepo.list(limit);
      }
    
      @Override
      public List<DataTable> count(Integer limit) { 
        return dataTableRepo.count(limit);
      }
    
      @Override
      public DataTable last() { 
        return dataTableRepo.list(1).get(0);
      }
    }

package com.wretchant.csdnchart.service;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.entity.UserInfo;
    
    public interface UserInfoService { 
    
      @InfoLog("创建用户信息")
      UserInfo create(UserInfo userInfo);
    
      @InfoLog("查询最新的用户信息")
      UserInfo findTop();
    }

package com.wretchant.csdnchart.service;
    
    import com.wretchant.csdnchart.entity.UserInfo;
    import com.wretchant.csdnchart.repo.UserInfoRepo;
    import org.springframework.stereotype.Service;
    
    @Service
    public class UserInfoServiceImpl implements UserInfoService { 
    
      private final UserInfoRepo userInfoRepo;
    
      public UserInfoServiceImpl(UserInfoRepo userInfoRepo) { 
        this.userInfoRepo = userInfoRepo;
      }
    
      @Override
      public UserInfo create(UserInfo userInfo) { 
        userInfo.setVersion(userInfoRepo.count());
        return userInfoRepo.saveAndFlush(userInfo);
      }
    
      @Override
      public UserInfo findTop() { 
        return userInfoRepo.findTop();
      }
    }

package com.wretchant.csdnchart.timer;
    
    import com.wretchant.csdnchart.annotation.InfoLog;
    import com.wretchant.csdnchart.entity.CsdnConfig;
    import com.wretchant.csdnchart.entity.DataTableDto;
    import com.wretchant.csdnchart.service.ConnService;
    import com.wretchant.csdnchart.service.DataTableService;
    import org.jsoup.nodes.Document;
    import org.jsoup.nodes.Element;
    import org.jsoup.select.Elements;
    import org.springframework.scheduling.annotation.Async;
    import org.springframework.scheduling.annotation.Scheduled;
    import org.springframework.stereotype.Service;
    
    @Service
    public class DataTimer { 
    
      private final CsdnConfig csdnConfig;
      private final ConnService connService;
      private final DataTableService dataTableService;
    
      public DataTimer(
          ConnService connService, CsdnConfig csdnConfig, DataTableService dataTableService) { 
        this.connService = connService;
        this.csdnConfig = csdnConfig;
        this.dataTableService = dataTableService;
      }
    
      @InfoLog("开始爬取数据")
      @Async
      @Scheduled(fixedRate = 1000 * 60 * 5)
      public void execute() throws Exception { 
        Document document = connService.conn(csdnConfig.getCsdnUrl());
        Element asideProfile = document.body().getElementById("asideProfile");
        Elements gradeBox = asideProfile.select(".grade-box");
        Elements info = asideProfile.select(".item-tiling");
    
        Elements gradeColumn = gradeBox.select("dl");
        String level = gradeColumn.get(0).select("dd a").attr("title").substring(0, 1);
        String visitNumber = gradeColumn.get(1).select("dd").attr("title");
        String integral = gradeColumn.get(2).select("dd").attr("title");
        String top = gradeColumn.get(3).attr("title");
    
        Elements infoColumn = info.select("dl");
        String htmlValue = infoColumn.select("dd span").html();
        String[] values = htmlValue.split("\n");
        String articleNumber = values[0];
        String fans = values[1];
        String likeNumber = values[2];
        String commentNumber = values[3];
    
        DataTableDto dto = new DataTableDto();
        dto.setLevel(Integer.valueOf(level));
        dto.setVisitNumber(Integer.valueOf(visitNumber));
        dto.setIntegral(Integer.valueOf(integral));
        dto.setTop(Integer.valueOf(top));
        dto.setArticleNumber(Integer.valueOf(articleNumber));
        dto.setFans(Integer.valueOf(fans));
        dto.setLikeNumber(Integer.valueOf(likeNumber));
        dto.setCommentNumber(Integer.valueOf(commentNumber));
        dataTableService.create(dto);
      }
    }

package com.wretchant.csdnchart;
    
    import org.springframework.boot.SpringApplication;
    import org.springframework.boot.autoconfigure.SpringBootApplication;
    
    @SpringBootApplication
    public class CsdnChartApplication { 
    
      public static void main(String[] args) { 
        SpringApplication.run(CsdnChartApplication.class, args);
      }
    }

<!DOCTYPE html>
    <html lang="en" xmlns="http://www.w3.org/1999/xhtml" xmlns:th="http://www.thymeleaf.org">
    <head>
        <meta charset="UTF-8">
        <title>数据</title>
    
        <style> body { text-align: center; width: 100%; } table { padding-right: 20px; width: 100%; min-width: 700px; text-align: center; border-spacing: unset; } td { border: none; } thead { background: rgba(0, 57, 117, 0.9); color: aliceblue; font-size: 14px; font-weight: 700; } thead td { padding-left: 10px; padding-right: 10px; padding-top: 5px; padding-bottom: 5px; } tbody { background: rgba(3, 21, 46, 0.9); font-size: 12px; font-weight: 400; } tbody td { color: azure; padding: 3px; border-bottom: blueviolet 1px solid; border-right: blueviolet 1px solid; } </style>
    </head>
    <body>
    
    
    <table>
    
        <thead>
        
        
        <tr>
            <td>时间</td>
    
            <td>原创</td>
            <td>粉丝</td>
            <td>喜欢</td>
            <td>评论</td>
    
            <td>等级</td>
            <td>访问</td>
            <td>积分</td>
            <td>排名</td>
    
    
        </tr>
        </thead>
        <tbody>
        <tr th:each="data:${list}">
            <td th:text="${#temporals.format(data.gmtCreate,'yyyy-MM-dd') +' - '+ data.timePoint+'点'}">0</td>
    
            <td><span th:text="${ '+'+data.articleNumber}"></span></td>
            <td><span th:text="${ '+'+data.fans}"></span></td>
            <td><span th:text="${ '+'+data.likeNumber}"></span></td>
            <td><span th:text="${ '+'+data.commentNumber}"></span></td>
    
            <td><span th:text="${ '+'+data.level}"></span></td>
            <td><span th:text="${ '+'+data.visitNumber}"></span></td>
            <td><span th:text="${ '+'+data.integral}"></span></td>
            <td><span th:text="${ '+'+data.top}"></span></td>
    
    
        </tr>
        </tbody>
    
    </table>
    
    </body>
    </html>

<!DOCTYPE html>
    <html lang="en" xmlns="http://www.w3.org/1999/xhtml" xmlns:th="http://www.thymeleaf.org">
    <head>
        <meta charset="UTF-8">
        <title>数据</title>
    
        <link href="https://cdn.bootcss.com/twitter-bootstrap/4.3.1/css/bootstrap.min.css" rel="stylesheet">
    
        <style> body { text-align: center; width: 100%; } table { padding-right: 20px; width: 60%; min-width: 700px; text-align: center; border-spacing: unset; } td { border: none; } thead { background: rgba(0, 57, 117, 0.9); color: aliceblue; font-size: 14px; font-weight: 700; } thead td { padding-left: 10px; padding-right: 10px; padding-top: 5px; padding-bottom: 5px; } tbody { background: rgba(3, 21, 46, 0.9); font-size: 12px; font-weight: 400; } tbody td { color: azure; padding: 3px; border-bottom: blueviolet 1px solid; border-right: blueviolet 1px solid; } a { color: chartreuse; cursor: pointer; } td { max-width: 650px; } </style>
    </head>
    <body>
    
    
    <table class="m-auto">
    
        <thead>
        
        
        <tr>
    
            <td>名称</td>
            <td>评论</td>
            <td>访问</td>
            <td>文章地址</td>
    
    
        </tr>
        </thead>
        <tbody>
        <tr th:each="data:${list}">
    
            <td th:text="${data.articleName}"></td>
            <td th:text="${data.commentNumber}"></td>
            <td th:text="${data.readNumber }"></td>
            <td><a th:href="${data.url}" target="_blank" th:text="${data.url}"></a></td>
    
    
        </tr>
        </tbody>
    
    </table>
    
    </body>
    </html>

<!DOCTYPE html>
    <html lang="en" xmlns="http://www.w3.org/1999/xhtml" >
    <head>
        <meta charset="UTF-8">
        <title>报表</title>
    
    
        <script src="https://cdn.bootcss.com/echarts/4.3.0-rc.2/echarts-en.common.js"></script>
        <script src="https://cdn.bootcss.com/jquery/3.4.1/jquery.min.js"></script>
    
        <script> function setChart(id, option) { var myChart = echarts.init(document.getElementById(id)); option.backgroundColor = "#2c343c"; option.textStyle = { color: 'rgba(255, 255, 255, 0.3)' }; option.toolbox = { show: true, feature: { dataZoom: { yAxisIndex: 'none' }, dataView: { readOnly: false}, magicType: { type: ['line', 'bar']}, restore: { }, saveAsImage: { } } }; option.tooltip = { trigger: 'axis' }; myChart.setOption(option); } function url(position) { return '/api/data/t/' + [[${ size}]] + '?' + 'dataTableEnum=' + position; } </script>
    
        <style> .box { width: 49%; min-width: 400px; height: 350px; padding-bottom: 100px; } .left { float: left; } .right { float: right; } .title { font-size: 14px; font-weight: 300; color: blueviolet; padding-bottom: 10px; } </style>
    
    </head>
    <body>
    
    
    <div class="box left">
        <div class="title">访问数据增长图</div>
        <div id="visitNumber" style="height:400px;"></div>
        <script> $.get(url('VISIT_NUMBER'), function (res) { option = { xAxis: { type: 'category', data: res.t.datetime }, yAxis: { min: res.t.min - 2, max: res.t.max + 2 }, series: [{ data: res.t.data, type: 'line' }] }; setChart('visitNumber', option); }); </script>
    </div>
    
    <div class="box right">
        <div class="title">积分增长曲线图</div>
        <div id="integral" style="height:400px;"></div>
        <script> $.get(url('INTEGRAL'), function (res) { option = { xAxis: { type: 'category', data: res.t.datetime }, yAxis: { min: res.t.min - 2, max: res.t.max + 2 }, series: [{ data: res.t.data, type: 'line' }] }; setChart('integral', option); }); </script>
    </div>
    
    <div class="box left">
        <div class="title">排名增长图</div>
        <div id="top" style="height:400px;"></div>
        <script> $.get(url('TOP'), function (res) { option = { xAxis: { type: 'category', data: res.t.datetime }, yAxis: { min: res.t.min - 2, max: res.t.max + 2 }, series: [{ data: res.t.data, type: 'line' }] }; setChart('top', option); }); </script>
    </div>
    
    <div class="box right">
        <div class="title">等级增长曲线图</div>
        <div id="level" style="height:400px;"></div>
        <script> $.get(url('LEVEL'), function (res) { option = { xAxis: { type: 'category', data: res.t.datetime }, yAxis: { min: res.t.min - 2, max: res.t.max + 2 }, series: [{ data: res.t.data, type: 'line' }] }; setChart('level', option); }); </script>
    </div>
    
    
    <div class="box left">
        <div class="title">原创文章增长图</div>
        <div id="articleNumber" style="height:400px;"></div>
        <script> $.get(url('ARTICLE_NUMBER'), function (res) { option = { xAxis: { type: 'category', data: res.t.datetime }, yAxis: { min: res.t.min - 2, max: res.t.max + 2 }, series: [{ data: res.t.data, type: 'line' }] }; setChart('articleNumber', option); }); </script>
    </div>
    
    <div class="box right">
        <div class="title">粉丝增长曲线图</div>
        <div id="fans" style="height:400px;"></div>
        <script> $.get(url('FANS'), function (res) { option = { xAxis: { type: 'category', data: res.t.datetime }, yAxis: { min: res.t.min - 2, max: res.t.max + 2 }, series: [{ data: res.t.data, type: 'line' }] }; setChart('fans', option); }); </script>
    </div>
    
    
    <div class="box left">
        <div class="title">喜欢增长图</div>
        <div id="likeNumber" style="height:400px;"></div>
        <script> $.get(url('LIKE_NUMBER'), function (res) { option = { xAxis: { type: 'category', data: res.t.datetime }, yAxis: { min: res.t.min - 2, max: res.t.max + 2 }, series: [{ data: res.t.data, type: 'line' }] }; setChart('likeNumber', option); }); </script>
    </div>
    
    <div class="box right">
        <div class="title">评论增长曲线图</div>
        <div id="commentNumber" style="height:400px;"></div>
        <script> $.get(url('COMMENT_NUMBER'), function (res) { option = { xAxis: { type: 'category', data: res.t.datetime }, yAxis: { min: res.t.min - 2, max: res.t.max + 2 }, series: [{ data: res.t.data, type: 'line' }] }; setChart('commentNumber', option); }); </script>
    </div>
    
    </body>
    </html>

<!DOCTYPE html>
    <html lang="en" xmlns="http://www.w3.org/1999/xhtml" xmlns:th="http://www.thymeleaf.org">
    <head>
        <meta charset="UTF-8">
        <title>数据</title>
    
        <style> body { text-align: center; width: 100%; } table { padding-right: 20px; width: 100%; min-width: 700px; text-align: center; border-spacing: unset; } td { border: none; } thead { background: rgba(0, 57, 117, 0.9); color: aliceblue; font-size: 14px; font-weight: 700; } thead td { padding-left: 10px; padding-right: 10px; padding-top: 5px; padding-bottom: 5px; } tbody { background: rgba(3, 21, 46, 0.9); font-size: 12px; font-weight: 400; } tbody td { color: azure; padding: 3px; border-bottom: blueviolet 1px solid; border-right: blueviolet 1px solid; } </style>
    </head>
    <body>
    
    
    <table>
    
        <thead>
        
        
        <tr>
            <td>时间</td>
    
            <td>原创</td>
            <td>粉丝</td>
            <td>喜欢</td>
            <td>评论</td>
    
            <td>等级</td>
            <td>访问</td>
            <td>积分</td>
            <td>排名</td>
    
    
        </tr>
        </thead>
        <tbody>
        <tr th:each="data:${list}">
            <td th:text="${#temporals.format(data.gmtCreate,'yyyy-MM-dd HH:mm:ss')}">0</td>
    
            <td><span th:text="${data.articleNumber}"></span><span th:text="${data.articleNumberAdd eq 0 ? '':'(+'+data.articleNumberAdd+')'}"></span></td>
            <td><span th:text="${data.fans}"></span><span th:text="${data.fansAdd eq 0 ? '':'(+'+data.fansAdd+')'}"></span>
            </td>
            <td><span th:text="${data.likeNumber}"></span><span th:text="${data.likeNumberAdd eq 0 ? '':'(+'+data.likeNumberAdd+')'}"></span></td>
            <td><span th:text="${data.commentNumber}"></span><span th:text="${data.commentNumberAdd eq 0 ? '':'(+'+data.commentNumberAdd+')'}"></span></td>
    
            <td><span th:text="${data.level}"></span><span th:text="${data.levelAdd eq 0 ? '':'(+'+data.levelAdd+')'}"></span></td>
            <td><span th:text="${data.visitNumber}"></span><span th:text="${data.visitNumberAdd eq 0 ? '':'(+'+data.visitNumberAdd+')'}"></span></td>
            <td><span th:text="${data.integral}"></span><span th:text="${data.integralAdd eq 0 ? '':'(+'+data.integralAdd+')'}"></span></td>
            <td><span th:text="${data.top}"></span><span th:text="${data.topAdd eq 0 ? '':'(+'+data.topAdd+')'}"></span>
            </td>
    
    
        </tr>
        </tbody>
    
    </table>
    
    </body>
    </html>

<!DOCTYPE html>
    <html lang="en" xmlns="http://www.w3.org/1999/xhtml" xmlns:th="http://www.thymeleaf.org">
    <head>
        <meta charset="UTF-8">
    
        <link href="https://cdn.bootcss.com/twitter-bootstrap/4.3.1/css/bootstrap.min.css" rel="stylesheet">
        <script src="https://cdn.bootcss.com/echarts/4.3.0-rc.2/echarts-en.common.js"></script>
        <script src="https://cdn.bootcss.com/jquery/3.4.1/jquery.min.js"></script>
        <title>首页</title>
    
    
        <script> function setChart(id, option) { var myChart = echarts.init(document.getElementById(id)); option.backgroundColor = "#2c343c"; option.textStyle = { color: 'rgba(255, 255, 255, 0.3)' }; option.toolbox = { show: true, feature: { dataZoom: { yAxisIndex: 'none' }, dataView: { readOnly: false}, magicType: { type: ['line', 'bar']}, restore: { }, saveAsImage: { } } }; option.tooltip = { trigger: 'axis' }; myChart.setOption(option); } function url(position) { return '/api/data/t/' + 30 + '?' + 'dataTableEnum=' + position; } </script>
    
        <style> .main { width: 60%; min-width: 900px; min-height: 1080px; background: beige; } .info-box { height: 170px; width: 100%; background: aliceblue; } .grand-box { padding: 30px; width: 100%; background: aliceblue; } .info-box img { width: 160px; height: 160px; } .head-img { padding-left: 50px; padding-top: 10px; } h1 { margin-top: 10px; } h6 { margin-top: 10px; } .table { background: darkseagreen; height: 380px; } .title { padding-top: 15px; font-size: 14px; font-weight: 300; color: blueviolet; padding-bottom: 10px; } .all-table { float: right; padding-right: 20px; } .all-table input { width: 50px; border: none; border-bottom: blueviolet 1px solid; text-align: center; background: beige; } a { font-size: 14px; font-weight: 300; cursor: pointer; color: red; } </style>
    </head>
    <body>
    
    
    <div class="main m-auto">
    
        
        <div class="info-box">
            <div class="row">
                <div class="col-md-3">
                    <img src="http://b-ssl.duitang.com/uploads/item/201511/07/20151107201311_TYGkm.jpeg" alt="头像" class="head-img">
                </div>
                <div class="col-md-9">
                    <h1 th:text="${userInfo.nickname}">简简单单OnlineZuozuo</h1>
                    <h6 th:text="${userInfo.motto}">未闻万里蓬莱，而窥先圣遗智。故，以此生筑梦，奔而逐之；以泰山之伟，攀而登之；以静雅素心，处世为人。------zuozuo 著...</h6>
    
                    <h6 class="title"><a href="/web/article">查看全部的文章 - - ></a></h6>
                </div>
            </div>
        </div>
    
    
        <div class="title">
            具体的数据
        </div>
        <div class="grand-box">
            <div class="row">
                <div class="col-md-3" th:text="${ '原创：'+last.articleNumber}">0</div>
                <div class="col-md-3" th:text="${ '粉丝：'+last.fans}">0</div>
                <div class="col-md-3" th:text="${ '喜欢：'+last.likeNumber}">0</div>
                <div class="col-md-3" th:text="${ '评论：'+last.commentNumber}">0</div>
            </div>
            <div class="row">
                <div class="col-md-3" th:text="${ '等级：'+last.level}">0</div>
                <div class="col-md-3" th:text="${ '访问：'+last.visitNumber}">0</div>
                <div class="col-md-3" th:text="${ '积分：'+last.integral}">0</div>
                <div class="col-md-3" th:text="${ '排名：'+last.top}">0</div>
            </div>
            <br>
            <div class="row">
                <div class="col-md-12" th:text="${ '原转比：'+last.articleNumber+'(原创) / ' + count+'(总文章数) = ' + #numbers.formatPercent(last.articleNumber / (count * 1.00) ,3,2) }">
                    0
                </div>
            </div>
        </div>
    
        <div class="title">
            访问通道
        </div>
        <div class="grand-box">
            <div class="row">
                <div class="col-md-3"><a href="/web/conduit/VISIT_MOST">最高访问10篇 - - ></a></div>
                <div class="col-md-3"><a href="/web/conduit/NEW_MOST">最新文章10篇 - - ></a></div>
                <div class="col-md-3"><a href="/web/conduit/COMMENT_MOST">评论最多10篇 - - ></a></div>
            </div>
        </div>
    
    
        
        <div class="title">
            访问增量简报
    
            <div class="all-table">以<input id="chart-data-length" type="number" value="30"><a onclick="chart()">条数据长度查看全部报表
                - - ></a></div>
        </div>
        <div class="table" id="visitNumber"></div>
        <script> $.get(url('VISIT_NUMBER'), function (res) { option = { toolbox: { feature: { saveAsImage: { } } }, tooltip: { trigger: 'axis' }, xAxis: { type: 'category', data: res.t.datetime }, yAxis: { min: res.t.min - 2, max: res.t.max + 2 }, series: [{ data: res.t.data, type: 'line' }] }; setChart('visitNumber', option); }); </script>
    
        
        <div class="title">
            写作趋势
        </div>
        <div class="table" id="archival_storage"></div>
        <script> $.get(url('VISIT_NUMBER'), function (res) { option = { xAxis: { type: 'category', data: [[${ dates}]] }, yAxis: { min: [[${ min}]] - 2, max: [[${ max}]] + 2 }, series: [{ data: [[${ datas}]], type: 'line', markPoint: { data: [ { type: 'max', name: '最大值'}, { type: 'min', name: '最小值'} ] }, markLine: { data: [ { type: 'average', name: '平均值'} ] } }] }; setChart('archival_storage', option); }); </script>
    
    </div>
    
    <script> function chart() { var choose = $("#chart-data-length"); window.location.href = "/web/chart/" + choose.val(); } </script>
    
    
    </body>
    </html>

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly93cmV0Y2hhbnQuYmxvZy5jc2RuLm5ldA_size_16_color_FFFFFF_t_70]: /images/20230601/47702b41660249d2bd720a706cb381ec.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly93cmV0Y2hhbnQuYmxvZy5jc2RuLm5ldA_size_16_color_FFFFFF_t_70 1]: /images/20230601/5f7df5fbbb574f5db96503b0cc696e33.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly93cmV0Y2hhbnQuYmxvZy5jc2RuLm5ldA_size_16_color_FFFFFF_t_70 2]: /images/20230601/bfc7013e7a5a40aba07b3d294e269a7b.png
[https_github.com_FrankZuozuo_csdn-chart]: https://github.com/FrankZuozuo/csdn-chart