Spring Boot WebMagic 入库时 mapper注入提示空指针，以及正确的操作

深碍√TFBOYSˉ_ 2023-06-19 11:29 12阅读 0赞

# 本屌研究了一整天 #

# 网上说 实现Pipeline接口中的 process是多线程的，所以注入后不是同一个对象，无所报空指针 #

贴出代码

## 1.启动类 ##

@SpringBootApplication
    @EnableScheduling//开启定时任务
    @MapperScan(basePackages = { "com.xianbaovip.project"})
    public class ProjectApplication { 
        public static void main(String[] args) { 
            SpringApplication.run(ProjectApplication.class, args);
        }
    }

### 重点是@EnableScheduling注解， ###

## 2.WebMagic 的爬取 PageProcessor ##

@Component
    public class WebMagicSpider1 implements PageProcessor { 
    
        private Site site = Site.me().setRetryTimes(3).setSleepTime(100);
        @Override
        public Site getSite() { 
            return site;
        }
    
        public void process(Page page) { 
              this.saveMessage(page);
        }
    
    	//WebMagic的入库方法
        @Autowired
        private  GatherPipeline gatherPipeline; 
    
        private  void saveMessage(Page page){ 
            page.putField("Content", page.getHtml().xpath("//div[@class='comment-body']/div/p[1]").all());
            page.putField("Source", constant.reptileSource1);
        }
    	//initialDelay 延迟多久执行第一次任务
    	//fixedDelay 相隔多久执行上一次任务
        @Scheduled(initialDelay = 1000,fixedDelay = 100*1000)
        public void process() { 
            Spider.create(new WebMagicSpider1())
                    .addUrl("http://www.xianbaoi.vip")
                    .addPipeline(this.gatherPipeline)
                    .thread(1)
                    .run();
    
        }
    }

## 重点 ##

1.@Component注解  
2.@Scheduled定时任务

## 2.WebMagic 的爬取 Pipeline ##

@Component
    public class GatherPipeline implements Pipeline { 
    	//自己封装的入库方法
        @Autowired
        private GatherMapper gatherMapper;
        @Override
        public void process(ResultItems resultItems, Task task) { 
            for (Map.Entry<String, Object> entry : resultItems.getAll().entrySet()) { 
    
                if (entry.getKey().contains("Content")) { 
                    List<String> value =  (List<String>)entry.getValue();
                    //取值入库
                    for (String Content : value) { 
                        Gather gather = new Gather();
                        gather.setGatherContent(Content);
                        gather.setGatherSource(sourceString);
                        gather.setGatherTitle("test");
                        gather.setCreateTime(new Date());
                        this.gatherMapper.insert(gather);
    
                    }
    
    
                }
    
            }
    
    
            }
            }

## 重点 ##

# 1.调用mapper的时候 使用`this.` #