Java 爬取网易云MV视频

Love The Way You Lie 2021-09-27 15:58 490阅读 0赞

# 一、环境 #

JDK 1.8+

推荐一个Java的学习网站： [http://how2j.cn?p=17361][http_how2j.cn_p_17361]

# 二、 步骤 #

**1. 打开网易云音乐中的其中一首歌页面，然后打开开发者工具，查看mv链接地址**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70][]

**2. 由上面可知道 歌曲mv视频的id 是 303284， 所以完整的视频地址是：[http://music.163.com/\#/mv?id=303284][http_music.163.com_mv_id_303284]**

**注意： 地址中是没有\#的，，真实地址是  [http://music.163.com/][http_music.163.com]mv?id= mv视频id  **

**3. 打开mv链接地址后，查看网页框架源代码**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70 1][]

**4. 发现视频的地址就是这里<meta content='">中，用Jsoup进行解析，这里的url可以用URLDecode 进行解码，我们可以复制url到在线解码网站去进行解码， 在线加/解码 地址： [http://www.convertstring.com/zh\_CN/EncodeDecode/UrlDecode][http_www.convertstring.com_zh_CN_EncodeDecode_UrlDecode]**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70 2][]

**解码后的url是这样的：**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70 3][]

**复制解码后的网址进行访问：**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70 4][]

**视频的源播放地址就出来了，这时候就可以进行下载**

**下面给出源码（Java）**

# 三、 源代码 #

maven配置

<dependency>
                <groupId>org.apache.httpcomponents</groupId>
                <artifactId>httpclient</artifactId>
                <version>4.5.2</version>
            </dependency>
    
            
            
            <dependency>
                <groupId>org.jsoup</groupId>
                <artifactId>jsoup</artifactId>
                <version>1.11.3</version>
            </dependency>
    
            
            <dependency>
                <groupId>net.sf.json-lib</groupId>
                <artifactId>json-lib</artifactId>
                <version>2.3</version>
                <classifier>jdk15</classifier>
            </dependency>

package netease;
    
    import org.apache.http.HttpHeaders;
    import org.jsoup.Jsoup;
    import org.jsoup.nodes.Document;
    import org.jsoup.nodes.Element;
    
    import java.io.*;
    import java.net.MalformedURLException;
    import java.net.URL;
    import java.net.URLConnection;
    import java.net.URLDecoder;
    import java.util.HashMap;
    import java.util.UUID;
    
    public class MVTest {
        static final String[] user_Agent = {
                "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36",
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.101 Safari/537.36",
                "Mozilla/5.0 (X11; U; Linux x86_64; zh-CN; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10",
                "Mozilla/5.0 (iPad; U; CPU OS 4_2_1 like Mac OS X; zh-cn) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5",
                "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Acoo Browser; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506)",
                "Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2; .NET CLR 3.0.04506.30)",
                "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN) AppleWebKit/523.15 (KHTML, like Gecko, Safari/419.3) Arora/0.3 (Change: 287 c9dfb30)",
                "Mozilla/5.0 (X11; U; Linux; en-US) AppleWebKit/527+ (KHTML, like Gecko, Safari/419.3) Arora/0.6",
                "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.2pre) Gecko/20070215 K-Ninja/2.1.1",
                "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9) Gecko/20080705 Firefox/3.0 Kapiko/3.0",
                "Mozilla/5.0 (X11; Linux i686; U;) Gecko/20070322 Kazehakase/0.4.5",
                "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.8) Gecko Fedora/1.9.0.8-1.fc10 Kazehakase/0.5.6",
                "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11",
                "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_3) AppleWebKit/535.20 (KHTML, like Gecko) Chrome/19.0.1036.7 Safari/535.20",
                "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; fr) Presto/2.9.168 Version/11.52",
                "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.11 TaoBrowser/2.0 Safari/536.11",
                "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.71 Safari/537.1 LBBROWSER",
                "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; LBBROWSER)",
        };
    
        static final String neteaseReferer = "http://music.163.com/";
        static final String neteeaseHost = "music.163.com";
    
        public static void main(String[] args) throws IOException {
            String url = Common.neteaseReferer+"mv?id=303284";  //mv 视频地址
    
            //请求header 设置
            HashMap<String, String> config = new HashMap<>();
            config.put(HttpHeaders.USER_AGENT, user_Agent[0]);
            config.put(HttpHeaders.REFERER, neteaseReferer);
            config.put(HttpHeaders.HOST, neteeaseHost);
            Document res = Jsoup.connect(url).headers(config).get();
    
            //解析框架源代码中的 mv 播放url
            Element element = res.select("meta[property=og:video]").first();
            String downloadURL = URLDecoder.decode( element.attr("content"), "UTF-8" );
            System.err.println(downloadURL);
    
            downloadFile(downloadURL, UUID.randomUUID().toString()+".mp4");
        }
    
        /**
         * 文件下载保存
         * @param downloadURL  下载链接
         * @param savePath     保存路径
         * @throws MalformedURLException
         * @throws MalformedURLException
         */
        public static void downloadFile(String downloadURL, String savePath) throws MalformedURLException, MalformedURLException {
            // 下载网络文件
            int byteread = 0;
            URL url = new URL(downloadURL.trim());
            try {
                URLConnection conn = url.openConnection();
                InputStream inStream = conn.getInputStream();
                File file = new File(savePath);
                if(!file.exists()){
                    if(!file.getParentFile().exists()) {
                        //若文件夹不存在，创建文件夹
                        file.getParentFile().mkdirs();  //注意不是 mkdir ， mkdirs是创建多级目录
                    }
                }
                FileOutputStream fs = new FileOutputStream(file);
    
                byte[] buffer = new byte[1204];
                while ((byteread = inStream.read(buffer)) != -1) {
                    fs.write(buffer, 0, byteread);
                }
            } catch (FileNotFoundException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

运行后当前项目目录下出现刚刚下载的MP4文件

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70 5][]

[http_how2j.cn_p_17361]: http://how2j.cn?p=17361
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70]: /images/20210923/d766c1d9baf948f9809ed38af98b502e.png
[http_music.163.com_mv_id_303284]: http://music.163.com/#/mv?id=303284
[http_music.163.com]: https://music.163.com/#/mv?id=303284
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70 1]: /images/20210923/d206d71e0374420fa9d5509420634228.png
[http_www.convertstring.com_zh_CN_EncodeDecode_UrlDecode]: http://www.convertstring.com/zh_CN/EncodeDecode/UrlDecode
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70 2]: /images/20210923/f9ea8160b8404a08ab76e22cdbc8e31d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70 3]: /images/20210923/ccf37a6a9f6d4180b7cc0062651d28bd.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70 4]: /images/20210923/e18a21591438428db8f96b0d7f35e7e3.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2F3YXdhd2F3YWE_size_27_color_FFFFFF_t_70 5]: /images/20210923/b02d5be1207447a9ba1bdfb5d1c3ac77.png