第 11 章直接内存

悠悠 2022-12-04 01:14 192阅读 0赞

# 第 11 章 直接内存 #

## 1、直接内存概述 ##

> **直接内存**

1.  不是虚拟机运行时数据区的一部分，也不是《Java虚拟机规范》中定义的内存区域。
2.  直接内存是在Java堆外的、直接向系统申请的内存区间。
3.  来源于NIO，通过存在堆中的DirectByteBuffer操作Native内存
4.  通常，访问直接内存的速度会优于Java堆。即读写性能高。
5.  因此出于性能考虑，读写频繁的场合可能会考虑使用直接内存。
6.  Java的NIO库允许Java程序使用直接内存，用于数据缓冲区

> **代码示例**

*  代码

/**
     *  IO                  NIO (New IO / Non-Blocking IO)
     *  byte[] / char[]     Buffer
     *  Stream              Channel
     *
     * 查看直接内存的占用与释放
     * @author shkstart  shkstart@126.com
     * @create 2020  0:22
     */
    public class BufferTest {
        
        private static final int BUFFER = 1024 * 1024 * 1024;//1GB
    
        public static void main(String[] args){
        
            //直接分配本地内存空间
            ByteBuffer byteBuffer = ByteBuffer.allocateDirect(BUFFER);
            System.out.println("直接内存分配完毕，请求指示！");
    
            Scanner scanner = new Scanner(System.in);
            scanner.next();
    
            System.out.println("直接内存开始释放！");
            byteBuffer = null;
            System.gc();
            scanner.next();
        }
    }

*  呐，直接占用了 1G 的本地内存

![image-20200729214811954][]

*  释放后，Java程序的内存占用明显减少

![image-20200729214909938][]

## 2、BIO 与 NIO ##

> **非直接缓存区（BIO）**

原来采用BIO的架构，在读写本地文件时，我们需要从用户态切换成内核态

![image-20200709170907611][]

> **直接缓冲区（NIO）**

NIO 直接操作物理磁盘，省去了中间商赚差价

![image-20200729215128430][]

> **代码测试**

*  测试代码：分别使用 BIO 和 NIO 复制大文件，看看用时差别

/**
     * @author shkstart  shkstart@126.com
     * @create 2020  0:04
     */
    public class BufferTest1 {
        
    
        private static final String TO = "F:\\test\\异界BD中字.mp4";
        private static final int _100Mb = 1024 * 1024 * 100;
    
        public static void main(String[] args) {
        
            long sum = 0;
            String src = "F:\\test\\异界BD中字.mp4";
            for (int i = 0; i < 3; i++) {
        
                String dest = "F:\\test\\异界BD中字_" + i + ".mp4";
                // sum += io(src,dest);//54606
                sum += directBuffer(src, dest);//50244
            }
    
            System.out.println("总花费的时间为：" + sum);
        }
    
        private static long directBuffer(String src, String dest) {
        
            long start = System.currentTimeMillis();
    
            FileChannel inChannel = null;
            FileChannel outChannel = null;
            try {
        
                inChannel = new FileInputStream(src).getChannel();
                outChannel = new FileOutputStream(dest).getChannel();
    
                ByteBuffer byteBuffer = ByteBuffer.allocateDirect(_100Mb);
                while (inChannel.read(byteBuffer) != -1) {
        
                    byteBuffer.flip();//修改为读数据模式
                    outChannel.write(byteBuffer);
                    byteBuffer.clear();//清空
                }
            } catch (IOException e) {
        
                e.printStackTrace();
            } finally {
        
                if (inChannel != null) {
        
                    try {
        
                        inChannel.close();
                    } catch (IOException e) {
        
                        e.printStackTrace();
                    }
    
                }
                if (outChannel != null) {
        
                    try {
        
                        outChannel.close();
                    } catch (IOException e) {
        
                        e.printStackTrace();
                    }
    
                }
            }
    
            long end = System.currentTimeMillis();
            return end - start;
    
        }
    
        private static long io(String src, String dest) {
        
            long start = System.currentTimeMillis();
    
            FileInputStream fis = null;
            FileOutputStream fos = null;
            try {
        
                fis = new FileInputStream(src);
                fos = new FileOutputStream(dest);
                byte[] buffer = new byte[_100Mb];
                while (true) {
        
                    int len = fis.read(buffer);
                    if (len == -1) {
        
                        break;
                    }
                    fos.write(buffer, 0, len);
                }
            } catch (IOException e) {
        
                e.printStackTrace();
            } finally {
        
                if (fis != null) {
        
                    try {
        
                        fis.close();
                    } catch (IOException e) {
        
                        e.printStackTrace();
                    }
    
                }
                if (fos != null) {
        
                    try {
        
                        fos.close();
                    } catch (IOException e) {
        
                        e.printStackTrace();
                    }
    
                }
            }
    
    
            long end = System.currentTimeMillis();
    
            return end - start;
        }
    }

--------------------

**深入 ByteBuffer 源码**

*  ByteBuffer.allocateDirect() 方法

public static ByteBuffer allocateDirect(int capacity) {
        
        return new DirectByteBuffer(capacity);
    }

*  DirectByteBuffer 类的构造器用到了 Unsafe 类分配本地内存

DirectByteBuffer(int cap) {
                           // package-private
    
        super(-1, 0, cap, cap);
        boolean pa = VM.isDirectMemoryPageAligned();
        int ps = Bits.pageSize();
        long size = Math.max(1L, (long)cap + (pa ? ps : 0));
        Bits.reserveMemory(size, cap);
    
        long base = 0;
        try {
        
            base = unsafe.allocateMemory(size);
        } catch (OutOfMemoryError x) {
        
            Bits.unreserveMemory(size, cap);
            throw x;
        }
        unsafe.setMemory(base, size, (byte) 0);
        if (pa && (base % ps != 0)) {
        
            // Round up to page boundary
            address = base + ps - (base & (ps - 1));
        } else {
        
            address = base;
        }
        cleaner = Cleaner.create(this, new Deallocator(base, size, cap));
        att = null;
    
    
    
    }

## 3、直接内存与 OOM ##

1.  直接内存也可能导致OutofMemoryError异常
2.  由于直接内存在Java堆外，因此它的大小不会直接受限于-Xmx指定的最大堆大小，但是系统内存是有限的，Java堆和直接内存的总和依然受限于操作系统能给出的最大内存。
3.  直接内存的缺点为：
    
     *  分配回收成本较高
     *  不受JVM内存回收管理
4.  直接内存大小可以通过MaxDirectMemorySize设置
5.  如果不指定，默认与堆的最大值-Xmx参数值一致

--------------------

**代码示例 1**

*  代码

/**
     * 本地内存的OOM:  OutOfMemoryError: Direct buffer memory
     *
     * @author shkstart  shkstart@126.com
     * @create 2020  0:09
     */
    public class BufferTest2 {
        
        private static final int BUFFER = 1024 * 1024 * 20;//20MB
    
        public static void main(String[] args) {
        
            ArrayList<ByteBuffer> list = new ArrayList<>();
    
            int count = 0;
            try {
        
                while(true){
        
                    ByteBuffer byteBuffer = ByteBuffer.allocateDirect(BUFFER);
                    list.add(byteBuffer);
                    count++;
                    try {
        
                        Thread.sleep(100);
                    } catch (InterruptedException e) {
        
                        e.printStackTrace();
                    }
                }
            } finally {
        
                System.out.println(count);
            }
    
    
        }
    }

*  本地内存持续增长，直至程序抛出异常：java.lang.OutOfMemoryError: Direct buffer memory

![image-20200729215912722][]

*  异常信息

"C:\Program Files\Java\jdk1.8.0_144\bin\java" "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2017.3.1\lib\idea_rt.jar=1296:C:\Program Files\JetBrains\IntelliJ IDEA 2017.3.1\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\Java\jdk1.8.0_144\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\jaccess.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\jfxrt.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\localedata.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\nashorn.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunec.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunmscapi.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\zipfs.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\plugin.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\rt.jar;C:\Users\Heygo\Desktop\JVMDemo\out\production\chapter11" com.atguigu.java.BufferTest2
    180
    Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory
    	at java.nio.Bits.reserveMemory(Bits.java:694)
    	at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
    	at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
    	at com.atguigu.java.BufferTest2.main(BufferTest2.java:21)
    
    Process finished with exit code 1

--------------------

**代码示例 2**

*  代码：直接通过 Unsafe 类申请本地内存

/**
     * -Xmx20m -XX:MaxDirectMemorySize=10m
     * @author shkstart  shkstart@126.com
     * @create 2020  0:36
     */
    public class MaxDirectMemorySizeTest {
        
        private static final long _1MB = 1024 * 1024;
    
        public static void main(String[] args) throws IllegalAccessException {
        
            Field unsafeField = Unsafe.class.getDeclaredFields()[0];
            unsafeField.setAccessible(true);
            Unsafe unsafe = (Unsafe)unsafeField.get(null);
            while(true){
        
                unsafe.allocateMemory(_1MB);
            }
    
        }
    }

*  JVM 参数

-Xmx20m -XX:MaxDirectMemorySize=10m

*  抛出 OOM 异常

"C:\Program Files\Java\jdk1.8.0_144\bin\java" "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2017.3.1\lib\idea_rt.jar=1362:C:\Program Files\JetBrains\IntelliJ IDEA 2017.3.1\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\Java\jdk1.8.0_144\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\jaccess.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\jfxrt.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\localedata.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\nashorn.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunec.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunmscapi.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\zipfs.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\plugin.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_144\jre\lib\rt.jar;C:\Users\Heygo\Desktop\JVMDemo\out\production\chapter11" com.atguigu.java.MaxDirectMemorySizeTest
    Exception in thread "main" java.lang.OutOfMemoryError
    	at sun.misc.Unsafe.allocateMemory(Native Method)
    	at com.atguigu.java.MaxDirectMemorySizeTest.main(MaxDirectMemorySizeTest.java:20)
    
    Process finished with exit code 1

> **JDK8 中元空间直接使用本地内存**

![image-20200729221624142][]

[image-20200729214811954]: /images/20221123/05dea35f177e469b9ad13cc3c4cdcc39.png
[image-20200729214909938]: /images/20221123/b26605b20f254d9b8ea0780517cf1b07.png
[image-20200709170907611]: /images/20221123/acd043dd07a84859a5f2223535a410e6.png
[image-20200729215128430]: /images/20221123/390af0c333b842b296146ea9d5cceadc.png
[image-20200729215912722]: /images/20221123/863bd2622cf84b3e9b63917450f7cf53.png
[image-20200729221624142]: /images/20221123/164183a3d26e4b8cb54e121ec5ed3719.png