glibc动态内存管理

Myth丶恋晨 2023-07-06 10:25 6阅读 0赞

# 前言 #

为了更好了解glibc的内存分配机制以及堆溢出类题目，特地来glibc挖坑。

**本文章分析的源码来自[glibc 2.29][]**

glibc详细信息如下：

*  version: 2.29.9000
 *  release: development

# GNU分配器简述 #

**翻译自** [reference][]

GNU C库的malloc函数实现来自ptmalloc（pthreads malloc），而pthmalloc又用dlmalloc（Doug Lea malloc）实现。malloc函数有两种不同的内存分配方法，这取决于开辟空间的大小或用户传入的具体参数，最常见的分配方法是从一大块连续的内存区域分配一部分出来，并管理这些相邻的区块以优化其使用和减少浪费不可用块。传统系统的堆被设置为一大块内存区域，但是GNU C库的malloc实现保留了多块这样的内存区域来优化多线程的堆内存使用，每一个这样的内存区域在程序中叫做Arena（竞技场？）

与其它版本相反，GNU C库中的malloc不会将块（chunk）的大小向上取整为2的n次幂，无论块大小如何，相邻的块都可以合并为一块空闲块。这使得malloc的实现适合于各种分配模式且不会在整个段中导致大量的内存浪费。多个Arena的存在使得多个线程能够在独自的Arena中同时分配内存，因此性能得到了提高。

另外一种一种分配方式是用来分配一块很大的内存块的（比一页的大小还要大），此时要完成这种内存分配需要用到mmap（anonymous or via /dev/zero; see [Memory-mapped I/O][Memory-mapped I_O]）。这对于释放的这些大块能够立即返还给系统具有很好的优势，因此，即使在调用free释放内存之后，也不会发生大块“锁定”在较小块之间的情况。 mmap的大小阈值是动态的，可以根据程序的分配模式进行调整。 mallopt通过M\_MMAP\_THRESHOLD可静态调整阈值，而mmap的使用可通过M\_MMAP\_MAX完全禁用。请参见[Malloc Tunable Parameters][]。

GNU Allocator的更详细的技术描述在GNU C库Wiki中。参考[https://sourceware.org/glibc/wiki/MallocInternals][https_sourceware.org_glibc_wiki_MallocInternals]。 你也可以使用自定义的malloc来代替GNU C库中内置的内存分配器。请参阅[Replacing malloc][]。

# 术语 #

[reference][reference 1]

## arena ##

通过sbrk或mmap系统调用为线程分配的堆区，按线程的类型可以分为2类：

*  main arena：主线程建立的arena；
 *  thread arena：子线程建立的arena；

## bin ##

一个用来保存Free chunk链表的表头信息的指针数组，按所悬挂链表的类型可以分为4类:

*  Fast bin
 *  Unsorted bin
 *  Small bin
 *  Large bin

## chunk ##

chunk：逻辑上划分的一小块内存，根据作用不同分为4类：

*  Allocated chunk：即分配给用户且未释放的内存块；
 *  Free chunk：即用户已经释放的内存块；
 *  Top chunk
 *  Last Remainder chunk

## 三者关系图 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM2OTYzMjE0_size_16_color_FFFFFF_t_70]

# 术语详解 #

## arena ##

在一个或多个线程之间共享的结构，它包含对一个或多个堆的引用，以及这些堆中“空闲”的块的链表。分配给每个arena的线程将从该arena的空闲列表中分配内存。

https://blog.csdn.net/qq\_41453285/article/details/96847761

### main arena ###

### thread arena ###

## fastbin ##

链表数组包含最近释放的小块，fastbin并不是双向链表结构，而是更加快速的单向链表结构，因为fastbin的块不会从链表的中间移除，所以也不需要双向链表。fastbin与常规bin不同，常规的bin使用FIFO顺序进行处理，而fastbin使用更快的LIFO的处理顺序，因为在fastbin瞬态使用的上下文环境中排序并不重要。

### chunk ###

source code

// INTERNAL_SIZE_T宏定义为size_t
    
    struct malloc_chunk { 
    
      INTERNAL_SIZE_T      mchunk_prev_size;  /* Size of previous chunk (if free). */
      INTERNAL_SIZE_T      mchunk_size;       /* Size in bytes, including overhead. */
    
      struct malloc_chunk* fd;         /* double links -- used only if free. */
      struct malloc_chunk* bk;
    
      /* Only used for large blocks: pointer to next larger size. */
      struct malloc_chunk* fd_nextsize; /* double links -- used only if free. */
      struct malloc_chunk* bk_nextsize;
    };

chunk的默认对其方式是以`2*sizeof(size_t)`字节对其，也就是说在32 bits程序中以2\*4=8字节对其，在64 bits程序中以2\*8=16字节对齐，其在malloc.c的原文注释如下：

Alignment:                              2 * sizeof(size_t) (default)
       
       (i.e., 8 byte alignment with 4byte size_t). This suffices for
       nearly all current machines and C compilers. However, you can
       define MALLOC_ALIGNMENT to be wider than this if necessary.

### glibc中的chunk的大小 ###

*  最小的chunk的大小  
    chunk需要保证放下mchunk\_prev\_size、mchunk\_size，fd和bk以及符合chunk的对齐方式，因此在32位系统中，最小的chunk为16字节，在64位系统中，最小的chunk为32字节

Minimum allocated size: 4-byte ptrs:  16 bytes    (including 4 overhead)
    			  8-byte ptrs:  24/32 bytes (including, 4/8 overhead)
    
           When a chunk is freed, 12 (for 4byte ptrs) or 20 (for 8 byte
           ptrs but 4 byte size) or 24 (for 8/8) additional bytes are
           needed; 4 (8) for a trailing size field and 8 (16) bytes for
           free list pointers. Thus, the minimum allocatable size is
           16/24/32 bytes.
    
           Even a request for zero bytes (i.e., malloc(0)) returns a
           pointer to something of the minimum allocatable size.
    
           The maximum overhead wastage (i.e., number of extra bytes
           allocated than were requested in malloc) is less than or equal
           to the minimum size, except for requests >= mmap_threshold that
           are serviced via mmap(), where the worst case wastage is 2 *
           sizeof(size_t) bytes plus the remainder from a system page (the
           minimal mmap unit); typically 4096 or 8192 bytes.

*  最大块的大小

Maximum allocated size:  4-byte size_t: 2^32 minus about two pages
    			   8-byte size_t: 2^64 minus about two pages
    
           It is assumed that (possibly signed) size_t values suffice to
           represent chunk sizes. `Possibly signed' is due to the fact
           that `size_t' may be defined on a system as either a signed or
           an unsigned type. The ISO C standard says that it must be
           unsigned, but a few systems are known not to adhere to this.
           Additionally, even when size_t is unsigned, sbrk (which is by
           default used to obtain memory from system) accepts signed
           arguments, and may not be able to handle size_t-wide arguments
           with negative sign bit.  Generally, values that would
           appear as negative after accounting for overhead and alignment
           are supported only via mmap(), which does not have this
           limitation.
    
           Requests for sizes outside the allowed range will perform an optional
           failure action and then return null. (Requests may also
           also fail because a system is out of memory.)

[glibc 2.29]: https://ftp.gnu.org/gnu/libc/
[reference]: https://www.gnu.org/software/libc/manual/html_node/The-GNU-Allocator.html#The-GNU-Allocator
[Memory-mapped I_O]: https://www.gnu.org/software/libc/manual/html_node/Memory_002dmapped-I_002fO.html#Memory_002dmapped-I_002fO
[Malloc Tunable Parameters]: https://www.gnu.org/software/libc/manual/html_node/Malloc-Tunable-Parameters.html#Malloc-Tunable-Parameters
[https_sourceware.org_glibc_wiki_MallocInternals]: https://sourceware.org/glibc/wiki/MallocInternals
[Replacing malloc]: https://www.gnu.org/software/libc/manual/html_node/Replacing-malloc.html#Replacing-malloc
[reference 1]: https://blog.csdn.net/maokelong95/article/details/52006379
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM2OTYzMjE0_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200218155803548.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM2OTYzMjE0,size_16,color_FFFFFF,t_70