// HACKER NEWS — CYBERSECURITY
Comparison of Malloc() Algorithms
Multithreaded programs often do not scale because the heap is a
bottleneck.
When multiple threads simultaneously allocate or deallocate memory from
the allocator, the allocator will serialize them. Programs making
intensive use of the allocator actually slow down as the number of
processors increases.
Malloc (libc) is the worst memory allocation API to use.
Programs should avoid, if possible, allocating/deallocating memory too
often and in particular whenever a packet is received.
In the Linux kernel there are available kernel/driver patches for
recycling skbuff (kernel memory used to store incoming/outgoing
packets).
Using PF_RING (into the driver) for copying packets from the NIC to the
circular buffer without any memory allocation increases the capture
performance (around 10%) and reduces congestion issues.
Basic design of malloc() is to dynamically pre-allocate a pool of memory
from the OS in which applications can then take smaller pieces from.
malloc() is a standard API having a choice of different allocation
algorithms and to mitigate the expensive OS system calls (typically done
at program initialization time) during allocation of its system memory.
The first memory allocation scheme started with a stack-based memory
allocation.
Next came the dynamic-based memory allocation scheme where linked-list
and bucket-heap mechanism are used to divide the private-heap using size
class approach.
Soon, garbage collection algorithm introduced the initial backend of the
memory allocation scheme. Frontend covers the usual malloc() API, et
al.
In 2006, a third pool was introduced (after operating system memory pool
and library-based memory pool) called the “arena”. Arena is a
jemalloc-term and is intended to deal with different memory types such
as different-speed memory bank or NUMA-architecture, as well as memory
tied to specific to each of the multiple CPU core or even CPU infinity.