FastMM4-AVX Allocation Size Overflow in FullDebugMode

Technical Details

Root Cause

DebugGetMem added the debug overhead to the caller's size before any bound was applied:

Result := FastGetMem(ASize + FullDebugBlockOverhead);

When the sum wrapped, FastGetMem could allocate a small block. The footer address, derived from the original size, then pointed outside it. DebugReallocMem had the same wrap in its in-place size test. DebugAllocMem calls DebugGetMem and already handles nil.

For FreePascal, the affected interval is:

ASize > High(ptruint) - FullDebugBlockOverhead

Compilers using a signed parameter must also reject negative sizes and values above High(NativeInt) - FullDebugBlockOverhead.

FullDebugBlockOverhead is 136 bytes on 32-bit and 256 bytes on 64-bit: the 128-byte or 240-byte full-debug header, one NativeUInt trailer, and one trailing free-block pointer.

Impact In Practice

Exposure requires FullDebugMode and a path producing a size in this narrow interval, for example through underflow, signed conversion, or attacker-influenced arithmetic. The allocator cannot detect arithmetic that already wrapped to a small value.

The Original FastMM4 Was Affected As Well

The fork inherited both expressions from original FastMM4 . Upstream master before commit b09f74b and forks carrying those expressions unchanged are affected. A 32-bit FullDebugMode test of the vulnerable upstream code returned an undersized block and then terminated:

=== Test 3: Boundary Conditions ===
Attempting to allocate: $FFFFFFFF bytes (High(NativeUInt))
[FAIL] High(NativeUInt) allocation - VULNERABILITY: got pointer $7FE91940
Runtime error 204 at 00402B88
                    

Two Further Defects Found On The Way

Signed Promotion In AllocateLargeBlock

Looking for a probe that only a checked build would catch turned up a defect on the ordinary path, with nothing to do with FullDebugMode. AllocateLargeBlock pads the caller's size:

LLargeUsedBlockSize := (ASize + LargeBlockHeaderSize + LargeBlockGranularity - 1 + BlockHeaderSize)
  and LargeBlockGranularityMask;
                    

ASize is NativeUInt and the constants are untyped, so FreePascal widens the whole expression to int64. A 64-bit size at or above 2^63 then overflows that signed intermediate. MaxSafeLargeBlockSize does not stop it: at $FFFFFFFFFFE00000 it sits only 2 MB below the top of the range and passes almost the entire upper half through. Unchecked, the mask hid the wrap; with overflow checking on, the process was terminated from inside the allocator instead of receiving nil. Each constant is now added as NativeUInt, so the arithmetic stays in the type the size already has. 32-bit is unaffected, because a 32-bit NativeUInt cannot reach 2^63 inside the widened intermediate.

Truncated Size In The POSIX VirtualAlloc Shim

The POSIX shims for VirtualAlloc and VirtualFree declared dwSize as Cardinal, so a 64-bit size was reduced modulo 2^32 before valloc saw it, and valloc is declared there as taking size_t, which is pointer-sized. An oversized request therefore came back as a small, valid block rather than failing. Both parameters are now NativeUInt. The macOS shim in FastMM_OSXUtil.pas already used SIZE_T, so the non-macOS one was the outlier rather than the intended design.

Reproduction

On unmodified master before the fix, with FreePascal on Windows, place the matching precompiled FullDebugMode DLL beside the test executable and run:

cd Tests/Simple
fpc -B -Sd -Twin64 -Px86_64 -dFullDebugMode -dNoMessageBoxes IntegerOverflowTest.dpr
.\IntegerOverflowTest.exe
                    

For 32-bit, replace -Twin64 -Px86_64 with -Twin32 -Pi386. The top-of-range checks fail on both pointer widths and the run exits 1:

[FAIL] High(NativeUInt) allocation - VULNERABILITY: got pointer $7FF4FDEB8CF0
[FAIL] High(NativeUInt)-1 allocation - VULNERABILITY: got pointer $7FF4FDEB8E10
                    

Without -dNoMessageBoxes the corruption is reported interactively instead, as a modify-after-free during a later GetMem and a corrupted block header during a later FreeMem, naming the same addresses.

Adding -Criot, which turns range and overflow checking on, changes the outcome rather than merely reporting it: the wrap no longer produces a small block quietly, the process is terminated from inside the allocator with runtime error 215.

Fix

The size is refused before the addition rather than after it, in each compiler's own parameter type. That type is unsigned under FreePascal and signed under other supported compilers, where a size that has already wrapped arrives as a negative number and one just below High(NativeInt) would overflow the addition itself. Both are refused:

{$IFDEF FPC}
    LSizeRefused := ASize > (High(ptruint) - FullDebugBlockOverhead);
{$ELSE}
    LSizeRefused := (ASize < 0) or (ASize > (High(NativeInt) - NativeInt(FullDebugBlockOverhead)));
{$ENDIF}
    if LSizeRefused then
      Result := nil
    else
      Result := FastGetMem(ASize + FullDebugBlockOverhead);
                    

Writing the test in the parameter's own type rather than in NativeUInt is what makes the addition below it exact instead of merely harmless, and it is what lets the unit be compiled with overflow checking on: FastMM4 sets no {$Q} or {$R} of its own and inherits whatever the program compiling it asks for.

DebugReallocMem takes the same test as the first operand of its existing comparison. FastMM forces short-circuit Boolean evaluation, so the addition in the last operand is reached only for a size that fits it, and an unsafe size falls through to DebugGetMem, which refuses it and leaves the caller's existing block untouched.

The LSizeRefused flag also keeps the two kinds of nil apart. A size the guard refuses never reaches the allocator, so it says nothing about the address space, and the branch that releases AddressSpaceSlackPtr now runs only when the allocator actually tried and failed. That reserve exists so that a later genuine out-of-memory can still be reported.

Fix Verification

Every configuration was built twice, once from unmodified master and once from the fixed tree, with the allocation probes driven through DebugGetMem and DebugReallocMem directly.

The fixed tree runs 14 FullDebugMode checks and 8 ordinary ones and passes all of them on FreePascal 3.2.2 and 3.3.1, on both pointer widths, plain and with -Criot.

The checked runs are the sharpest evidence and were not predicted when the advisory was first written: with checking on, the wrap does not silently produce a small block, it terminates the process from inside the allocator. Continuous integration had never caught it because IntegerOverflowTest was run neither with FullDebugMode nor with range and overflow checking. Four Windows FullDebugMode jobs now run it, two of them with -Criot, each copying the tracked DLL beside the executable and stopping on its own compiler's exit code.

Workarounds

• Upgrade to v1.0.14 or later, which is the complete fix, or do not ship a build with FullDebugMode enabled. FullDebugMode is a development aid; an ordinary release build never reaches the affected code.
• Validate allocation sizes in application code before calling GetMem, ReallocMem or SetLength, rejecting any size above a reasonable application-specific maximum well below High(NativeUInt).
• Compile with range and overflow checking on. Arithmetic checking does not remove the defect, but it turns a silent undersized allocation into an immediate, diagnosable termination.
• Upstream FastMM4 master carries the guards in commit b09f74b. Its latest release, 4.993, predates the fix and must be patched or used without FullDebugMode.

CWE Mapping

• CWE-190: Integer Overflow or Wraparound. The debug overhead is added to the caller's size before the sum is bounded, so a size near the top of the range wraps to a small value.
• CWE-680: Integer Overflow to Buffer Overflow. The wrapped sum is what the underlying allocator is asked for, so the block returned is far smaller than the size recorded for it.
• CWE-787: Out-of-bounds Write. The block footer address is computed from the recorded size, so the footer is written outside the block that was allocated.

Timeline

Credit

Found and fixed by Maxim Masiutin, maintainer of FastMM4-AVX. The address space slack defect in the first version of the guard, and the truncating POSIX VirtualAlloc shim, were both pointed out by CodeRabbit during review of pull request 164.

Other FastMM4-AVX Security Advisories