CVE-2009-3609: Integer Overflow

An integer overflow flaw exists in xpdf's ImageStream::ImageStream (Stream.cc) when calculating size of the imgLine buffer:

320 nVals = width nComps; 321 if (nBits == 1) { 322 imgLineSize = (nVals + 7) & ~7; 323 } else { 324 imgLineSize = nVals; 325 } 326 imgLine = (Guchar )gmallocn(imgLineSize, sizeof(Guchar));

width and nComps used go compute nVals value come from the input PDF file. Their multiplication may overflow / wrap, resulting in smaller imgLine buffer allocation than expected.

ImageStream always uses nVals as an upper bound when writing data into imgLine. Therefore, no buffer overflow occurs in ImageStream, but NULL pointer dereference may occur (gmallocn returns NULL when called with imgLineSize 0).

Pointer to imgLine is also returned out of ImageStream class instance from ImageStream::getLine() method. Callers of the method may later over-read allocated buffer, but no caller over-writing it was identified.

This code was introduced in xpdf in some early versions (exists in 0.9x), so is likely to appear in all applications embedding / forking xpdf.