You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There is a segmentation fault in CABAC_encoder_bitstream::append_byte and CABAC_encoder_bitstream::write_startcode if the call to realloc fails in check_size_and_resize
When executing CABAC_encoder_bitstream::check_size_and_resize, there is no check to ensure the call to realloc is successful.
In case it is not, it will return NULL that will be set as the value of data_mem. Later on, the variable is accessed in CABAC_encoder_bitstream::append_byte:
Summary
There is a segmentation fault in CABAC_encoder_bitstream::append_byte and CABAC_encoder_bitstream::write_startcode if the call to realloc fails in check_size_and_resize
Tested with:
./enc265/enc265 -i <yuv file>
Analysis
When executing
CABAC_encoder_bitstream::check_size_and_resize
, there is no check to ensure the call torealloc
is successful.In case it is not, it will return NULL that will be set as the value of
data_mem
. Later on, the variable is accessed inCABAC_encoder_bitstream::append_byte
:and
CABAC_encoder_bitstream::write_startcode
:resulting in a segmentation fault due to trying to access a NULL pointer.
Impact
Patch
In order to prevent a crash, a check for
data_mem
not being NULL should be added that can handle the case appropriately.The text was updated successfully, but these errors were encountered: