Xine-lib contains an integer overflow vulnerability while parsing malformed STTS atoms of Quicktime movie files. The vulnerability may be exploited by a (remote) attacker to execute arbitrary code in the context of an application using the xine library.
[1] The unsigned int variable "trak->time_to_sample_count" is filled with user supplied data from the media file.
[2] In the lines 1548 and 1549 an integer overflow happens as the first argument to calloc() is calculated with the addition "trak->time_to_sample_count+1". A user supplied "trak->time_to_sample_count" of UINT_MAX (0xffffffff) will cause an integer overflow within the first parameter of calloc() and therefore only allocate a 0 byte buffer. Please notice that calloc(0, sizeof(time_to_sample_table_t)) will not return a NULL pointer but a pointer into the legal heap on at least platforms like Windows and Linux.
[3] The value of "trak->time_to_sample_count" is used as a counter in this for() loop.
[4] User controlled data from the quicktime movie file gets copied into thepreviously allocated heap buffer (see [2]). As "j" is used as an array index and the for() loop is executed until "j < trak->time_to_sample_count" it is possible to overflow the heap buffer with user controlled data from the quicktime movie file.
[5] Same as [4]
Solution:
Upgrade to xine-lib version 1.1.16.3 or newer.
Disclosure Timeline:
2009/03/05 - xine-lib maintainers notified (bugs.xine-project.org)
2009/04/03 - Public disclosure of vulnerability details by xine-lib maintainers
2009/04/04 - Release date of this security advisory
