fix integer overflows and uncaught EOVERFLOW in printf core
this patch fixes a large number of missed internal signed-overflow
checks and errors in determining when the return value (output length)
would exceed INT_MAX, which should result in EOVERFLOW. some of the
issues fixed were reported by Alexander Cherepanov; others were found
in subsequent review of the code.
aside from the signed overflows being undefined behavior, the
following specific bugs were found to exist in practice:
- overflows computing length of floating point formats with huge
explicit precisions, integer formats with prefix characters and huge
explicit precisions, or string arguments or format strings longer
than INT_MAX, resulted in wrong return value and wrong %n results.
- literal width and precision values outside the range of int were
misinterpreted, yielding wrong behavior in at least one well-defined
case: string formats with precision greater than INT_MAX were
sometimes truncated.
- in cases where EOVERFLOW is produced, incorrect values could be
written for %n specifiers past the point of exceeding INT_MAX.
in addition to fixing these bugs, we now stop producing output
immediately when output length would exceed INT_MAX, rather than
continuing and returning an error only at the end.
2 files changed