tty: Clarify re-open behavior of master ptys

Re-opening master ptys is not allowed. Once opened and for the remaining
lifetime of the master pty, its tty count is 1. If its tty count has
dropped to 0, then the master pty was closed and TTY_CLOSING was set,
and destruction may begin imminently.

Besides the normal case of a legacy BSD pty master being re-opened
(which always returns -EIO), this code is only reachable in 2 degenerate
cases:
1. The pty master is the controlling terminal (this is possible through
   the TIOCSCTTY ioctl). pty masters are not designed to be controlling
   terminals and it's an oversight that tiocsctty() ever let that happen.
   The attempted open of /dev/tty will always fail. No known program does
   this.
2. The legacy BSD pty slave was opened first. The slave open will fail
   in pty_open() and tty_release() will commence. But before tty_release()
   claims the tty_mutex, there is a very small window where a parallel
   master open might succeed. In a test of racing legacy BSD slave and
   master parallel opens, where:
      slave open attempts:  10000   success:4527  failure:5473
      master open attempts: 11728   success:5789  failure:5939
   only 8 master open attempts would have succeeded reaching this code and
   successfully opened the master pty. This case is not possible with
   SysV ptys.

Always return -EIO if a master pty is re-opened or the slave is opened
first and the master opened in parallel (for legacy BSD ptys).

Furthermore, now that changing the slave's count is not required,
the tty_lock is sufficient for preventing concurrent changes to the
tty being re-opened (or failing re-opening).

Reviewed-by: Alan Cox <alan@linux.intel.com>
Signed-off-by: Peter Hurley <peter@hurleysoftware.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
1 file changed