BUG#17259750 - STACK CORRUPTION IN VIO_IO_WAIT ON MAC OS X

Description & Fix:
On OS X, vio_io_wait is implemented using
select system call (as per analysis in bug#11748945).
The select system call cannot handle file descriptors
greater than or equal to FD_SETSIZE. This causes
stack corruption when FD_SET is used on this range
of file descriptors.
This fix is check if fd exceeds or equals FD_SETSIZE in
vio_io_wait and return failure. Also if the connected
file descriptor exceeds or equal FD_SETSIZE, do not
accept the connection on OS X.

Author: thayumanavar77

sql/conn_handler/socket_connection.cc

@@ -921,6 +921,17 @@ Channel_info* Mysqld_socket_listener::listen_for_connection_event()
     return NULL;
   }
 
+#ifdef __APPLE__
+  if (mysql_socket_getfd(connect_sock) >= FD_SETSIZE)
+  {
+    sql_print_warning("File Descriptor %d exceedeed FD_SETSIZE=%d",
+                      mysql_socket_getfd(connect_sock), FD_SETSIZE);
+    connection_errors_internal++;
+    (void) mysql_socket_close(connect_sock);
+    return NULL;
+  }
+#endif
+
 #ifdef HAVE_LIBWRAP
   if (!is_unix_socket)
   {

vio/viosocket.c

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2001, 2013, Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2001, 2015, Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or
    modify it under the terms of the GNU General Public License
@@ -821,6 +821,11 @@ int vio_io_wait(Vio *vio, enum enum_vio_io_event event, int timeout)
   if (fd == INVALID_SOCKET)
     DBUG_RETURN(-1);
 
+#ifdef __APPLE__
+  if (fd >= FD_SETSIZE)
+    DBUG_RETURN(-1);
+#endif
+
   /* Convert the timeout, in milliseconds, to seconds and microseconds. */
   if (timeout >= 0)
   {