Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pkg.test() is failing on Windows 64 bit #40

Closed
bmharsha opened this issue Feb 22, 2016 · 5 comments
Closed

Pkg.test() is failing on Windows 64 bit #40

bmharsha opened this issue Feb 22, 2016 · 5 comments

Comments

@bmharsha
Copy link
Member

Pkg.test() is failing while executing prepared statements when the value is NULL, I've included the exact error message below

julia> Pkg.test("MySQL")
INFO: Testing MySQL

*** Running Basic Test ***

SUCCESS: Create database
SUCCESS: Create user
SUCCESS: Grant privilege
SUCCESS: Create table
SUCCESS: Insert
Last insert id was 1
SUCCESS: Update
SUCCESS: Insert Null row

 *** Results as Dataframe:
5x10 DataFrames.DataFrame
| Row | ID | Name   | Salary  | JoinDate   | LastLogin           |
|-----|----|--------|---------|------------|---------------------|
| 1   | 1  | "John" | 10000.5 | 2015-08-03 | 2015-09-05T12:31:30 |
| 2   | 2  | "Tom"  | 20000.3 | 2015-08-04 | 2015-10-12T13:12:14 |
| 3   | 3  | "Jim"  | 25000.0 | 2015-06-02 | 2015-09-05T10:05:10 |
| 4   | 4  | "Tim"  | 25000.0 | 2015-07-25 | 2015-10-10T12:12:25 |
| 5   | 5  | NA     | NA      | NA         | NA                  |

| Row | LunchTime           | OfficeNo | JobType      | Senior | empno |
|-----|---------------------|----------|--------------|--------|-------|
| 1   | 1970-01-01T12:00:00 | 1        | "HR"         | 0x01   | 1301  |
| 2   | 1970-01-01T13:00:00 | 12       | "HR"         | 0x01   | 1422  |
| 3   | 1970-01-01T12:30:00 | 45       | "Management" | NA     | 1567  |
| 4   | 1970-01-01T12:30:00 | 56       | "Accounts"   | 0x01   | 3200  |
| 5   | NA                  | NA       | NA           | NA     | NA    |

 *** Expected Result:
5x10 DataFrames.DataFrame
| Row | ID | Name   | Salary  | JoinDate   | LastLogin           |
|-----|----|--------|---------|------------|---------------------|
| 1   | 1  | "John" | 10000.5 | 2015-08-03 | 2015-09-05T12:31:30 |
| 2   | 2  | "Tom"  | 20000.3 | 2015-08-04 | 2015-10-12T13:12:14 |
| 3   | 3  | "Jim"  | 25000.0 | 2015-06-02 | 2015-09-05T10:05:10 |
| 4   | 4  | "Tim"  | 25000.0 | 2015-07-25 | 2015-10-10T12:12:25 |
| 5   | 5  | NA     | NA      | NA         | NA                  |

| Row | LunchTime           | OfficeNo | JobType      | Senior | empno |
|-----|---------------------|----------|--------------|--------|-------|
| 1   | 1970-01-01T12:00:00 | 1        | "HR"         | 0x01   | 1301  |
| 2   | 1970-01-01T13:00:00 | 12       | "HR"         | 0x01   | 1422  |
| 3   | 1970-01-01T12:30:00 | 45       | "Management" | NA     | 1567  |
| 4   | 1970-01-01T12:30:00 | 56       | "Accounts"   | 0x01   | 3200  |
| 5   | NA                  | NA       | NA           | NA     | NA    |

 *** Results using Iterator:

(1,Nullable("John"),Nullable(10000.5f0),Nullable(2015-08-03),Nullable(2015-09-05
T12:31:30),Nullable(1970-01-01T12:00:00),Nullable(1),Nullable("HR"),Nullable(0x0
1),Nullable(1301))
(2,Nullable("Tom"),Nullable(20000.25f0),Nullable(2015-08-04),Nullable(2015-10-12
T13:12:14),Nullable(1970-01-01T13:00:00),Nullable(12),Nullable("HR"),Nullable(0x
01),Nullable(1422))
(3,Nullable("Jim"),Nullable(25000.0f0),Nullable(2015-06-02),Nullable(2015-09-05T
10:05:10),Nullable(1970-01-01T12:30:00),Nullable(45),Nullable("Management"),Null
able{UInt8}(),Nullable(1567))
(4,Nullable("Tim"),Nullable(25000.0f0),Nullable(2015-07-25),Nullable(2015-10-10T
12:12:25),Nullable(1970-01-01T12:30:00),Nullable(56),Nullable("Accounts"),Nullab
le(0x01),Nullable(3200))
(5,Nullable{AbstractString}(),Nullable{Float32}(),Nullable{Date}(),Nullable{Date
Time}(),Nullable{DateTime}(),Nullable{Int8}(),Nullable{AbstractString}(),Nullabl
e{UInt8}(),Nullable{Int16}())

 *** Results as tuples:

Tuple[(1,Nullable("John"),Nullable(10000.5f0),Nullable(2015-08-03),Nullable(2015
-09-05T12:31:30),Nullable(1970-01-01T12:00:00),Nullable(1),Nullable("HR"),Nullab
le(0x01),Nullable(1301)),(2,Nullable("Tom"),Nullable(20000.25f0),Nullable(2015-0
8-04),Nullable(2015-10-12T13:12:14),Nullable(1970-01-01T13:00:00),Nullable(12),N
ullable("HR"),Nullable(0x01),Nullable(1422)),(3,Nullable("Jim"),Nullable(25000.0
f0),Nullable(2015-06-02),Nullable(2015-09-05T10:05:10),Nullable(1970-01-01T12:30
:00),Nullable(45),Nullable("Management"),Nullable{UInt8}(),Nullable(1567)),(4,Nu
llable("Tim"),Nullable(25000.0f0),Nullable(2015-07-25),Nullable(2015-10-10T12:12
:25),Nullable(1970-01-01T12:30:00),Nullable(56),Nullable("Accounts"),Nullable(0x
01),Nullable(3200)),(5,Nullable{AbstractString}(),Nullable{Float32}(),Nullable{D
ate}(),Nullable{DateTime}(),Nullable{DateTime}(),Nullable{Int8}(),Nullable{Abstr
actString}(),Nullable{UInt8}(),Nullable{Int16}())]
SUCCESS: Drop table
SUCCESS: Drop user
SUCCESS: Drop database

*** Running Prepared Statement Test ***

Success: Create database
Success: Create user
Success: Grant privilege
Success: Create table
Affected rows after insert_values(): 4
Last insert id was 4
Affected rows after update_values(): 2
Success: Insert Null row

 *** Results as dataframe:
5x10 DataFrames.DataFrame
| Row | ID | Name   | Salary  | JoinDate   | LastLogin           |
|-----|----|--------|---------|------------|---------------------|
| 1   | 1  | "John" | 10000.5 | 2015-08-03 | 2015-09-05T12:31:30 |
| 2   | 2  | "Tom"  | 20000.3 | 2015-08-04 | 2015-10-12T13:12:14 |
| 3   | 3  | "Jim"  | 25000.0 | 2015-06-02 | 2015-09-05T10:05:10 |
| 4   | 4  | "Tim"  | 25000.0 | 2015-07-25 | 2015-10-10T12:12:25 |
| 5   | 5  | "Tim"  | 25000.0 | 2015-07-25 | 2015-10-10T12:12:25 |

| Row | LunchTime           | OfficeNo | JobType | Senior | empno |
|-----|---------------------|----------|---------|--------|-------|
| 1   | 1970-01-01T12:00:00 | 1        | "L\f"     | 0x31   | 1301  |
| 2   | 1970-01-01T13:00:00 | 12       | "L\f"     | 0x31   | 1422  |
| 3   | 1970-01-01T12:30:00 | 45       | "L\f"     | 0x31   | 1567  |
| 4   | 1970-01-01T12:30:00 | 56       | "L\f"     | 0x31   | 3200  |
| 5   | 1970-01-01T12:30:00 | 56       | "L\f"     | 0x31   | 3200  |

 *** Expected result:
5x10 DataFrames.DataFrame
| Row | ID | Name   | Salary  | JoinDate   | LastLogin           |
|-----|----|--------|---------|------------|---------------------|
| 1   | 1  | "John" | 10000.5 | 2015-08-03 | 2015-09-05T12:31:30 |
| 2   | 2  | "Tom"  | 20000.5 | 2015-08-04 | 2015-10-12T13:12:14 |
| 3   | 3  | "Jim"  | 25000.0 | 2015-06-02 | 2015-09-05T10:05:10 |
| 4   | 4  | "Tim"  | 25000.0 | 2015-07-25 | 2015-10-10T12:12:25 |
| 5   | 5  | NA     | NA      | NA         | NA                  |

| Row | LunchTime           | OfficeNo | JobType | Senior | empno |
|-----|---------------------|----------|---------|--------|-------|
| 1   | 1970-01-01T12:00:00 | 1        | NA      | NA     | 1301  |
| 2   | 1970-01-01T13:00:00 | 12       | NA      | NA     | 1422  |
| 3   | 1970-01-01T12:30:00 | 45       | NA      | NA     | 1567  |
| 4   | 1970-01-01T12:30:00 | 56       | NA      | NA     | 3200  |
| 5   | NA                  | NA       | NA      | NA     | NA    |
ERROR: LoadError: LoadError: test failed: dfisequal(dframe,DataFrameResultsPrep)

 in expression: dfisequal(dframe,DataFrameResultsPrep)
 in error at error.jl:21
while loading C:\Users\Administrator\AppData\Local\JuliaPackages\Feb2016\v0.4\My
SQL\test\test_prep.jl, in expression starting on line 139
while loading C:\Users\Administrator\AppData\Local\JuliaPackages\Feb2016\v0.4\My
SQL\test\runtests.jl, in expression starting on line 8
================================[ ERROR: MySQL ]================================


failed process: Process(`'C:\Users\Administrator\AppData\Local\Julia-0.4.3\bin\j
ulia' --check-bounds=yes --code-coverage=none --color=yes 'C:\Users\Administrato
r\AppData\Local\JuliaPackages\Feb2016\v0.4\MySQL\test\runtests.jl'`, ProcessExit
ed(1)) [1]

================================================================================

ERROR: MySQL had test errors
 in error at error.jl:21
Configuration
  • OS = Windows 7, 64 bit
  • Julia version 0.4.3
  • MySQL version 5.7
@amellnik
Copy link
Contributor

I also can't run the tests on x64 windows with a similar error at the same place. It looks like NAs are not being returned correctly. I also had to fiddle with init_test() to get that far as the method for making sure the user doesn't exist before the test doesn't seem to work.

The error is in reading the results - the values appear in MySQL correctly (and the results-as-tuples test works):

image

@amellnik
Copy link
Contributor

A bit more isolation: it's specifically related to bound select statements and not to either tuples or dataframes:

image

image

Running the second cell repeatedly produced different types of garbage in place of the nulls each time.

As best I can tell, the prepared statement version calls MySQL.mysql_result_to_dataframe(hndl) and at some point bindarr[i].is_null_value is not being set correctly, but that's where I got lost.

@nkottary
Copy link
Member

The MySQL bug that I mentioned in the comment in init_test is fixed in MySQL 5.7, so the last two lines in that function can be replaced with:

mysql_execute(hndl, "DROP USER IF EXISTS 'test'@'127.0.0.1';" )

The NA's not showing up is caused by the fact that Culong on Windows 64-bit in UInt32 and on Linux 64-bit is UInt64. So the offsets in unsafe_store! for MYSQL_BIND in mysql_bind_array are wrong.

@nkottary
Copy link
Member

It looks like I made a mistake while defining the MYSQL_BIND type. I declared variables declared as unsigned long long in MySQL C headers as Culong in Julia instead of Culonglong. This should fix the issue.

@amellnik
Copy link
Contributor

I tried to make this change but I don't actually see where there should be a Culonglong. My copy of mysql.h in my c connector has

typedef struct st_mysql_bind
{
  unsigned long *length;          /* output length pointer */
  my_bool       *is_null;     /* Pointer to null indicator */
  void      *buffer;      /* buffer to get/put data */
  /* set this if you want to track data truncations happened during fetch */
  my_bool       *error;
  unsigned char *row_ptr;         /* for the current data position */
  void (*store_param_func)(NET *net, struct st_mysql_bind *param);
  void (*fetch_result)(struct st_mysql_bind *, MYSQL_FIELD *,
                       unsigned char **row);
  void (*skip_result)(struct st_mysql_bind *, MYSQL_FIELD *,
              unsigned char **row);
  /* output buffer length, must be set when fetching str/binary */
  unsigned long buffer_length;
  unsigned long offset;           /* offset position for char/binary fetch */
  unsigned long length_value;     /* Used if length is 0 */
  unsigned int  param_number;     /* For null count and error messages */
  unsigned int  pack_length;      /* Internal length for packed data */
  enum enum_field_types buffer_type;    /* buffer type */
  my_bool       error_value;      /* used if error is 0 */
  my_bool       is_unsigned;      /* set if integer type is unsigned */
  my_bool   long_data_used;   /* If used with mysql_send_long_data */
  my_bool   is_null_value;    /* Used if is_null is 0 */
  void *extension;
} MYSQL_BIND;

and I don't see a mysql_bind.h anywhere. I think I see two fields in MYSQL_STMT that should be Clonglongs though if I'm reading things correctly:

immutable MYSQL_STMT # This is different in mariadb header file.
    mem_root::MEM_ROOT
    list::LIST
    mysql::Ptr{Void}
    params::MYSQL_BIND
    bind::MYSQL_BIND
    fields::MYSQL_FIELD
    result::MYSQL_DATA
    data_cursor::MYSQL_ROWS

    affected_rows::Culonglong #Changed
    insert_id::Culonglong #Changed
    stmt_id::Culong
    flags::Culong
    prefetch_rows::Culong

    server_status::Cuint
    last_errno::Cuint
    param_count::Cuint
    field_count::Cuint
    state::Cuint
    last_error::Ptr{Cchar}
    sqlstate::Ptr{Cchar}
    send_types_to_server::Cint
    bind_param_done::Cint
    bind_result_done::Cuchar
    unbuffered_fetch_cancelled::Cint
    update_max_length::Cint
    extension::Ptr{Cuchar}
end

For the offsets I see

immutable MYSQL_BIND
    length::Ptr{Culong} # 8
    is_null::Ptr{Cchar} # 8 
    buffer::Ptr{Void} # 8
    error::Ptr{Cchar} # 8
    row_ptr::Ptr{Cuchar} # 8
    store_param_func ::Ptr{Void} # 8
    fetch_result ::Ptr{Void} # 8
    skip_result ::Ptr{Void} # 8
    buffer_length::Culong  # sizeof(Clong)
    offset::Culong # sizeof(Clong)
    length_value::Culong # sizeof(Clong)
    param_number :: Cuint # 4
    pack_length :: Cuint # 4
    buffer_type :: Cint # 4
    error_value :: Cchar # 1
    is_unsigned :: Cchar # 1
    long_data_used :: Cchar # 1
    is_null_value :: Cchar 
    extension :: Ptr{Void}

so it looks like the offset should be 8*8 + sizeof(Clong)*3 + 4*3 + 3 -- 103 for Linux and 91 for x64 Windows.

Miraculously tests pass for me after this change, but someone should check and make sure I didn't break it for other systems. I'll submit a pull request mostly to see if it passes travis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants