Merge branch 'master' into test-linear

This commit is contained in:
Roland Bracewell Shoemaker 2016-02-16 14:41:04 -08:00
commit 736376173a
28 changed files with 995 additions and 2673 deletions

View File

@ -55,6 +55,7 @@ env:
matrix:
- RUN="integration vet lint fmt migrations"
- RUN="unit"
- RUN="godep-restore"
script:
- bash test.sh

6
Godeps/Godeps.json generated
View File

@ -1,6 +1,6 @@
{
"ImportPath": "github.com/letsencrypt/boulder",
"GoVersion": "go1.5.2",
"GoVersion": "go1.5",
"Packages": [
"./..."
],
@ -89,8 +89,8 @@
},
{
"ImportPath": "github.com/go-sql-driver/mysql",
"Comment": "v1.2-88-ga197e5d",
"Rev": "a197e5d40516f2e9f74dcee085a5f2d4604e94df"
"Comment": "v1.2-171-g267b128",
"Rev": "267b128680c46286b9ca13475c3cca5de8f79bd7"
},
{
"ImportPath": "github.com/golang/mock/gomock",

View File

@ -4,6 +4,7 @@ go:
- 1.2
- 1.3
- 1.4
- 1.5
- tip
before_script:

View File

@ -15,22 +15,31 @@ Aaron Hopkins <go-sql-driver at die.net>
Arne Hormann <arnehormann at gmail.com>
Carlos Nieto <jose.carlos at menteslibres.net>
Chris Moos <chris at tech9computers.com>
Daniel Nichter <nil at codenode.com>
DisposaBoy <disposaboy at dby.me>
Frederick Mayle <frederickmayle at gmail.com>
Gustavo Kristic <gkristic at gmail.com>
Hanno Braun <mail at hannobraun.com>
Henri Yandell <flamefew at gmail.com>
Hirotaka Yamamoto <ymmt2005 at gmail.com>
INADA Naoki <songofacandy at gmail.com>
James Harr <james.harr at gmail.com>
Jian Zhen <zhenjl at gmail.com>
Joshua Prunier <joshua.prunier at gmail.com>
Julien Lefevre <julien.lefevr at gmail.com>
Julien Schmidt <go-sql-driver at julienschmidt.com>
Kamil Dziedzic <kamil at klecza.pl>
Kevin Malachowski <kevin at chowski.com>
Leonardo YongUk Kim <dalinaum at gmail.com>
Luca Looz <luca.looz92 at gmail.com>
Lucas Liu <extrafliu at gmail.com>
Luke Scott <luke at webconnex.com>
Michael Woolnough <michael.woolnough at gmail.com>
Nicola Peduzzi <thenikso at gmail.com>
Runrioter Wung <runrioter at gmail.com>
Soroush Pour <me at soroushjp.com>
Stan Putrya <root.vagner at gmail.com>
Stanley Gunawan <gunawan.stanley at gmail.com>
Xiaobing Jiang <s7v7nislands at gmail.com>
Xiuming Chen <cc at cxm.cc>

View File

@ -12,10 +12,21 @@ Bugfixes:
- Enable microsecond resolution on TIME, DATETIME and TIMESTAMP (#249)
- Fixed handling of queries without columns and rows (#255)
- Fixed a panic when SetKeepAlive() failed (#298)
- Support receiving ERR packet while reading rows (#321)
- Fixed reading NULL length-encoded integers in MySQL 5.6+ (#349)
- Fixed absolute paths support in LOAD LOCAL DATA INFILE (#356)
- Actually zero out bytes in handshake response (#378)
- Fixed race condition in registering LOAD DATA INFILE handler (#383)
- Fixed tests with MySQL 5.7.9+ (#380)
- QueryUnescape TLS config names (#397)
- Fixed "broken pipe" error by writing to closed socket (#390)
New Features:
- Support for returning table alias on Columns() (#289)
- Support for returning table alias on Columns() (#289, #359, #382)
- Placeholder interpolation, can be actived with the DSN parameter `interpolateParams=true` (#309, #318)
- Support for uint64 parameters with high bit set (#332, #345)
- Cleartext authentication plugin support (#327)
## Version 1.2 (2014-06-03)

View File

@ -30,7 +30,7 @@ A MySQL-Driver for Go's [database/sql](http://golang.org/pkg/database/sql) packa
## Features
* Lightweight and [fast](https://github.com/go-sql-driver/sql-benchmark "golang MySQL-Driver performance")
* Native Go implementation. No C-bindings, just pure Go
* Connections over TCP/IPv4, TCP/IPv6 or Unix domain sockets
* Connections over TCP/IPv4, TCP/IPv6, Unix domain sockets or [custom protocols](http://godoc.org/github.com/go-sql-driver/mysql#DialFunc)
* Automatic handling of broken connections
* Automatic Connection Pooling *(by database/sql package)*
* Supports queries larger than 16MB
@ -93,6 +93,8 @@ This has the same effect as an empty DSN string:
```
Alternatively, [Config.FormatDSN](https://godoc.org/github.com/go-sql-driver/mysql#Config.FormatDSN) can be used to create a DSN string by filling a struct.
#### Password
Passwords can consist of any character. Escaping is **not** necessary.
@ -123,6 +125,16 @@ Default: false
`allowAllFiles=true` disables the file Whitelist for `LOAD DATA LOCAL INFILE` and allows *all* files.
[*Might be insecure!*](http://dev.mysql.com/doc/refman/5.7/en/load-data-local.html)
##### `allowCleartextPasswords`
```
Type: bool
Valid Values: true, false
Default: false
```
`allowCleartextPasswords=true` allows using the [cleartext client side plugin](http://dev.mysql.com/doc/en/cleartext-authentication-plugin.html) if required by an account, such as one defined with the [PAM authentication plugin](http://dev.mysql.com/doc/en/pam-authentication-plugin.html). Sending passwords in clear text may be a security problem in some configurations. To avoid problems if there is any possibility that the password would be intercepted, clients should connect to MySQL Server using a method that protects the password. Possibilities include [TLS / SSL](#tls), IPsec, or a private network.
##### `allowOldPasswords`
```
@ -205,8 +217,20 @@ Default: UTC
Sets the location for time.Time values (when using `parseTime=true`). *"Local"* sets the system's location. See [time.LoadLocation](http://golang.org/pkg/time/#LoadLocation) for details.
Note that this sets the location for time.Time values but does not change MySQL's [time_zone setting](https://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html). For that see the [time_zone system variable](#system-variables), which can also be set as a DSN parameter.
Please keep in mind, that param values must be [url.QueryEscape](http://golang.org/pkg/net/url/#QueryEscape)'ed. Alternatively you can manually replace the `/` with `%2F`. For example `US/Pacific` would be `loc=US%2FPacific`.
##### `multiStatements`
```
Type: bool
Valid Values: true, false
Default: false
```
Allow multiple statements in one query. While this allows batch queries, it also greatly increases the risk of SQL injections. Only the result of the first query is returned, all other results are silently discarded.
##### `parseTime`
@ -219,6 +243,16 @@ Default: false
`parseTime=true` changes the output type of `DATE` and `DATETIME` values to `time.Time` instead of `[]byte` / `string`
##### `readTimeout`
```
Type: decimal number
Default: 0
```
I/O read timeout. The value must be a decimal number with an unit suffix ( *"ms"*, *"s"*, *"m"*, *"h"* ), such as *"30s"*, *"0.5m"* or *"1m30s"*.
##### `strict`
```
@ -239,7 +273,7 @@ Type: decimal number
Default: OS default
```
*Driver* side connection timeout. The value must be a string of decimal numbers, each with optional fraction and a unit suffix ( *"ms"*, *"s"*, *"m"*, *"h"* ), such as *"30s"*, *"0.5m"* or *"1m30s"*. To set a server side timeout, use the parameter [`wait_timeout`](http://dev.mysql.com/doc/refman/5.6/en/server-system-variables.html#sysvar_wait_timeout).
*Driver* side connection timeout. The value must be a decimal number with an unit suffix ( *"ms"*, *"s"*, *"m"*, *"h"* ), such as *"30s"*, *"0.5m"* or *"1m30s"*. To set a server side timeout, use the parameter [`wait_timeout`](http://dev.mysql.com/doc/refman/5.6/en/server-system-variables.html#sysvar_wait_timeout).
##### `tls`
@ -253,11 +287,21 @@ Default: false
`tls=true` enables TLS / SSL encrypted connection to the server. Use `skip-verify` if you want to use a self-signed or invalid certificate (server side). Use a custom value registered with [`mysql.RegisterTLSConfig`](http://godoc.org/github.com/go-sql-driver/mysql#RegisterTLSConfig).
##### `writeTimeout`
```
Type: decimal number
Default: 0
```
I/O write timeout. The value must be a decimal number with an unit suffix ( *"ms"*, *"s"*, *"m"*, *"h"* ), such as *"30s"*, *"0.5m"* or *"1m30s"*.
##### System Variables
All other parameters are interpreted as system variables:
* `autocommit`: `"SET autocommit=<value>"`
* `time_zone`: `"SET time_zone=<value>"`
* [`time_zone`](https://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html): `"SET time_zone=<value>"`
* [`tx_isolation`](https://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_tx_isolation): `"SET tx_isolation=<value>"`
* `param`: `"SET <param>=<value>"`
@ -319,7 +363,7 @@ import "github.com/go-sql-driver/mysql"
Files must be whitelisted by registering them with `mysql.RegisterLocalFile(filepath)` (recommended) or the Whitelist check must be deactivated by using the DSN parameter `allowAllFiles=true` ([*Might be insecure!*](http://dev.mysql.com/doc/refman/5.7/en/load-data-local.html)).
To use a `io.Reader` a handler function must be registered with `mysql.RegisterReaderHandler(name, handler)` which returns a `io.Reader` or `io.ReadCloser`. The Reader is available with the filepath `Reader::<name>` then.
To use a `io.Reader` a handler function must be registered with `mysql.RegisterReaderHandler(name, handler)` which returns a `io.Reader` or `io.ReadCloser`. The Reader is available with the filepath `Reader::<name>` then. Choose different names for different handlers and `DeregisterReaderHandler` when you don't need it anymore.
See the [godoc of Go-MySQL-Driver](http://godoc.org/github.com/go-sql-driver/mysql "golang mysql driver documentation") for details.

View File

@ -1,246 +0,0 @@
// Go MySQL Driver - A MySQL-Driver for Go's database/sql package
//
// Copyright 2013 The Go-MySQL-Driver Authors. All rights reserved.
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at http://mozilla.org/MPL/2.0/.
package mysql
import (
"bytes"
"database/sql"
"database/sql/driver"
"math"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
)
type TB testing.B
func (tb *TB) check(err error) {
if err != nil {
tb.Fatal(err)
}
}
func (tb *TB) checkDB(db *sql.DB, err error) *sql.DB {
tb.check(err)
return db
}
func (tb *TB) checkRows(rows *sql.Rows, err error) *sql.Rows {
tb.check(err)
return rows
}
func (tb *TB) checkStmt(stmt *sql.Stmt, err error) *sql.Stmt {
tb.check(err)
return stmt
}
func initDB(b *testing.B, queries ...string) *sql.DB {
tb := (*TB)(b)
db := tb.checkDB(sql.Open("mysql", dsn))
for _, query := range queries {
if _, err := db.Exec(query); err != nil {
if w, ok := err.(MySQLWarnings); ok {
b.Logf("Warning on %q: %v", query, w)
} else {
b.Fatalf("Error on %q: %v", query, err)
}
}
}
return db
}
const concurrencyLevel = 10
func BenchmarkQuery(b *testing.B) {
tb := (*TB)(b)
b.StopTimer()
b.ReportAllocs()
db := initDB(b,
"DROP TABLE IF EXISTS foo",
"CREATE TABLE foo (id INT PRIMARY KEY, val CHAR(50))",
`INSERT INTO foo VALUES (1, "one")`,
`INSERT INTO foo VALUES (2, "two")`,
)
db.SetMaxIdleConns(concurrencyLevel)
defer db.Close()
stmt := tb.checkStmt(db.Prepare("SELECT val FROM foo WHERE id=?"))
defer stmt.Close()
remain := int64(b.N)
var wg sync.WaitGroup
wg.Add(concurrencyLevel)
defer wg.Wait()
b.StartTimer()
for i := 0; i < concurrencyLevel; i++ {
go func() {
for {
if atomic.AddInt64(&remain, -1) < 0 {
wg.Done()
return
}
var got string
tb.check(stmt.QueryRow(1).Scan(&got))
if got != "one" {
b.Errorf("query = %q; want one", got)
wg.Done()
return
}
}
}()
}
}
func BenchmarkExec(b *testing.B) {
tb := (*TB)(b)
b.StopTimer()
b.ReportAllocs()
db := tb.checkDB(sql.Open("mysql", dsn))
db.SetMaxIdleConns(concurrencyLevel)
defer db.Close()
stmt := tb.checkStmt(db.Prepare("DO 1"))
defer stmt.Close()
remain := int64(b.N)
var wg sync.WaitGroup
wg.Add(concurrencyLevel)
defer wg.Wait()
b.StartTimer()
for i := 0; i < concurrencyLevel; i++ {
go func() {
for {
if atomic.AddInt64(&remain, -1) < 0 {
wg.Done()
return
}
if _, err := stmt.Exec(); err != nil {
b.Fatal(err.Error())
}
}
}()
}
}
// data, but no db writes
var roundtripSample []byte
func initRoundtripBenchmarks() ([]byte, int, int) {
if roundtripSample == nil {
roundtripSample = []byte(strings.Repeat("0123456789abcdef", 1024*1024))
}
return roundtripSample, 16, len(roundtripSample)
}
func BenchmarkRoundtripTxt(b *testing.B) {
b.StopTimer()
sample, min, max := initRoundtripBenchmarks()
sampleString := string(sample)
b.ReportAllocs()
tb := (*TB)(b)
db := tb.checkDB(sql.Open("mysql", dsn))
defer db.Close()
b.StartTimer()
var result string
for i := 0; i < b.N; i++ {
length := min + i
if length > max {
length = max
}
test := sampleString[0:length]
rows := tb.checkRows(db.Query(`SELECT "` + test + `"`))
if !rows.Next() {
rows.Close()
b.Fatalf("crashed")
}
err := rows.Scan(&result)
if err != nil {
rows.Close()
b.Fatalf("crashed")
}
if result != test {
rows.Close()
b.Errorf("mismatch")
}
rows.Close()
}
}
func BenchmarkRoundtripBin(b *testing.B) {
b.StopTimer()
sample, min, max := initRoundtripBenchmarks()
b.ReportAllocs()
tb := (*TB)(b)
db := tb.checkDB(sql.Open("mysql", dsn))
defer db.Close()
stmt := tb.checkStmt(db.Prepare("SELECT ?"))
defer stmt.Close()
b.StartTimer()
var result sql.RawBytes
for i := 0; i < b.N; i++ {
length := min + i
if length > max {
length = max
}
test := sample[0:length]
rows := tb.checkRows(stmt.Query(test))
if !rows.Next() {
rows.Close()
b.Fatalf("crashed")
}
err := rows.Scan(&result)
if err != nil {
rows.Close()
b.Fatalf("crashed")
}
if !bytes.Equal(result, test) {
rows.Close()
b.Errorf("mismatch")
}
rows.Close()
}
}
func BenchmarkInterpolation(b *testing.B) {
mc := &mysqlConn{
cfg: &config{
interpolateParams: true,
loc: time.UTC,
},
maxPacketAllowed: maxPacketSize,
maxWriteSize: maxPacketSize - 1,
buf: newBuffer(nil),
}
args := []driver.Value{
int64(42424242),
float64(math.Pi),
false,
time.Unix(1423411542, 807015000),
[]byte("bytes containing special chars ' \" \a \x00"),
"string containing special chars ' \" \a \x00",
}
q := "SELECT ?, ?, ?, ?, ?, ?"
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err := mc.interpolateParams(q, args)
if err != nil {
b.Fatal(err)
}
}
}

View File

@ -8,7 +8,11 @@
package mysql
import "io"
import (
"io"
"net"
"time"
)
const defaultBufSize = 4096
@ -18,17 +22,18 @@ const defaultBufSize = 4096
// The buffer is similar to bufio.Reader / Writer but zero-copy-ish
// Also highly optimized for this particular use case.
type buffer struct {
buf []byte
rd io.Reader
idx int
length int
buf []byte
nc net.Conn
idx int
length int
timeout time.Duration
}
func newBuffer(rd io.Reader) buffer {
func newBuffer(nc net.Conn) buffer {
var b [defaultBufSize]byte
return buffer{
buf: b[:],
rd: rd,
nc: nc,
}
}
@ -54,7 +59,13 @@ func (b *buffer) fill(need int) error {
b.idx = 0
for {
nn, err := b.rd.Read(b.buf[n:])
if b.timeout > 0 {
if err := b.nc.SetReadDeadline(time.Now().Add(b.timeout)); err != nil {
return err
}
}
nn, err := b.nc.Read(b.buf[n:])
n += nn
switch err {

View File

@ -8,7 +8,7 @@
package mysql
const defaultCollation byte = 33 // utf8_general_ci
const defaultCollation = "utf8_general_ci"
// A list of available collations mapped to the internal ID.
// To update this map use the following MySQL query:
@ -237,14 +237,14 @@ var collations = map[string]byte{
// A blacklist of collations which is unsafe to interpolate parameters.
// These multibyte encodings may contains 0x5c (`\`) in their trailing bytes.
var unsafeCollations = map[byte]bool{
1: true, // big5_chinese_ci
13: true, // sjis_japanese_ci
28: true, // gbk_chinese_ci
84: true, // big5_bin
86: true, // gb2312_bin
87: true, // gbk_bin
88: true, // sjis_bin
95: true, // cp932_japanese_ci
96: true, // cp932_bin
var unsafeCollations = map[string]bool{
"big5_chinese_ci": true,
"sjis_japanese_ci": true,
"gbk_chinese_ci": true,
"big5_bin": true,
"gb2312_bin": true,
"gbk_bin": true,
"sjis_bin": true,
"cp932_japanese_ci": true,
"cp932_bin": true,
}

View File

@ -9,9 +9,7 @@
package mysql
import (
"crypto/tls"
"database/sql/driver"
"errors"
"net"
"strconv"
"strings"
@ -23,9 +21,10 @@ type mysqlConn struct {
netConn net.Conn
affectedRows uint64
insertId uint64
cfg *config
cfg *Config
maxPacketAllowed int
maxWriteSize int
writeTimeout time.Duration
flags clientFlag
status statusFlag
sequence uint8
@ -33,27 +32,9 @@ type mysqlConn struct {
strict bool
}
type config struct {
user string
passwd string
net string
addr string
dbname string
params map[string]string
loc *time.Location
tls *tls.Config
timeout time.Duration
collation uint8
allowAllFiles bool
allowOldPasswords bool
clientFoundRows bool
columnsWithAlias bool
interpolateParams bool
}
// Handles parameters set in DSN after the connection is established
func (mc *mysqlConn) handleParams() (err error) {
for param, val := range mc.cfg.params {
for param, val := range mc.cfg.Params {
switch param {
// Charset
case "charset":
@ -69,27 +50,6 @@ func (mc *mysqlConn) handleParams() (err error) {
return
}
// time.Time parsing
case "parseTime":
var isBool bool
mc.parseTime, isBool = readBool(val)
if !isBool {
return errors.New("Invalid Bool value: " + val)
}
// Strict mode
case "strict":
var isBool bool
mc.strict, isBool = readBool(val)
if !isBool {
return errors.New("Invalid Bool value: " + val)
}
// Compression
case "compress":
err = errors.New("Compression not implemented yet")
return
// System Vars
default:
err = mc.exec("SET " + param + "=" + val + "")
@ -119,18 +79,27 @@ func (mc *mysqlConn) Close() (err error) {
// Makes Close idempotent
if mc.netConn != nil {
err = mc.writeCommandPacket(comQuit)
if err == nil {
err = mc.netConn.Close()
} else {
mc.netConn.Close()
}
mc.cleanup()
return
}
// Closes the network connection and unsets internal variables. Do not call this
// function after successfully authentication, call Close instead. This function
// is called before auth or on auth failure because MySQL will have already
// closed the network connection.
func (mc *mysqlConn) cleanup() {
// Makes cleanup idempotent
if mc.netConn != nil {
if err := mc.netConn.Close(); err != nil {
errLog.Print(err)
}
mc.netConn = nil
}
mc.cfg = nil
mc.buf.rd = nil
return
mc.buf.nc = nil
}
func (mc *mysqlConn) Prepare(query string) (driver.Stmt, error) {
@ -207,7 +176,7 @@ func (mc *mysqlConn) interpolateParams(query string, args []driver.Value) (strin
if v.IsZero() {
buf = append(buf, "'0000-00-00'"...)
} else {
v := v.In(mc.cfg.loc)
v := v.In(mc.cfg.Loc)
v = v.Add(time.Nanosecond * 500) // To round under microsecond
year := v.Year()
year100 := year / 100
@ -252,7 +221,7 @@ func (mc *mysqlConn) interpolateParams(query string, args []driver.Value) (strin
if v == nil {
buf = append(buf, "NULL"...)
} else {
buf = append(buf, '\'')
buf = append(buf, "_binary'"...)
if mc.status&statusNoBackslashEscapes == 0 {
buf = escapeBytesBackslash(buf, v)
} else {
@ -288,7 +257,7 @@ func (mc *mysqlConn) Exec(query string, args []driver.Value) (driver.Result, err
return nil, driver.ErrBadConn
}
if len(args) != 0 {
if !mc.cfg.interpolateParams {
if !mc.cfg.InterpolateParams {
return nil, driver.ErrSkip
}
// try to interpolate the parameters to save extra roundtrips for preparing and closing a statement
@ -339,7 +308,7 @@ func (mc *mysqlConn) Query(query string, args []driver.Value) (driver.Rows, erro
return nil, driver.ErrBadConn
}
if len(args) != 0 {
if !mc.cfg.interpolateParams {
if !mc.cfg.InterpolateParams {
return nil, driver.ErrSkip
}
// try client-side prepare to reduce roundtrip
@ -385,6 +354,7 @@ func (mc *mysqlConn) getSystemVar(name string) ([]byte, error) {
if err == nil {
rows := new(textRows)
rows.mc = mc
rows.columns = []mysqlField{{fieldType: fieldTypeVarChar}}
if resLen > 0 {
// Columns

View File

@ -24,6 +24,7 @@ const (
iERR byte = 0xff
)
// https://dev.mysql.com/doc/internals/en/capability-flags.html#packet-Protocol::CapabilityFlags
type clientFlag uint32
const (
@ -45,6 +46,13 @@ const (
clientSecureConn
clientMultiStatements
clientMultiResults
clientPSMultiResults
clientPluginAuth
clientConnectAttrs
clientPluginAuthLenEncClientData
clientCanHandleExpiredPasswords
clientSessionTrack
clientDeprecateEOF
)
const (
@ -78,6 +86,7 @@ const (
comStmtFetch
)
// https://dev.mysql.com/doc/internals/en/com-query-response.html#packet-Protocol::ColumnType
const (
fieldTypeDecimal byte = iota
fieldTypeTiny
@ -98,7 +107,8 @@ const (
fieldTypeBit
)
const (
fieldTypeNewDecimal byte = iota + 0xf6
fieldTypeJSON byte = iota + 0xf5
fieldTypeNewDecimal
fieldTypeEnum
fieldTypeSet
fieldTypeTinyBLOB
@ -132,7 +142,6 @@ const (
)
// http://dev.mysql.com/doc/internals/en/status-flags.html
type statusFlag uint16
const (

View File

@ -4,7 +4,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at http://mozilla.org/MPL/2.0/.
// Go MySQL Driver - A MySQL-Driver for Go's database/sql package
// Package mysql provides a MySQL driver for Go's database/sql package
//
// The driver should be used via the database/sql package:
//
@ -22,7 +22,7 @@ import (
"net"
)
// This struct is exported to make the driver directly accessible.
// MySQLDriver is exported to make the driver directly accessible.
// In general the driver is used via the database/sql package.
type MySQLDriver struct{}
@ -53,17 +53,19 @@ func (d MySQLDriver) Open(dsn string) (driver.Conn, error) {
maxPacketAllowed: maxPacketSize,
maxWriteSize: maxPacketSize - 1,
}
mc.cfg, err = parseDSN(dsn)
mc.cfg, err = ParseDSN(dsn)
if err != nil {
return nil, err
}
mc.parseTime = mc.cfg.ParseTime
mc.strict = mc.cfg.Strict
// Connect to Server
if dial, ok := dials[mc.cfg.net]; ok {
mc.netConn, err = dial(mc.cfg.addr)
if dial, ok := dials[mc.cfg.Net]; ok {
mc.netConn, err = dial(mc.cfg.Addr)
} else {
nd := net.Dialer{Timeout: mc.cfg.timeout}
mc.netConn, err = nd.Dial(mc.cfg.net, mc.cfg.addr)
nd := net.Dialer{Timeout: mc.cfg.Timeout}
mc.netConn, err = nd.Dial(mc.cfg.Net, mc.cfg.Addr)
}
if err != nil {
return nil, err
@ -81,37 +83,30 @@ func (d MySQLDriver) Open(dsn string) (driver.Conn, error) {
mc.buf = newBuffer(mc.netConn)
// Set I/O timeouts
mc.buf.timeout = mc.cfg.ReadTimeout
mc.writeTimeout = mc.cfg.WriteTimeout
// Reading Handshake Initialization Packet
cipher, err := mc.readInitPacket()
if err != nil {
mc.Close()
mc.cleanup()
return nil, err
}
// Send Client Authentication Packet
if err = mc.writeAuthPacket(cipher); err != nil {
mc.Close()
mc.cleanup()
return nil, err
}
// Read Result Packet
err = mc.readResultOK()
if err != nil {
// Retry with old authentication method, if allowed
if mc.cfg != nil && mc.cfg.allowOldPasswords && err == ErrOldPassword {
if err = mc.writeOldAuthPacket(cipher); err != nil {
mc.Close()
return nil, err
}
if err = mc.readResultOK(); err != nil {
mc.Close()
return nil, err
}
} else {
mc.Close()
return nil, err
}
// Handle response to auth packet, switch methods if possible
if err = handleAuthResult(mc, cipher); err != nil {
// Authentication failed and MySQL has already closed the connection
// (https://dev.mysql.com/doc/internals/en/authentication-fails.html).
// Do not send COM_QUIT, just cleanup and return the error.
mc.cleanup()
return nil, err
}
// Get max allowed packet size
@ -135,6 +130,38 @@ func (d MySQLDriver) Open(dsn string) (driver.Conn, error) {
return mc, nil
}
func handleAuthResult(mc *mysqlConn, cipher []byte) error {
// Read Result Packet
err := mc.readResultOK()
if err == nil {
return nil // auth successful
}
if mc.cfg == nil {
return err // auth failed and retry not possible
}
// Retry auth if configured to do so.
if mc.cfg.AllowOldPasswords && err == ErrOldPassword {
// Retry with old authentication method. Note: there are edge cases
// where this should work but doesn't; this is currently "wontfix":
// https://github.com/go-sql-driver/mysql/issues/184
if err = mc.writeOldAuthPacket(cipher); err != nil {
return err
}
err = mc.readResultOK()
} else if mc.cfg.AllowCleartextPasswords && err == ErrCleartextPassword {
// Retry with clear text password for
// http://dev.mysql.com/doc/refman/5.7/en/cleartext-authentication-plugin.html
// http://dev.mysql.com/doc/refman/5.7/en/pam-authentication-plugin.html
if err = mc.writeClearAuthPacket(); err != nil {
return err
}
err = mc.readResultOK()
}
return err
}
func init() {
sql.Register("mysql", &MySQLDriver{})
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,513 @@
// Go MySQL Driver - A MySQL-Driver for Go's database/sql package
//
// Copyright 2016 The Go-MySQL-Driver Authors. All rights reserved.
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at http://mozilla.org/MPL/2.0/.
package mysql
import (
"bytes"
"crypto/tls"
"errors"
"fmt"
"net"
"net/url"
"strings"
"time"
)
var (
errInvalidDSNUnescaped = errors.New("invalid DSN: did you forget to escape a param value?")
errInvalidDSNAddr = errors.New("invalid DSN: network address not terminated (missing closing brace)")
errInvalidDSNNoSlash = errors.New("invalid DSN: missing the slash separating the database name")
errInvalidDSNUnsafeCollation = errors.New("invalid DSN: interpolateParams can not be used with unsafe collations")
)
// Config is a configuration parsed from a DSN string
type Config struct {
User string // Username
Passwd string // Password
Net string // Network type
Addr string // Network address
DBName string // Database name
Params map[string]string // Connection parameters
Collation string // Connection collation
Loc *time.Location // Location for time.Time values
TLSConfig string // TLS configuration name
tls *tls.Config // TLS configuration
Timeout time.Duration // Dial timeout
ReadTimeout time.Duration // I/O read timeout
WriteTimeout time.Duration // I/O write timeout
AllowAllFiles bool // Allow all files to be used with LOAD DATA LOCAL INFILE
AllowCleartextPasswords bool // Allows the cleartext client side plugin
AllowOldPasswords bool // Allows the old insecure password method
ClientFoundRows bool // Return number of matching rows instead of rows changed
ColumnsWithAlias bool // Prepend table alias to column names
InterpolateParams bool // Interpolate placeholders into query string
MultiStatements bool // Allow multiple statements in one query
ParseTime bool // Parse time values to time.Time
Strict bool // Return warnings as errors
}
// FormatDSN formats the given Config into a DSN string which can be passed to
// the driver.
func (cfg *Config) FormatDSN() string {
var buf bytes.Buffer
// [username[:password]@]
if len(cfg.User) > 0 {
buf.WriteString(cfg.User)
if len(cfg.Passwd) > 0 {
buf.WriteByte(':')
buf.WriteString(cfg.Passwd)
}
buf.WriteByte('@')
}
// [protocol[(address)]]
if len(cfg.Net) > 0 {
buf.WriteString(cfg.Net)
if len(cfg.Addr) > 0 {
buf.WriteByte('(')
buf.WriteString(cfg.Addr)
buf.WriteByte(')')
}
}
// /dbname
buf.WriteByte('/')
buf.WriteString(cfg.DBName)
// [?param1=value1&...&paramN=valueN]
hasParam := false
if cfg.AllowAllFiles {
hasParam = true
buf.WriteString("?allowAllFiles=true")
}
if cfg.AllowCleartextPasswords {
if hasParam {
buf.WriteString("&allowCleartextPasswords=true")
} else {
hasParam = true
buf.WriteString("?allowCleartextPasswords=true")
}
}
if cfg.AllowOldPasswords {
if hasParam {
buf.WriteString("&allowOldPasswords=true")
} else {
hasParam = true
buf.WriteString("?allowOldPasswords=true")
}
}
if cfg.ClientFoundRows {
if hasParam {
buf.WriteString("&clientFoundRows=true")
} else {
hasParam = true
buf.WriteString("?clientFoundRows=true")
}
}
if col := cfg.Collation; col != defaultCollation && len(col) > 0 {
if hasParam {
buf.WriteString("&collation=")
} else {
hasParam = true
buf.WriteString("?collation=")
}
buf.WriteString(col)
}
if cfg.ColumnsWithAlias {
if hasParam {
buf.WriteString("&columnsWithAlias=true")
} else {
hasParam = true
buf.WriteString("?columnsWithAlias=true")
}
}
if cfg.InterpolateParams {
if hasParam {
buf.WriteString("&interpolateParams=true")
} else {
hasParam = true
buf.WriteString("?interpolateParams=true")
}
}
if cfg.Loc != time.UTC && cfg.Loc != nil {
if hasParam {
buf.WriteString("&loc=")
} else {
hasParam = true
buf.WriteString("?loc=")
}
buf.WriteString(url.QueryEscape(cfg.Loc.String()))
}
if cfg.MultiStatements {
if hasParam {
buf.WriteString("&multiStatements=true")
} else {
hasParam = true
buf.WriteString("?multiStatements=true")
}
}
if cfg.ParseTime {
if hasParam {
buf.WriteString("&parseTime=true")
} else {
hasParam = true
buf.WriteString("?parseTime=true")
}
}
if cfg.ReadTimeout > 0 {
if hasParam {
buf.WriteString("&readTimeout=")
} else {
hasParam = true
buf.WriteString("?readTimeout=")
}
buf.WriteString(cfg.ReadTimeout.String())
}
if cfg.Strict {
if hasParam {
buf.WriteString("&strict=true")
} else {
hasParam = true
buf.WriteString("?strict=true")
}
}
if cfg.Timeout > 0 {
if hasParam {
buf.WriteString("&timeout=")
} else {
hasParam = true
buf.WriteString("?timeout=")
}
buf.WriteString(cfg.Timeout.String())
}
if len(cfg.TLSConfig) > 0 {
if hasParam {
buf.WriteString("&tls=")
} else {
hasParam = true
buf.WriteString("?tls=")
}
buf.WriteString(url.QueryEscape(cfg.TLSConfig))
}
if cfg.WriteTimeout > 0 {
if hasParam {
buf.WriteString("&writeTimeout=")
} else {
hasParam = true
buf.WriteString("?writeTimeout=")
}
buf.WriteString(cfg.WriteTimeout.String())
}
// other params
if cfg.Params != nil {
for param, value := range cfg.Params {
if hasParam {
buf.WriteByte('&')
} else {
hasParam = true
buf.WriteByte('?')
}
buf.WriteString(param)
buf.WriteByte('=')
buf.WriteString(url.QueryEscape(value))
}
}
return buf.String()
}
// ParseDSN parses the DSN string to a Config
func ParseDSN(dsn string) (cfg *Config, err error) {
// New config with some default values
cfg = &Config{
Loc: time.UTC,
Collation: defaultCollation,
}
// [user[:password]@][net[(addr)]]/dbname[?param1=value1&paramN=valueN]
// Find the last '/' (since the password or the net addr might contain a '/')
foundSlash := false
for i := len(dsn) - 1; i >= 0; i-- {
if dsn[i] == '/' {
foundSlash = true
var j, k int
// left part is empty if i <= 0
if i > 0 {
// [username[:password]@][protocol[(address)]]
// Find the last '@' in dsn[:i]
for j = i; j >= 0; j-- {
if dsn[j] == '@' {
// username[:password]
// Find the first ':' in dsn[:j]
for k = 0; k < j; k++ {
if dsn[k] == ':' {
cfg.Passwd = dsn[k+1 : j]
break
}
}
cfg.User = dsn[:k]
break
}
}
// [protocol[(address)]]
// Find the first '(' in dsn[j+1:i]
for k = j + 1; k < i; k++ {
if dsn[k] == '(' {
// dsn[i-1] must be == ')' if an address is specified
if dsn[i-1] != ')' {
if strings.ContainsRune(dsn[k+1:i], ')') {
return nil, errInvalidDSNUnescaped
}
return nil, errInvalidDSNAddr
}
cfg.Addr = dsn[k+1 : i-1]
break
}
}
cfg.Net = dsn[j+1 : k]
}
// dbname[?param1=value1&...&paramN=valueN]
// Find the first '?' in dsn[i+1:]
for j = i + 1; j < len(dsn); j++ {
if dsn[j] == '?' {
if err = parseDSNParams(cfg, dsn[j+1:]); err != nil {
return
}
break
}
}
cfg.DBName = dsn[i+1 : j]
break
}
}
if !foundSlash && len(dsn) > 0 {
return nil, errInvalidDSNNoSlash
}
if cfg.InterpolateParams && unsafeCollations[cfg.Collation] {
return nil, errInvalidDSNUnsafeCollation
}
// Set default network if empty
if cfg.Net == "" {
cfg.Net = "tcp"
}
// Set default address if empty
if cfg.Addr == "" {
switch cfg.Net {
case "tcp":
cfg.Addr = "127.0.0.1:3306"
case "unix":
cfg.Addr = "/tmp/mysql.sock"
default:
return nil, errors.New("default addr for network '" + cfg.Net + "' unknown")
}
}
return
}
// parseDSNParams parses the DSN "query string"
// Values must be url.QueryEscape'ed
func parseDSNParams(cfg *Config, params string) (err error) {
for _, v := range strings.Split(params, "&") {
param := strings.SplitN(v, "=", 2)
if len(param) != 2 {
continue
}
// cfg params
switch value := param[1]; param[0] {
// Disable INFILE whitelist / enable all files
case "allowAllFiles":
var isBool bool
cfg.AllowAllFiles, isBool = readBool(value)
if !isBool {
return errors.New("invalid bool value: " + value)
}
// Use cleartext authentication mode (MySQL 5.5.10+)
case "allowCleartextPasswords":
var isBool bool
cfg.AllowCleartextPasswords, isBool = readBool(value)
if !isBool {
return errors.New("invalid bool value: " + value)
}
// Use old authentication mode (pre MySQL 4.1)
case "allowOldPasswords":
var isBool bool
cfg.AllowOldPasswords, isBool = readBool(value)
if !isBool {
return errors.New("invalid bool value: " + value)
}
// Switch "rowsAffected" mode
case "clientFoundRows":
var isBool bool
cfg.ClientFoundRows, isBool = readBool(value)
if !isBool {
return errors.New("invalid bool value: " + value)
}
// Collation
case "collation":
cfg.Collation = value
break
case "columnsWithAlias":
var isBool bool
cfg.ColumnsWithAlias, isBool = readBool(value)
if !isBool {
return errors.New("invalid bool value: " + value)
}
// Compression
case "compress":
return errors.New("compression not implemented yet")
// Enable client side placeholder substitution
case "interpolateParams":
var isBool bool
cfg.InterpolateParams, isBool = readBool(value)
if !isBool {
return errors.New("invalid bool value: " + value)
}
// Time Location
case "loc":
if value, err = url.QueryUnescape(value); err != nil {
return
}
cfg.Loc, err = time.LoadLocation(value)
if err != nil {
return
}
// multiple statements in one query
case "multiStatements":
var isBool bool
cfg.MultiStatements, isBool = readBool(value)
if !isBool {
return errors.New("invalid bool value: " + value)
}
// time.Time parsing
case "parseTime":
var isBool bool
cfg.ParseTime, isBool = readBool(value)
if !isBool {
return errors.New("invalid bool value: " + value)
}
// I/O read Timeout
case "readTimeout":
cfg.ReadTimeout, err = time.ParseDuration(value)
if err != nil {
return
}
// Strict mode
case "strict":
var isBool bool
cfg.Strict, isBool = readBool(value)
if !isBool {
return errors.New("invalid bool value: " + value)
}
// Dial Timeout
case "timeout":
cfg.Timeout, err = time.ParseDuration(value)
if err != nil {
return
}
// TLS-Encryption
case "tls":
boolValue, isBool := readBool(value)
if isBool {
if boolValue {
cfg.TLSConfig = "true"
cfg.tls = &tls.Config{}
} else {
cfg.TLSConfig = "false"
}
} else if vl := strings.ToLower(value); vl == "skip-verify" {
cfg.TLSConfig = vl
cfg.tls = &tls.Config{InsecureSkipVerify: true}
} else {
name, err := url.QueryUnescape(value)
if err != nil {
return fmt.Errorf("invalid value for TLS config name: %v", err)
}
if tlsConfig, ok := tlsConfigRegister[name]; ok {
if len(tlsConfig.ServerName) == 0 && !tlsConfig.InsecureSkipVerify {
host, _, err := net.SplitHostPort(cfg.Addr)
if err == nil {
tlsConfig.ServerName = host
}
}
cfg.TLSConfig = name
cfg.tls = tlsConfig
} else {
return errors.New("invalid value / unknown config name: " + name)
}
}
// I/O write Timeout
case "writeTimeout":
cfg.WriteTimeout, err = time.ParseDuration(value)
if err != nil {
return
}
default:
// lazy init
if cfg.Params == nil {
cfg.Params = make(map[string]string)
}
if cfg.Params[param[0]], err = url.QueryUnescape(value); err != nil {
return
}
}
}
return
}

View File

@ -19,18 +19,20 @@ import (
// Various errors the driver might return. Can change between driver versions.
var (
ErrInvalidConn = errors.New("Invalid Connection")
ErrMalformPkt = errors.New("Malformed Packet")
ErrNoTLS = errors.New("TLS encryption requested but server does not support TLS")
ErrOldPassword = errors.New("This server only supports the insecure old password authentication. If you still want to use it, please add 'allowOldPasswords=1' to your DSN. See also https://github.com/go-sql-driver/mysql/wiki/old_passwords")
ErrOldProtocol = errors.New("MySQL-Server does not support required Protocol 41+")
ErrPktSync = errors.New("Commands out of sync. You can't run this command now")
ErrPktSyncMul = errors.New("Commands out of sync. Did you run multiple statements at once?")
ErrPktTooLarge = errors.New("Packet for query is too large. You can change this value on the server by adjusting the 'max_allowed_packet' variable.")
ErrBusyBuffer = errors.New("Busy buffer")
ErrInvalidConn = errors.New("invalid connection")
ErrMalformPkt = errors.New("malformed packet")
ErrNoTLS = errors.New("TLS requested but server does not support TLS")
ErrOldPassword = errors.New("this user requires old password authentication. If you still want to use it, please add 'allowOldPasswords=1' to your DSN. See also https://github.com/go-sql-driver/mysql/wiki/old_passwords")
ErrCleartextPassword = errors.New("this user requires clear text authentication. If you still want to use it, please add 'allowCleartextPasswords=1' to your DSN")
ErrUnknownPlugin = errors.New("this authentication plugin is not supported")
ErrOldProtocol = errors.New("MySQL server does not support required protocol 41+")
ErrPktSync = errors.New("commands out of sync. You can't run this command now")
ErrPktSyncMul = errors.New("commands out of sync. Did you run multiple statements at once?")
ErrPktTooLarge = errors.New("packet for query is too large. Try adjusting the 'max_allowed_packet' variable on the server")
ErrBusyBuffer = errors.New("busy buffer")
)
var errLog Logger = log.New(os.Stderr, "[MySQL] ", log.Ldate|log.Ltime|log.Lshortfile)
var errLog = Logger(log.New(os.Stderr, "[mysql] ", log.Ldate|log.Ltime|log.Lshortfile))
// Logger is used to log critical error messages.
type Logger interface {

View File

@ -1,42 +0,0 @@
// Go MySQL Driver - A MySQL-Driver for Go's database/sql package
//
// Copyright 2013 The Go-MySQL-Driver Authors. All rights reserved.
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at http://mozilla.org/MPL/2.0/.
package mysql
import (
"bytes"
"log"
"testing"
)
func TestErrorsSetLogger(t *testing.T) {
previous := errLog
defer func() {
errLog = previous
}()
// set up logger
const expected = "prefix: test\n"
buffer := bytes.NewBuffer(make([]byte, 0, 64))
logger := log.New(buffer, "prefix: ", 0)
// print
SetLogger(logger)
errLog.Print("test")
// check result
if actual := buffer.String(); actual != expected {
t.Errorf("expected %q, got %q", expected, actual)
}
}
func TestErrorsStrictIgnoreNotes(t *testing.T) {
runTests(t, dsn+"&sql_notes=false", func(dbt *DBTest) {
dbt.mustExec("DROP TABLE IF EXISTS does_not_exist")
})
}

View File

@ -13,11 +13,14 @@ import (
"io"
"os"
"strings"
"sync"
)
var (
fileRegister map[string]bool
readerRegister map[string]func() io.Reader
fileRegister map[string]bool
fileRegisterLock sync.RWMutex
readerRegister map[string]func() io.Reader
readerRegisterLock sync.RWMutex
)
// RegisterLocalFile adds the given file to the file whitelist,
@ -32,17 +35,21 @@ var (
// ...
//
func RegisterLocalFile(filePath string) {
fileRegisterLock.Lock()
// lazy map init
if fileRegister == nil {
fileRegister = make(map[string]bool)
}
fileRegister[strings.Trim(filePath, `"`)] = true
fileRegisterLock.Unlock()
}
// DeregisterLocalFile removes the given filepath from the whitelist.
func DeregisterLocalFile(filePath string) {
fileRegisterLock.Lock()
delete(fileRegister, strings.Trim(filePath, `"`))
fileRegisterLock.Unlock()
}
// RegisterReaderHandler registers a handler function which is used
@ -61,18 +68,22 @@ func DeregisterLocalFile(filePath string) {
// ...
//
func RegisterReaderHandler(name string, handler func() io.Reader) {
readerRegisterLock.Lock()
// lazy map init
if readerRegister == nil {
readerRegister = make(map[string]func() io.Reader)
}
readerRegister[name] = handler
readerRegisterLock.Unlock()
}
// DeregisterReaderHandler removes the ReaderHandler function with
// the given name from the registry.
func DeregisterReaderHandler(name string) {
readerRegisterLock.Lock()
delete(readerRegister, name)
readerRegisterLock.Unlock()
}
func deferredClose(err *error, closer io.Closer) {
@ -86,9 +97,15 @@ func (mc *mysqlConn) handleInFileRequest(name string) (err error) {
var rdr io.Reader
var data []byte
if strings.HasPrefix(name, "Reader::") { // io.Reader
name = name[8:]
if handler, inMap := readerRegister[name]; inMap {
if idx := strings.Index(name, "Reader::"); idx == 0 || (idx > 0 && name[idx-1] == '/') { // io.Reader
// The server might return an an absolute path. See issue #355.
name = name[idx+8:]
readerRegisterLock.RLock()
handler, inMap := readerRegister[name]
readerRegisterLock.RUnlock()
if inMap {
rdr = handler()
if rdr != nil {
data = make([]byte, 4+mc.maxWriteSize)
@ -104,7 +121,10 @@ func (mc *mysqlConn) handleInFileRequest(name string) (err error) {
}
} else { // File
name = strings.Trim(name, `"`)
if mc.cfg.allowAllFiles || fileRegister[name] {
fileRegisterLock.RLock()
fr := fileRegister[name]
fileRegisterLock.RUnlock()
if mc.cfg.AllowAllFiles || fr {
var file *os.File
var fi os.FileInfo
@ -119,12 +139,12 @@ func (mc *mysqlConn) handleInFileRequest(name string) (err error) {
} else if fileSize <= mc.maxPacketAllowed {
data = make([]byte, 4+mc.maxWriteSize)
} else {
err = fmt.Errorf("Local File '%s' too large: Size: %d, Max: %d", name, fileSize, mc.maxPacketAllowed)
err = fmt.Errorf("local file '%s' too large: size: %d, max: %d", name, fileSize, mc.maxPacketAllowed)
}
}
}
} else {
err = fmt.Errorf("Local File '%s' is not registered. Use the DSN parameter 'allowAllFiles=true' to allow all files", name)
err = fmt.Errorf("local file '%s' is not registered", name)
}
}
@ -155,8 +175,8 @@ func (mc *mysqlConn) handleInFileRequest(name string) (err error) {
// read OK packet
if err == nil {
return mc.readResultOK()
} else {
mc.readPacket()
}
mc.readPacket()
return err
}

View File

@ -13,6 +13,7 @@ import (
"crypto/tls"
"database/sql/driver"
"encoding/binary"
"errors"
"fmt"
"io"
"math"
@ -47,9 +48,8 @@ func (mc *mysqlConn) readPacket() ([]byte, error) {
if data[3] != mc.sequence {
if data[3] > mc.sequence {
return nil, ErrPktSyncMul
} else {
return nil, ErrPktSync
}
return nil, ErrPktSync
}
mc.sequence++
@ -100,6 +100,12 @@ func (mc *mysqlConn) writePacket(data []byte) error {
data[3] = mc.sequence
// Write packet
if mc.writeTimeout > 0 {
if err := mc.netConn.SetWriteDeadline(time.Now().Add(mc.writeTimeout)); err != nil {
return err
}
}
n, err := mc.netConn.Write(data[:4+size])
if err == nil && n == 4+size {
mc.sequence++
@ -140,7 +146,7 @@ func (mc *mysqlConn) readInitPacket() ([]byte, error) {
// protocol version [1 byte]
if data[0] < minProtocolVersion {
return nil, fmt.Errorf(
"Unsupported MySQL Protocol Version %d. Protocol Version %d or higher is required",
"unsupported protocol version %d. Version %d or higher is required",
data[0],
minProtocolVersion,
)
@ -196,7 +202,11 @@ func (mc *mysqlConn) readInitPacket() ([]byte, error) {
// return
//}
//return ErrMalformPkt
return cipher, nil
// make a memory safe copy of the cipher slice
var b [20]byte
copy(b[:], cipher)
return b[:], nil
}
// make a memory safe copy of the cipher slice
@ -214,9 +224,11 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error {
clientLongPassword |
clientTransactions |
clientLocalFiles |
clientPluginAuth |
clientMultiResults |
mc.flags&clientLongFlag
if mc.cfg.clientFoundRows {
if mc.cfg.ClientFoundRows {
clientFlags |= clientFoundRows
}
@ -225,13 +237,17 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error {
clientFlags |= clientSSL
}
// User Password
scrambleBuff := scramblePassword(cipher, []byte(mc.cfg.passwd))
if mc.cfg.MultiStatements {
clientFlags |= clientMultiStatements
}
pktLen := 4 + 4 + 1 + 23 + len(mc.cfg.user) + 1 + 1 + len(scrambleBuff)
// User Password
scrambleBuff := scramblePassword(cipher, []byte(mc.cfg.Passwd))
pktLen := 4 + 4 + 1 + 23 + len(mc.cfg.User) + 1 + 1 + len(scrambleBuff) + 21 + 1
// To specify a db name
if n := len(mc.cfg.dbname); n > 0 {
if n := len(mc.cfg.DBName); n > 0 {
clientFlags |= clientConnectWithDB
pktLen += n + 1
}
@ -257,7 +273,14 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error {
data[11] = 0x00
// Charset [1 byte]
data[12] = mc.cfg.collation
var found bool
data[12], found = collations[mc.cfg.Collation]
if !found {
// Note possibility for false negatives:
// could be triggered although the collation is valid if the
// collations map does not contain entries the server supports.
return errors.New("unknown collation")
}
// SSL Connection Request Packet
// http://dev.mysql.com/doc/internals/en/connection-phase-packets.html#packet-Protocol::SSLRequest
@ -273,15 +296,18 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error {
return err
}
mc.netConn = tlsConn
mc.buf.rd = tlsConn
mc.buf.nc = tlsConn
}
// Filler [23 bytes] (all 0x00)
pos := 13 + 23
pos := 13
for ; pos < 13+23; pos++ {
data[pos] = 0
}
// User [null terminated string]
if len(mc.cfg.user) > 0 {
pos += copy(data[pos:], mc.cfg.user)
if len(mc.cfg.User) > 0 {
pos += copy(data[pos:], mc.cfg.User)
}
data[pos] = 0x00
pos++
@ -291,11 +317,16 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error {
pos += 1 + copy(data[pos+1:], scrambleBuff)
// Databasename [null terminated string]
if len(mc.cfg.dbname) > 0 {
pos += copy(data[pos:], mc.cfg.dbname)
if len(mc.cfg.DBName) > 0 {
pos += copy(data[pos:], mc.cfg.DBName)
data[pos] = 0x00
pos++
}
// Assume native client during response
pos += copy(data[pos:], "mysql_native_password")
data[pos] = 0x00
// Send Auth packet
return mc.writePacket(data)
}
@ -304,9 +335,9 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error {
// http://dev.mysql.com/doc/internals/en/connection-phase-packets.html#packet-Protocol::AuthSwitchResponse
func (mc *mysqlConn) writeOldAuthPacket(cipher []byte) error {
// User password
scrambleBuff := scrambleOldPassword(cipher, []byte(mc.cfg.passwd))
scrambleBuff := scrambleOldPassword(cipher, []byte(mc.cfg.Passwd))
// Calculate the packet lenght and add a tailing 0
// Calculate the packet length and add a tailing 0
pktLen := len(scrambleBuff) + 1
data := mc.buf.takeSmallBuffer(4 + pktLen)
if data == nil {
@ -322,6 +353,25 @@ func (mc *mysqlConn) writeOldAuthPacket(cipher []byte) error {
return mc.writePacket(data)
}
// Client clear text authentication packet
// http://dev.mysql.com/doc/internals/en/connection-phase-packets.html#packet-Protocol::AuthSwitchResponse
func (mc *mysqlConn) writeClearAuthPacket() error {
// Calculate the packet length and add a tailing 0
pktLen := len(mc.cfg.Passwd) + 1
data := mc.buf.takeSmallBuffer(4 + pktLen)
if data == nil {
// can not take the buffer. Something must be wrong with the connection
errLog.Print(ErrBusyBuffer)
return driver.ErrBadConn
}
// Add the clear password [null terminated string]
copy(data[4:], mc.cfg.Passwd)
data[4+pktLen-1] = 0x00
return mc.writePacket(data)
}
/******************************************************************************
* Command Packets *
******************************************************************************/
@ -405,8 +455,20 @@ func (mc *mysqlConn) readResultOK() error {
return mc.handleOkPacket(data)
case iEOF:
// someone is using old_passwords
return ErrOldPassword
if len(data) > 1 {
plugin := string(data[1:bytes.IndexByte(data, 0x00)])
if plugin == "mysql_old_password" {
// using old_passwords
return ErrOldPassword
} else if plugin == "mysql_clear_password" {
// using clear text password
return ErrCleartextPassword
} else {
return ErrUnknownPlugin
}
} else {
return ErrOldPassword
}
default: // Error otherwise
return mc.handleErrorPacket(data)
@ -470,6 +532,10 @@ func (mc *mysqlConn) handleErrorPacket(data []byte) error {
}
}
func readStatus(b []byte) statusFlag {
return statusFlag(b[0]) | statusFlag(b[1])<<8
}
// Ok Packet
// http://dev.mysql.com/doc/internals/en/generic-response-packets.html#packet-OK_Packet
func (mc *mysqlConn) handleOkPacket(data []byte) error {
@ -484,18 +550,21 @@ func (mc *mysqlConn) handleOkPacket(data []byte) error {
mc.insertId, _, m = readLengthEncodedInteger(data[1+n:])
// server_status [2 bytes]
mc.status = statusFlag(data[1+n+m]) | statusFlag(data[1+n+m+1])<<8
mc.status = readStatus(data[1+n+m : 1+n+m+2])
if err := mc.discardResults(); err != nil {
return err
}
// warning count [2 bytes]
if !mc.strict {
return nil
} else {
pos := 1 + n + m + 2
if binary.LittleEndian.Uint16(data[pos:pos+2]) > 0 {
return mc.getWarnings()
}
return nil
}
pos := 1 + n + m + 2
if binary.LittleEndian.Uint16(data[pos:pos+2]) > 0 {
return mc.getWarnings()
}
return nil
}
// Read Packets as Field Packets until EOF-Packet or an Error appears
@ -514,7 +583,7 @@ func (mc *mysqlConn) readColumns(count int) ([]mysqlField, error) {
if i == count {
return columns, nil
}
return nil, fmt.Errorf("ColumnsCount mismatch n:%d len:%d", count, len(columns))
return nil, fmt.Errorf("column count mismatch n:%d len:%d", count, len(columns))
}
// Catalog
@ -531,7 +600,7 @@ func (mc *mysqlConn) readColumns(count int) ([]mysqlField, error) {
pos += n
// Table [len coded string]
if mc.cfg.columnsWithAlias {
if mc.cfg.ColumnsWithAlias {
tableName, _, n, err := readLengthEncodedString(data[pos:])
if err != nil {
return nil, err
@ -603,6 +672,11 @@ func (rows *textRows) readRow(dest []driver.Value) error {
// EOF Packet
if data[0] == iEOF && len(data) == 5 {
// server_status [2 bytes]
rows.mc.status = readStatus(data[3:])
if err := rows.mc.discardResults(); err != nil {
return err
}
rows.mc = nil
return io.EOF
}
@ -630,7 +704,7 @@ func (rows *textRows) readRow(dest []driver.Value) error {
fieldTypeDate, fieldTypeNewDate:
dest[i], err = parseDateTime(
string(dest[i].([]byte)),
mc.cfg.loc,
mc.cfg.Loc,
)
if err == nil {
continue
@ -660,6 +734,10 @@ func (mc *mysqlConn) readUntilEOF() error {
if err == nil && data[0] != iEOF {
continue
}
if err == nil && data[0] == iEOF && len(data) == 5 {
mc.status = readStatus(data[3:])
}
return err // Err or EOF
}
}
@ -692,13 +770,13 @@ func (stmt *mysqlStmt) readPrepareResultPacket() (uint16, error) {
// Warning count [16 bit uint]
if !stmt.mc.strict {
return columnCount, nil
} else {
// Check for warnings count > 0, only available in MySQL > 4.1
if len(data) >= 12 && binary.LittleEndian.Uint16(data[10:12]) > 0 {
return columnCount, stmt.mc.getWarnings()
}
return columnCount, nil
}
// Check for warnings count > 0, only available in MySQL > 4.1
if len(data) >= 12 && binary.LittleEndian.Uint16(data[10:12]) > 0 {
return columnCount, stmt.mc.getWarnings()
}
return columnCount, nil
}
return 0, err
}
@ -760,7 +838,7 @@ func (stmt *mysqlStmt) writeCommandLongData(paramID int, arg []byte) error {
func (stmt *mysqlStmt) writeExecutePacket(args []driver.Value) error {
if len(args) != stmt.paramCount {
return fmt.Errorf(
"Arguments count mismatch (Got: %d Has: %d)",
"argument count mismatch (got: %d; has: %d)",
len(args),
stmt.paramCount,
)
@ -937,7 +1015,7 @@ func (stmt *mysqlStmt) writeExecutePacket(args []driver.Value) error {
if v.IsZero() {
val = []byte("0000-00-00")
} else {
val = []byte(v.In(mc.cfg.loc).Format(timeFormat))
val = []byte(v.In(mc.cfg.Loc).Format(timeFormat))
}
paramValues = appendLengthEncodedInteger(paramValues,
@ -946,7 +1024,7 @@ func (stmt *mysqlStmt) writeExecutePacket(args []driver.Value) error {
paramValues = append(paramValues, val...)
default:
return fmt.Errorf("Can't convert type: %T", arg)
return fmt.Errorf("can not convert type: %T", arg)
}
}
@ -964,6 +1042,28 @@ func (stmt *mysqlStmt) writeExecutePacket(args []driver.Value) error {
return mc.writePacket(data)
}
func (mc *mysqlConn) discardResults() error {
for mc.status&statusMoreResultsExists != 0 {
resLen, err := mc.readResultSetHeaderPacket()
if err != nil {
return err
}
if resLen > 0 {
// columns
if err := mc.readUntilEOF(); err != nil {
return err
}
// rows
if err := mc.readUntilEOF(); err != nil {
return err
}
} else {
mc.status &^= statusMoreResultsExists
}
}
return nil
}
// http://dev.mysql.com/doc/internals/en/binary-protocol-resultset-row.html
func (rows *binaryRows) readRow(dest []driver.Value) error {
data, err := rows.mc.readPacket()
@ -973,11 +1073,16 @@ func (rows *binaryRows) readRow(dest []driver.Value) error {
// packet indicator [1 byte]
if data[0] != iOK {
rows.mc = nil
// EOF Packet
if data[0] == iEOF && len(data) == 5 {
rows.mc.status = readStatus(data[3:])
if err := rows.mc.discardResults(); err != nil {
return err
}
rows.mc = nil
return io.EOF
}
rows.mc = nil
// Error otherwise
return rows.mc.handleErrorPacket(data)
@ -1057,7 +1162,7 @@ func (rows *binaryRows) readRow(dest []driver.Value) error {
case fieldTypeDecimal, fieldTypeNewDecimal, fieldTypeVarChar,
fieldTypeBit, fieldTypeEnum, fieldTypeSet, fieldTypeTinyBLOB,
fieldTypeMediumBLOB, fieldTypeLongBLOB, fieldTypeBLOB,
fieldTypeVarString, fieldTypeString, fieldTypeGeometry:
fieldTypeVarString, fieldTypeString, fieldTypeGeometry, fieldTypeJSON:
var isNull bool
var n int
dest[i], isNull, n, err = readLengthEncodedString(data[pos:])
@ -1094,13 +1199,13 @@ func (rows *binaryRows) readRow(dest []driver.Value) error {
dstlen = 8 + 1 + decimals
default:
return fmt.Errorf(
"MySQL protocol error, illegal decimals value %d",
"protocol error, illegal decimals value %d",
rows.columns[i].decimals,
)
}
dest[i], err = formatBinaryDateTime(data[pos:pos+int(num)], dstlen, true)
case rows.mc.parseTime:
dest[i], err = parseBinaryDateTime(num, data[pos:], rows.mc.cfg.loc)
dest[i], err = parseBinaryDateTime(num, data[pos:], rows.mc.cfg.Loc)
default:
var dstlen uint8
if rows.columns[i].fieldType == fieldTypeDate {
@ -1113,7 +1218,7 @@ func (rows *binaryRows) readRow(dest []driver.Value) error {
dstlen = 19 + 1 + decimals
default:
return fmt.Errorf(
"MySQL protocol error, illegal decimals value %d",
"protocol error, illegal decimals value %d",
rows.columns[i].decimals,
)
}
@ -1130,7 +1235,7 @@ func (rows *binaryRows) readRow(dest []driver.Value) error {
// Please report if this happens!
default:
return fmt.Errorf("Unknown FieldType %d", rows.columns[i].fieldType)
return fmt.Errorf("unknown field type %d", rows.columns[i].fieldType)
}
}

View File

@ -38,9 +38,13 @@ type emptyRows struct{}
func (rows *mysqlRows) Columns() []string {
columns := make([]string, len(rows.columns))
if rows.mc.cfg.columnsWithAlias {
if rows.mc != nil && rows.mc.cfg.ColumnsWithAlias {
for i := range columns {
columns[i] = rows.columns[i].tableName + "." + rows.columns[i].name
if tableName := rows.columns[i].tableName; len(tableName) > 0 {
columns[i] = tableName + "." + rows.columns[i].name
} else {
columns[i] = rows.columns[i].name
}
}
} else {
for i := range columns {
@ -61,6 +65,12 @@ func (rows *mysqlRows) Close() error {
// Remove unread packets from stream
err := mc.readUntilEOF()
if err == nil {
if err = mc.discardResults(); err != nil {
return err
}
}
rows.mc = nil
return err
}

View File

@ -10,6 +10,9 @@ package mysql
import (
"database/sql/driver"
"fmt"
"reflect"
"strconv"
)
type mysqlStmt struct {
@ -34,6 +37,10 @@ func (stmt *mysqlStmt) NumInput() int {
return stmt.paramCount
}
func (stmt *mysqlStmt) ColumnConverter(idx int) driver.ValueConverter {
return converter{}
}
func (stmt *mysqlStmt) Exec(args []driver.Value) (driver.Result, error) {
if stmt.mc.netConn == nil {
errLog.Print(ErrInvalidConn)
@ -94,9 +101,9 @@ func (stmt *mysqlStmt) Query(args []driver.Value) (driver.Rows, error) {
}
rows := new(binaryRows)
rows.mc = mc
if resLen > 0 {
rows.mc = mc
// Columns
// If not cached, read them and cache them
if stmt.columns == nil {
@ -110,3 +117,34 @@ func (stmt *mysqlStmt) Query(args []driver.Value) (driver.Rows, error) {
return rows, err
}
type converter struct{}
func (c converter) ConvertValue(v interface{}) (driver.Value, error) {
if driver.IsValue(v) {
return v, nil
}
rv := reflect.ValueOf(v)
switch rv.Kind() {
case reflect.Ptr:
// indirect pointers
if rv.IsNil() {
return nil, nil
}
return c.ConvertValue(rv.Elem().Interface())
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return rv.Int(), nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32:
return int64(rv.Uint()), nil
case reflect.Uint64:
u64 := rv.Uint()
if u64 >= 1<<63 {
return strconv.FormatUint(u64, 10), nil
}
return int64(u64), nil
case reflect.Float32, reflect.Float64:
return rv.Float(), nil
}
return nil, fmt.Errorf("unsupported type %T, a %s", v, rv.Kind())
}

View File

@ -13,28 +13,16 @@ import (
"crypto/tls"
"database/sql/driver"
"encoding/binary"
"errors"
"fmt"
"io"
"net"
"net/url"
"strings"
"time"
)
var (
tlsConfigRegister map[string]*tls.Config // Register for custom tls.Configs
errInvalidDSNUnescaped = errors.New("Invalid DSN: Did you forget to escape a param value?")
errInvalidDSNAddr = errors.New("Invalid DSN: Network Address not terminated (missing closing brace)")
errInvalidDSNNoSlash = errors.New("Invalid DSN: Missing the slash separating the database name")
errInvalidDSNUnsafeCollation = errors.New("Invalid DSN: interpolateParams can be used with ascii, latin1, utf8 and utf8mb4 charset")
)
func init() {
tlsConfigRegister = make(map[string]*tls.Config)
}
// RegisterTLSConfig registers a custom tls.Config to be used with sql.Open.
// Use the key as a value in the DSN where tls=value.
//
@ -60,7 +48,11 @@ func init() {
//
func RegisterTLSConfig(key string, config *tls.Config) error {
if _, isBool := readBool(key); isBool || strings.ToLower(key) == "skip-verify" {
return fmt.Errorf("Key '%s' is reserved", key)
return fmt.Errorf("key '%s' is reserved", key)
}
if tlsConfigRegister == nil {
tlsConfigRegister = make(map[string]*tls.Config)
}
tlsConfigRegister[key] = config
@ -69,228 +61,9 @@ func RegisterTLSConfig(key string, config *tls.Config) error {
// DeregisterTLSConfig removes the tls.Config associated with key.
func DeregisterTLSConfig(key string) {
delete(tlsConfigRegister, key)
}
// parseDSN parses the DSN string to a config
func parseDSN(dsn string) (cfg *config, err error) {
// New config with some default values
cfg = &config{
loc: time.UTC,
collation: defaultCollation,
if tlsConfigRegister != nil {
delete(tlsConfigRegister, key)
}
// TODO: use strings.IndexByte when we can depend on Go 1.2
// [user[:password]@][net[(addr)]]/dbname[?param1=value1&paramN=valueN]
// Find the last '/' (since the password or the net addr might contain a '/')
foundSlash := false
for i := len(dsn) - 1; i >= 0; i-- {
if dsn[i] == '/' {
foundSlash = true
var j, k int
// left part is empty if i <= 0
if i > 0 {
// [username[:password]@][protocol[(address)]]
// Find the last '@' in dsn[:i]
for j = i; j >= 0; j-- {
if dsn[j] == '@' {
// username[:password]
// Find the first ':' in dsn[:j]
for k = 0; k < j; k++ {
if dsn[k] == ':' {
cfg.passwd = dsn[k+1 : j]
break
}
}
cfg.user = dsn[:k]
break
}
}
// [protocol[(address)]]
// Find the first '(' in dsn[j+1:i]
for k = j + 1; k < i; k++ {
if dsn[k] == '(' {
// dsn[i-1] must be == ')' if an address is specified
if dsn[i-1] != ')' {
if strings.ContainsRune(dsn[k+1:i], ')') {
return nil, errInvalidDSNUnescaped
}
return nil, errInvalidDSNAddr
}
cfg.addr = dsn[k+1 : i-1]
break
}
}
cfg.net = dsn[j+1 : k]
}
// dbname[?param1=value1&...&paramN=valueN]
// Find the first '?' in dsn[i+1:]
for j = i + 1; j < len(dsn); j++ {
if dsn[j] == '?' {
if err = parseDSNParams(cfg, dsn[j+1:]); err != nil {
return
}
break
}
}
cfg.dbname = dsn[i+1 : j]
break
}
}
if !foundSlash && len(dsn) > 0 {
return nil, errInvalidDSNNoSlash
}
if cfg.interpolateParams && unsafeCollations[cfg.collation] {
return nil, errInvalidDSNUnsafeCollation
}
// Set default network if empty
if cfg.net == "" {
cfg.net = "tcp"
}
// Set default address if empty
if cfg.addr == "" {
switch cfg.net {
case "tcp":
cfg.addr = "127.0.0.1:3306"
case "unix":
cfg.addr = "/tmp/mysql.sock"
default:
return nil, errors.New("Default addr for network '" + cfg.net + "' unknown")
}
}
return
}
// parseDSNParams parses the DSN "query string"
// Values must be url.QueryEscape'ed
func parseDSNParams(cfg *config, params string) (err error) {
for _, v := range strings.Split(params, "&") {
param := strings.SplitN(v, "=", 2)
if len(param) != 2 {
continue
}
// cfg params
switch value := param[1]; param[0] {
// Enable client side placeholder substitution
case "interpolateParams":
var isBool bool
cfg.interpolateParams, isBool = readBool(value)
if !isBool {
return fmt.Errorf("Invalid Bool value: %s", value)
}
// Disable INFILE whitelist / enable all files
case "allowAllFiles":
var isBool bool
cfg.allowAllFiles, isBool = readBool(value)
if !isBool {
return fmt.Errorf("Invalid Bool value: %s", value)
}
// Use old authentication mode (pre MySQL 4.1)
case "allowOldPasswords":
var isBool bool
cfg.allowOldPasswords, isBool = readBool(value)
if !isBool {
return fmt.Errorf("Invalid Bool value: %s", value)
}
// Switch "rowsAffected" mode
case "clientFoundRows":
var isBool bool
cfg.clientFoundRows, isBool = readBool(value)
if !isBool {
return fmt.Errorf("Invalid Bool value: %s", value)
}
// Collation
case "collation":
collation, ok := collations[value]
if !ok {
// Note possibility for false negatives:
// could be triggered although the collation is valid if the
// collations map does not contain entries the server supports.
err = errors.New("unknown collation")
return
}
cfg.collation = collation
break
case "columnsWithAlias":
var isBool bool
cfg.columnsWithAlias, isBool = readBool(value)
if !isBool {
return fmt.Errorf("Invalid Bool value: %s", value)
}
// Time Location
case "loc":
if value, err = url.QueryUnescape(value); err != nil {
return
}
cfg.loc, err = time.LoadLocation(value)
if err != nil {
return
}
// Dial Timeout
case "timeout":
cfg.timeout, err = time.ParseDuration(value)
if err != nil {
return
}
// TLS-Encryption
case "tls":
boolValue, isBool := readBool(value)
if isBool {
if boolValue {
cfg.tls = &tls.Config{}
}
} else {
if strings.ToLower(value) == "skip-verify" {
cfg.tls = &tls.Config{InsecureSkipVerify: true}
} else if tlsConfig, ok := tlsConfigRegister[value]; ok {
if len(tlsConfig.ServerName) == 0 && !tlsConfig.InsecureSkipVerify {
host, _, err := net.SplitHostPort(cfg.addr)
if err == nil {
tlsConfig.ServerName = host
}
}
cfg.tls = tlsConfig
} else {
return fmt.Errorf("Invalid value / unknown config name: %s", value)
}
}
default:
// lazy init
if cfg.params == nil {
cfg.params = make(map[string]string)
}
if cfg.params[param[0]], err = url.QueryUnescape(value); err != nil {
return
}
}
}
return
}
// Returns the bool value of the input.
@ -487,7 +260,7 @@ func parseDateTime(str string, loc *time.Location) (t time.Time, err error) {
}
t, err = time.Parse(timeFormat[:len(str)], str)
default:
err = fmt.Errorf("Invalid Time-String: %s", str)
err = fmt.Errorf("invalid time string: %s", str)
return
}
@ -536,7 +309,7 @@ func parseBinaryDateTime(num uint64, data []byte, loc *time.Location) (driver.Va
loc,
), nil
}
return nil, fmt.Errorf("Invalid DATETIME-packet length %d", num)
return nil, fmt.Errorf("invalid DATETIME packet length %d", num)
}
// zeroDateTime is used in formatBinaryDateTime to avoid an allocation
@ -571,7 +344,7 @@ func formatBinaryDateTime(src []byte, length uint8, justTime bool) (driver.Value
switch len(src) {
case 8, 12:
default:
return nil, fmt.Errorf("Invalid TIME-packet length %d", len(src))
return nil, fmt.Errorf("invalid TIME packet length %d", len(src))
}
// +2 to enable negative time and 100+ hours
dst = make([]byte, 0, length+2)
@ -605,7 +378,7 @@ func formatBinaryDateTime(src []byte, length uint8, justTime bool) (driver.Value
if length > 10 {
t += "TIME"
}
return nil, fmt.Errorf("illegal %s-packet length %d", t, len(src))
return nil, fmt.Errorf("illegal %s packet length %d", t, len(src))
}
dst = make([]byte, 0, length)
// start with the date
@ -771,6 +544,10 @@ func skipLengthEncodedString(b []byte) (int, error) {
// returns the number read, whether the value is NULL and the number of bytes read
func readLengthEncodedInteger(b []byte) (uint64, bool, int) {
// See issue #349
if len(b) == 0 {
return 0, true, 1
}
switch b[0] {
// 251: NULL
@ -867,7 +644,7 @@ func escapeBytesBackslash(buf, v []byte) []byte {
pos += 2
default:
buf[pos] = c
pos += 1
pos++
}
}
@ -912,7 +689,7 @@ func escapeStringBackslash(buf []byte, v string) []byte {
pos += 2
default:
buf[pos] = c
pos += 1
pos++
}
}

View File

@ -1,346 +0,0 @@
// Go MySQL Driver - A MySQL-Driver for Go's database/sql package
//
// Copyright 2013 The Go-MySQL-Driver Authors. All rights reserved.
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at http://mozilla.org/MPL/2.0/.
package mysql
import (
"bytes"
"crypto/tls"
"encoding/binary"
"fmt"
"testing"
"time"
)
var testDSNs = []struct {
in string
out string
loc *time.Location
}{
{"username:password@protocol(address)/dbname?param=value", "&{user:username passwd:password net:protocol addr:address dbname:dbname params:map[param:value] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:false interpolateParams:false}", time.UTC},
{"username:password@protocol(address)/dbname?param=value&columnsWithAlias=true", "&{user:username passwd:password net:protocol addr:address dbname:dbname params:map[param:value] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:true interpolateParams:false}", time.UTC},
{"user@unix(/path/to/socket)/dbname?charset=utf8", "&{user:user passwd: net:unix addr:/path/to/socket dbname:dbname params:map[charset:utf8] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:false interpolateParams:false}", time.UTC},
{"user:password@tcp(localhost:5555)/dbname?charset=utf8&tls=true", "&{user:user passwd:password net:tcp addr:localhost:5555 dbname:dbname params:map[charset:utf8] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:false interpolateParams:false}", time.UTC},
{"user:password@tcp(localhost:5555)/dbname?charset=utf8mb4,utf8&tls=skip-verify", "&{user:user passwd:password net:tcp addr:localhost:5555 dbname:dbname params:map[charset:utf8mb4,utf8] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:false interpolateParams:false}", time.UTC},
{"user:password@/dbname?loc=UTC&timeout=30s&allowAllFiles=1&clientFoundRows=true&allowOldPasswords=TRUE&collation=utf8mb4_unicode_ci", "&{user:user passwd:password net:tcp addr:127.0.0.1:3306 dbname:dbname params:map[] loc:%p tls:<nil> timeout:30000000000 collation:224 allowAllFiles:true allowOldPasswords:true clientFoundRows:true columnsWithAlias:false interpolateParams:false}", time.UTC},
{"user:p@ss(word)@tcp([de:ad:be:ef::ca:fe]:80)/dbname?loc=Local", "&{user:user passwd:p@ss(word) net:tcp addr:[de:ad:be:ef::ca:fe]:80 dbname:dbname params:map[] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:false interpolateParams:false}", time.Local},
{"/dbname", "&{user: passwd: net:tcp addr:127.0.0.1:3306 dbname:dbname params:map[] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:false interpolateParams:false}", time.UTC},
{"@/", "&{user: passwd: net:tcp addr:127.0.0.1:3306 dbname: params:map[] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:false interpolateParams:false}", time.UTC},
{"/", "&{user: passwd: net:tcp addr:127.0.0.1:3306 dbname: params:map[] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:false interpolateParams:false}", time.UTC},
{"", "&{user: passwd: net:tcp addr:127.0.0.1:3306 dbname: params:map[] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:false interpolateParams:false}", time.UTC},
{"user:p@/ssword@/", "&{user:user passwd:p@/ssword net:tcp addr:127.0.0.1:3306 dbname: params:map[] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:false interpolateParams:false}", time.UTC},
{"unix/?arg=%2Fsome%2Fpath.ext", "&{user: passwd: net:unix addr:/tmp/mysql.sock dbname: params:map[arg:/some/path.ext] loc:%p tls:<nil> timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false columnsWithAlias:false interpolateParams:false}", time.UTC},
}
func TestDSNParser(t *testing.T) {
var cfg *config
var err error
var res string
for i, tst := range testDSNs {
cfg, err = parseDSN(tst.in)
if err != nil {
t.Error(err.Error())
}
// pointer not static
cfg.tls = nil
res = fmt.Sprintf("%+v", cfg)
if res != fmt.Sprintf(tst.out, tst.loc) {
t.Errorf("%d. parseDSN(%q) => %q, want %q", i, tst.in, res, fmt.Sprintf(tst.out, tst.loc))
}
}
}
func TestDSNParserInvalid(t *testing.T) {
var invalidDSNs = []string{
"@net(addr/", // no closing brace
"@tcp(/", // no closing brace
"tcp(/", // no closing brace
"(/", // no closing brace
"net(addr)//", // unescaped
"user:pass@tcp(1.2.3.4:3306)", // no trailing slash
//"/dbname?arg=/some/unescaped/path",
}
for i, tst := range invalidDSNs {
if _, err := parseDSN(tst); err == nil {
t.Errorf("invalid DSN #%d. (%s) didn't error!", i, tst)
}
}
}
func TestDSNWithCustomTLS(t *testing.T) {
baseDSN := "user:password@tcp(localhost:5555)/dbname?tls="
tlsCfg := tls.Config{}
RegisterTLSConfig("utils_test", &tlsCfg)
// Custom TLS is missing
tst := baseDSN + "invalid_tls"
cfg, err := parseDSN(tst)
if err == nil {
t.Errorf("Invalid custom TLS in DSN (%s) but did not error. Got config: %#v", tst, cfg)
}
tst = baseDSN + "utils_test"
// Custom TLS with a server name
name := "foohost"
tlsCfg.ServerName = name
cfg, err = parseDSN(tst)
if err != nil {
t.Error(err.Error())
} else if cfg.tls.ServerName != name {
t.Errorf("Did not get the correct TLS ServerName (%s) parsing DSN (%s).", name, tst)
}
// Custom TLS without a server name
name = "localhost"
tlsCfg.ServerName = ""
cfg, err = parseDSN(tst)
if err != nil {
t.Error(err.Error())
} else if cfg.tls.ServerName != name {
t.Errorf("Did not get the correct ServerName (%s) parsing DSN (%s).", name, tst)
}
DeregisterTLSConfig("utils_test")
}
func TestDSNUnsafeCollation(t *testing.T) {
_, err := parseDSN("/dbname?collation=gbk_chinese_ci&interpolateParams=true")
if err != errInvalidDSNUnsafeCollation {
t.Error("Expected %v, Got %v", errInvalidDSNUnsafeCollation, err)
}
_, err = parseDSN("/dbname?collation=gbk_chinese_ci&interpolateParams=false")
if err != nil {
t.Error("Expected %v, Got %v", nil, err)
}
_, err = parseDSN("/dbname?collation=gbk_chinese_ci")
if err != nil {
t.Error("Expected %v, Got %v", nil, err)
}
_, err = parseDSN("/dbname?collation=ascii_bin&interpolateParams=true")
if err != nil {
t.Error("Expected %v, Got %v", nil, err)
}
_, err = parseDSN("/dbname?collation=latin1_german1_ci&interpolateParams=true")
if err != nil {
t.Error("Expected %v, Got %v", nil, err)
}
_, err = parseDSN("/dbname?collation=utf8_general_ci&interpolateParams=true")
if err != nil {
t.Error("Expected %v, Got %v", nil, err)
}
_, err = parseDSN("/dbname?collation=utf8mb4_general_ci&interpolateParams=true")
if err != nil {
t.Error("Expected %v, Got %v", nil, err)
}
}
func BenchmarkParseDSN(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
for _, tst := range testDSNs {
if _, err := parseDSN(tst.in); err != nil {
b.Error(err.Error())
}
}
}
}
func TestScanNullTime(t *testing.T) {
var scanTests = []struct {
in interface{}
error bool
valid bool
time time.Time
}{
{tDate, false, true, tDate},
{sDate, false, true, tDate},
{[]byte(sDate), false, true, tDate},
{tDateTime, false, true, tDateTime},
{sDateTime, false, true, tDateTime},
{[]byte(sDateTime), false, true, tDateTime},
{tDate0, false, true, tDate0},
{sDate0, false, true, tDate0},
{[]byte(sDate0), false, true, tDate0},
{sDateTime0, false, true, tDate0},
{[]byte(sDateTime0), false, true, tDate0},
{"", true, false, tDate0},
{"1234", true, false, tDate0},
{0, true, false, tDate0},
}
var nt = NullTime{}
var err error
for _, tst := range scanTests {
err = nt.Scan(tst.in)
if (err != nil) != tst.error {
t.Errorf("%v: expected error status %t, got %t", tst.in, tst.error, (err != nil))
}
if nt.Valid != tst.valid {
t.Errorf("%v: expected valid status %t, got %t", tst.in, tst.valid, nt.Valid)
}
if nt.Time != tst.time {
t.Errorf("%v: expected time %v, got %v", tst.in, tst.time, nt.Time)
}
}
}
func TestLengthEncodedInteger(t *testing.T) {
var integerTests = []struct {
num uint64
encoded []byte
}{
{0x0000000000000000, []byte{0x00}},
{0x0000000000000012, []byte{0x12}},
{0x00000000000000fa, []byte{0xfa}},
{0x0000000000000100, []byte{0xfc, 0x00, 0x01}},
{0x0000000000001234, []byte{0xfc, 0x34, 0x12}},
{0x000000000000ffff, []byte{0xfc, 0xff, 0xff}},
{0x0000000000010000, []byte{0xfd, 0x00, 0x00, 0x01}},
{0x0000000000123456, []byte{0xfd, 0x56, 0x34, 0x12}},
{0x0000000000ffffff, []byte{0xfd, 0xff, 0xff, 0xff}},
{0x0000000001000000, []byte{0xfe, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00}},
{0x123456789abcdef0, []byte{0xfe, 0xf0, 0xde, 0xbc, 0x9a, 0x78, 0x56, 0x34, 0x12}},
{0xffffffffffffffff, []byte{0xfe, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}},
}
for _, tst := range integerTests {
num, isNull, numLen := readLengthEncodedInteger(tst.encoded)
if isNull {
t.Errorf("%x: expected %d, got NULL", tst.encoded, tst.num)
}
if num != tst.num {
t.Errorf("%x: expected %d, got %d", tst.encoded, tst.num, num)
}
if numLen != len(tst.encoded) {
t.Errorf("%x: expected size %d, got %d", tst.encoded, len(tst.encoded), numLen)
}
encoded := appendLengthEncodedInteger(nil, num)
if !bytes.Equal(encoded, tst.encoded) {
t.Errorf("%v: expected %x, got %x", num, tst.encoded, encoded)
}
}
}
func TestOldPass(t *testing.T) {
scramble := []byte{9, 8, 7, 6, 5, 4, 3, 2}
vectors := []struct {
pass string
out string
}{
{" pass", "47575c5a435b4251"},
{"pass ", "47575c5a435b4251"},
{"123\t456", "575c47505b5b5559"},
{"C0mpl!ca ted#PASS123", "5d5d554849584a45"},
}
for _, tuple := range vectors {
ours := scrambleOldPassword(scramble, []byte(tuple.pass))
if tuple.out != fmt.Sprintf("%x", ours) {
t.Errorf("Failed old password %q", tuple.pass)
}
}
}
func TestFormatBinaryDateTime(t *testing.T) {
rawDate := [11]byte{}
binary.LittleEndian.PutUint16(rawDate[:2], 1978) // years
rawDate[2] = 12 // months
rawDate[3] = 30 // days
rawDate[4] = 15 // hours
rawDate[5] = 46 // minutes
rawDate[6] = 23 // seconds
binary.LittleEndian.PutUint32(rawDate[7:], 987654) // microseconds
expect := func(expected string, inlen, outlen uint8) {
actual, _ := formatBinaryDateTime(rawDate[:inlen], outlen, false)
bytes, ok := actual.([]byte)
if !ok {
t.Errorf("formatBinaryDateTime must return []byte, was %T", actual)
}
if string(bytes) != expected {
t.Errorf(
"expected %q, got %q for length in %d, out %d",
bytes, actual, inlen, outlen,
)
}
}
expect("0000-00-00", 0, 10)
expect("0000-00-00 00:00:00", 0, 19)
expect("1978-12-30", 4, 10)
expect("1978-12-30 15:46:23", 7, 19)
expect("1978-12-30 15:46:23.987654", 11, 26)
}
func TestEscapeBackslash(t *testing.T) {
expect := func(expected, value string) {
actual := string(escapeBytesBackslash([]byte{}, []byte(value)))
if actual != expected {
t.Errorf(
"expected %s, got %s",
expected, actual,
)
}
actual = string(escapeStringBackslash([]byte{}, value))
if actual != expected {
t.Errorf(
"expected %s, got %s",
expected, actual,
)
}
}
expect("foo\\0bar", "foo\x00bar")
expect("foo\\nbar", "foo\nbar")
expect("foo\\rbar", "foo\rbar")
expect("foo\\Zbar", "foo\x1abar")
expect("foo\\\"bar", "foo\"bar")
expect("foo\\\\bar", "foo\\bar")
expect("foo\\'bar", "foo'bar")
}
func TestEscapeQuotes(t *testing.T) {
expect := func(expected, value string) {
actual := string(escapeBytesQuotes([]byte{}, []byte(value)))
if actual != expected {
t.Errorf(
"expected %s, got %s",
expected, actual,
)
}
actual = string(escapeStringQuotes([]byte{}, value))
if actual != expected {
t.Errorf(
"expected %s, got %s",
expected, actual,
)
}
}
expect("foo\x00bar", "foo\x00bar") // not affected
expect("foo\nbar", "foo\nbar") // not affected
expect("foo\rbar", "foo\rbar") // not affected
expect("foo\x1abar", "foo\x1abar") // not affected
expect("foo''bar", "foo'bar") // affected
expect("foo\"bar", "foo\"bar") // not affected
}

23
test.sh
View File

@ -7,7 +7,9 @@ fi
# The list of segments to run. To run only some of these segments, pre-set the
# RUN variable with the ones you want (see .travis.yml for an example).
# Order doesn't matter.
# Order doesn't matter. Note: godep-restore is specifically left out of the
# defaults, because we don't want to run it locally (would be too disruptive to
# GOPATH).
RUN=${RUN:-vet lint fmt migrations unit integration}
# The list of segments to hard fail on, as opposed to continuing to the end of
@ -281,4 +283,23 @@ if [[ "$RUN" =~ "integration" ]] ; then
end_context #integration
fi
# Run godep-restore (happens only in Travis) to check that the hashes in
# Godeps.json really exist in the remote repo and match what we have.
if [[ "$RUN" =~ "godep-restore" ]] ; then
start_context "godep-restore"
run_and_comment godep restore
# Run godep save and do a diff, to ensure that the version we got from
# `godep restore` matched what was in the remote repo. We only do this on
# builds of the main fork (not PRs from external contributors), because godep
# rewrites import paths to the path of the fork we're building from, which
# creates spurious diffs if we're not building from the main fork.
# Once we switch to Go 1.6's imports and don't need rewriting anymore, we can
# do this for all builds.
if [[ "${TRAVIS_REPO_SLUG}" == "letsencrypt/boulder" ]] ; then
run_and_comment godep save -r ./...
run_and_comment git diff --exit-code
fi
end_context #godep-restore
fi
exit ${FAILURE}

View File

@ -149,7 +149,7 @@
},
"pa": {
"dbConnect": "mysql+tcp://policy@localhost:3306/boulder_policy_integration",
"dbConnect": "mysql+tcp://policy@localhost:3306/boulder_policy_integration?readTimeout=800ms&writeTimeout=800ms",
"challenges": {
"http-01": true,
"tls-sni-01": true,
@ -237,7 +237,7 @@
},
"ocspResponder": {
"source": "mysql+tcp://ocsp_resp@localhost:3306/boulder_sa_integration",
"source": "mysql+tcp://ocsp_resp@localhost:3306/boulder_sa_integration?readTimeout=800ms&writeTimeout=800ms",
"path": "/",
"listenAddress": "localhost:4002",
"maxAge": "10s",

View File

@ -1 +1 @@
mysql+tcp://ocsp_update@localhost:3306/boulder_sa_integration
mysql+tcp://ocsp_update@localhost:3306/boulder_sa_integration?readTimeout=800ms&writeTimeout=800ms

View File

@ -1 +1 @@
mysql+tcp://policy@localhost:3306/boulder_policy_integration
mysql+tcp://policy@localhost:3306/boulder_policy_integration?readTimeout=800ms&writeTimeout=800ms

View File

@ -1 +1 @@
mysql+tcp://sa@localhost:3306/boulder_sa_integration
mysql+tcp://sa@localhost:3306/boulder_sa_integration?readTimeout=14s&writeTimeout=14s

View File

@ -8,6 +8,7 @@ go get \
golang.org/x/tools/cmd/vet \
golang.org/x/tools/cmd/cover \
github.com/golang/lint/golint \
github.com/tools/godep \
github.com/mattn/goveralls \
github.com/modocache/gover \
github.com/jcjones/github-pr-status \