Output write method may be called concurrently

[danielnelson]

Bug report

While no new metrics can be added during a flush due to metric_batch_size being reached (AddMetric), it is possible to reach the batch size during an interval flush, which will trigger a second call.

Relevant telegraf.conf:

System info:

Telegraf 1.3.3

Steps to reproduce:

1. Set interval to 500ms
2. Set flush interval to 1s
3. Add this patch:

--- a/plugins/outputs/influxdb/influxdb.go
+++ b/plugins/outputs/influxdb/influxdb.go
@@ -172,6 +172,7 @@ func (i *InfluxDB) Description() string {
 // Write will choose a random server in the cluster to write to until a successful write
 // occurs, logging each unsuccessful. If all servers fail, return error.
 func (i *InfluxDB) Write(metrics []telegraf.Metric) error {
+       time.Sleep(time.Duration(rand.Uint64()%3000) * time.Millisecond)
 
        bufsize := 0
        for _, m := range metrics {

4. go build --race ./cmd/telegraf
5. Run telegraf with influxdb output

Expected behavior:

Calls to write are serialized

Actual behavior:

Write called twice concurrently

Additional info:

WARNING: DATA RACE
Write at 0x00c4200211b0 by goroutine 55:
  github.com/influxdata/telegraf/metric.(*metric).SerializeTo()
      /home/dbn/src/influx/src/github.com/influxdata/telegraf/metric/metric.go:208 +0x265
  github.com/influxdata/telegraf/metric.(*reader).Read()
      /home/dbn/src/influx/src/github.com/influxdata/telegraf/metric/reader.go:61 +0x221
  github.com/influxdata/telegraf/plugins/outputs/influxdb/client.(*udpClient).WriteStream()
      /home/dbn/src/influx/src/github.com/influxdata/telegraf/plugins/outputs/influxdb/client/udp.go:83 +0xa3
  github.com/influxdata/telegraf/plugins/outputs/influxdb.(*InfluxDB).Write()
      /home/dbn/src/influx/src/github.com/influxdata/telegraf/plugins/outputs/influxdb/influxdb.go:189 +0x320
  github.com/influxdata/telegraf/internal/models.(*RunningOutput).write()
      /home/dbn/src/influx/src/github.com/influxdata/telegraf/internal/models/running_output.go:173 +0xc6
  github.com/influxdata/telegraf/internal/models.(*RunningOutput).AddMetric()
      /home/dbn/src/influx/src/github.com/influxdata/telegraf/internal/models/running_output.go:115 +0x2a1
  github.com/influxdata/telegraf/agent.(*Agent).flusher.func1()
      /home/dbn/src/influx/src/github.com/influxdata/telegraf/agent/agent.go:284 +0x275

Previous write at 0x00c4200211b0 by goroutine 63:
  [failed to restore the stack]

Goroutine 55 (running) created at:
  github.com/influxdata/telegraf/agent.(*Agent).flusher()
      /home/dbn/src/influx/src/github.com/influxdata/telegraf/agent/agent.go:292 +0x121
  github.com/influxdata/telegraf/agent.(*Agent).Run.func1()
      /home/dbn/src/influx/src/github.com/influxdata/telegraf/agent/agent.go:370 +0x7d