Commit Graph

96 Commits

Author SHA1 Message Date
79fb33ace9 Merge pull request #18766 from mikedanese/gcplogs
Add logging driver for Google Cloud Logging
Upstream-commit: 3c4d093babb6f4f4f8343f5c364c5229a9d96ba3
Component: engine
2016-03-02 02:09:44 +01:00
f87c5bb0dc Merge pull request #20825 from LK4D4/unused
Remove some unused structs and fields
Upstream-commit: 106793dcbe49aedd96d74b0c4b0e32a972032d06
Component: engine
2016-03-01 12:58:35 -08:00
307185d18b Merge pull request #20617 from nalind/journald-pkgconfig
Try to handle changing names for journal packages
Upstream-commit: 36401f20ceb8b689fb619df4f6c7219fe76b521c
Component: engine
2016-03-01 15:38:03 -05:00
cf6760a787 Remove some unused structs and fields
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Upstream-commit: 0a352e1a906fbf7592aa95d6327776236d13392a
Component: engine
2016-03-01 09:59:29 -08:00
461ed7624b daemon/logger: Add logging driver for Google Cloud Logging
Signed-off-by: Mike Danese <mikedanese@google.com>
Upstream-commit: ed1b9fa07a0b34315d2fa624b978d3f8627319c2
Component: engine
2016-03-01 08:06:10 -08:00
6324c7793e Merge pull request #20605 from mountkin/optimize-json-log-writer
make the json log writer much faster
Upstream-commit: ba5a282a83a603c3bb61cd40642af54d2088f747
Component: engine
2016-02-29 10:47:46 -08:00
828abfa863 add file poller panic fix from 1.10.2
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: f9524a4d24d6135ae1777738390e1f6a590a7a1c
Component: engine
2016-02-24 10:17:29 -05:00
0a8b8e629c Revert "use pubsub instead of filenotify to follow json logs"
This reverts commit b1594c59f5e0d1ac898eacde8d91b1ba33c2b626.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 91fdfdd53722179b21b80b9dfbe8bc2e09d0a7b1
Component: engine
2016-02-23 21:43:52 -05:00
006622c38a Try to handle changing names for journal packages
When checking if we have the development files for libsystemd's journal
APIs, check for either 'libsystemd >= 209' and 'libsystemd-journal'.  If
we find 'libsystemd', define the 'journald' tag, which defaults to using
the 'libsystemd.pc' file.  If we find the older 'libsystemd-journal',
define both the 'journald' and 'journald_compat' tags, which causes the
'libsystemd-journal.pc' file to be consulted instead.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
Upstream-commit: 6cdc4ba6cd5178037466c50ebe03a7eb111c43b1
Component: engine
2016-02-23 12:24:27 -05:00
1805b24705 make the json log writer much faster
Signed-off-by: Shijiang Wei <mountkin@gmail.com>
Upstream-commit: d7af03111495cadd71abd3a8b5805066ea688e9b
Component: engine
2016-02-23 19:28:06 +08:00
5aea778205 Merge pull request #19689 from cednation/etwlogs
Windows: Add ETW logging driver plug-in
Upstream-commit: c795d0bab9057fbf16ca70349bcc37e244be33b9
Component: engine
2016-02-16 15:01:17 -08:00
442a84b35d Windows: Add ETW logging driver plug-in
Signed-off-by: Cedric Davies <cedricda@microsoft.com>
Upstream-commit: 3fe60bbf95b60f1a1e847a48e1c9b9730e570dff
Component: engine
2016-02-16 13:24:49 -08:00
6ebcbbfdd8 use pubsub instead of filenotify to follow json logs
inotify event is trigged immediately there's data written to disk.
But at the time that the inotify event is received, the json line might
not fully saved to disk. If the json decoder tries to decode in such
case, an io.UnexpectedEOF will be trigged.
We used to retry for several times to mitigate the io.UnexpectedEOF error.
But there are still flaky tests caused by the partial log entries.

The daemon knows exactly when there are new log entries emitted. We can
use the pubsub package to notify all the log readers instead of inotify.

Signed-off-by: Shijiang Wei <mountkin@gmail.com>

try to fix broken test. will squash once tests pass

Signed-off-by: Shijiang Wei <mountkin@gmail.com>
Upstream-commit: b1594c59f5e0d1ac898eacde8d91b1ba33c2b626
Component: engine
2016-02-15 19:25:16 +08:00
154bed9787 Merge pull request #18041 from jnummelin/feature/fluent-logger-ignore-connect-error-flag
Added flag to ignore fluentd connect error on container start
Upstream-commit: 7a016578dbb5c9b8a43b43e1ce96785fa26baf56
Component: engine
2016-01-27 14:25:24 -08:00
8977875ac5 Add tag support to journald logging driver, closes #19556
Signed-off-by: Ivan Babrou <ibobrik@gmail.com>
Upstream-commit: 5a3351883b254d3690e9dcc5b89293bcee474493
Component: engine
2016-01-27 10:52:19 +00:00
9669dc677a Added flag to ignore fluentd connect error on container start
Signed-off-by: Jussi Nummelin <jussi.nummelin@gmail.com>

Changed buffer size to 1M and removed unnecessary fmt call

Signed-off-by: Jussi Nummelin <jussi.nummelin@gmail.com>

Updated docs for the new fluentd opts

Signed-off-by: Jussi Nummelin <jussi.nummelin@gmail.com>
Upstream-commit: 3cf82ff1ab14e1ddd2b629524e894ac359168388
Component: engine
2016-01-27 09:05:44 +02:00
5ce70c9580 only close LogDriver after LogCopier is done
this prevents the copier from sending messages in the buffer to the closed
driver. If the copied took longer than the timeout to drain the buffer, this
aborts the copier read loop and return back so we can cleanup resources
properly.

Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
Upstream-commit: 84e14754e1ef3b089442398a31c5c5813fa9a1b6
Component: engine
2016-01-18 17:47:57 +00:00
1c0d76c749 Add support for syslog over TLS.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 4b98193beab00bc6cf48762858570a1bd418c9ef
Component: engine
2016-01-11 12:25:34 -05:00
be7836d63c Fix race condition in JSONFileLogger.Log
Signed-off-by: Ingo Gottwald <in.gottwald@gmail.com>
Upstream-commit: f8c54200805a9b9088c90f0b46bfa848d7bae67c
Component: engine
2016-01-03 22:45:56 +01:00
c737574605 Merge pull request #18759 from mikedanese/doc-fix
daemon/logger: fix typo in godoc
Upstream-commit: 6bf8844f1179108b9fabd271a655bf9eaaf1ee8c
Component: engine
2015-12-17 19:26:41 -05:00
f814dbe783 daemon/logger: fix typo in godoc
Signed-off-by: Mike Danese <mikedanese@google.com>
Upstream-commit: 881a30c707141b06af0cc7a076b62ff338615f2b
Component: engine
2015-12-17 15:52:33 -08:00
3d517a11e5 Replace pkg/units with docker/go-units.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 4fef42ba206ac90346e6e0fe25bead3f77dc4b0f
Component: engine
2015-12-16 12:26:49 -05:00
7f43cd332d Move timeutils functions to the only places where they are used.
- Move time json marshaling to the jsonlog package: this is a docker
  internal hack that we should not promote as a library.
- Move Timestamp encoding/decoding functions to the API types: This is
  only used there. It could be a standalone library but I don't this
it's worth having a separated repo for this. It could introduce more
complexity than it solves.

Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 27220ecc6b1eedf650ca9cf94965cb0dc2054efd
Component: engine
2015-12-15 14:56:14 -05:00
3c4fcf6b7a Fix typos found across repository
Signed-off-by: Justas Brazauskas <brazauskasjustas@gmail.com>
Upstream-commit: 927b334ebfc786276a039e45ec097e71bf9a104c
Component: engine
2015-12-13 18:04:12 +02:00
46aa4e0b30 dont trim gelf log message
Signed-off-by: Marius Sturm <marius@graylog.com>
Upstream-commit: a31435c8a2c4560b6e1a0e84450c45490fa587be
Component: engine
2015-12-11 10:28:24 +01:00
fb726f0d76 Revert "prevent journald from being built on ARM"
This reverts commit 6f6f10a75f8b447637e8a89d685452871899e9c0, so that we
can apply a different workaround.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
Upstream-commit: 0ca6d77e6e17ea378ff04b59f971dc1338e0ddc2
Component: engine
2015-12-02 10:30:13 -05:00
8f5a861b89 prevent journald from being built on ARM
Signed-off-by: Govinda Fichtner <govinda.fichtner@googlemail.com>
Upstream-commit: 6f6f10a75f8b447637e8a89d685452871899e9c0
Component: engine
2015-11-21 15:17:31 +01:00
804455e1fb Merge pull request #17593 from daehyeok/refactoring_json_log
Refactoring json log driver
Upstream-commit: adce1981f4c7fe653ae76cdd1f84d75f88b273cd
Component: engine
2015-11-17 15:38:52 -08:00
cbf9321ca8 Merge pull request #17891 from splunk/splunk-logger-support-for-tag-env-labels
Allow configurable metadata for Splunk log driver
Upstream-commit: a3065fa48f889ef438a7d7339ebc8fe01aca10dd
Component: engine
2015-11-16 10:11:27 -08:00
c0f1567bbc Extract rotate file writer from json log driver
Signed-off-by: Daehyeok Mun <daehyeok@gmail.com>
Upstream-commit: 086c0b4a6616e8db8009a5d72411bcc6774adaf1
Component: engine
2015-11-14 16:15:44 -07:00
1a8e0677b9 Merge pull request #17417 from cpuguy83/15815_add_log_file_poller
Fallback to file polling for jsonlog reader on err
Upstream-commit: 42fa8e31284c76f485bed81ed2e1327d60e5cb13
Component: engine
2015-11-13 11:49:54 -08:00
bba768bb09 Allow configurable metadata for Splunk log driver
Add support of `tag`, `env` and `labels` for Splunk logging driver.
Removed from message `containerId` as it is the same as `tag`.

Signed-off-by: Denis Gladkikh <denis@gladkikh.email>
Upstream-commit: 26855c780184c528446957bd77821c6f4c74b343
Component: engine
2015-11-12 06:36:51 -08:00
48001c30bb dockerversion placeholder for library import
- Add a *version* file placeholder.
- Update autogen and builds to use it and an autogen build flag

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 8054a303870b81eebe05e38261c1b68197b68558
Component: engine
2015-11-09 19:32:46 +01:00
66c6bac35d Fallback to file polling for jsonlog reader on err
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c136a33c5bf233f4ce108a8a8481d78aeaec93cc
Component: engine
2015-11-05 10:05:05 -05:00
74b08273a6 increase the maxJSONDecodeRetry in json log reader
Signed-off-by: Shijiang Wei <mountkin@gmail.com>
Upstream-commit: a82f9ac81956c6b63772c9c720878d377dd9ee02
Component: engine
2015-11-02 17:51:22 +08:00
08bc723a70 Merge pull request #17568 from Microsoft/10662-journald
Windows: Fix journald compile error
Upstream-commit: 8d584da96e71691d2d70c493818c2b056f7553c5
Component: engine
2015-10-31 09:35:39 -07:00
bb88c5713c Windows: Fix journald compile error
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 5452954d89160985d4d896f4241f1838b4114413
Component: engine
2015-10-31 08:31:25 -07:00
7f7650a345 Merge pull request #17537 from mountkin/fix-logger-race
fix a race in json logger reader
Upstream-commit: ed6e3cf7043e7454e10e390bc1a33343e3615c5d
Component: engine
2015-10-30 23:00:17 -07:00
3e20fd363b fix a race in json logger reader
The json decoder starts to decode immediately an inotify event is
received.
But at the time the inotify event is trigged, the json log
entry might haven't been fully written to the disk.
In this case the decoder will return an "io.UnexpectedEOF" error, but
there is still data remaining in the decoder's buffer. And the data
should be passed to the decoder when the next inotify event is
triggered.

Signed-off-by: Shijiang Wei <mountkin@gmail.com>
Upstream-commit: e41eae8b42052d07432405cca113ffcf8cf49c06
Component: engine
2015-10-31 02:28:23 +08:00
2e804d8f73 Windows build broken on master (awslogs)
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 2716e3964d7e30e924e7f5da3d7923c9f286cff2
Component: engine
2015-10-30 10:02:32 -07:00
069f98c935 [awslogs] Auto-detect region on EC2 instances
Signed-off-by: Samuel Karp <skarp@amazon.com>
Upstream-commit: 8a6dfb26f30331231839c420f8cf8889c9ef35a8
Component: engine
2015-10-27 23:47:28 +00:00
32f8320544 [awslogs] Set User-Agent for Amazon CloudWatch Logs
Signed-off-by: Samuel Karp <skarp@amazon.com>
Upstream-commit: 480c9c01784d65a46f6ba62e8176e9b2a6a3feab
Component: engine
2015-10-27 23:47:19 +00:00
2e7de41d92 Update vendored aws-sdk-go
Signed-off-by: Samuel Karp <skarp@amazon.com>
Upstream-commit: 70083e6fc3a7f64755fcf4752e6d24758bdd0543
Component: engine
2015-10-27 23:43:51 +00:00
7a1aba432f Add Splunk logging driver #16207
Allow to send Splunk logs using Http Event Collector

Signed-off-by: Denis Gladkikh <denis@gladkikh.email>
Upstream-commit: 1f1dbf312ddc299bdad408ce3d6b71f4f5993652
Component: engine
2015-10-23 13:37:53 -07:00
1ba0019242 add labels/env log option for jsonfile
this allows jsonfile logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`.

Extra attributes are saved into `attrs` attributes for each log data.

Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
Upstream-commit: 0083f6e984894b4d3697c1ae63547c07eea697af
Component: engine
2015-10-12 21:12:46 +02:00
21e1fc0995 add labels/env log option for journald
this allows journald logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`

Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
Upstream-commit: 11a24f19c2da88b6c3b50114863f24c06c5ce2fd
Component: engine
2015-10-12 21:12:46 +02:00
8aebdf0c20 add labels/env log option for fluentd
this allows fluentd logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`

Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
Upstream-commit: 4cc8490283fdc8315f595be3e32951929e65bb40
Component: engine
2015-10-12 21:12:46 +02:00
d5bd4c9ab4 add labels/env log option for gelf
this allows gelf logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`

Additional log field will be prefixed with `_` as per gelf protocol
https://www.graylog.org/resources/gelf/

Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
Upstream-commit: 5794a0190d7505e998116f1321ae39b001d0e710
Component: engine
2015-10-12 21:12:46 +02:00
bc80b0014f add env and labels to log context
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
Upstream-commit: 656cdbb0e96a1f8531b118caedd8e9b3d281c201
Component: engine
2015-10-12 21:12:46 +02:00
72ffa04366 daemon: logger: error out on daemon start if invalid logger address
If an invalid logger address is provided on daemon start it will
silently fail. As syslog driver is doing, this check should be done on
daemon start and prevent it from starting even in other drivers.
This patch also adds integration tests for this behavior.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
Upstream-commit: e3c472426ff1dccf083e41dbbfdf5935921bf330
Component: engine
2015-09-20 16:20:25 +02:00