Commit Graph

253 Commits

Author SHA1 Message Date
822098983e Merge pull request #34520 from fnoeding/fixed-raw-splunk-logger
Fixed `raw` mode splunk logger
Upstream-commit: 5c57ca17d50c392b7ad627fc1215d7a7e91e0c41
Component: engine
2017-09-19 10:53:01 +02:00
18c29c5c5f Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: f7f101d57ef8cbf2d8723a18b7d723c5c5dd04b6
Component: engine
2017-09-12 12:09:59 -04:00
fb9a7edb75 Merge pull request #31727 from sascha-andres/31726-le-lion-only
Log payload only [logentries]
Upstream-commit: b3b630655dfe8bcaa35af9ece296189660ed22ae
Component: engine
2017-08-22 14:59:28 +02:00
de626e10fb Fix golint errors.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 9b47b7b1519c5f2138e2933fb1fc459eb00895c0
Component: engine
2017-08-18 14:23:44 -04:00
ce23144495 Fixed raw mode splunk logger
Splunk HEC does not accept log events with an empty string or a
whitespace-only string.

Signed-off-by: Florian Noeding <florian@noeding.com>
Upstream-commit: 5f6d6a5093a4db799f9c1a6bb82eed1eea13ec0c
Component: engine
2017-08-18 12:03:43 +02:00
30f1b651e2 Remove string checking in API error handling
Use strongly typed errors to set HTTP status codes.
Error interfaces are defined in the api/errors package and errors
returned from controllers are checked against these interfaces.

Errors can be wraeped in a pkg/errors.Causer, as long as somewhere in the
line of causes one of the interfaces is implemented. The special error
interfaces take precedence over Causer, meaning if both Causer and one
of the new error interfaces are implemented, the Causer is not
traversed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: ebcb7d6b406fe50ea9a237c73004d75884184c33
Component: engine
2017-08-15 16:01:11 -04:00
6ed8995fb3 Move pkg/templates away
- Remove unused function and variables from the package
- Remove usage of it from `profiles/apparmor` where it wasn't required
- Move the package to `daemon/logger/templates` where it's only used

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 9ef3b535974612b137abae062b7a8a0f7e969871
Component: engine
2017-08-08 18:16:41 +02:00
9282435e75 Log payload only [logentries]
If you want to use the advanced query in logentries
logging in json is very useful. If you do not need
the advanced information with
--log-opt line-only=true just logs the payload

Signed-off-by: Sascha Andres <sascha.andres@outlook.com>
Upstream-commit: 6362843e67f21274c3ee5a719d3a6393c5612699
Component: engine
2017-08-05 09:54:21 +02:00
d659edcaf5 Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 1009e6a40b295187e038b67e184e9c0384d95538
Component: engine
2017-07-31 13:16:46 -07:00
cf98674bff Fix awslogs driver repeating last event - #34292
Signed-off-by: Justin Menga <justin.menga@gmail.com>
Upstream-commit: 0fd5a0bab79f20f910cb7551ec34158a32e05f5a
Component: engine
2017-07-28 20:33:14 +12:00
b337882c5b Avoid using a map for log attributes
Having a map per log entry seemed heavier than necessary. These
attributes end up being sorted and serialized, so storing them in a map
doesn't add anything (there's no random access element). In SwarmKit,
they originate as a slice, so there's an unnecessary conversion to a map
and back.

This also fixes the sort comparator, which used to inefficiently split
the string on each comparison.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: b642b3f21f17cf50c79e464d3aedc93b2dbf0fb0
Component: engine
2017-07-18 19:01:20 -07:00
tim
812c72c3da Replaces fluentd-address string by constant
Signed-off-by: tim <tim@fewagainstmany.com>
Upstream-commit: cb972b1515428fd7a8ef573fa65af08a6fef669c
Component: engine
2017-07-16 23:06:57 -07:00
e95f4619cd [project] change syscall to /x/sys/unix|windows
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>

[s390x] switch utsname from unsigned to signed

per 33267e036f
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
Upstream-commit: 069fdc8a083cb1663e4f86fe3fd9b9a1aebc3e54
Component: engine
2017-07-11 08:00:32 -04:00
abfe031053 Merge pull request #33713 from x1022as/error_fd
fd leak and error handling
Upstream-commit: 3be2273d03e4b06dff4006b1a600c578c7aa09fc
Component: engine
2017-07-06 16:47:34 -07:00
74d8ac0d96 Merge pull request #33897 from cpuguy83/jsonfile_log_blocking
Fix log readers can block writes indefinitely
Upstream-commit: db8c2651b84c681579389bb57d309a3b210a7e78
Component: engine
2017-07-05 18:21:14 -07:00
2aedd8b160 Fix log readers can block writes indefinitely
Before this patch, a log reader is able to block all log writes
indefinitely (and other operations) by simply opening the log stream and
not consuming all the messages.

The reason for this is we protect the read stream from corruption by
ensuring there are no new writes while the log stream is consumed (and
caught up with the live entries).

We can get around this issue because log files are append only, so we
can limit reads to only the section of the file that was written to when
the log stream was first requested.

Now logs are only blocked until all files are opened, rather than
streamed to the client.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e2209185ed1c959131d4068ec7fc93e194dc0802
Component: engine
2017-07-05 14:04:52 -04:00
ec1ee882b3 Merge pull request #33322 from jsoref/spelling
Spelling
Upstream-commit: ff4f700f74450018f36d014f3cde0ff1b9c17fb3
Component: engine
2017-07-04 15:46:34 +02:00
63c7ce310a Merge pull request #33213 from gkze/master
GCP Logs: send log text as jsonPayload.message instead of jsonPayload.data
Upstream-commit: c2ad549533cf2ffdb8b93ce8eb0bad5c695b828c
Component: engine
2017-07-03 16:18:18 -04:00
643654c2f0 Spelling fixes
* additional
* ambiguous
* anonymous
* anything
* application
* because
* before
* building
* capabilities
* circumstances
* commit
* committer
* compresses
* concatenated
* config
* container
* container's
* current
* definition
* delimiter
* disassociates
* discovery
* distributed
* doesnotexist
* downloads
* duplicates
* either
* enhancing
* enumerate
* escapable
* exactly
* expect
* expectations
* expected
* explicitly
* false
* filesystem
* following
* forbidden
* git with
* healthcheck
* ignore
* independent
* inheritance
* investigating
* irrelevant
* it
* logging
* looking
* membership
* mimic
* minimum
* modify
* mountpoint
* multiline
* notifier
* outputting
* outside
* overridden
* override
* parsable
* plugins
* precedence
* propagation
* provided
* provides
* registries
* repositories
* returning
* settings
* should
* signals
* someone
* something
* specifically
* successfully
* synchronize
* they've
* thinking
* uninitialized
* unintentionally
* unmarshaling
* unnamed
* unreferenced
* verify

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 39bcaee47b8a284a46b761afe218ba7deda0d482
Component: engine
2017-07-03 13:13:09 -07:00
3a59ac4d9f Fix wrong filemode for rotate log files
the filemode should be 0640 but not 06400

Signed-off-by: Lei Jitang <leijitang@huawei.com>
Upstream-commit: 96ea8eaa1592856e0829c3465e64f93911d3e389
Component: engine
2017-07-03 03:49:22 -04:00
29123f2ea7 Fix stderr logging for journald and syslog
logger.PutMessage, added in #28762 (v17.04.0-ce), clears msg.Source. So journald
and syslog were treating stderr messages as if they were stdout.

Signed-off-by: David Glasser <glasser@davidglasser.net>
Upstream-commit: 917050c5728f2fb9958ccb3ab66a23766f741adc
Component: engine
2017-06-26 14:37:43 -07:00
c41db1dd55 fd leak and error handling
Signed-off-by: Deng Guangxing <dengguangxing@huawei.com>
Upstream-commit: 0042f992d80b59b97380bbbbeb271fd3030edd90
Component: engine
2017-06-23 15:38:35 +08:00
0e43fa4b98 * standardize timeouts for log reads and writes for logger adapter tests
* use an assertion framework in logger adapter tests

Signed-off-by: Royce Remer <royceremer@gmail.com>
Upstream-commit: b3add005d5135f64670f6502357cde5bd8d03f75
Component: engine
2017-06-18 18:47:45 -07:00
a53424fe86 Logging driver should receive same file in start/stop request
Signed-off-by: Peter Bücker <peter.buecker@gmail.com>
Upstream-commit: e908e1a357b435d7fab497d51cdd3e58458a0590
Component: engine
2017-06-08 10:05:52 +02:00
a1a5b0a48d move multireader out of /pkg
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
Upstream-commit: 2445e6b99d4beecb25d556d9a099bdf47703e174
Component: engine
2017-06-01 16:22:00 -07:00
e9185f10cf Don't log error if file is already closed
When closing the log-file, and the file is already
closed, there's no need to log an error.

This patch adds a `closed` boolean to check if the
file was closed, and if so, skip closing the file.
This prevents errors like this being logged:

    level=error msg="Error closing logger: invalid argument"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 07b51ed300429e88871e40b4d67dc031e2e8901c
Component: engine
2017-05-29 17:42:37 +02:00
17712f857a Improved poor memory efficiency of awslogs
Signed-off-by: YAMASAKI Masahide <masahide.y@gmail.com>
Upstream-commit: 524f30634018ce619da61aa0a13dad245e098226
Component: engine
2017-05-18 23:30:00 +09:00
bbf5e9962e GCP Logs: send log line as jsonPayload.message instead of jsonPayload.data
Signed-off-by: gkze <george@bugsnag.com>
Upstream-commit: d42c268fc3983d19e9dbd800948e7c4445c425f0
Component: engine
2017-05-15 16:35:53 -07:00
2d5cb49e4e Update comments describing new behaviour
Signed-off-by: Justin Menga <justin.menga@gmail.com>
Upstream-commit: dc87490b632949394307e25dce17c8db6cf664c6
Component: engine
2017-05-15 11:08:16 +12:00
2373c13c5a Code review changes
Signed-off-by: Justin Menga <justin.menga@gmail.com>
Upstream-commit: 3a4cf2b076a8f04f54b6edff4f661f5c3b2c8157
Component: engine
2017-05-15 10:28:18 +12:00
6083b78b58 Add missing month in datetime map
Signed-off-by: Justin Menga <justin.menga@gmail.com>
Upstream-commit: bf1c377f6012f3db714e0f848d70e776ef1226a6
Component: engine
2017-05-13 22:10:51 +12:00
3ac82c3d80 Append new line to multiline events
Signed-off-by: Justin Menga <justin.menga@gmail.com>
Upstream-commit: 4374f16667708118c126cd71f733493b0070e807
Component: engine
2017-05-13 17:54:04 +12:00
7dd003d1da Code review changes
Signed-off-by: Justin Menga <justin.menga@gmail.com>
Upstream-commit: 8ef9c19ecdd16881289a63e5e89de3f5c83def66
Component: engine
2017-05-13 17:54:04 +12:00
38049ee803 awslogs: use github.com/pkg/errors to wrap an error
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: f775005a17c7ae0d3851cc86f46b7c46aafa27d8
Component: engine
2017-05-13 17:54:04 +12:00
17b96f5ff1 Formatting
Signed-off-by: Justin Menga <justin.menga@gmail.com>
Upstream-commit: 6f073e352247217a4cc5ee825676a9a496655455
Component: engine
2017-05-13 17:54:04 +12:00
4221bdc95f Add awslogs benchmarks
Signed-off-by: Justin Menga <justin.menga@gmail.com>
Upstream-commit: 84b03660dab03ac366d99ba817a5ec8523808386
Component: engine
2017-05-13 17:54:04 +12:00
db722ea7f6 Add awslogs multiline support
Signed-off-by: Justin Menga <justin.menga@gmail.com>
Upstream-commit: ab74038df93156ef27c03d6e4490bca5c3359423
Component: engine
2017-05-13 17:54:04 +12:00
ddcae36da7 gcplogging driver MRPB set
Signed-off-by: Grzegorz Jaśkiewicz <gj.jaskiewicz@gmail.com>
Upstream-commit: d925e5047067c2014a3769d41bebd5c837e4cae3
Component: engine
2017-05-09 14:24:43 +02:00
0411944839 Merge pull request #31856 from jim-minter/more_races
Resolve a set of race conditions in logging and attaching code
Upstream-commit: 5eca7f7c5d7678f6548cd7d90199d67ce98d4c98
Component: engine
2017-04-26 16:04:02 -07:00
740b35daa9 Prevent ContainerLogs from hanging if container doesn't run for long
Signed-off-by: Jim Minter <jminter@redhat.com>
Upstream-commit: 4fdb17c77765fb44ae263d5012bb79bb8bd15456
Component: engine
2017-04-20 12:27:51 +01:00
62eff90f69 Add logdrivers to /info
This is required for swarmkit to be able to filter based on log driver.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 17abacb8946ed89496fcbf07a0288fafe24cb7b0
Component: engine
2017-04-11 18:07:15 -04:00
276ea6a456 Implement plugins for logging drivers
Logging plugins use the same HTTP interface as other plugins for basic
command operations meanwhile actual logging operations are handled (on
Unix) via a fifo.

The plugin interface looks like so:

```go
type loggingPlugin interface {
  StartLogging(fifoPath string, loggingContext Context) error
  StopLogging(fifoPath)
```

This means a plugin must implement `LoggingDriver.StartLogging` and
`LoggingDriver.StopLogging` endpoints and be able to consume the passed
in fifo.

Logs are sent via stream encoder to the fifo encoded with protobuf.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 27bd6842f8518780b99fdb42f8e7f84c17856d87
Component: engine
2017-04-10 13:17:20 -04:00
334f054f13 refactor logs and support service logs /w tty
Refactor container logs system to make communicating log messages
internally much simpler. Move responsibility for marshalling log
messages into the REST server. Support TTY logs. Pave the way for fixing
the ambiguous bytestream format. Pave the way for fixing details.

Signed-off-by: Drew Erny <drew.erny@docker.com>
Upstream-commit: 1044093bb0aa12eb8972361a93b9bc8c4ddd857b
Component: engine
2017-04-06 17:54:11 -07:00
3f185ca5af Merge pull request #27565 from rothrock/env-by-regex
[splunk] Use a regex to match environment variables
Upstream-commit: 1325f667eeb42b717c2f9d369f2ee6d701a280e3
Component: engine
2017-03-31 10:46:48 -04:00
47709fd338 Merge pull request #31083 from xulike666/fight-for-readability
fix some typos from module contrib to man
Upstream-commit: 57fd478169071b09f0e6657cc1c2568d80c1bed3
Component: engine
2017-03-03 15:13:01 +01:00
458406a7f3 Merge pull request #31212 from fate-grand-order/fixed
use t.Fatal() to output the err message and fix typo
Upstream-commit: a6576b8a2bff0feeb67918403caa3df0c4c37673
Component: engine
2017-02-27 12:26:35 +09:00
9632b0f1e8 use t.Fatal() to output the err message where the values used for formatting
text does not appear to contain a placeholder

Signed-off-by: Helen Xie <chenjg@harmonycloud.cn>
Upstream-commit: 2a8d6368d4a930203b93f75914173ab65bf3b0bc
Component: engine
2017-02-23 10:16:22 +08:00
4fcd45b6de Merge pull request #31263 from nalind/journal-doublefree
Synchronize the cursor returned by followJournal
Upstream-commit: b3e7f7013719be7d11d513335ff55802eaf543ce
Component: engine
2017-02-22 17:45:49 -08:00
53075c0a1f Synchronize the cursor returned by followJournal
Make sure that the cursor value returned by followJournal() is the last
of the values returned by its goroutine's calls to drainJournal() by
waiting for it, rather than returning a value that may be superceded by
another if we're singalling the goroutine that it should exit by closing
a pipe.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Upstream-commit: d57c330617efb97cad736a3e4ede82bb46ebbbf2
Component: engine
2017-02-22 16:22:00 -05:00
73607300a9 Prevent freeing a possible invalid pointer from journald
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
Upstream-commit: 81630df85439adab1836e83a7e645d67b1c0eb83
Component: engine
2017-02-22 07:54:10 -08:00