Commit Graph

29 Commits

Author SHA1 Message Date
Peter Wilson
ae774b93d3
VAULT-18160: Audit - options and defaults (#22295)
* options for syslog

* fix bug in default value for HMACAccessor

* backend device cleanup

* socket backend options update

* options: prefix removed check, added default file mode

* fix option setting for elision

* fix test for prefix and whitespace
2023-09-11 08:41:29 +00:00
hashicorp-copywrite[bot]
0b12cdcfd1
[COMPLIANCE] License changes (#22290)
* Adding explicit MPL license for sub-package.

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Adding explicit MPL license for sub-package.

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Updating the license from MPL to Business Source License.

Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.

* add missing license headers

* Update copyright file headers to BUS-1.1

* Fix test that expected exact offset on hcl file

---------

Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Sarah Thompson <sthompson@hashicorp.com>
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
2023-08-10 18:14:03 -07:00
Marc Boudreau
b07eff0998
VAULT-18934: Record individual metrics for each Auditing Event Pipeline (#22266)
* add sink wrapper to take telemetry measures

* make use of sinkwrapper
2023-08-10 09:49:55 -04:00
Peter Wilson
32b6a5c840
VAULT-18157: Audit events: Log Test Message (#22263)
* manually crank the nodes to log a test message (if we have them configured)
* try to extract the duplicated manual processing of nodes
2023-08-09 21:04:31 +01:00
Peter Wilson
30114d0038
Moved HeaderFormatter to an Option for NewEntryFormatter, updated tests (#22116) 2023-07-28 17:07:20 +01:00
Marc Boudreau
769fe0b67d
VAULT-17079: Adding Hash Function and HeaderAdjuster to EntryFormatter (#22042)
* add hashfunc field to EntryFormatter struct and adjust NewEntryFormatter function and tests

* add HeaderAdjuster interface and require it in EntryFormatter
dquote> adjust all references to NewEntryFormatter to include a HeaderAdjuster parameter

* replace use of hash function in AuditedHeadersConfig's ApplyConfig method with Salter interface instance

* fixup! replace use of hash function in AuditedHeadersConfig's ApplyConfig method with Salter interface instance

* review feedback

* Go doc typo

* add another test function

---------

Co-authored-by: Peter Wilson <peter.wilson@hashicorp.com>
2023-07-27 12:31:52 -04:00
Peter Wilson
050759f661
VAULT-17078: Implement Register and Deregister Audit Devices for EventLogger Framework (#21898)
* begin refactoring of event package into audit package

* audit options additions

* rename option structs

* Trying to remove 'audit' from the start of names.

* typo

* typo

* typo

* newEvent required params

* typo

* comments on noop sink

* more refactoring - merge json/jsonx formatters

* fix file backend and tests

* Moved unexported funcs to formatter, fixed file tests

* typos, comments, moved func

* fix corehelpers

* fix backends (syslog, socket)

* Moved some sinks back to generic event package.

* return of the file sink

* remove unneeded sink params/return vars

* Implement Register and Deregister Audit Devices for EventLogger Framework (#21940)

* add function to create StdoutSinkNode

* add boolean argument to audit Factory function

* create eventlogger nodes in backend factory functions

* simplify NewNoopSink function and remove DiscardSinkNode

* make the sanity test in the file backend mutually exclusive based on useEventLogger value

* remove test cases that no longer made sense and were failing

* NewFileSink attempts to open file for sanity check

* fix FileSink tests and update FileSink to remove discard, stdout but add /dev/null

* Moved WithPrefix from FileSink to EventFormatter

* move prefix in backend

* NewFormatterConfig and Options (tests fixed)

* Little tidy up

* add test where audit file is created with useEventLogger set to true

* only create eventlogger.Node instances when useEventLogger is true
fix failing test due to invalid string conversion of FileMode value

* moved variable definition to more appropriate scope

---------

Co-authored-by: Marc Boudreau <marc.boudreau@hashicorp.com>
2023-07-24 09:27:09 -04:00
Peter Wilson
31074bc448
VAULT-18284: Audit refactor packages (#21972)
* initial git mv to rename 'audit' packages

* remove 'Audit' prefix from structs inside audit package

* refactor of event/audit pacakges

* EventFormatter => EntryFormatter

* 'AuditFormat' => EntryFormat

* Use NewFormatterConfig func

---------

Co-authored-by: Marc Boudreau <marc.boudreau@hashicorp.com>
2023-07-20 13:32:06 -04:00
Marc Boudreau
7103bc2cdb
VAULT-17078: Add useEventLogger Argument to Audit Factory Functions (#21962)
* add useEventLogger argument to audit Factory functions

* adjusting Factory functions defined in tests

* fixup! adjusting Factory functions defined in tests
2023-07-20 11:23:21 -04:00
Peter Wilson
637679a327
VAULT-17080: refactor prior to introducing JSON formatting nodes for audit (#21708)
* Split out interfaces for salter/formatter/writer
2023-07-11 15:30:24 +01:00
Hamid Ghaf
e55c18ed12
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
Max Bowsher
aeb1b1e72f
Add option 'elide_list_responses' to audit backends (#18128)
This PR relates to a feature request logged through HashiCorp commercial
support.

Vault lacks pagination in its APIs. As a result, certain list operations
can return **very** large responses.  The user's chosen audit sinks may
experience difficulty consuming audit records that swell to tens of
megabytes of JSON.

In our case, one of the systems consuming audit log data could not cope,
and failed.

The responses of list operations are typically not very interesting, as
they are mostly lists of keys, or, even when they include a "key_info"
field, are not returning confidential information. They become even less
interesting once HMAC-ed by the audit system.

Some example Vault "list" operations that are prone to becoming very
large in an active Vault installation are:

    auth/token/accessors/
    identity/entity/id/
    identity/entity-alias/id/
    pki/certs/

In response, I've coded a new option that can be applied to audit
backends, `elide_list_responses`. When enabled, response data is elided
from audit logs, only when the operation type is "list".

For added safety, the elision only applies to the "keys" and "key_info"
fields within the response data - these are conventionally the only
fields present in a list response - see logical.ListResponse, and
logical.ListResponseWithInfo. However, other fields are technically
possible if a plugin author writes unusual code, and these will be
preserved in the audit log even with this option enabled.

The elision replaces the values of the "keys" and "key_info" fields with
an integer count of the number of entries. This allows even the elided
audit logs to still be useful for answering questions like "Was any data
returned?" or "How many records were listed?".
2023-01-11 16:15:52 -05:00
Brian Kassouf
230ed81150
Fix some linting errors (#12860) 2021-10-18 17:29:47 -07:00
Jeff Mitchell
861454e0ed
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090)
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
Mark Gritter
97e62cd3d6
Send a test message before committing a new audit device. (#10520)
* Send a test message before committing a new audit device.
Also, lower timeout on connection attempts in socket device.
* added changelog
* go mod vendor (picked up some unrelated changes.)
* Skip audit device check in integration test.
Co-authored-by: swayne275 <swayne@hashicorp.com>
2020-12-16 16:00:32 -06:00
ncabatoff
6c836bcd9b
Allow plugins to submit audit requests/responses via extended SystemView (#6777)
Move audit.LogInput to sdk/logical.  Allow the Data values in audited
logical.Request and Response to implement OptMarshaler, in which case
we delegate hashing/serializing responsibility to them.  Add new
ClientCertificateSerialNumber audit request field.

SystemView can now be cast to ExtendedSystemView to expose the Auditor
interface, which allows submitting requests and responses to the audit
broker.
2019-05-22 18:52:53 -04:00
Jeff Mitchell
170521481d
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Vishal Nayak
e2bb2ec3b9
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Brian Kassouf
c0815bd2b0
Add context to the NewSalt function (#4102) 2018-03-08 11:21:11 -08:00
Calvin Leung Huang
01eecf9d1a
Non-HMAC audit values (#4033)
* Add non-hmac request keys

* Update comment

* Initial audit request keys implementation

* Add audit_non_hmac_response_keys

* Move where req.NonHMACKeys gets set

* Minor refactor

* Add params to auth tune endpoints

* Sync cache on loadCredentials

* Explicitly unset req.NonHMACKeys

* Do not error if entry is nil

* Add tests

* docs: Add params to api sections

* Refactor audit.Backend and Formatter interfaces, update audit broker methods

* Add audit_broker.go

* Fix method call params in audit backends

* Remove fields from logical.Request and logical.Response, pass keys via LogInput

* Use data.GetOk to allow unsetting existing values

* Remove debug lines

* Add test for unsetting values

* Address review feedback

* Initialize values in FormatRequest and FormatResponse using input values

* Update docs

* Use strutil.StrListContains

* Use strutil.StrListContains
2018-03-02 12:18:39 -05:00
Brian Kassouf
8142b42d95 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Jeff Mitchell
4efff56640 Don't dial on backend startup; retry dials at log time so that transient (#2934)
network failures are worked around. Also, during a reconnect always
close the existing connection.

Fixes #2931
2017-07-06 10:18:18 -04:00
Jeff Mitchell
dd26071875 Delay salt initialization for audit backends 2017-05-23 20:36:20 -04:00
Jeff Mitchell
df575f0b3a Rename helper 'duration' to 'parseutil'. (#2449)
Add a ParseBool function that accepts various kinds of ways of
specifying booleans.

Have config use ParseBool for UI and disabling mlock/cache.
2017-03-07 11:21:22 -05:00
Tommy Murphy
57aac16cd2 audit: support a configurable prefix string to write before each message (#2359)
A static token at the beginning of a log line can help systems parse
logs better. For example, rsyslog and syslog-ng will recognize the
'@cee: ' prefix and will parse the rest of the line as a valid json message.
This is useful in environments where there is a mix of structured and
unstructured logs.
2017-02-10 16:56:28 -08:00
Brian Kassouf
541c53d354 Added a single retry after a reconnection 2017-02-06 11:38:38 -08:00
Brian Kassouf
aa32568aa9 Update the docs and move the logic for reconnecting into its own function 2017-02-04 16:55:17 -08:00
Brian Kassouf
b32cb4bedf Add write deadline and a Reload function 2017-02-02 15:44:56 -08:00
Harrison Harnisch
6da4806582 add socket audit backend 2017-02-02 14:21:48 -08:00