17834 Commits

Author SHA1 Message Date
John-Michael Faircloth
c121728a69
plugin automation: use gh cli and add labels (#21657)
* plugin automation: use gh cli and add labels

* remove unneeded fields

* reorder steps

* remove comment

* fix typo

* add some validation
2023-07-12 16:28:56 -04:00
Ryan Cragun
a98c0d9cbe
actions: always cache all required Go modules (#21792)
* Make sure that we always download all of the required modules.
* Fix actions/set-up-go path for UI test
* Fix broken go.mod in hcp_link

Signed-off-by: Ryan Cragun <me@ryan.ec>
2023-07-12 20:21:09 +00:00
Kuba Wieczorek
216f497c33
Don't fail tests if GitHub API doesn't return the expected response in CI (#21791) 2023-07-12 19:34:48 +01:00
Ryan Cragun
c43345c452
[QT-589] Use the go module cache between CI and build (#21764)
In order to reliably store Go test times in the Github Actions cache we
need to reduce our cache thrashing by not using more than 10gb over all
of our caches. This change reduces our cache usage significantly by
sharing Go module cache between our Go CI workflows and our build
workflows. We lose our per-builder cache which will result in a bit of
performance hit, but we'll enable better automatic rebalancing of our CI
workflows. Overall we should see a per branch reduction in cache sizes
from ~17gb to ~850mb.

Some preliminary investigation into this new strategy:

Prior build workflow strategy on a cache miss:
  Download modules: ~20s
  Build Vault: ~40s
  Upload cache: ~30s
  Total: ~1m30s

Prior build workflow strategy on a cache hit:
  Download and decompress modules and build cache: ~12s
  Build Vault: ~15s
  Total: ~28s

New build workflow strategy on a cache miss:
  Download modules: ~20
  Build Vault: ~40s
  Upload cache: ~6s
  Total: ~1m6s

New build workflow strategy on a cache hit:
  Download and decompress modules: ~3s
  Build Vault: ~40s
  Total: ~43s

Expected time if we used no Go caching:
  Download modules: ~20
  Build Vault: ~40s
  Total: ~1m

Signed-off-by: Ryan Cragun <me@ryan.ec>
2023-07-12 17:55:16 +00:00
Kuba Wieczorek
5c5c2abc63
VAULT-12958 Add link to logs to the test failure summary in CI (#21736) 2023-07-12 18:29:31 +01:00
miagilepner
bfa93fdeda
VAULT-14696: Pin 3rd party actions (#21782)
* use shas instead of versions and fix milestones

* remove trailing space
2023-07-12 17:58:45 +02:00
Mike Palmiotto
702c521489
Auto-balancing test runners (#21745)
Integrate the `test-go` workflow with `gotestsum tool ci-matrix`. The
tool uses the output of `go list ./...` along with timing files emitted
by `gotestsum` to generate a test matrix of 16 runners with evenly
distributed runtimes.

We intentionally ignore binary, docker-based test files for the initial
matrix creation and then inject a 17th runner, dedicated to building
Vault and running the entire binary test suite together. This avoids
duplication of build overhead when binary tests are rebalanced across
multiple runners in the generated matrix.

In order to maintain test results from previous runs, we cache the test
results after every run of `gotestsum`. Each cache entry occupies ~36MB
after compression on enterprise, at the time of this commit.

We'll have to keep an eye on this to make sure timing data is not
evicted from the cache, but in theory it should be toward the top of the
LRU entries.
2023-07-12 11:19:50 -04:00
Hamid Ghaf
c5fbc19d36
run go tests if verify-changes is skipped (#21776)
* run go tests if verify-changes is skipped

* experiment for a draft pr

* revert to old behaviour

* add always
2023-07-12 07:17:26 -07:00
Peter Wilson
b40a39c4cd
VAULT-17080: audit formatter node (JSON) (#21769)
* Export AuditFormatter, improve tests

* Correct issues in 'Date' for tests
2023-07-12 11:19:24 +00:00
Peter Wilson
71a6e1e3fa
VAULT-17081: audit formatter node (JSONx) (#21762)
* JSONX audit format events
2023-07-12 11:53:47 +01:00
Austin Gebauer
bf19846b18
Adds replication state helper to framework.Backend (#21743)
* Adds replication state helper to framework.Backend

* Fix test

* adds changelog
2023-07-11 15:22:28 -07:00
Steven Clark
11691b3b37
Allow PKI enterprise to influence TestProperAuthing test case (#21761)
- Allow enterprise repository to hook into and influence the proper
   authing test so we don't have to add values to OSS for enterprise
   work
2023-07-11 13:11:28 -04:00
Hamid Ghaf
3443ff1cc1
use verify changes for docs to skip tests (#21620)
* use verify changes for docs to skip tests

* add verify-changes to the needed jobs

* skip go tests for doc/ui only changes

* fix a job ref

* change names, remove script

* remove ui conditions

* separate flags

* feedback
2023-07-11 09:06:48 -07:00
Chelsea Shaw
5a4e95db62
UI: Fix SSH role edit clearing out object values (#21739)
* Remove logic that skips sending object if not changed

* Add changelog
2023-07-11 10:28:02 -05:00
Peter Wilson
637679a327
VAULT-17080: refactor prior to introducing JSON formatting nodes for audit (#21708)
* Split out interfaces for salter/formatter/writer
2023-07-11 15:30:24 +01:00
Mike Palmiotto
a9778be3f2
Only run fmtcheck on changed files pre-commit (#21747)
The pre-commit hook was taking quite a while to run. Let's just check
changed files to avoid unnecessary `go fmt` on the entire tree.
2023-07-11 10:26:16 -04:00
Steven Clark
1bed33a399
Add the ability to add enterprise only backend features in PKI - OSS (#21748)
To allow us to support CIEPS backend state, allow the backend to
contain enterprise only state variables. Also allow us to implement
enterprise only hooks into the various backend functions to initialize,
periodicFunc, cleanup and invalidate.
2023-07-11 10:05:23 -04:00
Chelsea Shaw
863945353d
UI: Fix enterprise test (#21737) 2023-07-11 08:57:06 -05:00
miagilepner
a29ba45a3a
VAULT-15958: Fix pr/no-milestone (#21713)
* fix milestone

* only pr milestone check and fix actionlint

* fix actionlint

* add comments saying that the event types work
2023-07-11 13:35:10 +00:00
Sarah Chavis
ee4424fa3a
Add note about user lockout defaults (#21744) 2023-07-11 09:16:15 -04:00
Max Winslow
d1c943b255
add sys/plugins to root-only path (#21599)
Co-authored-by: Yoko Hyakuna <yoko@hashicorp.com>
2023-07-10 21:50:54 +00:00
Kuba Wieczorek
ece2995ee1
Ensure test failure summaries job doesn't get skipped (#21709) 2023-07-10 17:47:48 +00:00
Alexander Scheel
31a71f6db4
Remove fmtcheck from prep (#21724)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-07-10 17:40:19 +00:00
divyaac
9ace8751ff
Ent only ADP Metrics (#21681)
* Ent only ADP Metrics

* Added change log

* Changed changelog name

* Restored previous impl

* Moved to mount_util

* Change impl

* Add same file

* Moved to registry_util

* Edited corehelpers mock registry

* Edited chagnelog

* Edited changelog

* Edited build tag

* Added back function

* Delete core.go.rej

* Edited mount

* Changed spacing
2023-07-10 10:28:42 -07:00
Alexander Scheel
b0fb3b1420
pre-commit hook for formatting backend code (#21682)
* Add backend format linting to pre-commit hook

By taking a slight penalty with each commit, we can ensure that
contributors follow the format behavior by default (if they run hooks),
making accidental PRs without proper formatting less likely.
Additionally, fix gofmtcheck to align with the Makefile, fixing the
corresponding fmtcheck target for use with the hook.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix formatting errors

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-07-10 16:49:47 +00:00
Max Bowsher
3bf1299814
Fix aspects of auth/token/create request parsing (#18556)
* Fix aspects of `auth/token/create` request parsing

Fixes #18550

Currently, the `auth/token/create` family of APIs (`create`,
`create-orphan`, `create/{role}`) does non-standard parsing of requests,
by directly using `mapstructure.WeakDecode(request.Data, ...)` instead
of using the standard `framework.FieldData` abstraction.

Furthermore, the fields declared for these APIs are incorrect, leading
to inappropriate OpenAPI generation, and inappropriate warnings about
ignored parameters.

Detailed changes:

* Factor out triplicated definitions of common fields across these three
  APIs.

* Remove incorrect `role_name` field from `create-orphan`.

* Add missing `lease` deprecated field.

* Rename incorrectly named `metadata` field to `meta`, and change from
  `TypeMap` to `TypeKVPairs` to reflect actual underlying Go type is
  `map[string]string`.

* Remove entirely incorrect `format` field.

* Add declarative `Default: true` to `renewable` field, to match
  behaviour currently implemented in code.

* Having fixed the field definitions to match current usage, remove the
  secondary decoding of the request via `mapstructure` inside
  `handleCreateCommon`, and migrate to using `FieldData` APIs like
  a normal operation function.

* Add changelog

* Rephrase comment.
2023-07-10 12:15:03 -04:00
Steven Clark
e29842e49e
Address data races in TestCertStorageMetrics due to plugin reloading (#21679)
- There's a race within the Plugin reloading mechanism that isn't
   trivial to address. To silence some of the failures, switch this
   test to use sealing of the cores instead of the plugin reload
   mechanism
2023-07-10 09:31:34 -04:00
miagilepner
5919645a70
Switch test-completed to always() (#21654) 2023-07-10 12:34:02 +02:00
Steven Clark
bce4b7a3e5
Attempt to address autoloading issues within Bind backend tests (#21694)
- We've seen a few issues with bind's auto-loading of configuration
   too quickly at bad times leading to it having partial configurations
   or not all files/permissions being restored properly during it's read
   attempt.
 - See if the freeze/thaw rndc commands will help out with these timing
   issues
2023-07-07 16:29:39 -04:00
Rebecca Willett
a053c616ba
Add some additional information to the Slack notification for Enos failures (#21624) 2023-07-07 19:40:42 +00:00
Violet Hynes
d18242dae4
Run on all branches (#21683) 2023-07-07 19:40:05 +00:00
Jordan Reimer
96fd024d15
adds fixed height to error-icon class to fix another safari style bug with stretched icons (#21678) 2023-07-07 13:03:43 -06:00
Nick Cabatoff
3d7aab7a34
Rework stubmaker logic so that if the funcs are found in Go, we don't attempt to write the file. (#21636) 2023-07-07 13:16:27 -04:00
Violet Hynes
87d37fecb7
Run only on OSS (#21669) 2023-07-07 17:01:31 +00:00
Angel Garbarino
9c8a7422ad
Bug Fix: Surface Error for when cookie settings prevent localStorage (#21503)
* initial fix

* changelog

* clean up

---------

Co-authored-by: Chelsea Shaw <82459713+hashishaw@users.noreply.github.com>
2023-07-07 10:21:00 -06:00
Jason O'Donnell
237b9f7140
Add documentation for DR reindex endpoint (#21446)
* Add dr reindex doc

* Format
2023-07-07 11:36:38 -04:00
Alexander Scheel
072f0dd7c8
Add explicit tests for Extensions, other Subject attributes (#21651)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-07-07 11:29:22 -04:00
Kuba Wieczorek
5defa54bd4
VAULT-17592 Extract failed Go test results across runners (#21625) 2023-07-07 16:18:29 +01:00
Chelsea Shaw
c7d3f8c191
UI: PKI show missing info on generated certificate (#21635) 2023-07-07 14:51:05 +00:00
Marc Boudreau
bf9ec97c59
Introduce Experiment to Toggle Between Legacy Auditing Backends/EventLogger (#21628)
* introduce experiment to toggle between legacy auditing backends and eventlogger

* provide changelog file
2023-07-07 10:01:40 -04:00
Steven Clark
e83b9e2bde
Fix data race between updating a route entry's tainted status and incoming requests - OSS (#21640)
* Fix data race within route entry tainting with requests - OSS

 - There is a data race between a mount being mounted with incoming
   requests and checking the mount's tainted status.

* Add cl
2023-07-07 09:14:33 -04:00
Violet Hynes
34d1d200ee
VAULT-17736 Add HashiCorp contributed label to HC contributed PRs (#21629)
* VAULT-17736 Add HashiCorp contributed label to HC contributed PRs

* VAULT-17736 finish my thought

* VAULT-17736 gh pr edit

* VAULT-17736 update formatting

* VAULT-17736 quote for linting happiness

* VAULT-17736 try without apostrophes?

* VAULT-17736 try with quotes?

* VAULT-17736 try one line run

* VAULT-17736 Oh, that was it

* VAULT-17736 No more ready_for_review
2023-07-07 13:03:19 +00:00
Kuba Wieczorek
95b44add74
Group CI artifacts per test type to reduce noise on GHA run summary page (#21648) 2023-07-07 13:46:53 +01:00
Sarah Chavis
a2de4c75cd
[Docs] Client count doc updates for ACME update in 1.14 (#21541)
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Brian Shumate <brianshumate@users.noreply.github.com>
2023-07-07 08:32:31 -04:00
Steven Clark
1a2eaf0de3
Add remediation steps to the PKI health-check docs (#21364)
* Add remediation steps to the PKI health-check docs

* Apply suggestions from code review

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Implement PR feedback

* Apply suggestions from code review

Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>

---------

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
2023-07-06 19:38:51 -04:00
John-Michael Faircloth
506db7b9bf
workflows: update plugin check job to use branch as ref (#21638)
* workflows: update plugin check job to use branch as ref

* make actionlint happy
2023-07-06 21:18:52 +00:00
Steven Clark
9b9017a926
Add the ability for enterprise to override PKI backend attributes during setup (#21618) 2023-07-06 16:16:17 -04:00
Austin Gebauer
304ecfc8e9
secrets/azure: upgrade to v0.16.1 for bug fix (#21631)
* secrets/azure: upgrade to v0.16.1 for bug fix

* adds changelog
2023-07-06 12:57:29 -07:00
Nick Cabatoff
502c2830c0
Add code that would break ent if merged prior to stubmaker (#21634) 2023-07-06 19:23:21 +00:00
Chelsea Shaw
15b5dd0a4e
UI: octanify lazy paginated query (#21602) 2023-07-06 13:32:01 -05:00