Commit Graph

55 Commits

Author SHA1 Message Date
Jesse
2655afd2bd
memory align structs (#635)
feat: align most structs and add lint rule to enforce struct alignment
2022-10-19 06:19:15 +01:00
Jesse-Cameron
c450a168d4 perf: add size hint to map creation
in order to avoid unnecessary hashmap resizing, add size parameters when
creating maps
2022-09-16 18:26:18 +01:00
Angus Lees
856bd58872 Add 'importbin' statement
Add `importbin` statement.  Similar to `importstr` but the result is
an array of numbers (all integers 0-255).
2022-03-03 22:49:02 +00:00
Stanisław Barzowski
5899996502 Make go linter happy again. 2021-05-19 15:10:49 +02:00
Leandro López
eeca44cd27 Allocate resulting slice size instead of dynamically growing it
We know for sure how big the resulting slice will be, so instead of
having Go perform checks and reallocations whenever the slice grows
too small by using `append` we calculate the resulting size earlier
and directly index each element into the output slice. This result in
only two allocations all the time, no matter how big the resulting
slice will be. With it also come big performance improvements:

name                                        old time/op    new time/op    delta
ToOrderedSlice/1_unique_identifiers-8          173ns ±10%     150ns ± 4%  -13.16%  (p=0.000 n=10+9)
ToOrderedSlice/10_unique_identifiers-8        1.24µs ± 6%    0.51µs ± 5%  -58.95%  (p=0.000 n=9+9)
ToOrderedSlice/100_unique_identifiers-8       20.4µs ±20%     4.0µs ± 5%  -80.25%  (p=0.000 n=10+9)
ToOrderedSlice/1000_unique_identifiers-8       253µs ± 9%      40µs ± 5%  -84.19%  (p=0.000 n=10+10)
ToOrderedSlice/10000_unique_identifiers-8     3.44ms ± 9%    0.37ms ± 5%  -89.32%  (p=0.000 n=9+9)
ToOrderedSlice/100000_unique_identifiers-8    48.6ms ±13%     3.6ms ± 4%  -92.49%  (p=0.000 n=10+10)

name                                        old alloc/op   new alloc/op   delta
ToOrderedSlice/1_unique_identifiers-8          40.0B ± 0%     40.0B ± 0%     ~     (all equal)
ToOrderedSlice/10_unique_identifiers-8          520B ± 0%      184B ± 0%  -64.62%  (p=0.000 n=10+10)
ToOrderedSlice/100_unique_identifiers-8       4.10kB ± 0%    1.82kB ± 0%  -55.75%  (p=0.000 n=10+10)
ToOrderedSlice/1000_unique_identifiers-8      32.8kB ± 0%    16.4kB ± 0%  -49.95%  (p=0.000 n=10+10)
ToOrderedSlice/10000_unique_identifiers-8      826kB ± 0%     164kB ± 0%  -80.16%  (p=0.000 n=10+10)
ToOrderedSlice/100000_unique_identifiers-8    9.25MB ± 0%    1.61MB ± 0%  -82.64%  (p=0.000 n=10+10)

name                                        old allocs/op  new allocs/op  delta
ToOrderedSlice/1_unique_identifiers-8           2.00 ± 0%      2.00 ± 0%     ~     (all equal)
ToOrderedSlice/10_unique_identifiers-8          6.00 ± 0%      2.00 ± 0%  -66.67%  (p=0.000 n=10+10)
ToOrderedSlice/100_unique_identifiers-8         9.00 ± 0%      2.00 ± 0%  -77.78%  (p=0.000 n=10+10)
ToOrderedSlice/1000_unique_identifiers-8        12.0 ± 0%       2.0 ± 0%  -83.33%  (p=0.000 n=10+10)
ToOrderedSlice/10000_unique_identifiers-8       21.0 ± 0%       2.0 ± 0%  -90.48%  (p=0.000 n=10+10)
ToOrderedSlice/100000_unique_identifiers-8      31.0 ± 0%       2.0 ± 0%  -93.55%  (p=0.000 n=10+10)

For ToSLice we use the same changes except the sort operation. In this
case the results are even more impressive: allocations are stable at
just 1 per operation, regardless of set size, and performance is
orders of magnitude faster:

name                                 old time/op    new time/op    delta
ToSlice/1_unique_identifiers-8         83.0ns ± 1%    92.4ns ± 0%    +11.37%  (p=0.000 n=9+8)
ToSlice/10_unique_identifiers-8         295ns ± 4%     749ns ± 3%   +154.21%  (p=0.000 n=10+10)
ToSlice/100_unique_identifiers-8       2.33µs ± 2%    4.28µs ± 2%    +83.62%  (p=0.000 n=9+8)
ToSlice/1000_unique_identifiers-8      25.4µs ±10%    34.7µs ± 1%    +36.51%  (p=0.000 n=10+10)
ToSlice/10000_unique_identifiers-8      215µs ± 2%     543µs ± 1%   +152.15%  (p=0.000 n=9+10)
ToSlice/100000_unique_identifiers-8    2.05ms ± 2%    7.17ms ± 1%   +249.53%  (p=0.000 n=9+9)

name                                 old alloc/op   new alloc/op   delta
ToSlice/1_unique_identifiers-8          16.0B ± 0%     16.0B ± 0%       ~     (all equal)
ToSlice/10_unique_identifiers-8          160B ± 0%      496B ± 0%   +210.00%  (p=0.000 n=10+10)
ToSlice/100_unique_identifiers-8       1.79kB ± 0%    4.08kB ± 0%   +127.68%  (p=0.000 n=10+10)
ToSlice/1000_unique_identifiers-8      16.4kB ± 0%    32.8kB ± 0%    +99.90%  (p=0.000 n=10+10)
ToSlice/10000_unique_identifiers-8      164kB ± 0%     826kB ± 0%   +404.12%  (p=0.000 n=8+10)
ToSlice/100000_unique_identifiers-8    1.61MB ± 0%    9.25MB ± 0%   +475.93%  (p=0.000 n=10+10)

name                                 old allocs/op  new allocs/op  delta
ToSlice/1_unique_identifiers-8           1.00 ± 0%      1.00 ± 0%       ~     (all equal)
ToSlice/10_unique_identifiers-8          1.00 ± 0%      5.00 ± 0%   +400.00%  (p=0.000 n=10+10)
ToSlice/100_unique_identifiers-8         1.00 ± 0%      8.00 ± 0%   +700.00%  (p=0.000 n=10+10)
ToSlice/1000_unique_identifiers-8        1.00 ± 0%     11.00 ± 0%  +1000.00%  (p=0.000 n=10+10)
ToSlice/10000_unique_identifiers-8       1.00 ± 0%     20.00 ± 0%  +1900.00%  (p=0.000 n=10+10)
ToSlice/100000_unique_identifiers-8      1.00 ± 0%     30.00 ± 0%  +2900.00%  (p=0.000 n=10+10)

Signed-off-by: Leandro López <leandro.lopez@grafana.com>
2021-03-25 15:44:43 +01:00
Leandro López
067dc391aa Add IdentifierSet.ToOrderedSlice and ToSlice benchmarks
This will be useful later on when implementing the performance
improvements to compare how much we gain.

Signed-off-by: Leandro López <leandro.lopez@grafana.com>
2021-03-25 15:44:43 +01:00
Stanisław Barzowski
b70cbd441a Anonymous Import Locations
We used to treat dummy paths like <stdin>, <std>, <extvar> as
real import locations, which causes obvious problem for importers.

After this change we consistently pass "" (an empty string) as location
for non-imported paths.

We exposed new functions to properly handle all paths.

The first function family, EvaluateFile* which allow evaluating
a Jsonnet program at a specified importable path. It should
be used in most cases.

The second function family, EvaluateAnonymousSnippet* allows evaluating
a snippet without an importable path. It should be used for situations
like the code passed as a commandline argument, stdin, etc.

The old function family, EvaluateSnippet* is now deprecated, but
works the same as before, i.e. the passed filenames are treated
as imported paths.

Changes are required to custom importers to make sure they satisfy
the refined contract.

Fixes #329.
2020-09-08 17:27:47 +02:00
Stanisław Barzowski
3a245f70d4 Preparation for linter.
* Extract some test utilities to a separate package.
* Rename some test utilities.
* Internally expose DirectChildren.
* Add LocationRange to some non-expr AST parts,
  such as local binds, parameters and object fields.
* Add end-of-file-reached testcases.
2020-03-15 14:38:40 +01:00
Dave Cunningham
724650d358
Add native golang formatter (#388)
Add native golang formatter
2020-03-10 22:19:49 +00:00
Stanisław Barzowski
9ada769ce4 Make golangci-lint happy
Some of the suggestions are minor bug fixes (missing error handling).
2020-03-05 14:54:27 +01:00
Dave Cunningham
0e67cc3c68 Refactor to allow interleaving optional and positional params 2020-03-05 09:33:21 +01:00
Dave Cunningham
8b2b1ba839 Avoid float overflow errors at parse time. 2020-02-27 23:06:53 +00:00
Dave Cunningham
dc5c94ac78
Add fodder to AST nodes (#319)
* Preserve fodder during parsing.
2019-09-24 12:11:22 +01:00
Stanisław Barzowski
e4e1d9112a Fix Bazel config 2019-09-09 13:31:08 +02:00
Stanisław Barzowski
c77f22c99f Eliminate the unnecessary ast.StdAst
The initially empty ast.StdAst was created to break the circular
dependency. The generation of stdlib AST used to depend on
the primary "jsonnet" package, which meant that "jsonnet"
could not depend on it directly. Hence stdlib needed to be put
in the ast package. Now dumpstdlibast no longer depends on Jsonnet,
so we can get rid of this complication.

All code using ast.StdAst should now use astgen.StdAst.
2019-09-01 21:17:29 +02:00
Stanisław Barzowski
82f949e7fe Extract AST processing to separate packages
Making it independent from the jsonnet package breaks the circular
dependency during stdast generation.
2019-08-28 20:16:47 +01:00
Steve Harris
7614fd519c Correct a few complaints reported by "staticcheck" 2019-06-15 01:34:11 +02:00
Steve Harris
7a5297a884 Dump standard library into "astgen" package
Export all fields in struct types in the "ast" package to allow
generating program source to reconstruct their complete values in a
separate package.
2019-06-15 01:34:11 +02:00
Steve Harris
ad72eb0fbb Allow building the Go program with Bazel 2019-06-15 01:34:11 +02:00
Dave Cunningham
df7753e2c7 Bring up-to-date with cpp version 2019-05-22 20:43:37 +01:00
Dave Cunningham
6fa1db7d85 Update cpp submodule 2019-05-16 10:12:09 +01:00
Stanisław Barzowski
33b6dcfa53
Keep object locals only once in AST (#263)
Keep object locals only once in AST

For example this reduces the size of stdlib ast file
roughly 3x. Note that this change doesn't regenerate the stdlib,
so that the diff here is sane.

It is likely to slightly improve performance of code using
a lot of locals (~10% on bench.05.gen.jsonnet).

The desugaring is more strightforward now, and we're back
to desugaring each node exactly once.
2019-05-07 22:43:55 +02:00
Stanisław Barzowski
c195e570ef Share std.jsonnet with cpp-jsonnet
So far `std.jsonnet` needed to be updated seprately from
the cpp-jsonnet submodule. Since we should update it anyway
at the same time (to make sure the tests are running fine),
we can just as well get it directly from there, eliminating
the extra step of copying the new `std.jsonnet` version.

This change updates the cpp-jsonnet used.
2019-04-16 20:35:19 +02:00
Stanisław Barzowski
9348a5d2b1 Update cpp-jsonnet submodule and stdlib
Current go-jsonnet is not really 0.12.1 - it even
reports 0.12.0. This brings all the good stuff
from recent cpp-jsonnet commits and actually syncs
the version.
2019-03-04 09:45:11 -05:00
Stanisław Barzowski
7012604878 Update stdlib 2018-12-12 21:01:20 +00:00
Dave Cunningham
95e1c1790a
Lex fodder in a manner compatible with C++ (#231)
* Lex fodder in a manner compatible with C++
2018-06-11 14:40:28 -04:00
Dave Cunningham
d9b833b8a0
Bump C++ version, implement std.trace, fix parsing of -$, update std (#224) 2018-05-23 22:24:12 -04:00
Roger Peppe
fde815f6a1 godoc.org-compatible ast package
Change the dump code so that it hides the values of variable
definitions if they're large. This means that godoc.org should
be able to deal with the output, and the godoc output is
readable without needing to read through a huge struct literal definition.

Other approaches might be to always generate an extra variable
(seems unnecessary) or to pass to writer explicitly to the dump
methods rather than swapping s.w out temporarily. The former
seems unnecessarily intrusive to the usual output; the latter
seemed unnecessarily intrusive to the source itself.
YMMV.
2018-04-12 14:06:52 -04:00
Dave Cunningham
0b46b25f0f Update to cpp jsonnet 2018-03-16 22:20:53 -04:00
Dave Cunningham
6efde9677d Handle base case of cloneForSpec 2018-03-15 13:56:17 -04:00
Dave Cunningham
405726fae2
Clone ASTs to avoid aliasing and double-unescaping (#210)
* Clone ASTs to avoid aliasing and double-unescaping
2018-03-07 11:23:32 -05:00
Dave Cunningham
5cc426ed04 Bring submodule and stdlib to cpp master 2018-03-06 23:49:02 -05:00
Marcelo Cantos
88519c3704 Apply more golint recommendations (#201)
* Apply more golint recommendations

* Update dumpstdlibast.go to include StdAst comment

* Improve dump.go package comment.
2018-02-28 01:11:18 -05:00
Marcelo Cantos
ed5f280c59 Apply golint recommendations (#196)
* Apply golint recommendations
2018-02-26 14:12:25 -05:00
Stanisław Barzowski
44770e75dd Add std.sign and fix division by zero in stdlib 2018-02-21 20:31:04 -05:00
Dave Cunningham
2ea01b24d8
Introduce a Parens AST (#189)
* Introduce a Parens AST
2018-02-16 13:01:14 -05:00
Dave Cunningham
f46dea2835 Update std.jsonnet 2018-02-01 14:21:39 -05:00
Dave Cunningham
741f9f06a2 Update stdlib and ||| fix 2018-01-23 23:26:28 -05:00
Dave Cunningham
7c8f4d0b12
Call top level function even if there are no TLAs (match cpp semantics) (#169)
* Call top level function even if there are no TLAs (match cpp semantics)
2018-01-10 20:43:55 -05:00
Liang Mingqiang
ce84685fb0 Sort FreeVariables slice (#148)
* sort FreeVariables slice

* regenerate the standard library AST
2017-11-09 08:55:36 -05:00
Dave Cunningham
b04e73e163 Fix unparsing of enums, avoid static type error 2017-11-08 11:28:38 -05:00
Liang Mingqiang
aeda6138e5 experiment dump ast to source code (#101)
* Add facility to dump part of Go heap as Go code.

* Generate stdlib AST as Go code

* Use hardcoded stdlib AST instead of parsing it
2017-11-07 23:31:11 -05:00
Stanisław Barzowski
ba0f236b14 Optional eval (#92)
* Optional arguments
2017-10-10 00:06:14 -04:00
Stanisław Barzowski
f0f70419f8 Clean up some TODOs
Some were stale, some were transformed into issues, some were fixed
2017-10-05 22:13:09 -04:00
Stanisław Barzowski
80ce6ac112 Don't escape block strings (#98)
* Don't escape block strings
2017-10-03 14:29:12 -04:00
Stanisław Barzowski
c3459153df Location, error formatting and stack trace improvements (#59)
* Location, error formatting and stack trace improvements

* Static context for AST nodes
* Thunks no longer need `name`
* Prototype for showing snippets in error messages (old format still
available)
* Use ast.Function to represent methods and local function sugar.
* Change tests so that the error output is pretty
2017-10-03 14:27:44 -04:00
Stanisław Barzowski
a1b8248e84 Fix comprehensions (#90)
* Fix comprehensions

Only first and last forspec was used due to a parser bug.
2017-09-29 23:33:55 -04:00
Stanisław Barzowski
0b52ea4d40 Support for various kinds of strings in imports 2017-09-29 12:34:21 -04:00
Stanisław Barzowski
0c86c9e109 Support of +:, in, in super
Also profiling
2017-09-28 22:21:46 -04:00
Stanisław Barzowski
78b4794523 Object comprehensions 2017-09-27 18:00:30 -04:00