dns/generate.go
Tom Thorogood 274da7d3ef
Add new ZoneParser API (#794)
* Improve ParseZone tests

* Add new ZoneParser API

* Use the ZoneParser API directly in ReadRR

* Merge parseZoneHelper into ParseZone

* Make generate string building slightly more efficient

* Add SetDefaultTTL method to ZoneParser

This makes it possible for external consumers to implement ReadRR.

* Make $INCLUDE directive opt-in

The $INCLUDE directive opens a user controlled file and parses it as
a DNS zone file. The error messages may reveal portions of sensitive
files, such as:
	/etc/passwd: dns: not a TTL: "root0:0:root:/root:/bin/bash" at line: 1:31
	/etc/shadow: dns: not a TTL: "root:$6$<redacted>::0:99999:7:::" at line: 1:125

Both ParseZone and ReadRR are currently opt-in for backward
compatibility.

* Disable $INCLUDE support in ReadRR

ReadRR and NewRR are often passed untrusted input. At the same time,
$INCLUDE isn't really useful for ReadRR as it only ever returns the
first record.

This is a breaking change, but it currently represents a slight
security risk.

* Document the need to drain the ParseZone chan

* Cleanup the documentation of NewRR, ReadRR and ParseZone

* Document the ZoneParser API

* Deprecated the ParseZone function

* Add whitespace to ZoneParser.Next

* Remove prevName field from ZoneParser

This doesn't track anything meaningful as both zp.prevName and h.Name
are only ever set at the same point and to the same value.

* Use uint8 for ZoneParser.include field

It has a maximum value of 7 which easily fits within uint8.

This reduces the size of ZoneParser from 160 bytes to 152 bytes.

* Add setParseError helper to ZoneParser

* Surface $INCLUDE os.Open error in error message

* Rename ZoneParser.include field to includeDepth

* Make maximum $INCLUDE depth a const

* Add ParseZone and ZoneParser benchmarks

* Parse $GENERATE directive with a single ZoneParser

This should be more efficient than calling NewRR for each generated
record.

* Run go fmt on generate_test.go

* Add a benchmark for $GENERATE directives

* Use a custom reader for generate

This avoids the overhead and memory usage of building the zone string.

name         old time/op    new time/op    delta
Generate-12     165µs ± 4%     157µs ± 2%   -5.06%  (p=0.000 n=25+25)

name         old alloc/op   new alloc/op   delta
Generate-12    42.1kB ± 0%    31.8kB ± 0%  -24.42%  (p=0.000 n=20+23)

name         old allocs/op  new allocs/op  delta
Generate-12     1.56k ± 0%     1.55k ± 0%   -0.38%  (p=0.000 n=25+25)

* Return correct ParseError from generateReader

The last commit made these regular errors while they had been
ParseErrors before.

* Return error message as string from modToPrintf

This is slightly simpler and they don't need to be errors.

* Skip setting includeDepth in generate

This sub parser isn't allowed to use $INCLUDE directives anyway.

Note: If generate is ever changed to allow $INCLUDE directives, then
      this line must be added back. Without doing that, it would be
      be possible to exceed maxIncludeDepth.

* Make generateReader errors sticky

ReadByte should not be called after an error has been returned, but
this is cheap insurance.

* Move file and lex fields to end of generateReader

These are only used for creating a ParseError and so are unlikely to be
accessed.

* Don't return offset with error in modToPrintf

Along for the ride, are some whitespace and style changes.

* Add whitespace to generate and simplify step

* Use a for loop instead of goto in generate

* Support $INCLUDE directives inside $GENERATE directives

This was previously supported and may be useful. This is now more
rigorous as the maximum include depth is respected and relative
$INCLUDE directives are now supported from within $GENERATE.

* Don't return any lexer tokens after read error

Without this, read errors are likely to be lost and become parse errors
of the remaining str. The $GENERATE code relies on surfacing errors from
the reader.

* Support $INCLUDE in NewRR and ReadRR

Removing $INCLUDE support from these is a breaking change and should
not be included in this pull request.

* Add test to ensure $GENERATE respects $INCLUDE support

* Unify TestZoneParserIncludeDisallowed with other tests

* Remove stray whitespace from TestGenerateSurfacesErrors

* Move ZoneParser SetX methods above Err method

* $GENERATE should not accept step of 0

If step is allowed to be 0, then generateReader (and the code it
replaced) will get stuck in an infinite loop.

This is a potential DOS vulnerability.

* Fix ReadRR comment for file argument

I missed this previosuly. The file argument is also used to
resolve relative $INCLUDE directives.

* Prevent test panics on nil error

* Rework ZoneParser.subNext

This is slightly cleaner and will close the underlying *os.File even
if an error occurs.

* Make ZoneParser.generate call subNext

This also moves the calls to setParseError into generate.

* Report errors when parsing rest of $GENERATE directive

* Report proper error location in $GENERATE directive

This makes error messages much clearer.

* Simplify modToPrintf func

Note: When width is 0, the leading 0 of the fmt string is now excluded.
      This should not alter the formatting of numbers in anyway.

* Add comment explaining sub field

* Remove outdated error comment from generate
2018-10-20 11:47:56 +10:30

243 lines
4.7 KiB
Go

package dns
import (
"bytes"
"fmt"
"io"
"strconv"
"strings"
)
// Parse the $GENERATE statement as used in BIND9 zones.
// See http://www.zytrax.com/books/dns/ch8/generate.html for instance.
// We are called after '$GENERATE '. After which we expect:
// * the range (12-24/2)
// * lhs (ownername)
// * [[ttl][class]]
// * type
// * rhs (rdata)
// But we are lazy here, only the range is parsed *all* occurrences
// of $ after that are interpreted.
func (zp *ZoneParser) generate(l lex) (RR, bool) {
token := l.token
step := 1
if i := strings.IndexByte(token, '/'); i >= 0 {
if i+1 == len(token) {
return zp.setParseError("bad step in $GENERATE range", l)
}
s, err := strconv.Atoi(token[i+1:])
if err != nil || s <= 0 {
return zp.setParseError("bad step in $GENERATE range", l)
}
step = s
token = token[:i]
}
sx := strings.SplitN(token, "-", 2)
if len(sx) != 2 {
return zp.setParseError("bad start-stop in $GENERATE range", l)
}
start, err := strconv.Atoi(sx[0])
if err != nil {
return zp.setParseError("bad start in $GENERATE range", l)
}
end, err := strconv.Atoi(sx[1])
if err != nil {
return zp.setParseError("bad stop in $GENERATE range", l)
}
if end < 0 || start < 0 || end < start {
return zp.setParseError("bad range in $GENERATE range", l)
}
zp.c.Next() // _BLANK
// Create a complete new string, which we then parse again.
var s string
for l, ok := zp.c.Next(); ok; l, ok = zp.c.Next() {
if l.err {
return zp.setParseError("bad data in $GENERATE directive", l)
}
if l.value == zNewline {
break
}
s += l.token
}
r := &generateReader{
s: s,
cur: start,
start: start,
end: end,
step: step,
file: zp.file,
lex: &l,
}
zp.sub = NewZoneParser(r, zp.origin, zp.file)
zp.sub.includeDepth, zp.sub.includeAllowed = zp.includeDepth, zp.includeAllowed
zp.sub.SetDefaultTTL(defaultTtl)
return zp.subNext()
}
type generateReader struct {
s string
si int
cur int
start int
end int
step int
mod bytes.Buffer
escape bool
eof bool
file string
lex *lex
}
func (r *generateReader) parseError(msg string, end int) *ParseError {
r.eof = true // Make errors sticky.
l := *r.lex
l.token = r.s[r.si-1 : end]
l.column += r.si // l.column starts one zBLANK before r.s
return &ParseError{r.file, msg, l}
}
func (r *generateReader) Read(p []byte) (int, error) {
// NewZLexer, through NewZoneParser, should use ReadByte and
// not end up here.
panic("not implemented")
}
func (r *generateReader) ReadByte() (byte, error) {
if r.eof {
return 0, io.EOF
}
if r.mod.Len() > 0 {
return r.mod.ReadByte()
}
if r.si >= len(r.s) {
r.si = 0
r.cur += r.step
r.eof = r.cur > r.end || r.cur < 0
return '\n', nil
}
si := r.si
r.si++
switch r.s[si] {
case '\\':
if r.escape {
r.escape = false
return '\\', nil
}
r.escape = true
return r.ReadByte()
case '$':
if r.escape {
r.escape = false
return '$', nil
}
mod := "%d"
if si >= len(r.s)-1 {
// End of the string
fmt.Fprintf(&r.mod, mod, r.cur)
return r.mod.ReadByte()
}
if r.s[si+1] == '$' {
r.si++
return '$', nil
}
var offset int
// Search for { and }
if r.s[si+1] == '{' {
// Modifier block
sep := strings.Index(r.s[si+2:], "}")
if sep < 0 {
return 0, r.parseError("bad modifier in $GENERATE", len(r.s))
}
var errMsg string
mod, offset, errMsg = modToPrintf(r.s[si+2 : si+2+sep])
if errMsg != "" {
return 0, r.parseError(errMsg, si+3+sep)
}
if r.start+offset < 0 || r.end+offset > 1<<31-1 {
return 0, r.parseError("bad offset in $GENERATE", si+3+sep)
}
r.si += 2 + sep // Jump to it
}
fmt.Fprintf(&r.mod, mod, r.cur+offset)
return r.mod.ReadByte()
default:
if r.escape { // Pretty useless here
r.escape = false
return r.ReadByte()
}
return r.s[si], nil
}
}
// Convert a $GENERATE modifier 0,0,d to something Printf can deal with.
func modToPrintf(s string) (string, int, string) {
// Modifier is { offset [ ,width [ ,base ] ] } - provide default
// values for optional width and type, if necessary.
var offStr, widthStr, base string
switch xs := strings.Split(s, ","); len(xs) {
case 1:
offStr, widthStr, base = xs[0], "0", "d"
case 2:
offStr, widthStr, base = xs[0], xs[1], "d"
case 3:
offStr, widthStr, base = xs[0], xs[1], xs[2]
default:
return "", 0, "bad modifier in $GENERATE"
}
switch base {
case "o", "d", "x", "X":
default:
return "", 0, "bad base in $GENERATE"
}
offset, err := strconv.Atoi(offStr)
if err != nil {
return "", 0, "bad offset in $GENERATE"
}
width, err := strconv.Atoi(widthStr)
if err != nil || width < 0 || width > 255 {
return "", 0, "bad width in $GENERATE"
}
if width == 0 {
return "%" + base, offset, ""
}
return "%0" + widthStr + base, offset, ""
}