Download progress messages will not currently appear via the syslog
console (or any other line buffering console) until the download
completes and the newline character is printed.
Use the ASCII unit separator (US) character to indicate the end of
each per-second progress message, and modify the line console to react
to this character by sending out the incomplete line.
Suggested-by: Christian I. Nilsson <nikize@gmail.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
For a UEFI HTTP boot, we set the current working URI based on the
loaded image device path. The autoexec.ipxe script will be fetched
from the same directory as the iPXE binary itself.
For a BIOS or UEFI PXE boot, we do not explicitly set a current
working URI, but rely on the fact that registering the cached DHCP
settings block will cause the TFTP code to set the current working URI
to "tftp://${next-server}/". The autoexec.ipxe script will therefore
be fetched from the default directory (which is most probably the root
directory) of the TFTP server.
When using a UEFI shim, the shim will always fetch iPXE from the same
directory as the shim itself. This leads to a somewhat unintuitive
requirement for a UEFI PXE boot: the shim and iPXE must be placed in
the same directory, but the corresponding autoexec.ipxe script must be
placed in the root directory.
As with the loaded image device path for a UEFI HTTP boot, the
existence of a cached DHCP packet gives us a way to construct the URI
of our own binary. We can therefore choose to use this to set the
current working URI, so that the autoexec.ipxe script may be placed in
the same directory as the iPXE binary itself. This is the least
surprising location, and avoids the need for lengthy explanations in
documentation.
Choose to set the current working URI at the point that the cached
DHCP packet is recorded, rather than the point at which it is applied
and registered as a settings block. This avoids some awkward corner
cases (such as failing to find a matching network device for the
DHCPACK), and naturally ensures that we retrieve the next-server
address and filename from the same DHCP packet. We rely on the order
in which cached DHCP packets are recorded to impose a priority
ordering: later packets (e.g. PxeBSACK) will override earlier ones.
To avoid breaking existing setups that do place the autoexec.ipxe
script in the root directory, we modify the fetching logic to first
attempt to retrieve autoexec.ipxe from the current working URI, then
from the root directory of that URI.
As with commit a69afd7 ("[tftp] Use TFTP server URI only if no other
working URI is set"), this is technically a breaking change in
behaviour, but the new behaviour is almost certainly less surprising
than the existing behaviour. Scripts that rely on the current working
URI being set to the root of the TFTP server can use absolute URIs
(i.e. add an initial slash): this is more explicit and will work on
iPXE builds both before and after this change.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
It is unintuitive to have to include an "ifopen" at the start of an
autoexec.ipxe script. Provide a mechanism for upper-layer drivers to
mark a network device to be opened automatically upon registration,
and do so for the device to which the cached DHCPACK is applied.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When searching for a shortcut key, search first from the currently
selected menu item and then from the start of the list.
This allows several ways for a shortcut key to be meaningfully used
multiple times within the same menu. For example, two sections may
have the same shortcut key:
item --key s --gap (S)ection 1
item ...
item ...
item --key s --gap (S)ection 2
item ...
With the above menu, repeated "s" keypresses would cycle through the
sections.
As another example, entries within different sections may have the
same shortcut keys. For example:
item --key d --gap (D)ebian
item --key s debst Debian (s)table release
item --key u debun Debian (u)nstable release
item --key f --gap (F)edora
item --key s fedst Fedora (s)table release
item --key u fedun Fedora (u)nstable release
With the above menu, a shortcut key sequence such as "f", "s" can be
used to select an entry within a specific section, avoiding the need
to choose shortcut keys that are globally unique within the menu.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow data to be successfully written (and discarded) to a void data
transfer buffer, rather than throwing an error. This allows a void
data transfer buffer to be used when determining the length of a file
downloaded from a TFTP server that does not support the "tsize" option
defined in RFC 2349.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Record the maximum size required when writing into a data transfer
buffer. This allows the maximum size to be determined even if
allocation fails (e.g. due to a fixed-size buffer or an out-of-memory
condition).
In the case of a fixed-size buffer (which may already be larger than
required), this allows the caller to determine the actual size used
for written data.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some older distributions (such as RHEL 8) provide their AArch64
kernels as gzip-compressed EFI binaries (with no self-decompressing
EFI stub present). We therefore enable support for gzip images by
default for arm64 EFI builds.
Review the files used to implement the gzip (and zlib) formats and
mark these as permitted for UEFI Secure Boot.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Enabling the GDB debugger functionality would provide an immediate and
trivial Secure Boot exploit. Mark all GDB-related files as explicitly
forbidden for UEFI Secure Boot.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Mark dynamic keyboard map support and the "pciscan", "usbscan", and
"time" commands as permitted for UEFI Secure Boot, on the basis that
these features have previously been present in binaries signed by
Microsoft.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some past security reviews carried out for UEFI Secure Boot signing
submissions have covered specific drivers or functional areas of iPXE.
Mark all of the files comprising these areas as permitted for UEFI
Secure Boot.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Mark all files used in a standard build of bin-x86_64-efi/snponly.efi
as permitted for UEFI Secure Boot. These files represent the core
functionality of iPXE that is guaranteed to have been included in
every binary that was previously subject to a security review and
signed by Microsoft. It is therefore legitimate to assume that at
least these files have already been reviewed to the required standard
multiple times.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit 2d180ce ("[tcp] Update maximum window size to 2MB") increased
the TCP window size to avoid filling the TCP window on typical modern
links.
The total heap size is only 512kB. Given that RX I/O buffers are
typically subject to alignment constraints, it is plausible that we
may be able to actually buffer only 256kB of data before having to
discard queued out-of-order packets.
On a low latency network, this behaviour is not a problem: the sender
will rapidly retransmit the lost or discarded packets. On a high
latency network, the sender's congestion control algorithm will end up
calculating a congestion window that is substantially smaller than our
advertised 2MB, which will result in a drastic reduction in actual
throughput.
We do not want to increase the heap size arbitrarily, since we still
have the constraint that memory used by iPXE may be permanently lost
to the operating system (depending on how the operating system is
booted). However, the cost of keeping the heap size down to 512kB is
no longer acceptable given that large downloads over high-speed
wide-area networks are now routine.
Increase the heap size from 512kB to 4MB. This should be sufficient
to hold an entire 2MB TCP window for a single connection under most
sensible conditions. For example:
* 1460-byte MSS => 1436 packets => 2872kB of 2kB RX I/O buffers
* 8960-byte MSS => 234 packets => 3744kB of 16kB RX I/O buffers
The notable exception is that of a network where jumbo frames are in
use, but the TCP connection ends up using a standard 1460-byte MSS.
If this is found to be an issue in practice, then one possible
solution would be to shrink (or reallocate) I/O buffers for
out-of-order queued data.
Experimentation shows that before this change, an induced latency of
25ms (representative of a typical connection to a public cloud
provider) would cause the download speed to vary unpredictably between
2MB/s and 25MB/s. After this change, the speed in this test scenario
remains consistently high at 25MB/s.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some systems (observed on an AWS EC2 m7i.metal-24xl instance in
eu-south-2) use the newer "16550-compatible with parameters defined in
Generic Address Structure" type value. (There does not appear to be
any particular reason why the newer value needs to be used: the UART
is still a standard 16550 with single-byte registers.)
Accept this additional type value for a 16550-compatible UART.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Assume that on any platforms where port I/O is used (i.e. x86), a port
I/O address may be used directly for the combined MMIO and port I/O
accessors without requiring an explicit mapping operation.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The BIOS may provide an ACPI Serial Port Console Redirection (SPCR)
table to describe the serial port to be used for early boot messages.
Add support for parsing the SPCR and instantiating a 16550-based UART.
We do not currently attempt to support other types of UART, since iPXE
does not yet have drivers for other types.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
An ACPI Generic Address Structure (GAS) may be used to describe the
location of a peripheral such as an early boot console. Add the
relevant definitions and provide acpi_ioremap() as a helper function
to map a region described using this structure.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Make the current baud rate (if specified) a property of the UART, to
allow the default_serial_console() function to specify the default
baud rate as well as the default UART device.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Many region types (e.g. I2C bus addresses) can only ever contain a
single region with no size cells specified. Provide fdt_reg() to
reduce boilerplate in this common use case.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commands were originally ordered by functional group (e.g. keeping the
image management commands together), with arrays used to impose a
functionally meaningful order within the group.
As the number of commands and functional groups has expanded over the
years, this has become essentially useless as an organising principle.
Switch to sorting commands alphabetically (using the linker table
mechanism).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Consumption of phandles will be in the form of locating a functional
device (e.g. a GPIO device, or an I2C device, or a reset controller)
by phandle, rather than locating the device tree node to which the
phandle refers.
Repurpose fdt_phandle() to obtain the phandle value (instead of
searching by phandle), and record this value as the bus location
within the generic device structure.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When debugging is enabled for the device tree or memory map parsing
code, the active serial console UART variable will be accessed during
early initialisation, before the .bss section has been zeroed.
Place this variable in the .data section (by providing an explicit
initialiser), so that reading this variable is well defined even
during early initialisation.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The data cache must be invalidated twice for RX DMA buffers: once
before passing ownership to the DMA device (in case the cache happens
to contain dirty data that will be written back at an undefined future
point), and once after receiving ownership from the DMA device (in
case the CPU happens to have speculatively accessed data in the buffer
while it was owned by the hardware).
Only the used portion of the buffer needs to be invalidated after
completion, since we do not care about data within the unused portion.
Update the DMA API to include the used length as an additional
parameter to dma_unmap(), and add the necessary second cache
invalidation pass to the RISC-V DMA API implementation.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Cache management operations must generally be performed on virtual
addresses rather than physical addresses.
Change the address parameter in dma_map() to be a virtual address, and
make dma() the API-level primitive instead of dma_phys().
Signed-off-by: Michael Brown <mcb30@ipxe.org>
On platforms where DMA devices are not in the same coherency domain as
the CPU cache, we must ensure that DMA I/O buffers do not share
cachelines with other data.
Align the start and end of I/O buffers to IOB_ZLEN, which is larger
than any cacheline size we expect to encounter.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add support for probing a device based on the path or alias found in
the "/chosen/stdout-path" node, and using a consequently instantiated
UART as the default serial console.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow the platform configuration to provide a mechanism for
identifying the serial console UART. Provide two globally available
mechanisms: "null" (i.e. no serial console), and "fixed" (i.e. use
whatever is specified by COMCONSOLE in config/serial.h).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Remove the assumption that all platforms use a fixed number of 16550
UARTs identifiable by a simple numeric index. Create an abstraction
allowing for dynamic instantiation and registration of any number of
arbitrary UART models.
The common case of the serial console on x86 uses a single fixed UART
specified at compile time. Avoid unnecessarily dragging in the
dynamic instantiation code in this use case by allowing COMCONSOLE to
refer to a single static UART object representing the relevant port.
When selecting a UART by command-line argument (as used in the
"gdbstub serial <port>" command), allow the UART to be specified as
either a numeric index (to retain backwards compatiblity) or a
case-insensitive port name such as "COM2".
Signed-off-by: Michael Brown <mcb30@ipxe.org>
In the context of serial consoles, the use of any frame formats other
than the standard 8 data bits, no parity, and one stop bit is so rare
as to be nonexistent.
Remove the almost certainly unused support for custom frame formats.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Non-permitted name characters such as a colon are sometimes used to
separate alias names or paths from additional metadata, such as the
baud rate for a UART in the "/chosen/stdout-path" property.
Support the use of such alias names and paths by allowing any
character not permitted in a property name to terminate a property or
node name match. (This is a very relaxed matching rule that will
produce false positive matches on invalid input, but this is unlikely
to cause problems in practice.)
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Devices with only 32-bit DMA addressing are relatively common even on
systems with 64-bit CPUs. Limit relocation of iPXE to 32-bit address
space so that I/O buffers and other DMA allocations will be accessible
by 32-bit devices.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Provide DBGC_MEMMAP() as a replacement for memmap_dump(), allowing the
colour used to match other messages within the same message group.
Retain a dedicated colour for output from memmap_dump_all(), on the
basis that it is generally most useful to visually compare full memory
dumps against previous full memory dumps.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Use the terminology "min" and "max" for addresses covered by a memory
region descriptor, since this is sufficiently intuitive to generally
not require further explanation.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
We currently rely on implicit detection of the external heap region.
The INT 15 memory map mangler relies on examining the corresponding
in-use memory region, and the initrd reshuffler relies on performing a
separate detection of the largest free memory block after startup has
completed.
Replace these with explicit public symbols to describe the external
heap region.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
If the external heap ends up at the top of the system memory map then
leave a gap after the heap to ensure that no block ends up being
allocated with either a start or end address of zero, since this is
frequently confusing to both code and humans.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow for relocation to a region at the very end of the physical
address space (where the next address wraps to zero).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow an initrd location to be specified in our constructed device
tree via the "linux,initrd-start" and "linux,initrd-end" properties.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Use image_replace() to transfer execution to the extracted image,
rather than calling image_exec() directly. This allows the original
archive image to be freed immediately if it was marked as an
automatically freeable image (e.g. via "chain --autofree").
In particular, this ensures that in the case of an archive image
containing another archive image (such as an EFI zboot kernel wrapper
image containing a gzip-compressed kernel image), the intermediate
extracted image will be freed as early as possible, since extracted
images are always marked as automatically freeable.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add an implementation of umalloc() using the generalised model of a
heap, placing the external heap in the largest usable region obtained
from the system memory map.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Size-tracked pointers allocated via umalloc() have historically been
aligned to a page boundary, as have the edges of the hidden memory
region covering the external heap.
Allow the block and size-tracked pointer alignments to be specified as
heap configuration parameters.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Create a generic model of a heap as a list of free blocks with
optional methods for growing and shrinking the heap.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
